I have an API endpoint that accepts some data from the client. There is also a 1 minute timer which is visible to the client.
What I hope to achieve is this:
Whilst the timer is active ( > 0 ) any posts sent to the API are kept in storage or in array ( or something ). Once the timer reaches zero, The client can no longer make a request to the API and any requests that were made and stored whilst the timer was active, are now processed through a function - For sake of example lets just say that this function logs all the data to the screen.
Perhaps i'm thinking of this in the wrong way, but how do I sync a front and backend timer so both the server and the client side know when to stop processing POST requests and to let the server know that it's time to process all the data that was sent during that 1 minute.
var express = require("express");
var app = express();
app.post("/api/data", function(req, res){
// do something here - no clue
});
app.listen(process.env.PORT, () => {
console.log("server running on port: ", process.env.PORT);
});
Apologies if I've explained this poorly.
Appreciate any help I can get, thank you :)
What do you want to achieve, because letting the client decide something is the worst choice you can make.
Maybe pass the server time to the client. Calculate the difference and then start counting. But check on the server if the client is actually allowed to post.
or let the server calculate how many seconds there are left until countdown reached and pass these to the client. But still server needs to check the valid time.
I would pick one of those. Let the server be the deciding factor and don't depend on the client. Since you can easily change PC time.
When the timer on the client side reaches 0 send a api call to the server. When the server gets this api call puts a boolean to false. If this boolean is false, you ignore data. What I mean in code is something like this
var ignore = false;
app.post("/api/data/stop", function(req,res){
ignore = true;
});
app.post("/api/data", function(req, res){
if(!ignore){
// listening to data
}
});
app.listen(process.env.PORT, () => {
console.log("server running on port: ", process.env.PORT);
});
Related
I send JSON requests one by one to the nodejs server. After 6th request, server can't reply to the client immediately and then it takes a little while(15 seconds or little bit more and send back to me answer 200 ok) It occurs a writing json value into MongoDB and time is important option for me in terms with REST call. How can I find the error in this case? (which tool or script code can help me?) My server side code is like that
var controlPathDatabaseSave = "/save";
app.use('/', function(req, res) {
console.log("req body app use", req.body);
var str= req.path;
if(str.localeCompare(controlPathDatabaseSave) == 0)
{
console.log("controlPathDatabaseSave");
mongoDbHandleSave(req.body);
res.setHeader('Content-Type', 'application/json');
res.write('Message taken: \n');
res.write('Everything all right with database saving');
res.send("OK");
console.log("response body", res.body);
}
});
My client side code as below:
function saveDatabaseData()
{
console.log("saveDatabaseData");
var oReq = new XMLHttpRequest();
oReq.open("POST", "http://192.168.80.143:2800/save", true);
oReq.setRequestHeader("Content-type", "application/json;charset=UTF-8");
oReq.onreadystatechange = function() {//Call a function when the state changes.
if(oReq.readyState == 4 && oReq.status == 200) {
console.log("http responseText", oReq.responseText);
}
}
oReq.send(JSON.stringify({links: links, nodes: nodes}));
}
--Mongodb save code
function mongoDbHandleSave(reqParam){
//Connect to the db
MongoClient.connect(MongoDBURL, function(err, db)
{
if(!err)
{
console.log("We are connected in accordance with saving");
} else
{
return console.dir(err);
}
/*
db.createCollection('user', {strict:true}, function(err, collection) {
if(err)
return console.dir(err);
});
*/
var collection = db.collection('user');
//when saving into database only use req.body. Skip JSON.stringify() function
var doc = reqParam;
collection.update(doc, doc, {upsert:true});
});
}
You can see my REST call in google chrome developer editor. (First six call has 200 ok. Last one is in pending state)
--Client output
--Server output
Thanks in advance,
Since it looks like these are Ajax requests from a browser, each browser has a limit on the number of simultaneous connections it will allow to the same host. Browsers have varied that setting over time, but it is likely in the 4-6 range. So, if you are trying to run 6 simultaneous ajax calls to the same host, then you may be running into that limit. What the browser does is hold off on sending the latest ones until the first ones finish (thus avoiding sending too many at once).
The general idea here is to protect servers from getting beat up too much by one single client and thus allow the load to be shared across many clients more fairly. Of course, if your server has nothing else to do, it doesn't really need protecting from a few more connections, but this isn't an interactive system, it's just hard-wired to a limit.
If there are any other requests in process (loading images or scripts or CSS stylesheets) to the same origin, those will count to the limit too.
If you run this in Chrome and you open the network tab of the debugger, you could actually see on the timline exactly when a given request was sent and when its response was received. This should show you immediately whether the later requests are being held up at the browser or at the server.
Here's an article on the topic: Maximum concurrent connections to the same domain for browsers.
Also, keep in mind that, depending upon what your requests do on the server and how the server is structured, there may be a maximum number of server requests that can efficiently processed at once. For example, if you had a blocking, threaded server that was configured with one thread for each of four CPUs, then once the server has four requests going at once, it may have to queue the fifth request until the first one is done causing it to be delayed more than the others.
I'm running a small Angular application with a Node/Express backend.
In one of my Angular factories (i.e. on the client side) I make a $http request to Github to return user info. However, a Github-generated key (which is meant to be kept secret) is required to do this.
I know I can't use process.env.XYZ on the client side. I'm wondering how I could keep this api key a secret? Do I have to make the request on the back end instead? If so, how do I transfer the returned Github data to the front end?
Sorry if this seems simplistic but I am a relative novice, so any clear responses with code examples would be much appreciated. Thank you
Unfortunately you have to proxy the request on your backend to keep the key secret. (I am assuming that you need some user data that is unavailable via an unauthenticated request like https://api.github.com/users/rsp?callback=foo because otherwise you wouldn't need to use API keys in the first place - but you didn't say specifically what you need to do so it is just my guess).
What you can do is something like this: In your backend you can add a new route for your frontend just for getting the info. It can do whatever you need - using or not any secret API keys, verify the request, process the response before returning to your client etc.
Example:
var app = require('express')();
app.get('/github-user/:user', function (req, res) {
getUser(req.params.user, function (err, data) {
if (err) res.json({error: "Some error"});
else res.json(data);
});
});
function getUser(user, callback) {
// a stub function that should do something more
if (!user) callback("Error");
else callback(null, {user:user, name:"The user "+user});
}
app.listen(3000, function () {
console.log('Listening on port 3000');
});
In this example you can get the user info at:
http://localhost:3000/github-user/abc
The function getUser should make an actual request to GitHub and before you call it you can change if that is really your frontend that is making the request e.g. by cheching the "Referer" header or other things, validate the input etc.
Now, if you only need a public info then you may be able to use a public JSON-P API like this - an example using jQuery to make things simple:
var user = prompt("User name:");
var req = $.getJSON('https://api.github.com/users/'+user);
req.then(function (data) {
console.log(data);
});
See DEMO
In browser javascript is pathetically broken in that the only way to make requests is using script tags and jsonp. To make this useful, I'm trying to make a nodejs server that, given a callback name and address, loads the page at the address and pads it in a call to callback and serves the result. However, I know next to nothing about nodejs. If the server's response is loaded from a script tag it would result in actually loading a web page. Currently, I'm writing the request as localhost:8000/callback/address so a script tag might be <script src="localhost:8000/alert/https://www.google.com" type="text/javascript"></script>. Here is my code for the server:
var http = require("http");
var request = require("request");
var server = http.createServer(function(req, res){
req.on("end", function(){
console.log("alive");
var url = req.url;
var i = url.indexOf("/", 1);
request(url.substring(i + 1), function(err, ret, body){
res.writeHead(200);
res.write(url.substring(1, i) + "(\"" + body + "\");");
res.end();
});
});
});
server.listen(8000);
Why does this stay loading for a very long time but never actually load? By using console.log() it seems as if the req.on("end") callback is never even called.
If you don't care about any request data, you could just add req.resume(); after you add your end event handler.
The reason it's getting "stuck" is that since node v0.10, streams start out in a paused state, so you need to unpause them by reading from them in some way. req.resume(); accomplishes this. Once there is nothing left in the request stream (which there could be nothing), the end event will be emitted.
I have a node.js process that uses a large number of client requests to pull information from a website. I am using the request package (https://www.npmjs.com/package/request) since, as it says: "It supports HTTPS and follows redirects by default."
My problem is that after a certain period of time, the requests begin to hang. I haven't been able to determine if this is because the server is returning an infinite data stream, or if something else is going on. I've set the timeout, but after some number of successful requests, some of them eventually get stuck and never complete.
var options = { url: 'some url', timeout: 60000 };
request(options, function (err, response, body) {
// process
});
My questions are, can I shut down a connection after a certain amount of data is received using this library, and can I stop the request from hanging? Do I need to use the http/https libraries and handle the redirects and protocol switching myself in order the get the kind of control I need? If I do, is there a standardized practice for that?
Edit: Also, if I stop the process and restart it, they pick right back up and start working, so I don't think it is related to the server or the machine the code is running on.
Note that in request(options, callback), the callback will be fired when request is completed and there is no way to break the request.
You should listen on data event instead:
var request = require('request')
var stream = request(options);
var len = 0
stream.on('data', function(data) {
// TODO process your data here
// break stream if len > 1000
len += Buffer.byteLength(data)
if (len > 1000) {
stream.abort()
}
})
Update: I posted this code here, after I added all (to 99%) possibilities one by one, and it still gave me a 120sec timeout...buffled.
So, ok, I figured it takes exactly 120sec (ok, 122sec) on my Windows 7 machine, until the FIN handshake is started. I want to do it immediately. HTTP RFC793 says
FIN: No more data from sender
Looks to me I do not send any data anymore. I tried all this bloated code, still a Hello World server...
var http = require('http')
var server = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'})
res.write('HELLO WORLD!\n')
res.end()
res.setTimeout(0)
req.abort() // 1) TypeError: Object #<IncomingMessage> has no method 'abort'
req.on('socket', function (socket) {
socket.setTimeout(0)
socket.on('timeout', function() {
req.abort()
})
})
})
server.timeout = 0
server.setTimeout(0)
server.listen(1337, '192.168.0.101')
So how to do 1) ? (Actually sends a RST like this...)
And how to do the whole thing HTTP conform?
Of course in the end I will be lucky to use nodejs as in websocket stuff and all this, but if conversion on my website means a thing of two minutes, and I have a million concurrent users (huh?), sending a FIN immediately could mean I have two million concurrent users (do the math). ;) Ok, to be on the sure side: Sending a FIN means the socket is closed?
Ah, eah, ok, since you are on, how do I console.log(res) or console.log(req)? It gives me [object Object]. (Update: I tried console.log(res.toSource()), gives me a TypeError?
Edit: Found the answer here.
If you want to close the connection, send a connection: close header. If you do this, then it will not leave the connection open for reuse.