Node.js: wait for the response from an async call - javascript

I am working on an API through which i should be able to add a list of user as JIRA watchers (so i am talking to the JIRa REST API)
Here is the function to do this:
for (var i = 0; i < votes.totalValue; i++) {
var voter = {
user : votes.rawVoteData[i].user,
value : votes.rawVoteData[i].value,
email : votes.rawVoteData[i].email,
fname : votes.rawVoteData[i].fname,
lname : votes.rawVoteData[i].fname
};
// Add these users as watchers to the Jira
jira.jira.addWatcher(issueId, voter.user, function(err, result){
// TODO: Return to callback
rep++;
console.log('user='+voter.user);
console.log(result);
});
}
The votes object has the list of users shown above. Now the problem is that when i execute this function, due to async nature of node, the for loop gets executed completely then all the async calls get fired at once. (which doesn't work for me since the JIRa REST API does not seem to support this)
I want to change the above code so that i make the call to the addWatcher function for a user one at a time & make the next addWAtcher call for the next user only if the previous async call has returned(. i.e. move on to the next async call one by one for each user and not fire all calls to addwatcher all at once.)
How can i do this?
Please advise,
thanks!

you should take a look at async module.
You can use async.each() function. The function takes an array of items, then iterates over them calling a wrapper function which accepts the item as an argument. When all the calls are complete, you specify a final function to be called.
other solution can be async.parallel() (in case you don't mind running them in parallel.
Credit for this great tutorial

Related

Ignore multiple successive function calls in Typescript from within the function?

I am trying to implement autocomplete using Handontable (Angular) by fetching data from the server as the user types.
I notice the API calls are made every time the user input changes but I would like to limit the number of API calls by waiting 1 second for the user to stop typing before making the call.
I have done this in the past when I controlled the event by using debounceTime but not sure how to implement that here.
...
column.source = function(query, callback) {
$component.dataService.getValidValues().subscribe(
arg => {
callback(arg);
},
err => {
...
}
);
}
...
Adding debounceTime(1000) here doesn't prevent multiple calls from happening.
$component.dataService.getValidValues().debounceTime(1000).subscribe(...)
As already explained by others you need to debounce the input. In your case this would be the invocation of the function.
One way to achieve that is using a subject that you create somewhere in your code:
const sourceRequest = new Subject();
sourceRequest.debounceTime(1000).subscribe(callback => {...});
The code you currently have inside function(query, callback) { goes into subscribe. The column definition is then changed to this:
column.source = function(query, callback) {
sourceRequest.next(callback);
}
Assuming that .getValidValues() is the function that makes the request to the remote server, you are debouncing the stream of events coming from this function. You want to debounce the stream of events coming from the user input, which limits the number of calls to .getValidValues().
Try something like this
$component.dataService.debounceTime(1000).getValidValues().subscribe(...)
The only thing I can think of is for you to create a standard Javascript debounce function that wraps the function you are assigning to column.source, kind of like this:
https://stackblitz.com/edit/angular-mq9jyv?file=src%2Fapp%2Fapp.component.ts

Stopping synchronous function after 2 seconds

I'm using the npm library jsdiff, which has a function that determines the difference between two strings. This is a synchronous function, but given two large, very different strings, it will take extremely long periods of time to compute.
diff = jsdiff.diffWords(article[revision_comparison.field], content[revision_comparison.comparison]);
This function is called in a stack that handles an request through Express. How can I, for the sake of the user, make the experience more bearable? I think my two options are:
Cancelling the synchronous function somehow.
Cancelling the user request somehow. (But would this keep the function still running?)
Edit: I should note that given two very large and different strings, I want a different logic to take place in the code. Therefore, simply waiting for the process to finish is unnecessary and cumbersome on the load - I definitely don't want it to run for any long period of time.
fork a child process for that specific task, you can even create a queu to limit the number of child process that can be running in a given moment.
Here you have a basic example of a worker that sends the original express req and res to a child that performs heavy sync. operations without blocking the main (master) thread, and once it has finished returns back to the master the outcome.
Worker (Fork Example) :
process.on('message', function(req,res) {
/* > Your jsdiff logic goes here */
//change this for your heavy synchronous :
var input = req.params.input;
var outcome = false;
if(input=='testlongerstring'){outcome = true;}
// Pass results back to parent process :
process.send(req,res,outcome);
});
And from your Master :
var cp = require('child_process');
var child = cp.fork(__dirname+'/worker.js');
child.on('message', function(req,res,outcome) {
// Receive results from child process
console.log('received: ' + outcome);
res.send(outcome); // end response with data
});
You can perfectly send some work to the child along with the req and res like this (from the Master): (imagine app = express)
app.get('/stringCheck/:input',function(req,res){
child.send(req,res);
});
I found this on jsdiff's repository:
All methods above which accept the optional callback method will run in sync mode when that parameter is omitted and in async mode when supplied. This allows for larger diffs without blocking the event loop. This may be passed either directly as the final parameter or as the callback field in the options object.
This means that you should be able to add a callback as the last parameter, making the function asynchronous. It will look something like this:
jsdiff.diffWords(article[x], content[y], function(err, diff) {
//add whatever you need
});
Now, you have several choices:
Return directly to the user and keep the function running in the background.
Set a 2 second timeout (or whatever limit fits your application) using setTimeout as outlined in this
answer.
If you go with option 2, your code should look something like this
jsdiff.diffWords(article[x], content[y], function(err, diff) {
//add whatever you need
return callback(err, diff);
});
//if this was called, it means that the above operation took more than 2000ms (2 seconds)
setTimeout(function() { return callback(); }, 2000);

How to develop node.js run-time strategy?

Node.js approach is event driven and I was wondering how would you tackle the problem of when to fire off an event?
Lets say that we have some actions on a web application: create some data, serve pages, receive data etc.
How would you lay out these events? In a threaded system the design is rather "simple". You dedicated threads to specific set of tasks and you go down the road of thread synchronization. While these task are at low on demand the threads sit idle and do nothing. When they are needed they run their code. While this road has issues it's well documented and kind of solved.
I find it hard to wrap my head around the node.js event way of doing things.
I have 10 request coming in, but I haven't created any data so I can't serve anying, creating data is a long action and another 5 client wants to send data. What now?
I've created the following untested code which is basically a pile of callbacks which get registered and should be executed. There will be some kind of a pile manager that will run and decide which code does it want to execute now. All the callback created by that callback can be added "naturally" to the even loop. It should also register it's self so the event loop could give the control back to it. Other things like static content and what ever can be bound differently.
How can I register a call back to be the last call in the current event loop state?
Is this a good way to solve this issue?
The most important thing to remember when coming from a threaded environment is that in node you don't wait for an action to finish happening, instead you tell it what to do when it is done. To do this you use a callback, this is a variable which contains a function to execute, or a pointer to a function if you like.
For example:
app.get('/details/:id?', function (req, res) {
var id = req.params.ucid,
publish = function (data) {
res.send(data);
};
service.getDetails(id, publish);
});
You can then invoke the publish method from within your get details method once you have created the required data.
getDetail : function (id, callback) {
var data = makeMyData(id);
callback(data)
}
Which will then publish your data back to the response object. Because of the event loop node will continue to serve requests to this url without interrupting the data generation from the first request
The answer chosen is the most correct, there is but one minor code change and that is:
Change this function from this:
getDetail : function (id, callback) {
var data = makeMyData(id);
callback(data)
}
To that:
getDetail : function (id, callback) {
var data = makeMyData(id);
setTimeout(callback, 0, data);
}
Update 2019:
In order to comply with community standard I've broken off an update to a new answer.
I've used setTimeout because I wanted to defer the callback to the back of the event loop. Another option I've used was process.nextTick(), this helped to defer the callback to the end of the current event processed.
For example:
getDetail : function (id, callback) {
var data = makeMyData(id);
process.nextTick(((info)=> callback(info))(data))
}

javascript outside of scope

Based on chrome developer tools a breakpoints I think I'm dealing with a scope issue I can figure out. Is it the way I define the function? The script below is an include js file and the array ' timeStamp I want available for use in other functions without having to call my loadData function everytime.
The timeStamp array goes undefined once it leaves the for loop before it even leaves the function.
var timeStamp = []; // Want this array to be global
function loadData (url){
$.getJSON(url, function(json) {
for (var i=0;i<json.length;i++){
timeStamp.push(json[i].TimeStamp);
}
console.log(inputBITS); //returns the value
});
console.log(inputBITS); //undefined
}
Thank you for anyhelp
It looks like the issue is that getJSON is asynchronous. When it executes and finishes and your code continues on, it indicates only the START of the networking operation to retrieve the data. The actual networking operation does not complete until some time later.
When it does complete, the success handler is called (as specified as the second argument to your getJSON() call) and you populate the timeStamp array. ONLY after that success handler has been called is the timeStamp array valid.
As such, you cannot use the timeStamp array in code that immediately follows the getJSON() call (it hasn't been filled in yet). If other code needs the timeStamp array, you should call that code from the success handler or use some other timing mechanism to make sure that the code that uses the timeStamp array doesn't try to use it until AFTER the success handler has been called and the timeStamp array has been populated.
It is possible to make some Ajax calls be synchronous instead of asynchronous, but that is generally a very bad idea because it locks up the browser during the entire networking operation which is very unfriendly to the viewer. It is much better to fix the coding logic to work with asynchronous networking.
A typical design pattern for an ajax call like this is as follows:
function loadData (url){
$.getJSON(url, function(json) {
// this will execute AFTER the ajax networking finishes
var timeStamp = [];
for (var i=0;i<json.length;i++) {
timeStamp.push(json[i].TimeStamp);
}
console.log(timeStamp);
// now call other functions that need timeStamp data
myOtherFunc(timeStamp);
});
// this will execute when the ajax networking has just been started
//
// timeStamp data is NOT valid here because
// the ajax call has not yet completed
// You can only use the ajax data inside the success handler function
// or in any functions that you call from there
}
And here's another person who doesn't understand basic AJAX...
getJSON is asynchronous. Meaning, code keeps running after the function call and before the successful return of the JSON request.
You can "fix" this by forcing the request to be synchronous with an appropriate flag, but that's a really bad idea for many reasons (the least of which is that you're violating the basic idea of AJAX). The best way is to remember how AJAX works and instead put all your code that should be executed when the AJAX returns, in the right place.

Sequential web service call not working

A little (!) bit of background before I can get to the question :
I am implementing a web based search solution. Technology used: javascript (jquery), .net, html etc. etc.
All my web service calls are done through javascript (cross domain ws call). I have few sequential web service calls which all have different success callback function.
I am not able to digest - when i call those ws individually in seperate places they are returning me proper results but sequentially sometime they are giving and sometime not.
sample code: this is not giving expected results all the time.
function submitSearchRequest(_queryString, Stores) {
if (Stores[1].length>0) {
//generate 'searchRequestForArtifact' request object
getSearchResponse("successcallForArtifact", _searchRequestForArtifact);
}
if (Stores[2].length > 0) {
//generate 'searchRequestForPerson' request object
getSearchResponse("successcallForPerson", _searchRequestForPerson);
}
}
function successcallForArtifact(response)
{
//show the results
}
function successcallForPerson(response)
{
//show the results
}
}
If you need sequentially you will need to kick off each search only after one has returned. Currently you are making async calls, meaning it gets kicked off then continues with the code. Currently if the second call is simply faster the order will be off. You will either need to make a sync call or simply have the order enforced by calling the second search from the success function for the artifact.
If you are using JQuery which it seems you are you can set the async parameter to false which will force the order you want but it will slow the overall performance of your page. See this question.

Categories