I'm trying to build a scraper using Node.js where I need some help implementing the callback pattern properly.
I'm having trouble implementing it the right what. What I am looking to do is something like this
var client = function(){};
client.prototype.init = function() {
// do something
}
client.prototype.A = function(callback) {
// do something
// run callback
if(recursive_condition_fulfilled)
myclient.A(callback)
}
var myClient = new client();
var myCallback = function(callback){
// do something
}
myClient.init();
myClient.A(myCallback);
The problem arises when I have an async function B() which runs inside A()
client.prototype.A = function(callback) {
// do something
this.B();
// wait for B
// once B is done
// run callback()
// then run the following
if(recursive_condition_fulfilled)
myclient.A(callback)
}
client.prototype.B = function(callback) {
// do something
// run callback
}
How do I implement the later part when B is resolved? (where it says)
// wait for B
// once B is done
// run callback()
Related
I think there is an easy solution for this, but for some reason I am not getting the expected results. My functions look like this:
var functionA = function(callback) {
loadData(fromURL1); // takes some time
loadData(fromURL2); // takes some time
callback(); // Should be called AFTER the loadData() functions are finished
}
var myCallBackFunction = function() {
// this function is called AFTER functionA() is finished
alert("All my loaded data from URL1 and URL2");
}
window.onload = function() {
functionA(myCallBackFunction);
}
Unfortunately, the callback() function above doesn't wait for loadData() to finish, and then just calls the alert with empty data.
I read a lot of online examples, but I think I am still missing something obvious.
If the loadData()s are async operations, you can do two things:
Using $.ajaxComplete():
var functionA = function(callback) {
loadData(fromURL1); // takes some time
loadData(fromURL2); // takes some time
$.ajaxComplete(function () {
callback(); // Should be called AFTER the loadData() functions are finished
});
}
Or chaining the functions:
var functionA = function(callback) {
loadData(fromURL1, function () {
loadData(fromURL2, function () {
callback(); // Should be called AFTER the loadData() functions are finished
}); // takes some time
}); // takes some time
}
I have two asynchronous functions, which use parameters already acquired somehow:
var a = getUserinput();
var b = getUserinput();
var c = getUserinput();
var d = getUserinput();
var func1 = function(valA, valB){
var deferred = $q.deferred;
//http call with valA, valb...
// when complete, resolve deferred
return deferred.promise;
}
var func2 = function(valC, valD){
var deferred = $q.deferred;
//http call with valC, valD...
// when complete, resolve deferred
return deferred.promise;
}
I want to execute func1 and then func2. The problem is that I can't find a way to do this successfully while being able to provide parameters to func2.
For example, if I do
func1(a, b).then(func2(c, d));
func2 executes before func1 is done.
How do I execute func2 only after func1 is done?
func
Provide anonymous function and place func2 invocation in it:
func1(a, b).then(function() {
func2(c, d);
});
If it helps, I recently took the first release of the rogue written for UNIX in C and rewrote it for javascript to work in a browser. I used a technic called continuation to be able to wait for key entry by the user because in javascript the are no interrupts.
So I would have a piece of code like this:
void function f() {
// ... first part
ch = getchar();
// ... second part
}
that would be transformed in
function f() {
// ... first part
var ch = getchar(f_cont1);
return;
// the execution stops here
function f_cont1 () {
// ... second part
}
}
the continuation is then stored to be reuse on a keypressed event. With closures everything would be restarted where it stoped.
Suppose you have the following JS function:
function YourProxy($orm, $usr) {
this.addToDB = function(obj) {
/* Do some validation on obj */
return function(callback){
var oo = $orm.createNew(obj);
oo.save(options, function(err,ok){
if(err) callback(err);
callback(null,ok);
}
}
}
}
which you can use on node.js with ES6 generators to wait for that operation to happen with something like:
function *(){
var yourProxy = new YourProxy();
try {
var result = yield yourProxy.addToDB(anObject);
} catch(e) {
/* Something went wrong sync. Here you have err from save's callback */
}
/* result contains ok, the one from save's callback */
}
To test that I've done something like this, using mocha and sinon (and mocha-sinon):
describe('addToDB', function(){
it('adds the object to the db', function(){
var callback = sinon.spy();
myProxy.addToDB(anObject)(callback);
expect( callback ).to.be.calledOnce;
});
});
but all I got is that the callback is never called because addToDB() exits before the save's callback gets called.
How would you test that?
Try using co-mocha and yield the generator as you did it in your example.
describe('addToDB', function(){
it('adds the object to the db', function* (){
var callback = sinon.spy();
yield myProxy.addToDB(anObject)(callback);
expect( callback ).to.be.calledOnce;
});
});
I have a handler (callback), an object to handle and four functions, which collect the data to object. In my case I wish to asynchronously call four data retrievers and when execution of all four is complete, handle the resulting object (something similar to the following):
var data = {};
function handle (jsObj) {}
// data retrieving
function getColorData () {}
function getSizeData () {}
function getWeightData () {}
function getExtraData () {}
data.color = getColorData();
data.size = getSizeData();
data.weight = getWeightData();
data.extra = getExtraData();
handle( data );
Of course, this code will not work properly. And if I chain data retrieving functions, they will be called one after another, right?
All four functions should be called asynchronously, cause they are being executed for too long to call them one by one.
Updated:
Thanks to everybody for your suggestions! I prefered $.Deferred(), but I found it slightly difficult to make it work the way I need. What I need is to asynchronously make a view, which requires four kinds of data (extraData, colorData, sizeData & weightData) and I have three objects: App, Utils & Tools.
Just a small description: view is created by calling App.getStuff passed App.handleStuff as a callback. Callback in the body of App.getStuff is called only $.when(App.getExtraData(), App.getColorData(), App.getSizeData(), App.getWeightData()). Before that Utils.asyncRequest passed Tools.parseResponse as a callback is called.
So, now the question is should I create four deferred objects inside each App.get*Data() and also return deferred.promise() from each of them?
And should I deferred.resolve() in the last function in my order (Tools.parseResponse for App.getExtraData in my example)?
var view,
App,
Utils = {},
Tools = {};
// Utils
Utils.asyncRequest = function (path, callback) {
var data,
parseResponse = callback;
// do something with 'data'
parseResponse( data );
};
// Tools
Tools.parseResponse = function (data) {
var output = {};
// do something to make 'output' from 'data'
/* So, should the deferred.resolve() be done here? */
deferred.resolve(output);
/// OR deferred.resolve();
/// OR return output;
};
// App
App = {
// Only one method really works in my example
getExtraData : function () {
var deferred = new jQuery.Deferred();
Utils.asyncRequest("/dir/data.txt", Tools.parseResponse);
return deferred.promise();
},
// Others do nothing
getColorData : function () { /* ... */ },
getSizeData : function () { /* ... */ },
getWeightData : function () { /* ... */ }
};
App.getStuff = function (callback) {
$.when(
App.getExtraData(),
App.getColorData(),
App.getSizeData(),
App.getWeightData()
)
.then(function (extraData, colorData, sizeData, weightData) {
var context,
handleStuff = callback;
// do something to make all kinds of data become a single object
handleStuff( context );
});
};
App.handleStuff = function (stuff) { /* ... */ };
/// RUN
view = App.getStuff( App.handleStuff );
I did not expect the code in my example above to work, it is for illustrative purposes.
I've been trying to solve this for quiet a long time and it still gives no result. The documentation for jQuery.Deferred() and discussions around this, unfortunately, did not help me. So, I would be very glad and greatful for any help or advise.
Conceptually, you would use a counter that gets incremented as each asynchronous call completes. The main caller should proceed after the counter has been incremented by all the asynchronous calls.
I think what you're looking for are Promises / Deferreds.
With promises you can write something like:
when(getColorData(), getSizeData(), getWeightData(), getExtraData()).then(
function (colorData, sizeData, weightData, extraData) {
handle(/*..*/);
}
)
The get*Data() functions will return a promise that they fulfill when their assynchronous call is complete.
Ex:
function getData() {
var promise = new Promise();
doAjax("getData", { "foo": "bar" }, function (result) {
promise.resolve(result);
});
return promise;
}
The when simply counts the number arguments, if all it's promises are resolved, it will call then with the results from the promises.
jQuery has an OK implementation: http://api.jquery.com/jQuery.when/
What I could suggest for this scenario would be something like that.
write a function like this
var completed = 0;
checkHandler = function() {
if(completed == 4) {
handle(data);
}
}
where completed is the number of positive callbacks you must receive.
As soon as every function receives a callback you can increment the "completed" counter and invoke the checkHandler function. and you're done!
in example
function getColorData() {
$.get('ajax/test.html', function(data) {
completed++;
checkHandler();
});
}
I have a Javascript object that requires 2 calls out to an external server to build its contents and do anything meaningful. The object is built such that instantiating an instance of it will automatically make these 2 calls. The 2 calls share a common callback function that operates on the returned data and then calls another method. The problem is that the next method should not be called until both methods return. Here is the code as I have implemented it currently:
foo.bar.Object = function() {
this.currentCallbacks = 0;
this.expectedCallbacks = 2;
this.function1 = function() {
// do stuff
var me = this;
foo.bar.sendRequest(new RequestObject, function(resp) {
me.commonCallback(resp);
});
};
this.function2 = function() {
// do stuff
var me = this;
foo.bar.sendRequest(new RequestObject, function(resp) {
me.commonCallback(resp);
});
};
this.commonCallback = function(resp) {
this.currentCallbacks++;
// do stuff
if (this.currentCallbacks == this.expectedCallbacks) {
// call new method
}
};
this.function1();
this.function2();
}
As you can see, I am forcing the object to continue after both calls have returned using a simple counter to validate they have both returned. This works but seems like a really poor implementation. I have only worked with Javascript for a few weeks now and am wondering if there is a better method for doing the same thing that I have yet to stumble upon.
Thanks for any and all help.
Unless you're willing to serialize the AJAX there is no other way that I can think of to do what you're proposing. That being said, I think what you have is fairly good, but you might want to clean up the structure a bit to not litter the object you're creating with initialization data.
Here is a function that might help you:
function gate(fn, number_of_calls_before_opening) {
return function() {
arguments.callee._call_count = (arguments.callee._call_count || 0) + 1;
if (arguments.callee._call_count >= number_of_calls_before_opening)
fn.apply(null, arguments);
};
}
This function is what's known as a higher-order function - a function that takes functions as arguments. This particular function returns a function that calls the passed function when it has been called number_of_calls_before_opening times. For example:
var f = gate(function(arg) { alert(arg); }, 2);
f('hello');
f('world'); // An alert will popup for this call.
You could make use of this as your callback method:
foo.bar = function() {
var callback = gate(this.method, 2);
sendAjax(new Request(), callback);
sendAjax(new Request(), callback);
}
The second callback, whichever it is will ensure that method is called. But this leads to another problem: the gate function calls the passed function without any context, meaning this will refer to the global object, not the object that you are constructing. There are several ways to get around this: You can either close-over this by aliasing it to me or self. Or you can create another higher order function that does just that.
Here's what the first case would look like:
foo.bar = function() {
var me = this;
var callback = gate(function(a,b,c) { me.method(a,b,c); }, 2);
sendAjax(new Request(), callback);
sendAjax(new Request(), callback);
}
In the latter case, the other higher order function would be something like the following:
function bind_context(context, fn) {
return function() {
return fn.apply(context, arguments);
};
}
This function returns a function that calls the passed function in the passed context. An example of it would be as follows:
var obj = {};
var func = function(name) { this.name = name; };
var method = bind_context(obj, func);
method('Your Name!');
alert(obj.name); // Your Name!
To put it in perspective, your code would look as follows:
foo.bar = function() {
var callback = gate(bind_context(this, this.method), 2);
sendAjax(new Request(), callback);
sendAjax(new Request(), callback);
}
In any case, once you've made these refactorings you will have cleared up the object being constructed of all its members that are only needed for initialization.
I can add that Underscore.js has a nice little helper for this:
Creates a version of the function that will only be run after first
being called count times. Useful for grouping asynchronous responses,
where you want to be sure that all the async calls have finished,
before proceeding.
_.after(count, function)
The code for _after (as-of version 1.5.0):
_.after = function(times, func) {
return function() {
if (--times < 1) {
return func.apply(this, arguments);
}
};
};
The license info (as-of version 1.5.0)
There is barely another way than to have this counter. Another option would be to use an object {} and add a key for every request and remove it if finished. This way you would know immediately which has returned. But the solution stays the same.
You can change the code a little bit. If it is like in your example that you only need to call another function inside of commonCallback (I called it otherFunction) than you don't need the commonCallback. In order to save the context you did use closures already. Instead of
foo.bar.sendRequest(new RequestObject, function(resp) {
me.commonCallback(resp);
});
you could do it this way
foo.bar.sendRequest(new RequestObject, function(resp) {
--me.expectedCallbacks || me.otherFunction(resp);
});
That's some good stuff Mr. Kyle.
To put it a bit simpler, I usually use a Start and a Done function.
-The Start function takes a list of functions that will be executed.
-The Done function gets called by the callbacks of your functions that you passed to the start method.
-Additionally, you can pass a function, or list of functions to the done method that will be executed when the last callback completes.
The declarations look like this.
var PendingRequests = 0;
function Start(Requests) {
PendingRequests = Requests.length;
for (var i = 0; i < Requests.length; i++)
Requests[i]();
};
//Called when async responses complete.
function Done(CompletedEvents) {
PendingRequests--;
if (PendingRequests == 0) {
for (var i = 0; i < CompletedEvents.length; i++)
CompletedEvents[i]();
}
}
Here's a simple example using the google maps api.
//Variables
var originAddress = "*Some address/zip code here*"; //Location A
var formattedAddress; //Formatted address of Location B
var distance; //Distance between A and B
var location; //Location B
//This is the start function above. Passing an array of two functions defined below.
Start(new Array(GetPlaceDetails, GetDistances));
//This function makes a request to get detailed information on a place.
//Then callsback with the **GetPlaceDetailsComplete** function
function GetPlaceDetails() {
var request = {
reference: location.reference //Google maps reference id
};
var PlacesService = new google.maps.places.PlacesService(Map);
PlacesService.getDetails(request, GetPlaceDetailsComplete);
}
function GetPlaceDetailsComplete(place, status) {
if (status == google.maps.places.PlacesServiceStatus.OK) {
formattedAddress = place.formatted_address;
Done(new Array(PrintDetails));
}
}
function GetDistances() {
distService = new google.maps.DistanceMatrixService();
distService.getDistanceMatrix(
{
origins: originAddress,
destinations: [location.geometry.location], //Location contains lat and lng
travelMode: google.maps.TravelMode.DRIVING,
unitSystem: google.maps.UnitSystem.IMPERIAL,
avoidHighways: false,
avoidTolls: false
}, GetDistancesComplete);
}
function GetDistancesComplete(results, status) {
if (status == google.maps.DistanceMatrixStatus.OK) {
distance = results[0].distance.text;
Done(new Array(PrintDetails));
}
}
function PrintDetails() {
alert(*Whatever you feel like printing.*);
}
So in a nutshell, what we're doing here is
-Passing an array of functions to the Start function
-The Start function calls the functions in the array and sets the number of PendingRequests
-In the callbacks for our pending requests, we call the Done function
-The Done function takes an array of functions
-The Done function decrements the PendingRequests counter
-If their are no more pending requests, we call the functions passed to the Done function
That's a simple, but practicle example of sychronizing web calls. I tried to use an example of something that's widely used, so I went with the Google maps api. I hope someone finds this useful.
Another way would be to have a sync point thanks to a timer. It is not beautiful, but it has the advantage of not having to add the call to the next function inside the callback.
Here the function execute_jobs is the entry point. it take a list of data to execute simultaneously. It first sets the number of jobs to wait to the size of the list. Then it set a timer to test for the end condition (the number falling down to 0). And finally it sends a job for each data. Each job decrease the number of awaited jobs by one.
It would look like something like that:
var g_numJobs = 0;
function async_task(data) {
//
// ... execute the task on the data ...
//
// Decrease the number of jobs left to execute.
--g_numJobs;
}
function execute_jobs(list) {
// Set the number of jobs we want to wait for.
g_numJobs = list.length;
// Set the timer (test every 50ms).
var timer = setInterval(function() {
if(g_numJobs == 0) {
clearInterval(timer);
do_next_action();
}
}, 50);
// Send the jobs.
for(var i = 0; i < list.length; ++i) {
async_task(list[i]));
}
}
To improve this code you can do a Job and JobList classes. The Job would execute a callback and decrease the number of pending jobs, while the JobList would aggregate the timer and call the callback to the next action once the jobs are finished.
I shared the same frustration. As I chained more asynchronous calls, it became a callback hell. So, I came up with my own solution. I'm sure there are similar solutions out there, but I wanted to create something very simple and easy to use. Asynq is a script that I wrote to chain asynchronous tasks. So to run f2 after f1, you can do:
asynq.run(f1, f2)
You can chain as many functions as you want. You can also specify parameters or run a series of tasks on elements in an array too. I hope this library can solve your issues or similar issues others are having.