There is array of objects, which are expanded with parallel ajax requests. When last request is done array should be processed. The only solution i see is:
function expandArray(objects, callback){
number_of_requests=objects.length-1;
for(i in objects){
$.getJSON(request,function(){
//expanding array
if(--number_of_reuests==0){
callback();
}
});
}
}
But as requests are executed in parallel there is chance of race condition. Variable number_of_requests can be edited by two "threads" simultaneously. How to avoid chance of race condition?
Is it possible to rework your AJAX so that it all goes in one request? That part of the system will be the biggest bottleneck and can get tricky (as you've found out), so the less requests you make, the better.
The only other way I could think would be if each request mutated its related object, setting a flag value or something, and then you looped through all the objects to check if all the flags had been set yet.
Isn't Javascript single threaded? The kind of condition you talk of wouldn't occur.
It's more complex, but I would suggest using a third function to monitor the results. Something like this:
1) Start monitor - using an appropriate interval monitor (using an interval to test here is important - a simple loop would lock the JS engine up tight) the results of requests A and B.
2) Call request A.
3) Call request B.
4) Request B finishes - callback function sets a "B All Done!" value.
5) Request A finishes - callback function sets an "A All Done!" value.
6) Monitor recognizes that both requests are completed and calls function which uses data from both.
Worded more simply this is "dependencies" (multiple calls)to "monitor" (the function that checks dependencies) to "completion" (the action to take when all is ready).
You can create the callbacks and the completion function as nested functions to maintain encapsulation and reduce "little" in the global scope.
You can extend your dependencies as far as you like.
Related
This question already has an answer here:
Why delaying evaluation can transform impure functions into pure ones?
(1 answer)
Closed 11 months ago.
I'm now reading "mostly adequate guide to functional programming" by Professor Frisby, and I was wondering how can we do pure fetching, and I saw this piece of code here in the book
const pureHttpCall = memoize((url, params) => () => $.getJSON(url, params));
And says
The interesting thing here is that we don't actually make the http call - we instead return a function that will do so when called. This function is pure because it will always return the same output given the same input: the function that will make the particular http call given the url and params.
where that left me with some confusion, I don't understand how pureHttpCall is now pure and still have impure underlying code (the fetching part).
So what am I missing here that makes the code pure and functional?
What makes the function pure is this:
memoize( ... )
Memoizing is a special kind of caching. You may be familiar with caching. That is, you store the result of an expensive operation in a variable and if that variable already exist return the value of that variable instead of performing the expensive operation. Memoizing is exactly that.
The main difference between memoizing and caching is that memoizing is a permanent cache - most commonly used caching algorithm has an invalidation process: it could be time based (delete the cache if it is older than x), count based (delete the cache if there are more than x items cached), memory based (delete the cache if the cache uses more than x megabytes of RAM) etc. Memoizing never deletes the cache. Therefore the expensive operation is done only once.
The fact that an operation is done only once causes every call to pureHttpCall to be guaranteed to return the same result. This means that the function is now pure.
Yes you can do the same to Math.random() if you memoize it making every call to pureRandom return exactly the same number. And this would make pureRandom pure because it always returns the same result. I personally would not call it a "random" function.
A very simple implementation of memoize could be something like this:
function memoize (fn) {
let cache;
return function () {
if (cache === undefined) {
cache = fn();
}
return cache;
}
}
The above code is 100% synchronous for clarity but it is possible to write an asynchronous version of a memoization function.
As I understand it, the author is arguing here that because the caching layer is added, the function now has predictable results:
Once you have done a combination of the input arguments, that same combination will always return the same result it produced the first time.
So if you unplug your ethernet cable and do pureHttpCall("http://google.com") you will get an error response back. If you now replug your cable, then pureHttpCall("http://google.com") will still produce that exact same error you got the first time.
I don't think I agree though, because the there is an implicit dependency here. The function still depends on the environment when it is first called. E.g. If you restart your program between unpluggin and replugging your ethernet cable, you will get different results.
First off all, I'm a pythonist, not a javascripter - please be kind.
In a popular MVVM Javascript framework from a popular Internet search provider, there is a class called Scope.
This class has a method called $watch, used to register callbacks (called listeners) that are supposed to be called every time that Scope.$digest is fired. Calls to Scope.$watch returns a deregistration function for this listener.
Listeners are called in the following loop:
...
do { // "traverse the scopes" loop
if ((watchers = current.$$watchers)) {
// process our watches
length = watchers.length;
while (length--) {
try {
watch = watchers[length];
// Most common watches are on primitives, in which case we can short
// circuit it with === operator, only when === fails do we use .equals
if ((value = watch.get(current)) !== (last = watch.last)
...
Since a listener can be registered or destroyed from another listener, there are two race conditions here:
when a listener is deregistered between length = watchers.length and watch = watchers[length], watch will be undefined and the call to watch.get(current) will fail (undefined has no method get).
when a listener is registered by another listener it may be skipped.
I guess a simple check for the existence of watch would fix condition #1:
if (watch && (value = watch.get(current)) !== (last = watch.last)
I'm not sure how to solve condition #2. Shall not modify an array while iterating over it - my first thought was: why while (length--)? Then I read in the source comments:
Loop operations are optimized by using while(count--) { ... }
this means that in order to keep the same order of execution as addition
we have to add items to the array at the beginning (shift) instead of at
the end (push)
In Python I would probably try to solve it using queues. Two questions:
What is the idiomatic way of dealing with this problem in javascript when targeting maximum performance and memory conservation?
How should I go about unit tests for race conditions (or should I use end-to-end tests instead)?
[edit]
Samuel Neff commented that since JavaScript is not multi-threaded, this is not really a race condition.
A more objective question is: with performance and memory footprint in mind, what can I do to prevent bugs caused by a callback that modifies the very array I'm iterating over when using the while (length--) pattern to loop over an array of callbacks?
JavaScript is not multi-threaded. You don't have to worry about another "thread" modifying the objects between sequential lines of code.
It is possible to have race conditions if you depend on callbacks being called back in a particular order, but you don't have to worry about race conditions like in normal multi-threaded languages.
The way I have solved problems like this a few times in the past (when iterating through connections in server code, for example) is by adding new listeners (or in my case, connections) to a temporary array / "queue."
In my main loop I would go first go through the existing connections, deleting the ones which had been marked as dropped and dispatching other threads to deal with connections which needed servicing. Other threads could register connections to the queue array or mark connections as dropped from the main array but would not actually delete them. This took care of your first race condition.
The second race condition was solved by having my main loop atomically link the queue array to the end of the main array thus starting an empty queue array and extending the main array by the previous contents of the queue array. The main loop then finished going through the newly extended contents of the main array.
var recurse = function(steps, data, delay) {
if(steps == 0) {
console.log(data.length)
} else {
setTimeout(function(){
recurse(steps - 1, data, delay);
}, delay);
}
};
var myData = "abc";
recurse(8000, myData, 1);
What troubles me with this code is that I'm passing a string on 8000 times. Does this result in any kind of memory problem?
Also, If I run this code with node.js, it prints immediately, which is not what I would expect.
If you're worried about the string being copied 8,000 times, don't be, there's only one copy of the string; what gets passed around is a reference.
The bigger question is whether the object created when you call a function (called the "variable binding object" of the "execution context") is retained, because you're creating a closure, and which has a reference to the variable object for the context and thus keeps it in memory as long as the closure is still referenced somewhere.
And the answer is: Yes, but only until the timer fires, because once it does nothing is referencing the closure anymore and so the garbage collector can reclaim them both. So you won't have 8,000 of them outstanding, just one or two. Of course, when and how the GC runs is up to the implementation.
Curiously, just earlier today we had another question on a very similar topic; see my answer there as well.
It prints immediately because the program executes "immediately". On my Intel i5 machine, the whole operation takes 0.07s, according to time node test.js.
For the memory problems, and wether this is a "cheap infinite loop", you'll just have to experiment and measure.
If you want to create an asynchronous loop in node, you could use process.nextTick. It will be faster than setTimeout(func, 1).
In general Javascript does not support tail call optimization, so writing recursive code normally runs the risk of causing a stack overflow. If you use setTimeout like this, it effectively resets the call stack, so stack overflow is no longer a problem.
Performance will be the problem though, as each call to setTimeout generally takes a fair bit of time (around 10 ms), even if you set delay to 0.
The '1' is 1 millisecond. It might as well be a for loop. 1 second is 1000. I recently wrote something similar checking on the progress of a batch of processes on the back end and set a delay of 500. Older browsers wouldn't see any real difference between 1 and about 15ms if I remember correctly. I think V8 might actually process faster than that.
I don't think garbage collection will be happening to any of the functions until the last iteration is complete but these newer generations of JS JIT compilers are a lot smarter than the ones I know more about so it's possible they'll see that nothing is really going on after the timeout and pull those params from memory.
Regardless, even if memory is reserved for every instance of those parameters, it would take a lot more than 8000 iterations to cause a problem.
One way to safeguard against potential problems with more memory intensive parameters is if you pass in an object with the params you want. Then I believe the params will just be a reference to a set place in memory.
So something like:
var recurseParams ={ steps:8000, data:"abc", delay:100 } //outside of the function
//define the function
recurse(recurseParams);
//Then inside the function reference like this:
recurseParams.steps--
I have a function called checkStatus(x) where x is Id. How can I call this function n times asynchronously ? Without being dependent one on another to completed and for another to start?
I'm using jquery
EDIT:
I'm not sure if I used the correct term, but here is what I want. I have a list of ID's and I iterate trough a list, I want to execute this function for each ID.
But I don't want to wait for one id to finnish, and then to execute this function on another id. Is it possible to execute this function at the same time?
Javascript engines are single-threaded, meaning that only one piece of code is ever executing at once. Asynchronous features (AJAX, timeouts/intervals) cause different blocks of code to run in sequence, not in parallel (i.e. you'll never get any use out of multiple processor cores in Javascript).
The simplest way to produce asynchronous (non-blocking) code is using setTimeout (I strongly discourage using setInterval), as others have suggested, but there is no performance benefit to doing so. This simply ensures that your browser won't "hang" during slow JS computations, by allowing the browser's other tasks (such as page repainting and user input) the opportunity to run. It won't actually increase the speed of those computations (in fact, it slightly slows them, due to the small additional overhead of the timeouts).
It is possible to create separate threads in Javascript using web workers, but their capabilities are limited (for example, they cannot alter the DOM) and they are not yet supported by IE.
An example of a long-running, non-blocking task using "recursive" setTimeout calls:
function getStarted(elements) {
// this function must be inside the outer function
// so that `i` (below) can be accessed via a closure
function processElement() {
// do work on elements[i] here
// or pass it to another function
// this continues only if we haven't hit the end of the array,
// like the second and third clauses of a `for` loop
if (++i < elements.length) {
setTimeout(processElement, 0);
}
}
// don't bother with empty arrays
if (elements.length) {
// the same `i` is used each time processElement runs
// and acts just like a `for` loop index
var i = 0;
// optional: make a copy of the array so that it
// doesn't get modified while we're working
elements = elements.slice();
// even a zero-millisecond "delay" gives the browser the
// opportunity to run other code
setTimeout(processElement, 0);
}
}
Use the setTimeout or setInterval functions.
setTimeout(function(){checkStatus(x);}, 100);
setTimeout(function(){checkStatus(x);}, 200);
setTimeout(function(){checkStatus(x);}, 300);
I thought I would try and be clever and create a Wait function of my own (I realise there are other ways to do this). So I wrote:
var interval_id;
var countdowntimer = 0;
function Wait(wait_interval) {
countdowntimer = wait_interval;
interval_id = setInterval(function() {
--countdowntimer <=0 ? clearInterval(interval_id) : null;
}, 1000);
do {} while (countdowntimer >= 0);
}
// Wait a bit: 5 secs
Wait(5);
This all works, except for the infinite looping. Upon inspection, if I take the While loop out, the anonymous function is entered 5 times, as expected. So clearly the global variable countdowntimer is decremented.
However, if I check the value of countdowntimer, in the While loop, it never goes down. This is despite the fact that the anonymous function is being called whilst in the While loop!
Clearly, somehow, there are two values of countdowntimer floating around, but why?
EDIT
Ok, so I understand (now) that Javascript is single threaded. And that - sort of - answers my question. But, at which point in the processing of this single thread, does the so called asynchronous call using setInterval actually happen? Is it just between function calls? Surely not, what about functions that take a long time to execute?
There aren't two copies of the variable lying around. Javascript in web browsers is single threaded (unless you use the new web workers stuff). So the anonymous function never has the chance to run, because Wait is tying up the interpreter.
You can't use a busy-wait functions in browser-based Javascript; nothing else will ever happen (and they're a bad idea in most other environments, even where they're possible). You have to use callbacks instead. Here's a minimalist reworking of that:
var interval_id;
var countdowntimer = 0;
function Wait(wait_interval, callback) {
countdowntimer = wait_interval;
interval_id = setInterval(function() {
if (--countdowntimer <=0) {
clearInterval(interval_id);
interval_id = 0;
callback();
}
}, 1000);
}
// Wait a bit: 5 secs
Wait(5, function() {
alert("Done waiting");
});
// Any code here happens immediately, it doesn't wait for the callback
Edit Answering your follow-up:
But, at which point in the processing of this single thread, does the so called asynchronous call using setInterval actually happen? Is it just between function calls? Surely not, what about functions that take a long time to execute?
Pretty much, yeah — and so it's important that functions not be long-running. (Technically it's not even between function calls, in that if you have a function that calls three other functions, the interpreter can't do anything else while that (outer) function is running.) The interpreter essentially maintains a queue of functions it needs to execute. It starts starts by executing any global code (rather like a big function call). Then, when things happen (user input events, the time to call a callback scheduled via setTimeout is reached, etc.), the interpreter pushes the calls it needs to make onto the queue. It always processes the call at the front of the queue, and so things can stack up (like your setInterval calls, although setInterval is a bit special — it won't queue a subsequent callback if a previous one is still sitting in the queue waiting to be processed). So think in terms of when your code gets control and when it releases control (e.g., by returning). The interpreter can only do other things after you release control and before it gives it back to you again. And again, on some browsers (IE, for instance), that same thread is also used for painting the UI and such, so DOM insertions (for instance) won't show up until you release control back to the browser so it can get on with doing its painting.
When Javascript in web browsers, you really need to take an event-driven approach to designing and coding your solutions. The classic example is prompting the user for information. In a non-event-driven world, you could do this:
// Non-functional non-event-driven pseudo-example
askTheQuestion();
answer = readTheAnswer(); // Script pauses here
doSomethingWithAnswer(answer); // This doesn't happen until we have an answer
doSomethingElse();
That doesn't work in an event-driven world. Instead, you do this:
askTheQuestion();
setCallbackForQuestionAnsweredEvent(doSomethingWithAnswer);
// If we had code here, it would happen *immediately*,
// it wouldn't wait for the answer
So for instance, askTheQuestion might overlay a div on the page with fields prompting the user for various pieces of information with an "OK" button for them to click when they're done. setCallbackForQuestionAnswered would really be hooking the click event on the "OK" button. doSomethingWithAnswer would collect the information from the fields, remove or hide the div, and do something with the info.
Most Javascript implementation are single threaded, so when it is executing the while loop, it doesn't let anything else execute, so the interval never runs while the while is running, thus making an infinite loop.
There are many similar attempts to create a sleep/wait/pause function in javascript, but since most implementations are single threaded, it simply doesn't let you do anything else while sleeping(!).
The alternative way to make a delay is to write timeouts. They can postpone an execution of a chunk of code, but you have to break it in many functions. You can always inline functions so it makes it easier to follow (and to share variables within the same execution context).
There are also some libraries that adds some syntatic suggar to javascript making this more readable.
EDIT:
There's an excelent blog post by John Resig himself about How javascript timers work. He pretty much explains it in details. Hope it helps.
Actually, its pretty much guaranteed that the interval function will never run while the loop does as javascript is single-threaded.
There is a reason why no-one has made Wait before (and so many have tried); it simply cannot be done.
You will have to resort to braking up your function into bits and schedule these using setTimeout or setInterval.
//first part
...
setTimeout(function(){
//next part
}, 5000/*ms*/);
Depending on your needs this could (should) be implemented as a state machine.
Instead of using a global countdowntimer variable, why not just change the millisecond attribute on setInterval instead? Something like:
var waitId;
function Wait(waitSeconds)
{
waitId= setInterval(function(){clearInterval(waitId);}, waitSeconds * 1000);
}