I have a large array containing various data, and a function that performs some basic but somewhat time-consuming analysis on the array.
I use a forEach loop to iterate over the array. I want the function to return a specific element if a condition is met, otherwise false, using Promises since the function could take a considerable amount of time to complete:
var analyzeData = function (arr) {
return new Promise(function (resolve, reject) {
var answer = false;
arr.forEach(function (elm) {
if (elm.foo == true) {
answer = elm;
}
});
resolve(answer);
});
}
The issue I'm encountering is that I suspect that answer is being resolved before the forEach loop completes; i.e. before the forEach loop can finish checking every element's foo property, the function resolves to answer, which is by default false.
Is there a way to ensure that the code following the forEach loop waits for the loop to finish iterating, or loop through the array in some other manner?
JavaScript loop iteration is synchronous, and in your code you will always complete the loop before the call to resolve is made.
Note that JavaScript is also single threaded, so putting this in a promise won't make it run in the background. There will still be some period of time in which the UI not responsive to the user while this loop is executing.
You can offload some computation to background processes using web workers.
As an aside: can you exit the loop when the first item is found? If not, why not iterate in reverse (using a for-loop, not array reversal) and exit when the first item is found? It is still technically O(N) in the worst case, but the best case is O(1) if you bail out as soon as possible.
However to be more helpful, you will need to provide more information about what is happening. I would suggest putting together a small example on http://jsfiddle.net/ -- that can be a useful exercise in and of itself.
Related
I am writing some software where one can execute a list of actions. each of these actions basically is a Promise.
Each action might have a timer Trigger (where the next action is triggered when the timer ran down). Also, the list of actions might be loopable.
Now it might be possible, that a list of actions e.g. consists out of 2 items, each is executed 0 seconds after finishing the last one, and looping is enabled, thus actually generating an infinite loop.
This is no error in the application and thus be possible to do. My Code looks (extremely simplified) like this at the moment:
const actions = [...] // (() => Promise<void>)[]
const current = 0
const loop = true
const autoTrigger = true
const go() {
if(current > actions.length && loop) current = 0
current ++
return Promise.resolve().then(() => {
return actions[current - 1]()
}).then(() => {
if(autoTrigger) return go()
return Promise.resolve()
})
}
Is this code safe to run, assuming that it might not be aborted until it iterated a few hundred thousand times, or might it cause an error because the recursion gets too deep?
Yes, it is safe to run. Because promises don't call their .then() handlers until after the interpreter has returned back to an empty stack (even if they resolve immediately), there is no stack build-up with this type of recursive-like coding style - no matter how many times it recurses.
So, this is safe, regardless of how many iterations it runs. It could even run indefinitely without any stack build-up.
I am not sure I understand the value of the functional style looping/mapping if we can't use the break and continue keywords.
I can do this:
collections.users.models.forEach(function(item, index) {
//can't use break or continue...?
});
or I can do this:
for (var i = 0; i < collections.users.models.length; i++) {
if (user.username === collections.users.models[i].username) {
app.currentUser = collections.users.models[i];
break;
}
}
what's the advantage of the functional call if I can't use the break or continue keywords?
.each() or .forEach() are less flexible than a plain for loop. They just are.
As you have discovered, they offer you less control over the looping. They are a convenience ONLY when you want to iterate the entire set or when it really helps you to automatically have a new function context for your loop iteration code (sometimes useful in async operations) or when you enjoy the more declarative coding in that using the method expresses your coding intent a little more clearly and succinctly than the for loop. .forEach() will also automatically skip sparse elements of an array.
Other than those features, they are just a reduction in typing that sacrifices some looping control. Use them when you like one of the advantages and don't use it when you need more looping control.
FYI, for Javascript arrays, .some() and .every() attempt to get you back some looping control, though they are still not as flexible as a for loop.
If you use .some(), you can return true; to be the equivalent of break; (since that will stop the looping) and you can just return; to be the equivalent of continue; (since that will return from the callback and advance to the next iteration).
To be honest, there is not much of an "advantage" so much as a convenience. You can call return to force yourself out of a single iteration of a forEach loop, but this is not the same as the native for because you are still invoking a function. The convenience comes in the form of not having to define the parameterization of your loop (ie. start point, end point, step size) as well as providing the value of the index and item in the items iterated over.
It is worth mentioning that this convenience comes at the cost of for loop control (ie. start point, end point, step size), backwards iteration, the ability to break the entire loop, a small performance impedance, and yes, even cross browser compliance.
each expresses your intent: to iterate and perform a side effect for each item. each abstracts the process of iterating: initializing and incrementing a counter, fetching by array index. That's what the functional combinators do.
Your code snippet is a great example. Let's rewrite that in a functional style:
app.currentUser = collections.users.models.find(function (u) {
return u.username === user.username;
});
find clearly expresses your intent and abstracts the concept of iterating until you find an item matching your predicate.
You can't natively break from forEach or each callback function. However, there is a technique many people use to break from this with customized exception:
var BreakException = function(){};
try{
list.forEach(function(el){
if (meetCriteria(el)){
throw BreakException; // Break from forEach
}
});
}
catch (e){
// Catch break
if (e != BreakException)
throw e; // If not a break, but other exceptions, raise it
}
// Carry on
Simple approach above is creating a BreakException object and raise it whenever you want to break from the unbreakable loop. Catch that particular BreakException and carry on what your function does.
One thing to remember, always check if the caught exception is BreakException as you defined, since the other possible exceptions (like TypeError, etc) also got trapped by this catch statement. Don't swallow it without checking.
Using Array.prototype.some and Array.prototype.every to continue/break out of functional for loops is a good thing to do
[1,2,3,null,5].every(function(v){
if(!v) return false; // this will break out of the loop
// do your regular loopage lulz
});
This question already has answers here:
Best way to iterate over an array without blocking the UI
(4 answers)
Closed 8 years ago.
I have some javascript on my page that takes a very long time to execute (between 10-30 seconds)
The code basically look like that :
//Iterate over all the elements in the array
for(int i=0; i<array.length; i++){
//Complex stuff for each element
}
The problem is that while this code is executing, the UI is not responsive.
Is there any way to solve this issue? I know javascript has some sort of asynchrony but I never really used it before...
Also, the order the elements are processed must always be the order that they appear in the array.
EDIT : I tried the solution below with setTimeout() and in the "duplicate", but it still doesn't fix my problem. I guess it's because the size of my array is not very big BUT the calculation for each element is pretty big.
Also the UI I want to be responsive is an animated loading gif. Since the calculation for each item is too big, the animation is sloppy.
Create a variable and set it to the starting value for your counter.
Create a function that:
Does whatever the body of the loop does
Increments the counter
Tests to see if the counter is still under the length and, if it is, calls the function again
At this point you will have functionality equivalent to what you have already.
To pause between each call of the function, and allow time for other functions to fire, replace the direct call to the function with a call to setTimeout and use the function as the first argument.
var counter = 0;
function iterator () {
//Complex stuff for each element
counter++;
if (counter < array.length) {
setTimeout(iterator, 5);
}
}
Javascript uses something called the "Event Loop". Basically, there is a single thread that executes the code for each event as it occurs.
There is a function called setTimeout that you can use to make your Javascript code run in a future iteration of the loop. One trick that some developers use to keep the UI responsive during a long-running Javascript task is to periodically invoke code to be run with a timeout of zero. This cause the code to execute almost immediately without locking the event loop.
For example, you could write
var i = 0;
setTimeout(myFunction, 0);
function myFunction() {
// complex stuff for each element.
i++;
if (i < array.length)
setTimeout(myFunction, 0);
}
Be aware that on modern browsers, as long as Javascript code is executing, the UI will be unresponsive. Some browsers will even pop up a message urging the user to kill the script; very bad!
Also, in modern browsers, there is a new feature called "Web Workers". Web Workers allow for Javascript code to be executed in a separate thread. You can create as many threads as you like, so consider using web workers as well if your task is easily parallelizable.
If you need the browser to update the page in the middle of your JS code, you need to yield to the browser, e.g. with a setTimeout, in a continuation passing style:
function doStuff(i) {
// do some stuff with index i
if (i < array.length) {
setTimeout(doStuff.bind(window, i + 1), 0);
}
}
doStuff(0);
I tend to do this kind of thing like so:
function doStuff(arr) {
if (arr.length === 0) {
// do whatever you need to do when you're done
return;
}
// Do body of computation here
// Or use setTimeout if your browser doesn't do setImmediate
setImmediate(function () { doStuff(arr.slice(1)); });
}
doStuff(actualArrayOfThings);
Basically, break your loop body up, and use recursion to run each part of the loop in order. It's pretty easy to break this out into a helper function, since the loop body itself is what varies. Something like:
function eachAsync(arr, body) {
if (arr.length === 0) {
// do whatever you need to do when you're done
return;
}
body(arr[0]);
setImmediate(function () { eachAsync(arr.slice(1), body); });
}
eachAsync(actualArrayOfThings, doStuff);
If your task is to process data without interfering with the user interface you can use WebWorkers
https://developer.mozilla.org/en/docs/Web/Guide/Performance/Using_web_workers
Note it's supported by modern browsers.
Use setTimeOut() if the for loop is doing some tidy work. It depends upon the complex stuff executing for each iteration.
Also, consider using JQuery/AJAX for executing it asynchronously.
Overview
So I have pulled out a document from my database. Inside is a nested collection of objects. Each of the objects inside this nested collection has an '_id' attribute. I want to find one on these objects specifically by its '_id' using Javascript.
Example
http://jsfiddle.net/WilsonPage/tNngT/
Alternative Example
http://jsfiddle.net/WilsonPage/tNngT/3/
Questions
Is my example the best way of achieving this?
Will this block in Node.js?
Yes, if you only know a specific value which is contained by one of your objects (which are held in an Array) you need to loop over the whole structure and compare those values.
As you also did right, break the iteration when you found one (return in your example).
So my answer to your first question would be yes, in terms of performance this is the right and best way.
What I don't get is the "Async" example. You just moved the code and changed the structure. Your code is still "blocking" since you're using a normal for-loop to search. If that array would be huge, it would block your node-app for the amount of time the loop needs to finish.
To really make it asynchronously, you would need to get rid of any loop. You would need to loop over the structure with a runway-timer.
var findById = function(collection, _id, cb){
var coll = collection.slice( 0 ); // create a clone
(function _loop( data ) {
if( data._id === _id ) {
cb.apply( null, [ data ] );
}
else if( coll.length ) {
setTimeout( _loop.bind( null, coll.shift() ), 25 );
}
}( coll.shift() ));
};
And then use it like
findById( myCollection, 102, function( data ) {
console.log('MATCH -> ', data);
});
This technique (which is a simplified example), we are creating an self-invoking anonymous function and we pass in the first array item (using .shift()). We do our comparison and if we found the item we are looking for, execute a callback function the caller needs to provide. If we don't have a match but the array still contains elements (check for the .length), we create a timeout of 25ms using setTimeout and call our _loop function again, this time with the next array item, because .shift() gets and removes the first entry. We repeat that until either no items are left or we found the element. Since setTimeout gives other tasks in the JS main thread (on a browser, the UI thread) the chance to do things, we don't block and screw up the whole show.
As I said, this can get optimized. For instance, we can use a do-while loop within the _loop() method and also use Date.now() to do things until we go over the 50ms mark for instance. If we need longer than that, create a timeout the same way and repeat the above described operation again (so its like, do as much operation as possible within 50ms).
I'd pre-sort the array by each item's _id and at least implement a binary search, if not something a little more sophisticated if speed is really an issue.
You could try using binary search, in most cases it's faster than linear search. As jAndy said, you will still block with standard for loop, so take a look at some node asynchronous library. First that falls to my mind is async.js
I messed around with async.js to produce this solution to my problem. I have tried make it a reusable as possible so it is not just locked down to search the '_id' attribute.
My Solution:
http://jsfiddle.net/WilsonPage/yJSjP/3/
Assuming you can generate unique strings from your _id you could hash them out with js's native object.
findById = (collection, _id, callback, timeout = 500, step = 10000)->
gen = ->
hash = {}
for value, i in collection
hash[value._id] = value
unless i % step then yield i
hash[_id]
it = gen()
do findIt = ->
{done, value} = it.next()
if done then callback value
else
console.log "hashed #{(value/collection.length*100).toFixed 0}%"
setTimeout findIt, timeout
I have a function called checkStatus(x) where x is Id. How can I call this function n times asynchronously ? Without being dependent one on another to completed and for another to start?
I'm using jquery
EDIT:
I'm not sure if I used the correct term, but here is what I want. I have a list of ID's and I iterate trough a list, I want to execute this function for each ID.
But I don't want to wait for one id to finnish, and then to execute this function on another id. Is it possible to execute this function at the same time?
Javascript engines are single-threaded, meaning that only one piece of code is ever executing at once. Asynchronous features (AJAX, timeouts/intervals) cause different blocks of code to run in sequence, not in parallel (i.e. you'll never get any use out of multiple processor cores in Javascript).
The simplest way to produce asynchronous (non-blocking) code is using setTimeout (I strongly discourage using setInterval), as others have suggested, but there is no performance benefit to doing so. This simply ensures that your browser won't "hang" during slow JS computations, by allowing the browser's other tasks (such as page repainting and user input) the opportunity to run. It won't actually increase the speed of those computations (in fact, it slightly slows them, due to the small additional overhead of the timeouts).
It is possible to create separate threads in Javascript using web workers, but their capabilities are limited (for example, they cannot alter the DOM) and they are not yet supported by IE.
An example of a long-running, non-blocking task using "recursive" setTimeout calls:
function getStarted(elements) {
// this function must be inside the outer function
// so that `i` (below) can be accessed via a closure
function processElement() {
// do work on elements[i] here
// or pass it to another function
// this continues only if we haven't hit the end of the array,
// like the second and third clauses of a `for` loop
if (++i < elements.length) {
setTimeout(processElement, 0);
}
}
// don't bother with empty arrays
if (elements.length) {
// the same `i` is used each time processElement runs
// and acts just like a `for` loop index
var i = 0;
// optional: make a copy of the array so that it
// doesn't get modified while we're working
elements = elements.slice();
// even a zero-millisecond "delay" gives the browser the
// opportunity to run other code
setTimeout(processElement, 0);
}
}
Use the setTimeout or setInterval functions.
setTimeout(function(){checkStatus(x);}, 100);
setTimeout(function(){checkStatus(x);}, 200);
setTimeout(function(){checkStatus(x);}, 300);