The problem
The popular ava package features a simple and powerful testing framework for javascript:
import test from 'ava';
test('single step', t => {
t.is('bar', 'bar');
t.end();
});
Testing is dead simple for synchronous code. However, I am not sure how to write tests with multiple, consecutive, dependent steps.
For example, I would like to test a REST API. I would like to create a resource using POST, make sure it exists with GET call, and delete it with DELETE.
Needless to say, the order matters: the POST call must be finished before the GET call can start, and sometimes we even want a sleep time between steps.
What have I tried
Tried a series of setTimeouts where where the callback of each call is the next test step. It is fairly unreadable.
My question
What's the right way to write a test with multiple consecutive steps with ava?
These steps are asynchronous and consecutive
If you're looking to have all these steps in a single test, make sure you can get promises out of your request library. Then use async / await.
You can mix slow functions and logic into complex scenarios if you run your code via sequential executor nsynjs. You just need to write your code as if it was synchronous, and put it into function:
function process() {
var data = jQueryGetJSON(nsynjsCtx, "data/index.json").data; // ajax call to get some data
for(var i in data) { // data is ready at this point
var resp = jQueryGetJSON(nsynjsCtx, "data/"+data[i]).data; // another ajax call
// ... resp is ready here, do processing of resp...
nsynWait(nsynjsCtx,100); // wait 100ms
}
return someResult;
}
Then you run that function via nsynjs:
nsynjs.run(process,this,function (someResult) {
console.log('done');
// put t.end() inside this callback
});
See more examples here: https://github.com/amaksr/nsynjs/tree/master/examples
Related
I have alot of syncrhounous functions that i want to execute before that are basic ajax requests, these request will render html to the DOM.
In order to do this i had to execute all of this synchrounous
requests one by one. But i somehow want to these synchrounous functions asynchrounous all at the same time and wait for them to finnish in order to speed things up. This has to happen inside a synchrounous function. But my understanding is that this doesnt work in javascript, but i would like to hear what you guys have to say.
So my attempt was to add all of these synchrounous requests into asynchrounous promises and then do a Promise.all call. I cant wait for the promise.all().then because the main thread will keep on execute the rest of the code after this main synchrounous thread/function. So i wonder if there is a way to block the main thread in order to wait for these asynchrounous calls
heres a short illustration of what im talking about
var syncfunc = () => {
var getPromise = () => {
return new Promise((resolve) => {
var asyncAjaxRequest = async function() {
doSomeStuff();
resolve();
}
})
}
var promises = [getPromse(), getPromse(), getPromse()];
Promise.all(promises);
console.log('i want this console.log to execute after all promises executed doSomeStuff');
/**
*
* Promise.all(promises).then(() => {
// I cant use this, because its script in other files that will execute if i wait like this
})
*/
}
I know .then will execute when all resolves are done, but i basiacally want to block this synchrounous thread waiting for all other asynchrounous to finish.
If i could i would ofcourse change the structure into my needs, but the problem and the reason why im trying to do this is because im using sitevision framework, and want to add some content to the dom before a print module opens the print window. To call every function synchrounous is just not the way to go, its to slow. Ive also tried to set window.print = null to make the print function disabled, and then add the print function back when promises resolves, but it simply doesnt work
You cannot make an asynchronous operation turn into a synchronous one in plain Javascript (without external code). The event driven JS engine just doesn't work that way.
By definition, an asynchronous operation starts the operation (handing execution off to native code) and then returns back to the interpreter which then continues to execute the code that follows. The native code will add an event to JS event queue when it finishes to allow the interpreter event loop to service the completion of the asynchronous operation. If you were the create some sort of "block" such as a semi-infinite while loop, that would "block" the interpreter from executing more code, you end up in a stalemate. The loop that is blocking the interpreter prevents the JS interpreter from ever getting to the point where it can ever process the event that signals the end of the asynchronous operation. So, you have a loop waiting for something to finish, but the thing it's waiting for can't finish until the loop finishes - stalemate.
So, because of the single threaded event loop nature of the JS interpreter, you can't (purely in Javascript) block waiting for the end of an asynchronous operation.
Pretty much always, the correct design is to refactor the surrounding code/infrastructure to work with an asynchronous operation and asynchronous result (callback or promise).
If this is node.js, there are a couple of horrific hacks that can get you this result, but they block the entire interpreter so are almost never a desired design.
The first option involves writing a custom nodejs plugin (async operations done in native code) that provides a blocking interface that just doesn't return until the operation is done.
The second option involves using the synchronous child_process operations (such as child_process.execFileSync() to create a blocking child process, run your code in that child process and then continue when that process finishes.
Both I could consider pretty bad hacks and pretty much never the desired way to solve such a problem. But, I did want to show you what has to be done in order to block for an asynchronous operation (it has to be moved out of Javascript or out of the process).
If you can't figure out how to solve your real problem with non-blocking, asynchronous operations, I'd suggest you post a new question where you describe in detail exactly what the real problem is and we can help you find an asynchronous design that would work for your situation. If you post a link to the new question in a comment here, some of the people engaged here may check in on the new question and attempt to help.
You could use async/await to solve this. This is how you do it:
async function promiseSolver() {
var getPromise = () => {
return new Promise((resolve) => {
var asyncAjaxRequest = async function() {
doSomeStuff();
resolve();
}
})
}
var promises = [getPromse(), getPromse(), getPromse()];
await Promise.all(promises);
console.log('i want this console.log to execute after all promises executed doSomeStuff');
/**
*
* Promise.all(promises).then(() => {
// I cant use this, because its script in other files that will execute if i wait like this
})
*/
}
Basically, your code will wait until the .all is completed and then will continue with processing. Take into consideration that while the code execution is synchronous, the code will be non blocking.
Express documentation Production best practices: performance and reliability says:
Don’t use synchronous functions
Synchronous functions and methods tie up the executing process until
they return. A single call to a synchronous function might return in a
few microseconds or milliseconds, however in high-traffic websites,
these calls add up and reduce the performance of the app. Avoid their
use in production.
So my question is, in the context of node/express, if I have a function that accepts some static value and returns a calculated result (what I would normally consider a "synchronous function"), is it best practice to wrap that function inside a new Promise and resolve the result or does this create any significant unnecessary overhead? For example:
Current:
//inside my index.js
var myArgument = 'some long string';
var myResult = myFunction(myArgument);
function myFunction(myArgument){
var thisResult;
//some calculations
return thisResult;
}
New (and Improved?)
//inside my index.js
(async function() {
var myArgument = 'some long string';
var myResult = await myFunction(myArgument);
});
function myFunction(url) {
return new Promise((resolve, reject) => {
var thisResult;
//some calculations
if(thisResult){
resolve (thisResult);
} else {
reject (null)
}
});
}
Short answer: Nope.
The documentation is talking about not using synchronous versions of functions like readfileSync from nodeJS filesystem or bcrypt.compareSync for example. Synchronous calls block the event loop in nodeJS. So nothing happens while you are waiting for the synchronous call to finish. The whole program is on halt while this one method finishes. This is bad in a single threaded system like nodeJS.
There no reason to wrap functions that are just simple calculations or array manipulations with callbacks or promises.
Its just saying that if there's a library/method that offers synchronous version of the method, try to avoid that method.
Check out: https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
JavaScript execution in Node.js is single threaded, so concurrency
refers to the event loop's capacity to execute JavaScript callback
functions after completing other work. Any code that is expected to
run in a concurrent manner must allow the event loop to continue
running as non-JavaScript operations, like I/O, are occurring.
As an example, let's consider a case where each request to a web
server takes 50ms to complete and 45ms of that 50ms is database I/O
that can be done asynchronously. Choosing non-blocking asynchronous
operations frees up that 45ms per request to handle other requests.
This is a significant difference in capacity just by choosing to use
non-blocking methods instead of blocking methods.
The event loop is different than models in many other languages where
additional threads may be created to handle concurrent work.
Regarding additional overhead with wrapping everything in promises. The answer is still no.
You will experience no difference in
function sum(x,y) {
return x+y
}
const ans = sum(1,2)
console.log(ans) // 3
and
function sum(x,y) {
return Promise.resolve(x+y) // Shorthand for your new Promise
}
sum(1,2).then(ans => {
console.log(ans) //3
})
I have hundreds of master JS scripts which look like this and which I may not modify:
> some code filling parameter1
var response1=callAjax(parameter1);
> some code using response1 and filling parameter2
var response2=callAjax(parameter2);
> etc ..
The callAjax function can be changed or re-written. It currently calls Ajax in synchronous mode - which is deprecated - so it needs to be overhauled. Blocking the UI during the execution of the master file is a requirement (these are banking and similar applications so the end user is not allowed to edit any input fields or click any button during the process - a screen veil can be shown).
How can I modify the callAjax function to use async mode and leave the master scripts (hundreds or thousands of them) un-changed ?
I looked at using Promises or async/await but cannot figure how to use these in order to return the ajax response to the master scripts. Same with adding some global variables used as semaphores. Again, breaking the master scripts into several functions for each call is not allowed. These are plain Javascript, no jQuery.
How can I modify the callAjax function to use async mode and leave the master scripts (hundreds or thousands of them) un-changed ?
You can't. If you use asynchronous ajax, then you can't return the value from the function because the function will return BEFORE the value is available. You simply can't make an async operation into a synchronous function. For a lot more detail on returning asynchronous operations from a function, see How do I return the response from an asynchronous call?. You will see that all the available options involve calling a callback when the async value is done or returning a promise and using a callback with .then() on the promise to know when the value is done and what it is.
If your requirement is that you block the UI and that you can't change the calling code, then you have to stick with synchronous Javascript. No asynchronous operation will do either.
Otherwise, get rid of those requirements and write proper asynchronous calling code and don't block the UI in this way (you can protect the contents of the screen from editing or button presses during the ajax operation with different techniques).
So, my conclusion from the problem you've described is that you need to lift some of your current requirements because there is no solution that:
Blocks the UI
Uses only asynchronous Ajax
Returns the ajax obtained value directly from callAjax()
Does not change the calling code that calls callAjax()
You can't do all of those at once.
The forward moving design choice would be to return a promise from callAjax() and fix all the calling code that uses it to use that promise. For blocking the UI, you'd have to use some sort of UI protection (screen veil, you called it, or something like that).
My advice if you have a large body of code using this old synchronous ajax function is to create a new ajax function with a different name that returns a promise and deprecate use of the old one for any new code (make all new code use the new one). Then, as needed and guided by business goals, rewrite uses of the old function to use the new one one at a time. This will at least stop more code being written with the old, synchronous ajax code. And, it will give you an opportunity over time to convert code to the new interface as needed.
Note that the closest thing you could get to calling code that looks close to what you have (but still requires changes) would be using async/await (available in the ES7 specification or in transpilers). You'd return a promise from callAjax() and then, the caller can await the result:
// async function returns a promise
async function someFunction() {
try {
// some code filling parameter1
var response1 = await callAjax2(parameter1);
// some code using response1 and filling parameter2
var response2 = await callAjax2(parameter2);
// etc...
} catch(e) {
// handle error here
}
}
P.S. It looks like your current code doesn't have a good way to communicate back ajax errors.
I have a question regarding synchronous run of included function. For example, I have the following code.
partitions(dataset, num_of_folds, function(train, test, fold) {
train_and_test(train,test, function(err, results)
{
})
})
where partitions runs num_of_folds times, for deffierent fold it returns different train and test sets. I want to run every iteration of partitions only when train_and_test is finished. How to do so?
Given your previous question: Optimum async flow for cross validation in node.js, I understand that partition is your own function, which splits the dataset into several parts, and then runs the callback (basically train_and_test here) for each of those parts.
Your issue now is that train_and_test is asynchronous, but you want to wait for each invocation to finish (which is signalled by the its own callback being called, I assume) before running the next one.
One trivial solution is do change your code to keep state, and run the next invocation from the callback. For instance:
exports.partitions = function(dataset, numOfPartitions, callback) {
var testSetCount = dataset.length / numOfPartitions;
var iPartition=0;
var iteration = function() {
if (iPartition<numOfPartitions)
{
var testSetStart = iPartition*testSetCount;
var partition = exports.partition(dataset, testSetStart, testSetCount);
callback(partition.train, partition.test, iPartition, iteration);
iPartition++;
}
};
iteration();
};
You'll then need to pass the additional callback down to your asynchronous function:
partitions(dataset, num_of_folds, function(train, test, fold, callback) {
train_and_test(train,test, function(err, results)
{
callback();
})
});
Note that I haven't tested any of the code above.
I want to run every iteration of partitions only when train_and_test is finished. How to do so?
That's entirely up to how partitions calls the callback you're giving it. If partitions calls the callback normally, then it will wait for the callback to finish before proceeding. But if it calls is asynchronously, for instance via nextTick or setTimeout or similar, then the only way you'd be able to tell it to wait would be if it provides a means of telling it that.
If I read your question another way: If train_and_test is asynchronous, and partitions isn't designed to deal with asynchronous callbacks, you can't make it wait.
Is there some way to make a number of synchronous ajax call in the same time? Lets say I have a number of ajax calls that have no dependency of each other. I want them to run in the same time but I can't use asynchronous calls because of some reason.
No, there is no way. Either you run them synchronously, one after another, or asynchronously, all at the same time :) What you want is absolute opposite and it can't be done, sorry :(
What I think you want - is to make the ajax calls in parallel and then make another action.
So you need to use the Async operations in parallel option - sync.js
Async operations in parallel -
http://alexeypetrushin.github.io/synchronize/docs/index.html
var sync = require('synchronize')
function read(a, cb) {
setTimeout(function(){
cb(null, a)
}, 1000)
}
// Runs in parallel
sync.fiber(function() {
sync.parallel(function() {
// You must call defer() manually within
// a parallel operation.
read(1, sync.defer())
read(2, sync.defer())
});
var results = sync.await()
// results now contains [1,2]
});
You should probably just use asynchronous calls. And do some kind of check to see if they've all been completed before doing what your script needs to do next, if that is the concern here. For example by using a setTimeout() function, or by using a global counter variable that is incremented with each ajax response.
I'm not sure why you can't call it asynchronously but you can try using Web Workers instead
http://www.w3schools.com/html/html5_webworkers.asp