I'm pretty new to Javascript and am trying to wrap my head around the Javascript's asynchronicity concept.
my goal is,
responseStr = httpPost(parameters)
// then store responseStr in DB
However, the Javascript I can do is,
var callback = function(response) {
var str = '';
//another chunk of data has been recieved, so append it to `str`
response.on('data', function(chunk) {
str += chunk;
});
//the whole response has been recieved, so we just print it out here
response.on('end', function() {
console.log(str);
});
};
http.request(options, callback).end();
So I get the http post request's response into the local variable str, from above code, but I still cannot imagine how to pass it back to my httpPost caller into responseStr
Basically, the flow is,
Within my httpPost function, I sent the http request via http.request,
and it returns immediately,
somehow sometime later, the callback function populated the str variable bits by bits
but how to pass it back to my httpPost caller into responseStr when all done so that I can store it away? Thx.
Summary:
Thank you all who answered. I enjoy all your sense of humor greatly, upvoting you all.
PS. sharing a piece of comment I read the other day, about the Javascript's asynchronicity,
Today, we will examine four different methods of performing the same task asynchronously, in node.js...:
In parallel, using callback functions
Sequentially, using callback functions
In parallel, using promises
Sequentially, using promises
This will help you decide which to use for your particular situation. It is simply a matter of taste. If you want to know which is the best method -- the absolute best way to go -- is probably to switch to Golang.
:-)
There are a couple of problems with what you are trying to do. The first problem being that it is impossible to return a value from a non blocking function that is only available in the future. Don't fret though, once time travel is possible we won't have to worry about this issue ever again :) But until then, where do we go from here? There are a couple of things we can do to get the value.
The first (preferred) solution is to modify your program by simply moving all functionality that uses the value in the place where the value becomes available e.g. the callback. For example, suppose the only functionality that uses value is the function call do_stuff_with(value)
// original program
var value = get_value('param', callback);
do_stuff_with(value);
// changes to
get_value('param', function (value) {
do_stuff_with(value);
});
Usually the reponse I get from new programmers is "Oh no! That means I need to change my entire program! Is there anything else we can do. I don't want to change the code. It is perfect and is now a part of my family. You monster!" I must admit, this is exactly how I felt after being proposed this same solution. There is another solution to this problem you might want to use right away, but it will drastically affect the performance of your program.
The second (terrible, naive, you'll be sorry, don't do it) solution that successfully returns the value from the function call requires you to use a method that supports synchronous functionality. I only see synchronous code in all languages other than javascript, because javascript is single threaded. "Blasphemy, What the hell does that mean?" That means the entire javascript program needs to pause the entire process in order to wait for an http request to complete. It is not possible to open an http request without opening a new thread. Is that bad? Hell yea it is. That means the entire user interface will freeze during an http request. That means other tasks scheduled tasks will not be called on time. You know what it's like to play a video game that freezes all the time. It's terrible.
"But I have no imagination and I really want to write a bad program." - Fine, just modify your program to use a method that implements synchronous functionality instead of an asynchronous method that takes a callback. I have no idea where to get this, because I've never needed it.
// replace this method
var value = get_str_async_method('param', callback);
console.log(value) // undefined
// use one that is synchronous
// a callback is not needed for this method
var value = get_str_sync_method('param');
console.log(value); // reponse
It's pretty straightforward: you can't do that. 😀
This is one of those problems that you have to cope with in JavaScript. Your callbacks get called after code that got pushed into the event loop was executed, so the surrounding code in your program has already executed synchronously and is long gone.
You should dig into the event loop to get a better feel for this.
As things are arranged currently in your code, nested code is the most straightforward way to handle this. Whatever depends on str gets moved to or called from within the callback where str gets it's value.
Another way to handle this is to find a package that supports a promise based interface. This allows your asynchronous code to transform from the nested structure asynchronous callbacks force you into and flattens the code. It can even allow you to do something like what you're trying to do here.
const context = {};
doAsync()
.then(function(result) {
// Do something with result
context.str = result;
return doSomethingElseAsync();
})
.then(function(result) {
// No more nesting, more readable, easier to reason about
context.str2 = result;
return doAsync3(context.str);
});
if you want to store all data after getting it you can add it to the nested function as down below
var callback = function(response) {
var str = '';
//another chunk of data has been recieved, so append it to `str`
response.on('data', function(chunk) {
str += chunk;
});
//the whole response has been recieved, so we just print it out here
response.on('end', function() {
console.log(str);
responseStr = httpPost(parameters); //this line should be added here
});
};
http.request(options, callback).end();
Related
I'm quite new to using Javascript and particularly JSON, and I've been struggling to do this:
There is a JSON file on my web server which I am trying to access and parse into a JavaScript object. What I am trying to do is parse that JSON into an array and then further manipulate that array based on other user variables.
This is a sample of what the JSON looks like:
{"log":
[{
"name":"Al",
"entries":[8,12,16,19]},
{"name":"Steve",
"entries":[11,17,22]}]}
What I need to do is retrieve the array for one of the entries and store it in an array as a JavaScript object. What I have tried to do is this:
var entriesLogged;
fetch ('url to the json file').then(function(response){
return response.json();
}).then(function(data){
entriesLogged = data.log[0].entries;
});
However, I can't seem to get this to work and to assign the value to the variable in a way that persists outside of this scope. I have been able to output the value of the array using console.log, but I have not been able to actually work with and manipulate that data like an object. I'd ideally like to parse the JSON file from the server onto a global array.
Most of the tutorials I've come across so far have used the JSON file to output console logs or change the contents of html elements, however I need to retrieve the values from the JSON into a global array first.
Am I missing something here? Does anyone have advice on how to do this?
Best wishes,
Dom
Are you trying to manipulate the entriesLogged variable after the fetch promise chain? The fetch is asynchronous so this means that any code after the fetch chain will run before the fetch chain finishes.
var entriesLogged;
fetch('url to the json file').then(function(response){
return response.json();
}).then(function(data){
entriesLogged = data.log[0].entries;
});
// Code down here will run before the fetch chain finishes
// So if you're using entriesLogged down here, it will be null
console.log(entriesLogged); // entriesLogged is null
You might want to do something like:
.then(function(data){
entriesLogged = data.log[0].entries;
myFunction(entriesLogged);
});
function myFunction(myData) {
console.log(myData); // can access entriesLogged here
}
What is happening is that the call is async, it means that when you make that call a new thread is created and javascript automatically passes to the next line.
var entriesLogged;
fecth(something).then(function(data){entriesLogged = data[0]}); // this goes to another thread an takes e.g. 5 miliseconds but javascript does not wait for this to complete, it goes to the next line
console.log(entriesLogged); // this is executed before the call is finished
There are TONS of "answers" for variations of this question that don't actually solve the problem, they just describe the issue. (tl;dr solution is last block of code)
The issue is that .fetch('URL.json') and .json() are "asynchronous" calls. So the script will start working on that call, then continue on to the rest of the commands in your script while the .fetch() and .json() calls are still being worked on. If you then hit a command BEFORE one of these asynchronous calls is done, the rest of your script will have no data to work with.
There are tons of places to look for fully understanding the ins and outs of how async calls work in JS, for example here:
How do I return the response from an asynchronous call?
If you're like me and you just want the code to function synchronously, the quick and dirty solution is to create an async main loop and then to make sure you use the await keyword when calling asynchronous functions that you need to WAIT for the data to be populated on.
Example main loop:
(async () => {
// put main loop code here
})();
You would then put your code inside that main loop BUT you have to make sure that you're adding the await keyword in front of every asynchronous call you use. in the original example provided, you'll have to declare your .then() functions that need to use await as async as well. This is confusing if you don't fully understand the asynchronous nature of javascript and I'll simplify it later, but I wanted to provide a working version of the original code provided so you can understand what changed:
(async () => {
var entriesLogged;
await fetch('url_to_the_file.json')
.then( async function(response){
return await response.json();
})
.then( function(data){
entriesLogged = data;
});
})();
The .then() call is typically used for post-processing so you can do work inside of the asynchronous call... but since in this example we're deliberately making it synchronous, we can avoid all of that and clean up the code by doing individual simple commands:
// Start main loop
(async () => {
let response = await fetch('url_to_the_file.json');
let entriesLogged = await response.json();
})();
That should get you what you're looking for and hopefully save anyone like us the heartache of spending hours of time trying track down something that there is a simple solution for.
Also want to give a huge call out to the place I got the solution for this:
https://www.sitepoint.com/delay-sleep-pause-wait/
Note: I'm bootstrapping a reactjs app but this is a general JavaScript question.
I have a special module "locationUtils" that I am trying to keep in it's own package, but keeping that code separate is causing an eyesore with callbacks.
When I access one of it's methods I have to send a callback with it that only has one of its parameters initially defined, and in that method I get the other data parameter to initalize the other parameter.
Can I add in undefined parameters later like that in JavaScript, and is it good practice to initial parameters for a callback method as you go down the callback chain in general, or am I making a convoluted newbie mistake?
/********************Module 1******************************/
var bootStrapUI = function(callback) {
locationUtils.findData(findOtherData(callback));
}
//This gets called last to finalize bootstraping
var findOtherData = function(callback,originalFetchedData){
//use originalFetchedData to get more data
//bootStraping program with all rendering data
callback() //sends back a boolean confirming all fetched
}
/**********************Module2**********************************/
var findData = function(findOtherData){
var data = magicGetData();
findOtherData(findOtherData,data);//I initialized a param late here!
}
It's a good Javascript question, callbacks can become a serious hell for the uninitiated, particularly when they are nested and / or the order in which they return is important.
This is where promises come in: they are an essential tool for Javascript development and about to become part of the standard (in EcmaScript 6).
In essence: a promise is an object that is returned from a function with a method (callback) that is called when the asynchronous action (e.g. API call) has been completed. The difference between a promise and a callback is that promises allow you to structure how you handle the callbacks and, importantly, in what order.
I recently wrote a method that had to make 30 api calls with each call dependent on the results of the previous one (this was not a well designed api). Can you imagine trying to do that with callbacks? As it was, I created an array of promises and used jQuery.when() to handle things when all the api calls had completed.
For the moment we need to use a library for promises. jQuery: https://api.jquery.com/jquery.deferred/ is the obvious one but there are various other implementations that do much the same thing.
Update:
The question relates more specifically to the passing of arguments between callbacks and modifying the arguments as execution moves between them. This can be done easily by passing whatever info you need as an argument to your resolve method.
Typically it looks something like this (using jQuery):
var myAsyncMethod = function(info){
var deferred = $.Deferred();
$.getJSON(myUrl,
function(dataFromServer) {
// Do stuff with data
var newData = doSomething(dataFromServer);
deferred.resolve(newData);
});
});
return deferred.promise();
};
// Make initial method call
myAsyncMethod(myInitialData).then(
function(transformedData){
// transformed data from server is returned here.
}
);
I recently did a lot of coding in AngularJS. After some time it started to feel comfortable with it and also got really productive. But unfortunately there is this one thing I don't understand:
Within my project I need to get data through $http.get and a RESTful API server. This is where I started to stumble first. After implementing promise ($q.defer etc and .then) at functions which are processing data that's necessary to continue, I thought I conquered the problem.
But in this code:
$scope.getObservationsByLocations = function() {
var promise = $q.defer();
var locationCount = 0;
angular.forEach($scope.analysisData, function(loc) { // for each location
$http.get($scope.api + 'Device?_format=json', { // get all devices
params: {
location: loc.location.id
}
}).then(function (resultDevices) {
var data = angular.fromJson(resultDevices);
promise.resolve(data);
// for each device in this location
angular.forEach(angular.fromJson(resultDevices).data.entry.map(function (dev) {
http.get($scope.api + 'Observation?_format=json', { // get all observations
params: {
device: dev.resource.id
}
}).then(function (resultObservations) {
var observations = angular.fromJson(resultObservations);
// for each obervation of that device in this location
angular.forEach(observations.data.entry.map(function(obs) {
$scope.analysisData[locationCount].observations.push({observation: obs.resource});
}));
})
}))
});
locationCount++
});
return promise.promise
};
I can't understand in which order the commands are executed. Since I use the Webstorm IDE and it's debugging feature, it would be more accurate to say I don't know why the commands are executed in an order I don't understand.
Thinking simple, everything included in the forEach have to be executed before the return is reached, because $http.get's are connected through .then's. But following the debugging information, the function iterates over locationCount++ and even returns the promise before it goes deeper (meaning after the first .then() ).
What's that all about? Did I misunderstood this part of the AngularJS concept?
Or is this just really bad practice and I should reach out for a different solution?
If the context is important/interesting: Objects are based on i.e. https://www.hl7.org/fhir/2015May/location.html#5.15.3
With JavaScript you can create only single thread applications, although e.g. here they say it is not guaranteed to be like that.
But we are talking about the real world and real browsers, so your code sample is running as a single thread (by the way the same thread is also used for rendering your CSS and HTML, at least in Firefox).
When it comes to an asynchronous call
$http.get($scope.api + 'Device?_format=json', {
it says "hey, I can do that later". And it waits with that because it must go on with the current thread.
Then, once the current task is done with return it finally can start getting the remote data.
Proof? Check this fiddle:
console.log(1);
for (var i=0;i<1000000;i++) setTimeout(function(){
console.log(2);
},0);
console.log(3);
You see the spike with the for loop? This is the moment when it registers the setTimeout asynchronous calls. Still 3 is printed before 2 because the task is not done until the 3 is printed.
The $http.get is asynchronous, so depending on (among other things) how large the fetched data is, the time it takes to 'complete' the get is variable. Hence why there is no saying in what order they will be completed
I have this code:
var resources = myFunc();
myFunc2(resources);
The problem is that JavaScript calls myFunc() asynchronous, and then myFunc2(), but I don't have the results of myFunc() yet.
Is there a way to block the first call? Or a way to make this work?
The reason why this code doesn't work represents the beauty and pitfalls of async javascript. It doesn't work because it is not supposed to.
When the first line of code is executed, you have basically told node to go do something and let you know when it is done. It then moves on to execute the next line of code - which is why you don't have the response yet when you get here. For more on this, I would study the event-loop in greater detail. It's a bit abstract, but it might help you wrap your head around control flow in node.
This is where callbacks come in. A callback is basically a function you pass to another function that will execute when that second function is complete. The usual signature for a callback is (err, response). This enables you to check for errors and handle them accordingly.
//define first
var first = function ( callback ) {
//This function would do something, then
// when it is done, you callback
// if no error, hand in null
callback(err, res);
};
//Then this is how we call it
first ( function (err, res) {
if ( err ) { return handleError(err); }
//Otherwise do your thing
second(res)
});
As you might imagine, this can get complicated really quickly. It is not uncommon to end up with many nested callbacks which make your code hard to read and debug.
Extra:
If you find yourself in this situation, I would check out the async library. Here is a great tutorial on how to use it.
myFunc(), if asynchronous, needs to accept a callback or return a promise. Typically, you would see something like:
myFunc(function myFuncCallback (resources) {
myFunc2(resources);
});
Without knowing more about your environment and modules, I can't give you specific code. However, most asynchronous functions in Node.js allow you to specify a callback that will be called once the function is complete.
Assuming that myFunc calls some async function, you could do something like this:
function myFunc(callback) {
// do stuff
callSomeAsyncFunction(callback);
}
myFunc(myFunc2);
Is there a way to wait on a promise so that you can get the actual result from it and return that instead of returning the promise itself? I'm thinking of something similar to how the C# await keyword works with Tasks.
Here is an example of why I'd like to have a method like canAccess() that returns true or false instead of a promise so that it can be used in an if statement. The method canAccess() would make an AJAX call using $http or $resource and then somehow wait for the promise to get resolved.
The would look something like this:
$scope.canAccess = function(page) {
var resource = $resource('/api/access/:page');
var result = resource.get({page: page});
// how to await this and not return the promise but the real value
return result.canAccess;
}
Is there anyway to do this?
In general that's a bad idea. Let me tell you why. JavaScript in a browser is basically a single threaded beast. Come to think of it, it's single threaded in Node.js too. So anything you do to not "return" at the point you start waiting for the remote request to succeed or fail will likely involve some sort of looping to delay execution of the code after the request. Something like this:
var semaphore = false;
var superImportantInfo = null;
// Make a remote request.
$http.get('some wonderful URL for a service').then(function (results) {
superImportantInfo = results;
semaphore = true;
});
while (!semaphore) {
// We're just waiting.
}
// Code we're trying to avoid running until we know the results of the URL call.
console.log('The thing I want for lunch is... " + superImportantInfo);
But if you try that in a browser and the call takes a long time, the browser will think your JavaScript code is stuck in a loop and pop up a message in the user's face giving the user the chance to stop your code. JavaScript therefore structures it like so:
// Make a remote request.
$http.get('some wonderful URL for a service').then(function (results) {
// Code we're trying to avoid running until we know the results of the URL call.
console.log('The thing I want for lunch is... " + results);
});
// Continue on with other code which does not need the super important info or
// simply end our JavaScript altogether. The code inside the callback will be
// executed later.
The idea being that the code in the callback will be triggered by an event whenever the service call returns. Because event driven is how JavaScript likes it. Timers in JavaScript are events, user actions are events, HTTP/HTTPS calls to send and receive data generate events too. And you're expected to structure your code to respond to those events when they come.
Can you not structure your code such that it thinks canAccess is false until such time as the remote service call returns and it maybe finds out that it really is true after all? I do that all the time in AngularJS code where I don't know what the ultimate set of permissions I should show to the user is because I haven't received them yet or I haven't received all of the data to display in the page at first. I have defaults which show until the real data comes back and then the page adjusts to its new form based on the new data. The two way binding of AngularJS makes that really quite easy.
Use a .get() callback function to ensure you get a resolved resource.
Helpful links:
Official docs
How to add call back for $resource methods in AngularJS
You can't - there aren't any features in angular, Q (promises) or javascript (at this point in time) that let do that.
You will when ES7 happens (with await).
You can if you use another framework or a transpiler (as suggested in the article linked - Traceur transpiler or Spawn).
You can if you roll your own implementation!
My approach was create a function with OLD javascript objects as follows:
var globalRequestSync = function (pUrl, pVerbo, pCallBack) {
httpRequest = new XMLHttpRequest();
httpRequest.onreadystatechange = function () {
if (httpRequest.readyState == 4 && httpRequest.status == 200) {
pCallBack(httpRequest.responseText);
}
}
httpRequest.open(pVerbo, pUrl, false);
httpRequest.send(null);
};
I recently had this problem and made a utility called 'syncPromises'. This basically works by sending what I called an "instruction list", which would be array of functions to be called in order. You'll need to call the first then() to kick things of, dynamically attach a new .then() when the response comes back with the next item in the instruction list so you'll need to keep track of the index.
// instructionList is array.
function syncPromises (instructionList) {
var i = 0,
defer = $q.defer();
function next(i) {
// Each function in the instructionList needs to return a promise
instructionList[i].then(function () {
var test = instructionList[i++];
if(test) {
next(i);
}
});
}
next(i);
return defer.promise;
}
This I found gave us the most flexibility.
You can automatically push operations etc to build an instruction list and you're also able to append as many .then() responses handlers in the callee function. You can also chain multiple syncPromises functions that will all happen in order.