I have written below code to get the response data chunk in the variable called outBody, but it is showing undefined. Any suggestions please?
var options = {
host: 'jsonplaceholder.typicode.com',
port: 80,
path: '/todos/1'
};
var outBody;
http.get(options, function(res) {
console.log("Got response: " + res.statusCode);
res.on("data", function(chunk) {
outBody = chunk;
});
}).on('error', function(e) {
console.log("Got error: " + e.message);
});
console.log(outBody);
I actually want to use the variable outBody outside the http request. How can I use it?
Your code is showing undefined, because your code is a asyncronous code and you are logging data synchronously.
A simple fix is that put your console.log inside the data event listener like this
res.on("data", function(chunk) {
outBody = chunk;
console.log(outBody);
});
and if you want to use the outBody variable outside the http.get then you either has to use promises or a callback function inside another event listener called end , which fires when the response is completed.
Like this
using CallBack :-
//Your code
res.on("data", chunk => {
outBody += chuck
});
res.on("end", () => {
// this event fires when the response body is completely loaded ...
myFunction(outBody); // pass the respone body to your callback function
}
// remaing code
Wanna learn how to deal with this problem again, then learn about async / await and promises on internet
OR
Using promises / async await :-
const options = {...}
var outBody = "";
const req = new Promise( (resolve, reject) => {
http.get(options, res => {
res.on("data", chunk => { outBody += chunk });
res.on("end", () => resolve(outBody));
res.on("error", console.error);
});
});
req.then( data => {
myFunction(data);
})
.catch(err => console.error(err));
// or u can try async / await
(async function (){
const data = await req;
})();
Note : You had done a mistake i.e., inside the data event listener you overwrite the outBody with the new chunk. It is a mistake because the http class loads the response in chuck which means parts. It means you are losing data , I also fixed it by adding += in place of outBody = chunk, it simply appends the new part the older part which gives you the complete response body...
Still confused about the promises and async / await , then them here
The http request is asynchronous, the reason why console.log is logging undefined is because you log the result of the request before the request is done, only after the request is done outbody gets the value of chunk. In order to access the outbody value you should write the console.log inside the callback block which is called right after the data is returned from the request like this:
http.get(options, function(res) {
console.log("Got response: " + res.statusCode);
res.on("data", function(chunk) {
outBody = chunk;
console.log(outbody)
});
res.on("error",e=>{
console.log("Got error: " + e.message);
})
})
Related
Hopefully someone can point me to the right direction. I read up on waiting for functions to complete before continuing and I resolved myself to using await/async but I am just stuck now.
I tried to get the Async/Await process to work, tried to inject the await in various locations, with adjusting the functions to be async, but i can not get the PSA_Resultbody to return to the original request. Any pointers would be appreciated.
Thank you,
CE
PSA_Resultbody = ProcessPSAAPI(xmlpackage, PSA_Action);
console.log("3 - Returned data:" + PSA_Resultbody);
calls the below:
async function ProcessPSAAPI(xmlpackage, PSA_Action) { //psa action is part of the options
var options = {...};
var req = https.request(options, function (res) {
var chunks = [];
res.on("data", function (chunk) {
chunks.push(chunk);
});
res.on("end", function (chunk) {
var body = Buffer.concat(chunks);
console.log('0 - Start '+body.toString());
if(res.statusCode != 200) {
PSA_Resultcode = "Error: " +res.statusCode +" - "+ res.statusMessage;
} else {
PSA_Resultcode = "Success: " +res.statusCode +" - "+ res.statusMessage;
PSA_Resultbody = ParseResults(body.toString()); //parse the results for later use --SCRIPT NEEDS TO WAIT FOR RESULTBODY TO COMPLETE
console.log("1 -PSA_Resultbody as part of RES = "+PSA_Resultbody);
}
});
res.on("error", function (error) {
console.error(error);
PSA_Resultcode = res.statusCode +" - "+ res.statusMessage;
});
});
console.log('2 -RESULT BODY BEFORE SENDING BACK TO INITIATING FUNCTION: '+PSA_Resultbody);
req.write(xmlpackage);
req.end();
return PSA_Resultbody;
Based on the above, my console log order is: 3,2,0,1 in stead of 0,1,2,3.
0 and 1 will have the correct data, so the API Call does work, but 2 will be "undefined" and should have the same data that is in 1.
There's no way to await an event emitter, so using async in this case isn't going to be useful. You cannot "return" from inside an event either.
The solution here is to return a new custom promise and to use resolve() inside of the "end" event of your emitter.
It will look something like this:
function ProcessPSAAPI(xmlpackage, PSA_Action) {
return new Promise( (resolve, reject) => {
// other code
res.on("end", function (chunk) {
// other code
resolve(PSA_Resultbody);
});
res.on("error", function (error) {
// other code
reject(error);
});
});
}
Here's a quick tutorial on creating your own promises, which I've written to simplify comprehension of the subject (official docs are somewhat dry and complex imho).
I did not change your code. I just put the appropriate promise structure in to get you started. This should really be a lesson in promises. async await is a shorthand promise structure. A Promise is one way you wait on code. It can be thought of as an array of callbacks that will be executed when the Promise is resolved.
A simple promise works like this:
const myPromise = new Promise(function(resolve, reject) {
/* Your logic goes in here. It can be anything.
* But the important part to remember is that when you have success, resolve it.
* When you have a failure, reject it.
*/
someCallBackPattern(function(error, data) {
if(error) {
reject(error);
} else {
resolve(data);
}
});
});
// To get the data out you use 'then', and 'catch'. then has two arguments.
myPromise.then(function(data) {
// The first argument is the result from resolve.
}, function(err) {
// The second argument is the result from reject.
}).catch((err) => {
// you can also get to the error from the catch callback
});
This is kinda messy and complex. So there is async await.
async function() {
try {
const result = await myFunctionThatReturnsAPromise();
// result is the resolved data
} catch (err) {
// err is the rejected Error
}
}
function myFunctionThatReturnsAPromise() {
return new Promise((resolve, reject) => {
// your code
})
}
And thats how it works.
async function someFunction () { // You can not wait on results unless you are in an await function
PSA_Resultbody = await ProcessPSAAPI(xmlpackage, PSA_Action); // await on your results.
console.log("3 - Returned data:" + PSA_Resultbody);
}
function ProcessPSAAPI(xmlpackage, PSA_Action) { // This does not need to be async. Unless you are awaiting in it.
return new Promise((resolve, reject) => { // async await is a shorthand promise structure. Although you do not need to use promises. It really helps to get the structure correct.
var options = {...};
var req = https.request(options, function (res) {
var chunks = [];
res.on("data", function (chunk) {
chunks.push(chunk);
});
res.on("end", function (chunk) {
var body = Buffer.concat(chunks);
console.log('0 - Start '+body.toString());
if(res.statusCode != 200) {
PSA_Resultcode = "Error: " +res.statusCode +" - "+ res.statusMessage;
reject(new Error(PSA_Resultcode)); // Reject you errors
} else {
PSA_Resultcode = "Success: " +res.statusCode +" - "+ res.statusMessage;
PSA_Resultbody = ParseResults(body.toString()); //parse the results for later use --SCRIPT NEEDS TO WAIT FOR RESULTBODY TO COMPLETE
console.log("1 -PSA_Resultbody as part of RES = "+PSA_Resultbody);
resolve(PSA_Resultbody); // Resolve your result
}
});
res.on("error", function (error) {
console.error(error);
PSA_Resultcode = res.statusCode +" - "+ res.statusMessage;
reject(new Error(PSA_Resultcode)); // Reject you errors
});
});
console.log('2 -RESULT BODY BEFORE SENDING BACK TO INITIATING FUNCTION: '+PSA_Resultbody);
req.write(xmlpackage);
req.end();
})
}
I am trying to pass data out of my get request from resp.on function. I want to use 'var url' to make a separate get request from which I will parse data again. I am able to console.log variables from inside the function but not return (or access from outside). This seems to be a scoping or async issue.
const https = require('https');
https.get('https://collectionapi.metmuseum.org/public/collection/v1/objects', (resp) => {
let data = '';
// A chunk of data has been recieved.
resp.on('data', (chunk) => {
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
var json_data = JSON.parse(data);
var total = json_data.total
var random_objectID = Math.floor(Math.random()*total)
var url = 'https://collectionapi.metmuseum.org/public/collection/v1/objects/' + random_objectID
console.log(url);
});
}).on("error", (err) => {
console.log("Error: " + err.message);
})
//'url' becomes unknown here. I want to pass it to another get request.
It's both an async and a scope issue!
If you declare var url; in the outermost scope you'll be able to set it inside that callback as intended, but since that's happening asynchronously you won't be able to use the value outside the scope unless you are checking after the callback completes.
Alternatively, you could wrap the whole thing in a promise, e.g.
const https = require('https');
new Promise((resolve,reject)=>{
let targetUrl = 'https://collectionapi.metmuseum.org/public/collection/v1/objects';
https.get(targetUrl,resp=>{
// ...
resp.on('end', () => {
// ...
resolve(url);
});
});
}).then(url=>{
// do stuff with that URL
});
If your goal is to automate fetching data from web resources, I'd recommend checking out the request module, which also has a promisified variant.
So I started this simple project in node.js where the client sends a POST request containing in it's body a windows CMD command.
The server receives the POST request, extracts the CMD command, and after running it, responds with the output of that command.
This worked fine when I had one request being sent out, but then I set up a system to repeatedly ask the user for the command and then send the POST request with the inputted command as it's body.
Here is the client-side code for this: (server side not included as it's irrelevant)
var http = require("http");
var readline = require("readline");
var rl = readline.createInterface(process.stdin, process.stdout);
var options = {
hostname: "localhost",
port: 3001,
path: "/",
method: "POST",
headers: {
"Content-Type": "text/plain", // all command must be string
"Content-Length": 0 // changes based on what the command is
}
};
function requestCommand(hostname, port, command) {
// Some of the options of the request need to be changed based on
// what the command is
options.hostname = hostname;
options.port = port;
options.headers["Content-Length"] = command.length;
var req = http.request(options, function(res) {
console.log(`Got response. Status code: ${res.statusCode}`);
var resData = "";
res.setEncoding("utf-8");
res.on("data", function(chunk){
resData += chunk
});
res.on("end", function(){
return resData;
})
})
req.on("error", function (e){
return "\n\n\n ---------------\nERROR OCCURED IN THE REQUEST.\nREQUEST NOT SENT\n--------------" + e.stack;
})
req.write(command);
req.end();
}
rl.setPrompt("What command would you like to request from the server?: ");
rl.prompt();
rl.on("line", function(data){
if (data === "exit") {
console.log("\n\n\Exiting appplication...\n\n");
process.exit();
} else {
console.log("processing..");
var out = requestCommand("localhost", 3001, data);
console.log(`\n\n${out}\n\n`);
rl.prompt();
}
});
If I run this without creating the server first, instead of getting the error message, I get undefined.
Currently, I think this is because the requestCommand function ends before the error is handled (before the error event is emitted and the callback function that returns the error is called), since the callback function for http.request is obviously asynchronous, and before the server responds or an error is emitted, the function ends and hence return nothing (undefined)
So my question is: Can I keep that function running until the asynchronous command is done?
Or if this is not possible, is there a different approach to this? How would you send requests to this server upon a certain event fired by the user, such as data input?
EDIT: I am really not interested in using a 3rd party modules as I already can. This project is really pointless and only here for me to learn, so I am only using core modules for this. More specifically, I am only using HTTP for making the requests (not sync-request, e.t.c..)
requestCommand has to return a promise that gets resolved with resData and rejected on error:
function requestCommand(hostname, port, command) {
return new Promise((resolve, reject) => { // immeadiately return a Promise that can be resolved/rejected later
// Some of the options of the request need to be changed based on
// what the command is
options.hostname = hostname;
options.port = port;
options.headers["Content-Length"] = command.length;
var req = http.request(options, function(res) {
console.log(`Got response. Status code: ${res.statusCode}`);
var resData = "";
res.setEncoding("utf-8");
res.on("data", function(chunk){
resData += chunk
});
res.on("end", function(){
resolve(resData); // resolve, return won't work here
});
});
req.on("error", function (e){
reject(e); // reject here, don't handle it
});
req.write(command);
req.end();
});
}
That way, you can simply make the request handler async and then await the function call:
rl.on("line", async function(data){ // make the function async so that we can work with promises more easily
if (data === "exit") {
console.log("\n\n\Exiting appplication...\n\n");
process.exit();
} else {
console.log("processing..");
try {
var out = await requestCommand("localhost", 3001, data); // "await" lets async code look like sync code, but it is still async
console.log(`\n\n${out}\n\n`);
} catch(e) { // the reject() call will end up here
console.log("invalid command " + e.stack); // handle here
}
rl.prompt();
}
});
I have a problem when I call my function JavaScript I can not return my Buffer because it's in other function
index.js:
var client = require('./sender');
//envoyer le paquet au seveur blizzard.com
var received =client.sendAndreceive(bufenc);
console.log('received: '+received);
sender.js:
var Http = require('http');
var Url = require('url');
function sendAndreceive(data) {
console.log('##########################################################################');
// connexion au serveur
var options = {
//proxy
host: "proxya.u-pec.fr",
port: 3128,
//hostname : 'http://m.eu.mobileservice.blizzard.com',
//port : 80,
path : 'http://m.eu.mobileservice.blizzard.com/enrollment/enroll.htm',
method : 'POST',
headers: {
Host: "http://m.eu.mobileservice.blizzard.com"
}
};
var req = Http.request(options, callback );
req.on('error', function(e) {
console.log('problem with request: ' + e.message);
});
req.write(data);
req.end();
}
exports.sendAndreceive = sendAndreceive;
function callback (res) {
res.on('data', function(chunk) {
buf = new Buffer(chunk);
console.log('data length: ' + buf.length);
return buf; // my problem is her !!!!!!!!!!!!!!!!!!!!!!
});
}
output:
##########################################################################
received: undefined
data length: 45
Using the JavaScript callback pattern
You won't get a result as you returning the result from the callback function, which is nothing and wouldn't work either as your request to the server is asynchronously. Take a look at the example below, and see how it can work:
function callback (res, callback) {
res.on('data', function(chunk) {
buf = new Buffer(chunk);
console.log('data length: ' + buf.length);
// Make sure the argument is indeed a callable function
if (typeof callback === 'function') {
callback(buf);
}
});
}
Now what you do is simply implement a callback, that is executed when the result is finished.
var client = require('./sender');
//envoyer le paquet au seveur blizzard.com
client.sendAndreceive(bufenc, function(received) {
console.log('Received', received);
});
This JavaScript pattern is called the callback pattern, and can be very useful in these kinds of situations.
Using a promise
Another more sexy way of doing stuff like this, is through promises. I don't want to fully explain what a promise is or does (you can use search, right?), so I'll just show you an example:
// This is part of the node-promise module: https://github.com/kriszyp/node-promise
var deferred = require("promise").defer;
var client = require('./sender');
function callback (res) {
// Create a deffered object that will be returned
var deferred = defer();
res.on('data', function(chunk) {
buf = new Buffer(chunk);
console.log('data length: ' + buf.length);
// Now let the listeners know that this promise
// has been completed successfully, and return the buf
deffered.resolve(buf);
// Or if something went wrong, you could do the following:
// deffered.reject('Everything is wrong!');
});
return deferred;
}
//envoyer le paquet au seveur blizzard.com
client.sendAndreceive(bufenc)
.then(
// Function ended successfully
function(received) {
console.log('Received', received);
},
// Function returned an error
function(err) {
console.log('Oh noes!', err);
}
)
;
You are calling sendAndreceive(), and sendAndreceive() is starting a HTTP request that asynchronously calls callback(). So the return value of callback() is a return value given to the invoker inside the HTTP request object, while sendAndreceive() returns nothing.
There is no way to force asynchronous calls behave like one step-by-step-flow. Therefore I support the recommendation of Boy, using a callback.
OK, I see the promise pattern by Boy :-) cool. I will take a deeper look. It seems you can make asynchronous steps at least looking like synchronously written with constructs like .then(). Thx!
I'm somewhat new to javascript. Recently, I've been working on a node.js project, and I came across the problem of asynchronous callbacks in javascript. My code is below:
So ideally, I would like the request method to be caried out, THEN the callback from the res.on('data') event to be executed, and finally have callback for the .each() iterator to be called. Unfortunately, the asynchronous nature of javascript causes the program to exit the main code block even before the first callback is executed.
var req = https.request(options, function(res) {
// console.log("statusCode: ", res.statusCode);
// console.log("headers: ", res.headers);
console.log('1st call back exec');
res.on('data', function(d) {
console.log('2st call back exec');
var $ = cheerio.load(d);
//traversing method
$('.dddefault').each(function(i){
console.log('3st call back exec');
if(i == 3){
remainingSeats = parseInt($(this).text().trim());
console.log('inner0:'+remainingSeats);
return;
}
});
console.log('inner1:'+remainingSeats);
});
console.log('inner2:'+remainingSeats);
});
req.end();
req.on('error', function(e) {
console.error(e);
});
Can anyone point me in the right direction to get my code working in the way I had intended it to work? Thanks.
It shouldn't abruptly stop unless you get an unhandled error. And, chances are, that error would probably be from cheerio as you may not be giving it the entire document to try to parse.
Note: The req.on('error', ...) callback only receives errors that http or req throw. It won't catch errors from cheerio.
res is a Stream, so it won't necessarily arrive in a single chunk of 'data'. You'll want to collect any number of chunks that may arrive and work with the combined result once the stream has come to an 'end'.
var req = https.request(options, function(res) {
var body = [];
res.setEncoding('utf8');
res.on('data', function (chunk) {
body.push(chunk);
});
res.on('end', function () {
body = body.join('');
var $ = cheerio.load(body);
// etc.
});
You may also consider using request, which handles consuming the stream for you:
request(options, function (err, res, body) {
var $ = cheerio.load(body);
// etc.
});