How to use Node Worker in Wakanda - javascript

Up until now for me the concept of a Node Worker has been one of those things that sounds interesting and I will find out about one day.
Well that day has come and I am asking for some help.
I need to call a web service and then process the data returned. I can call the service with an XMLHttpRequest, but then I have to get useful data out of it.
There is a nice node module that both calls the service and returns the data in a useful form with one call.
I can set up Node worker (in Wakanda) to do this and verify that it works.
My problem is handling the asynchronous call in the proxy.
The call to the node module looks like this:
myModule.getData(param, (err, data) => {
// data is an object containing everything I want.
// if I get data I want to return it to the proxy
// if I get an err I want to return the error to the proxy
});
So my wrapper code looks something like this:
function doSomething(param){
// call proxy with param
// wait for result
// return result
}
This all sounds like something I should know how to do. However I think I am struggling with too many new things and getting myself absolutely confused.
PS: I did try Threadify but couldn't work out how to get the worker to return the error it received.
I would really appreciate any help or pointers here.

If I am correctly understanding your issue, you cannot return a value from a function AFTER an asynchronous call completes. You need to handle the data within the myModule.getData callback.
If you would rather handle it in a calling function (like doSomething), you can use a promise to "return" a value.
function myProxy(param) {
return new Promise((resolve, reject) => {
myModule.getData(param, (err, data) => {
if (!err) { // or however you determine an error occurred.
resolve(data); // if I get data I want to return it to the proxy
} else {
reject(err); // if I get an err I want to return the error to the proxy
}
});
});
}
function doSomething(param){
myProxy(param).then(data => {
// Handle the data here.
}).catch(err => {
// Handle the error here.
});
}

Related

Multiple fetch request in a raw

For a project, I have to generate a PDF that lists the vehicles in a DB. To do this, I use the JSPDF library.
1st step: I get data from a DB (via asynchronous requests on an API) and images on a server that I store in an Array.
2nd step: I call a function that generates the PDF with JSPDF.
The problem is that I need to have all my data retrieved before calling my generatePDF function otherwise the fields and images are empty because they have not yet been retrieved from the DB or the server.
A solution I found is to use the setTimeout function to put a delay between each call. However, this makes the code very slow and inflexible because you have to change the timeout manually depending on the number of data and images to retrieve. Moreover, it is impossible to determine exactly how long it will take to retrieve the data, especially since this can vary depending on the state of the network, so you have to allow for a margin which is often unnecessary.
Another solution is to use callbacks or to interweave fetch / ajax calls with .then / .done calls, but this becomes very complicated when it comes to retrieving the images since they are retrieved one by one and there are more than a hundred of them.
What would be the easiest way to do this in a clean and flexible way?
Thanks for your help and sorry for the long text, I tried to be as clear as possible :)
To do a series of asynchronous things in order, you start the next operation in the fulfillment handler of the previous operation.
An async function is the easiest way:
async function buildPDF() {
const response = await fetch("/path/to/the/data");
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`);
}
const data = await response.json(); // Or `.text()` or whatever
const pdf = await createPDF(data); // Assuming it returns a promise
// ...
}
If you can't use async functions in your environment and don't want to transpile, you can write your fulfullment handlers as callbacks:
function buildPDF() {
return fetch("/path/to/the/data")
.then(response => {
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`);
}
return response.json(); // Or `.text()` or whatever
})
.then(data => createPDF(data))
.then(pdf => {
// ...
});
}
Note that I'm returning the result of the chain, so that the caller can handle errors.

NodeJS - Sending SQL Results to Google Sheets

I am trying to write a script that will query a local database and send the results to a Google Spreadsheet. Where I am currently stuck and looking for help is how do I get the results of my query.
var myQuery = [];
function runQuery(query, callback) {
client.connect(function (err) {
if (err) throw err;
client.query(query, function (err, result) {
if (err) {
console.log(err);
}
callback({ result: result });
client.end(function (err) {
if (err) throw err;
});
});
});
}
runQuery('SELECT item_number FROM item WHERE item_number LIKE \'11-%\' LIMIT 10', function (resultsObject) {
myQuery = JSON.stringify(resultsObject.result.rows);
console.log('Did stringify work?');
console.log(myQuery);
});
console.log(myQuery);
My output:
Info: Start process (3:46:11 PM)
[]
Did stringify work?
[{"item_number":"11-0011"},{"item_number":"11-0012"},{"item_number":"11-1255"},{"item_number":"11-0052"},{"item_number":"11-0060"},{"item_number":"11-1256"},{"item_number":"11-1281"},{"item_number":"11-0659"},{"item_number":"11-0660"},{"item_number":"11-0054"}]
Info: End process (3:46:12 PM)
I think I understand what's happening, the scope of the variable myQuery is set and printed within the runQuery function just fine, but outside of that function it's not being actually set. How do I get around this?
Very new to JavaScript and NodeJS so I hope I'm using the correct terminology.
Thanks in advance!
The empty myQuery has already been consoled before you actually put values into it. This issue is caused by Javascript's Asynchronous mechanism. If you want to do something with the result, you have to use a callback function. Just like what you did when you accessed result from runQuery.
Another way to do it is promise. You can return a promise in some function and use the promise to access result.
If you decide to use promise, here is the code:
function getresult(parameters) {
return new Promise((resolve, reject) => {
runQuery(parameter, resolve, reject);
});
}
Using this function, you passed resolve as callback to runQuery. You do want to add another err callback function to catch err in runQuery. In promise you use reject as err callback.
When you want to do something with the result:
getresult(parameters)
.then((result) => {put your code here. result variable is your result})
.catch((err) => {do whatever with err});
This is not a scope issue. It is a timing issue. The query takes some time to execute. JavaScript is 'non-blocking' meaning it will start an asynchronous process (like doing a DB query - really virtually all I/O operations) and return the result later. In your code you are using a callback that is run once the data becomes available.
So, with that in mind, the way to think about your code is that you are doing things in this order:
1) declaring your function runQuery - this happens first since in JavaScript function declarations are moved to the top of the script they are in (a process called hoisting)
2) your variable myQuery is established and initialized (to an empty array)
3) your query is kicked off by a call to runQuery; this only starts the process
4) your console.log(myQuery) at the end of your script is run
5) some time later the query completes and your callback is called with the data where your other console.log() shows the returned data

Using Angular 2 to call jQuery based Javascript ajax library

I have a feeling that I'm just trying to fit a square peg into a round hole, but I'm trying to apply some things with Angular2 and Typescript, and I'm banging my head against a wall.
I've written a Javascript module that acts as an API client library to an API I'm consuming. It just packages some convenience things like setting up the correct API keys, switching keys based on certain desired data, etc. It's basically just a convenience library.
Most of the methods follow a pattern where you provide a query term and then execute a callback.
So for example:
API.searchAutocomplete("angular", function(err, data) {
// handle the data/error
});
Inside that method:
searchAutocomplete: function(query, callback) {
// set up request with data payload, url, headers, etc
$.ajax(settings)
.done(function(response) {
// callback with success
})
.fail(function () {
// callback with error
});
}
I'm struggling with trying to understand how to run this function in Typescript in an Angular service with a Promise (square peg round hole). Or should I just pass a callback within the service and treat it like it's Javascript?
My attempt:
public getAutocomplete(query:string): Promise < any > {
return new Promise((resolve, reject) => {
API.searchAutocomplete(query, function (err, result) {
if (err) {
reject(err);
return;
}
resolve(result);
});
});
}
Second, I've been able to load the library into my Angular app but I can't seem to actually make any of the requests. Even if I break in the console and access the library object it doesn't seem to actually make any network requests. Which I really don't understand.
Edit: I've sorted this part out.
When I made my service call return a promise, I had to subscribe to the promise otherwise I wouldn't execute it correctly. I think I still need to understand how to write my service call to return an observable and map the callback response.
As expected, I was trying to do more work than I should have.
This is pretty simple, just return an observable that calls the external library.
public autoCompleteResults(query: string): Observable<string[]> {
return new Observable<string[]>(observer => {
API.searchAutocomplete(query, function (err, result) {
if (err) {
console.log(err);
observer.next([]);
// OR
observer.error(err);
return;
}
observer.next(result);
});
});
}

How to return output variable of exec command back to client in Meteor? [duplicate]

I have an event triggering a Metor.call():
Meteor.call("runCode", myCode, function(err, response) {
Session.set('code', response);
console.log(response);
});
But my runCode function inside the server's Metheor.methods has inside it a callback too and I can't find a way to make it return something to response in the above code.
runCode: function(myCode) {
var command = 'pwd';
child = exec(command, function(error, stdout, stderr) {
console.log(stdout.toString());
console.log(stderr.toString());
// I Want to return stdout.toString()
// returning here causes undefined because runCode doesn't actually return
});
// I can't really return here because I don't have yet the valuer of stdout.toString();
}
I'd like a way to have the exec callback return something as runCode without setInterval which would work, but as a hacky way in my opinion.
You should use Future from fibers.
See docs here : https://npmjs.org/package/fibers
Essentially, what you want to do is wait until some asynchronous code is run, then return the result of it in a procedural fashion, this is exactly what Future does.
You will find out more here : https://www.eventedmind.com/feed/Ww3rQrHJo8FLgK7FF
Finally, you might want to use the Async utilities provided by this package : https://github.com/arunoda/meteor-npm, it will make your like easier.
// load future from fibers
var Future=Npm.require("fibers/future");
// load exec
var exec=Npm.require("child_process").exec;
Meteor.methods({
runCode:function(myCode){
// this method call won't return immediately, it will wait for the
// asynchronous code to finish, so we call unblock to allow this client
// to queue other method calls (see Meteor docs)
this.unblock();
var future=new Future();
var command=myCode;
exec(command,function(error,stdout,stderr){
if(error){
console.log(error);
throw new Meteor.Error(500,command+" failed");
}
future.return(stdout.toString());
});
return future.wait();
}
});

Most efficient way to receive json async responses?

(my case applies to C#, MVC, returning JSON, got jquery, angular), but I expect it applies to more than that.
I have a website where my angular/html/js calls ~7 services through Angular controllers and async-gets/displays data (weather, road conditions, etc). Some of these take longer than others (from ms to ~10s). I'd like to have a single call to my service which returns all of this data - but doesn't wait until the last call to return anything (10s).
Is there a way to make a single call, and return results as I have them and they get displayed accordingly? Do I need to have a repeating call which has a boolean like "IsMore=T" and calls the service again? (doesn't sound efficient).
Ideally, I'd like to keep a response channel open and keeping pumping results until it's done. Possible?
I'm not sure I understand completely, but I think you could just chain the response promises together, something like:
$scope.getWeatherData = function () {
return myWeatherService.get().then(function (resp) {
$scope.weatherData = resp;
});
}
$scope.getTrafficData = function () {
return myTrafficService.get().then(function (resp) {
$scope.trafficData = resp;
});
}
$scope.getWeatherData.then(getTrafficData).then(...chain others...);
This assumes that the service calls return a $http promise.
Anyway, whenever the promise comes in, the data will be on $scope, and hence the display will be updating as the promises arrive. It sounds like you might be using something like $q.all(), which would wait until all promises are resolved.
Buildilng on #reptilicus' answer:
$scope.getWeatherData = function () {
return myWeatherService.get().then(function (resp) {
$scope.weatherData = resp;
});
};
$scope.getTrafficData = function () {
return myTrafficService.get().then(function (resp) {
$scope.trafficData = resp;
});
};
$q.all([
$scope.getWeatherData(),
$scope.getTrafficData()
]).then(function () {
// do whatever's next, or nothing
});
... would request/receive both responses in parallel (if that's what you want). The relevant $scope property for each request will be populated when the related response is received, and the "do whatever's next" code will run once they are all complete.
Note, you need to inject $q to your controller constructor for this to work. :)
Edit: I just noticed that #reptilicus did mention $q.all. The difference between chaining the .thens and $q.all is that under chaining, one request wouldn't start until the previous was received....

Categories