We Keep Coding

vue.js: Where to put regularly repeating background query

I would like to query an API service every 15 seconds, so I can get data from a database and check whether something was changed. If there was a change, then my front end would update automatically because of how vue works.
while (true) {
setTimeout(function() {
QueryService.orders().then(response =>
this.orders = response.data
)
}, 15000)
}
My questions are:
Is this a good approach to solve such a problem at all?
What would be the best position in the code to place such a loop?
EDIT:
Using setInterval() seems to be the right way, but using a polling function with setInterval in the created() hook doesn't affect the data-table at all. It shows me "No data available":
data () {
return {
headers [
{ ... },
{ ... }
],
orders: []
}
created () {
setInterval(function() {
QueryService.orders().then(response => this.orders = response.data)
}, 15000)
}
Using the polling function without setInterval works and fills my data-table with data as usual:
created () {
QueryService.orders().then(response => this.orders = response.data)
}

For a simple and quick solution, I'd go with I'mOnlyVueman's answer. Here some example code I found from Vue.js polling using setINterval(). This example includes
pollData method initiated on created that dispatches a store action (which would call the API)
Canceling the poll as you navigate to another page using beforeDestroy
Code
data () {
return {
polling: null
}
},
methods: {
pollData () {
this.polling = setInterval(() => {
this.$store.dispatch('RETRIEVE_DATA_FROM_BACKEND')
}, 3000)
}
},
beforeDestroy () {
clearInterval(this.polling)
},
created () {
this.pollData()
}
But polling an API isn't very elegant and it doesn't scale well. You'll likely need to do something with Websockets, setting up your app to listen for events pushed from your API.
Here's info on Subscriptions in Vue-Apollo & GraphQL that Denis Tsoi mentioned.
Subscriptions are a GraphQL feature that allows the server to send
data to the clients when a specific event happens on the backend.
Subscriptions are usually implemented with WebSockets, where the
server holds a steady connection to the client. That is, the
Request-Response-Cycle that we used for all previous interactions with
the API is not used for subscriptions. Instead, the client initially
opens up a steady connection to the server by specifying which event
it is interested in. Every time this particular event happens, the
server uses the connection to push the data that’s related to the
event to the client.

A loop like this would go in the component's script within a mounted () lifecycle hook.
That would mean once the component loads your loop would trigger. For detailed guidance on this technique the Vue docs are a good first stop, as well as this article.

getJSON done callback

I have the function bellow called every 5 seconds to get data from the server, which is flask/python. My question is how can I adapt the getjson call to have callback when the data is successfully retrieved.
I know there's .done .fail and so on, but I was wondering if I can keep this structure and just add bellow it, but I don't know the syntax in this particular case, hope this isn't too confusing, thanks for reading, here's the code.
// get data from the server every getDataFromServerInterval milliseconds
var getDataFromServerInterval = 5000;
function getData(){
// request timesince table entries from server for user...
$.getJSON($SCRIPT_ROOT + '/_database', {
action: "getUserTable_timesince",
username: $('input[name="username"]').val()
}, function(data) { // do something with the response data
timesince_dataBuffer = data;
});
return false; // prevent get
}
// get data from the server every getDataFromServerInterval milliseconds
setInterval(getData, getDataFromServerInterval);

You could do something like this. Instead of processing the data in getData or using a callback, take advantage of the promise that $.getJSON returns. Have a separate function that is called by the timeout which calls for the data, then processes it. It neatly separates your code out into more managable functions.
var getDataFromServerInterval = 5000;
function getData() {
return $.getJSON($SCRIPT_ROOT + '/_database', {
action: "getUserTable_timesince",
username: $('input[name="username"]').val()
}
}
function wrangleData() {
getData().then(function (data) {
console.log(data);
});
}
setInterval(wrangleData, getDataFromServerInterval);

I found a partial solution, I realized that I can add a callback at the end of the function that handles the data received, which is somewhat equivalent to .done in a different getjson call structure, I'm not sure yet if the function gets called before or after the data is received.
// global timesince buffer, holds
var timesince_dataBuffer;
// get data from the server every getDataFromServerInterval milliseconds
var getDataFromServerInterval = 5000;
function getData(){
// request timesince table entries from server for user
$.getJSON($SCRIPT_ROOT + '/_database', {
action: "getUserTable_timesince",
username: $('input[name="username"]').val()
}, function(data) { // do something with the response data
timesince_dataBuffer = data;
updateEntryStruct(); // the hope is to call this when data is received
});
return false; // prevent get
}
// get data from the server every getDataFromServerInterval milliseconds
setInterval(getData, getDataFromServerInterval);

This is the solution I came up with.
var timesince_dataBuffer;
function getData(){
// gets user's entries from sql table
$.getJSON($SCRIPT_ROOT + '/_database', { // $SCRIPT_ROOT, root to the application
action: "getUserTable_timesince",
username: $('input[name="username"]').val()
}, function(data) { // if a response is sent, this function is called
timesince_dataBuffer = data;
updateEntryStruct(); // recreate the structure of each content, buttons etc
});
return false;
}
I get the data, put in a global variable, call another function which takes that data and re-creates a structure for each object received, this way I don't recreate parts of the structure which are static, most importantly the buttons.
Another function is called every 1 second, which updates the dynamic parts.
(formatted time) passed since
(event name)
Anyway, this is actually my final project in CS50, I started by communicating with the server via form submissions, refreshing the page each time the user pressed a button, then I did it by ajax, but I was sending requests to the server every 2 seconds, and having unresponsive buttons because I would keep re-creating the buttons themselves on a time interval.
And now the page feels responsive and efficient, it's been a great learning experience.
If anyone wants to check out the code, everything is here.
https://github.com/silvermirai/cs50-final-project
It's basically a bunch of random functionality that came to mind.
The application can be found here as of now.
http://ide502-silvermirai.cs50.io:8080/

Nodejs manage different threads

I'm a little bit newbie with Nodejs.
I'm working in a Nodejs - express solution.
I want to send and e-mail when some information is added to a MSSSQL database.
This is working well for me. The problem is that I want to check every five minutes that this information added to the database is modified or not, and if not, send another e-mail.
The call to add information to the db is this route:
router.post('/postlinevalidation', function(req, res) {
//insert function into mssql
silkcartCtrl.sendMail(req, res);
});
The controller part for sending the e-mail:
exports.sendMail = function(req, res) {
var emails = "";
fs.readFile('./config/email.conf', 'utf8', function (err,data) {
if (err) {
return logger.error(err);
}
emails = data;
});
var minutes = 5, the_interval = minutes * 60 * 1000;
var refreshId = setInterval(function() {
logger.info("I am doing my 5 minutes check FL_PENDIENTE");
var request = new sql.Request(req.dbsqlserver);
var sqlpendinglinesvalidation = "SELECT [FK_IDCHECK],[FK_IDPEDIDO],[BK_IDPROVEEDOR],[DE_PROVEEDOR]"+
",[FK_FAMILIA],[BK_FAMILIA],[FK_SUBFAMILIA],[BK_SUBFAMILIA],[FK_ARTICULO]"+
",[BK_ARTICULO],[FL_VALIDAR],[DT_FECHA],[FL_PENDIENTE],[DES_CHECK],[QNT_PROPUESTA],[FECHA]"+
"FROM TABLE"+
" WHERE [FL_PENDIENTE] = 1";
request.query(sqlpendinglinesvalidation, function (err, recordset) {
if (recordset.length > 0) {
var transporter = nodemailer.createTransport('smtps://user%40gmail.com:pwd#smtp.gmail.com');
var mailOptions = {
from: '"Mailer" <mail#mail.com>', // sender address
to: emails, // list of receivers
subject: 'Tienes compras pendientes de validar', // Subject line
text: 'Tienes compras pendientes de validar', // plaintext body
html: '<b>Tienes compras pendientes de validar.</b>' // html body
};
// send mail with defined transport object
transporter.sendMail(mailOptions, function(error, info){
if(error){
return logger.error(error);
}
logger.info('Message sent: ' + info.response);
});
} else {
clearInterval(refreshId);
return true;
}
});
}, the_interval);
};
As I said this is working well.
I control the five minutes withsetInterval
But I supossed every time the route postlinevalidation is called, a new thread is open, so I will have several setInterval processes running.
I want to know how to manage it. If the controller function exports.sendMail is running, when the route is called again, "kill this process", and start again exports.sendMail
Thanks in advance

But I supossed every time the route postlinevalidation is called, a
new thread is open, so I will have several setInterval processes
running.
No, this is not how node.js works. You don't get multiple threads because of multiple setInterval() timers.
node.js by itself is single threaded. So, each time a route is called, it just creates an event in the node.js event queue and they are served FIFO, one at a time. At any point that one of the route handlers makes an async call, it essentially "yields" control back and the next item in the event queue gets to run until it yields or finishes.
Timers like setInterval() also use the event queue so no additional threads are creates by setInterval(). It is possible that node.js modules that use native code may themselves use threads and node.js uses a small thread pool that it uses for disk managemnet, but neither of those have anything to do with setInterval().
If you explicitly want to create another execution context for a long running operation in node.js to separate it from the single node.js thread, then that is usually done with the child process module that is part of node.js. You create a new process (which can be a node.js process or some other program running in the process) and you can then communicate with that other process.
If the controller function exports.sendMail is running, when the route
is called again, "kill this process", and start again
exports.sendMail
This is something that would need to be an explicit feature of the nodemailer module in order for you to cancel an operation in process. How "in process" asynchronous operations are implemented and controlled is not a generic node.js thing, but is specific to how that specific module implements things and keeps track of things.
Looking into the code for the node-mailer and more specifically the smtp-connection module, it looks like it uses plain async node.js socket code. That means it does not create any new threads or processes on its own.
As for your setInterval() calls, you need to make sure that any body of code that creates a setInterval() keeps track of the interval timer ID and eventually clears the interval so it stops and you don't keep piling up more and more interval timers. Another possibility is that you have only one interval and it does checking for all outstanding operations (rather than have a separate interval for each one).

From a quick look, I think you don't really need to put the sendMail function inside postlinevalidation. If you want to control it, you could run it in a different script from the express app. You can use something like pm2 or parallelshell to run multiple scripts at the same time.

If you are using setInterval then you can use clearInterval to stop the setInterval based on your condition. Whenever you call a setInterval function, it returns an id using which you can stop the setInterval.
var interval = setInterval(doStuff, 5000);
function doStuff() {
if(your_condition) {
clearInterval(interval);
}
}

Handling interdependent and/or layered asynchronous calls

As an example, suppose I want to fetch a list of files from somewhere, then load the contents of these files and finally display them to the user. In a synchronous model, it would be something like this (pseudocode):
var file_list = fetchFiles(source);
if (!file_list) {
display('failed to fetch list');
} else {
for (file in file_list) { // iteration, not enumeration
var data = loadFile(file);
if (!data) {
display('failed to load: ' + file);
} else {
display(data);
}
}
}
This provides decent feedback to the user and I can move pieces of code into functions if I so deem necessary. Life is simple.
Now, to crush my dreams: fetchFiles() and loadFile() are actually asynchronous. The easy way out is to transform them into synchronous functions. But this is not good if the browser locks up waiting for calls to complete.
How can I handle multiple interdependent and/or layered asynchronous calls without delving deeper and deeper into an endless chain of callbacks, in classic reductio ad spaghettum fashion? Is there a proven paradigm to cleanly handle these while keeping code loosely coupled?

Deferreds are really the way to go here. They capture exactly what you (and a whole lot of async code) want: "go away and do this potentially expensive thing, don't bother me in the meantime, and then do this when you get back."
And you don't need jQuery to use them. An enterprising individual has ported Deferred to underscore, and claims you don't even need underscore to use it.
So your code can look like this:
function fetchFiles(source) {
var dfd = _.Deferred();
// do some kind of thing that takes a long time
doExpensiveThingOne({
source: source,
complete: function(files) {
// this informs the Deferred that it succeeded, and passes
// `files` to all its success ("done") handlers
dfd.resolve(files);
// if you know how to capture an error condition, you can also
// indicate that with dfd.reject(...)
}
});
return dfd;
}
function loadFile(file) {
// same thing!
var dfd = _.Deferred();
doExpensiveThingTwo({
file: file,
complete: function(data) {
dfd.resolve(data);
}
});
return dfd;
}
// and now glue it together
_.when(fetchFiles(source))
.done(function(files) {
for (var file in files) {
_.when(loadFile(file))
.done(function(data) {
display(data);
})
.fail(function() {
display('failed to load: ' + file);
});
}
})
.fail(function() {
display('failed to fetch list');
});
The setup is a little wordier, but once you've written the code to handle the Deferred's state and stuffed it off in a function somewhere you won't have to worry about it again, you can play around with the actual flow of events very easily. For example:
var file_dfds = [];
for (var file in files) {
file_dfds.push(loadFile(file));
}
_.when(file_dfds)
.done(function(datas) {
// this will only run if and when ALL the files have successfully
// loaded!
});

Events
Maybe using events is a good idea. It keeps you from creating code-trees and de-couples your code.
I've used bean as the framework for events.
Example pseudo code:
// async request for files
function fetchFiles(source) {
IO.get(..., function (data, status) {
if(data) {
bean.fire(window, 'fetched_files', data);
} else {
bean.fire(window, 'fetched_files_fail', data, status);
}
});
}
// handler for when we get data
function onFetchedFiles (event, files) {
for (file in files) {
var data = loadFile(file);
if (!data) {
display('failed to load: ' + file);
} else {
display(data);
}
}
}
// handler for failures
function onFetchedFilesFail (event, status) {
display('Failed to fetch list. Reason: ' + status);
}
// subscribe the window to these events
bean.on(window, 'fetched_files', onFetchedFiles);
bean.on(window, 'fetched_files_fail', onFetchedFilesFail);
fetchFiles();
Custom events and this kind of event handling is implemented in virtually all popular JS frameworks.

Sounds like you need jQuery Deferred. Here is some untested code that might help point you in the right direction:
$.when(fetchFiles(source)).then(function(file_list) {
if (!file_list) {
display('failed to fetch list');
} else {
for (file in file_list) {
$.when(loadFile(file)).then(function(data){
if (!data) {
display('failed to load: ' + file);
} else {
display(data);
}
});
}
}
});
I also found another decent post which gives a few uses cases for the Deferred object

If you do not want to use jQuery, what you could use instead are web workers in combination with synchronous requests. Web workers are supported across every major browser with the exception of any Internet Explorer version before 10.
Web Worker browser compatability
Basically, if you're not entirely certain what a web worker is, think of it as a way for browsers to execute specialized JavaScript on a separate thread without impacting the main thread (Caveat: On a single-core CPU, both threads will run in an alternating fashion. Luckily, most computers nowadays come equipped with dual-core CPUs). Usually, web workers are reserved for complex computations or some intense processing task. Just keep in mind that any code within the web worker CANNOT reference the DOM nor can it reference any global data structures that have not been passed to it. Essentially, web workers run independent of the main thread. Any code that the worker executes should be kept separate from the rest of your JavaScript code base, within its own JS file. Furthermore, if the web workers need specific data in order to properly work, you need to pass that data into them upon starting them up.
Yet another important thing worth noting is that any JS libraries that you need to use to load the files will need to be copied directly into the JavaScript file that the worker will execute. That means these libraries should first be minified(if they haven't been already), then copied and pasted into the top of the file.
Anyway, I decided to write up a basic template to show you how to approach this. Check it out below. Feel free to ask questions/criticize/etc.
On the JS file that you want to keep executing on the main thread, you want something like the following code below in order to invoke the worker.
function startWorker(dataObj)
{
var message = {},
worker;
try
{
worker = new Worker('workers/getFileData.js');
}
catch(error)
{
// Throw error
}
message.data = dataObj;
// all data is communicated to the worker in JSON format
message = JSON.stringify(message);
// This is the function that will handle all data returned by the worker
worker.onMessage = function(e)
{
display(JSON.parse(e.data));
}
worker.postMessage(message);
}
Then, in a separate file meant for the worker (as you can see in the code above, I named my file getFileData.js), write something like the following...
function fetchFiles(source)
{
// Put your code here
// Keep in mind that any requests made should be synchronous as this should not
// impact the main thread
}
function loadFile(file)
{
// Put your code here
// Keep in mind that any requests made should be synchronous as this should not
// impact the main thread
}
onmessage = function(e)
{
var response = [],
data = JSON.parse(e.data),
file_list = fetchFiles(data.source),
file, fileData;
if (!file_list)
{
response.push('failed to fetch list');
}
else
{
for (file in file_list)
{ // iteration, not enumeration
fileData = loadFile(file);
if (!fileData)
{
response.push('failed to load: ' + file);
}
else
{
response.push(fileData);
}
}
}
response = JSON.stringify(response);
postMessage(response);
close();
}
PS: Also, I dug up another thread which would better help you understand the pros and cons of using synchronous requests in combination with web workers.
Stack Overflow - Web Workers and Synchronous Requests

async is a popular asynchronous flow control library often used with node.js. I've never personally used it in the browser, but apparently it works there as well.
This example would (theoretically) run your two functions, returning an object of all the filenames and their load status. async.map runs in parallel, while waterfall is a series, passing the results of each step on to the next.
I am assuming here that your two async functions accept callbacks. If they do not, I'd require more info as to how they're intended to be used (do they fire off events on completion? etc).
async.waterfall([
function (done) {
fetchFiles(source, function(list) {
if (!list) done('failed to fetch file list');
else done(null, list);
});
// alternatively you could simply fetchFiles(source, done) here, and handle
// the null result in the next function.
},
function (file_list, done) {
var loadHandler = function (memo, file, cb) {
loadFile(file, function(data) {
if (!data) {
display('failed to load: ' + file);
} else {
display(data);
}
// if any of the callbacks to `map` returned an error, it would halt
// execution and pass that error to the final callback. So we don't pass
// an error here, but rather a tuple of the file and load result.
cb(null, [file, !!data]);
});
};
async.map(file_list, loadHandler, done);
}
], function(err, result) {
if (err) return display(err);
// All files loaded! (or failed to load)
// result would be an array of tuples like [[file, bool file loaded?], ...]
});
waterfall accepts an array of functions and executes them in order, passing the result of each along as the arguments to the next, along with a callback function as the last argument, which you call with either an error, or the resulting data from the function.
You could of course add any number of different async callbacks between or around those two, without having to change the structure of the code at all. waterfall is actually only 1 of 10 different flow control structures, so you have a lot of options (although I almost invariably end up using auto, which allows you to mix parallel and series execution in the same function via a Makefile like requirements syntax).

I had this issue with a webapp I'm working on and here's how I solved it (with no libraries).
Step 1: Wrote a very lightweight pubsub implementation. Nothing fancy. Subscribe, Unsubscribe, Publish and Log. Everything (with comments) adds up 93 lines of Javascript. 2.7kb before gzip.
Step 2: Decoupled the process you were trying to accomplish by letting the pubsub implementation do the heavy lifting. Here's an example:
// listen for when files have been fetched and set up what to do when it comes in
pubsub.notification.subscribe(
"processFetchedResults", // notification to subscribe to
"fetchedFilesProcesser", // subscriber
/* what to do when files have been fetched */
function(params) {
var file_list = params.notificationParams.file_list;
for (file in file_list) { // iteration, not enumeration
var data = loadFile(file);
if (!data) {
display('failed to load: ' + file);
} else {
display(data);
}
}
);
// trigger fetch files
function fetchFiles(source) {
// ajax call to source
// on response code 200 publish "processFetchedResults"
// set publish parameters as ajax call response
pubsub.notification.publish("processFetchedResults", ajaxResponse, "fetchFilesFunction");
}
Of course this is very verbose in the setup and scarce on the magic behind the scenes.
Here's some technical details:
I'm using setTimeout to handle triggering subscriptions. This way they run in a non-blocking fashion.
The call is effectively decoupled from the processing. You can write a different subscription to the notification "processFetchedResults" and do multiple things once the response comes through (for example logging and processing) while keeping them in very separate, tiny and easily-managed code blocks.
The above code sample doesn't address fallbacks or run proper checks. I'm sure it will require a bit of tooling to get to production standards. Just wanted to show you how possible it is and how library-independent your solution can be.
Cheers!

NowJS server event notifications

I'm trying to implement a system where an external server (SuperFeedr) sends a request to my server (running Node) and my server processes, then sends that data straight to the client in realtime using NowJS.
Problem is, I cannot access the everyonce namespace in my server functions since it has to be initialized after the listen() function is called which has to happen after the functions are declared. So basically:
Needs:
NowJS->Listen->Server functions->everyone variable->NowJS
Seems I have a dependency loop and I have no idea how to resolve it.

Start all of them independently. When one of them is up, put a reference to it into a shared parent scope. When e.g. the server receives a notification, just drop it if nowjs isn't ready yet. Simplified example:
var a, b;
initializeA(function(a_) {
a = a_
a.on('request', function(request, response) {
if (!b) {
// B isn't ready yet, drop the request
return response.end()
}
// ...
})
})
initializeB(function(b_) {
b = b_
b.on('request', function(request, response) {
if (!a) {
// A isn't ready yet, drop the request
return response.end()
}
// ...
})
})