We Keep Coding

Can I detect a hanging JS function and abort?

I am working on a tool that allows users to enter a regular expression for a find&replace, and then my tool will execute that find&replace and return the changed text. However, I recently ran into a situation where the find&replace simply froze, so I decided it would probably be best to somehow detect issues with regular expression matching, and abort after a certain amount of time has passed.
I've checked around, and what I was able to find using this answer was that the problem I'm experiencing was called 'catastrophic backtracking'. That's ideal to know, because that way I can make a minimal working example of where it goes wrong, however not ideal if the solution to change the regular expression isn't possible, because I have no control over the user's regex input (and there's no way I can write an advanced enough regex parser to limit the user's regex usage to exclude situations like this).
So in an attempt to solve this, I tried using promises, as suggested in this answer. I've made the 'catastrophic' match string in this example just long enough for the effect to hang my tab for a few seconds, without completely crashing the tab. Different computer specs may see different results though, I'm not sure.
Just one heads-up: Executing this code might freeze your current tab. PLEASE make sure you do not have a partial answer written when executing this code, as it might cause loss of work.
var PTest = function () {
return new Promise(function (resolve, reject) {
setTimeout(function() {
reject();
}, 100)
"xxxxxxxxxxxxxxxxxxxxxxxxx".match(/(x+x+)+y/)
resolve();
});
}
var myfunc = PTest();
myfunc.then(function () {
console.log("Promise Resolved");
}).catch(function () {
console.log("Promise Rejected");
});
On my computer, this causes the tab to freeze for about 4 seconds before showing "Promise Resolved" in the console.
My question now is: is it at all possible to "abort" the execution of a script like this, if execution takes too long (in the example: over 0.2 seconds)? I'd rather kill the regex find&replace than completely crash the tool, causing loss of work for the user.

I recommend using a Web Worker since it will run in its own sandbox: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API
The Web Worker is its own script that you need to include in your JavaScript, such as:
var worker = new Worker('/path/to/run-regex.js');
The following is untested code, but should get you going.
Your run-regex.js does the (potentially long running) regex match:
function regexMatch(str, regexStr, callback) {
let regex = new RegExp(regexStr);
let result = str.match(regex);
callback(result, '');
}
onmessage = function(e) {
let data = e.data;
switch (data.cmd) {
case 'match':
regexMatch(data.str, data.regex, function(result, err) {
postMessage({ cmd: data.cmd, result: result, err: err });
});
break;
case 'replace':
//regexMatch(data.str, data.regex, data.replace, function(result, err) {
// postMessage({ cmd: data.cmd, result: result, err: err });
//});
break;
default:
break;
postMessage({ err: 'Unknown command: ' + data.cmd });
}
}
In your own script, load the Web Worker, and add an event listener:
if(window.Worker) {
const myWorker = new Worker('/path/to/run-regex.js');
myWorker.onmessage = function(e) {
let data = e.data;
if(data.err) {
// handle error
} else {
// handle match result using data.result;
}
}
function regexMatch(str, regex) {
let data = { cmd: 'match', str: str, regex: regex.toString() };
myWorker.postMessage(data);
}
regexMatch('xxxxxxxxxxxxxxxxxxxxxxxxx', /(x+x+)+y/);
} else {
console.log('Your browser does not support web workers.');
}
With this, your main JavaScript thread is non blocking while the worker is working.
In case of a long running worker, you may add code to either:
ungracefully terminate the web worker using myWorker.terminate(), then restart it -- see Is it possible to terminate a running web worker?
or, try to close() from within the web worker scope -- see JavaScript Web Worker - close() vs terminate()

How to run an infinite blocking process in NodeJS?

I have a set of API endpoints in Express. One of them receives a request and starts a long running process that blocks other incoming Express requests.
My goal to make this process non-blocking. To understand better inner logic of Node Event Loop and how I can do it properly, I want to replace this long running function with my dummy long running blocking function that would start when I send a request to its endpoint.
I suppose, that different ways of making the dummy function blocking could cause Node manage these blockings differently.
So, my question is - how can I make a basic blocking process as a function that would run infinitely?

You can use node-webworker-threads.
var Worker, i$, x$, spin;
Worker = require('webworker-threads').Worker;
for (i$ = 0; i$ < 5; ++i$) {
x$ = new Worker(fn$);
x$.onmessage = fn1$;
x$.postMessage(Math.ceil(Math.random() * 30));
}
(spin = function(){
return setImmediate(spin);
})();
function fn$(){
var fibo;
fibo = function(n){
if (n > 1) {
return fibo(n - 1) + fibo(n - 2);
} else {
return 1;
}
};
return this.onmessage = function(arg$){
var data;
data = arg$.data;
return postMessage(fibo(data));
};
}
function fn1$(arg$){
var data;
data = arg$.data;
console.log("[" + this.thread.id + "] " + data);
return this.postMessage(Math.ceil(Math.random() * 30));
}
https://github.com/audreyt/node-webworker-threads

So, my question is - how can I make a basic blocking process as a function that would run infinitely?
function block() {
// not sure why you need that though
while(true);
}
I suppose, that different ways of making the dummy function blocking could cause Node manage these blockings differently.
Not really. I can't think of a "special way" to block the engine differently.
My goal to make this process non-blocking.
If it is really that long running you should really offload it to another thread.

There are short cut ways to do a quick fix if its like a one time thing, you can do it using a npm module that would do the job.
But the right way to do it is setting up a common design pattern called 'Work Queues'. You will need to set up a queuing mechanism, like rabbitMq, zeroMq, etc. How it works is, whenever you get a computation heavy task, instead of doing it in the same thread, you send it to the queue with relevant id values. Then a separate node process commonly called a 'worker' process will be listening for new actions on the queue and will process them as they arrive. This is a worker queue pattern and you can read up on it here:
https://www.rabbitmq.com/tutorials/tutorial-one-javascript.html
I would strongly advise you to learn this pattern as you would come across many tasks that would require this kind of mechanism. Also with this in place you can scale both your node servers and your workers independently.

I am not sure what exactly your 'long processing' is, but in general you can approach this kind of problem in two different ways.
Option 1:
Use the webworker-threads module as #serkan pointed out. The usual 'thread' limitations apply in this scenario. You will need to communicate with the Worker in messages.
This method should be preferable only when the logic is too complicated to be broken down into smaller independent problems (explained in option 2). Depending on complexity you should also consider if native code would better serve the purpose.
Option 2:
Break down the problem into smaller problems. Solve a part of the problem, schedule the next part to be executed later, and yield to let NodeJS process other events.
For example, consider the following example for calculating the factorial of a number.
Sync way:
function factorial(inputNum) {
let result = 1;
while(inputNum) {
result = result * inputNum;
inputNum--;
}
return result;
}
Async way:
function factorial(inputNum) {
return new Promise(resolve => {
let result = 1;
const calcFactOneLevel = () => {
result = result * inputNum;
inputNum--;
if(inputNum) {
return process.nextTick(calcFactOneLevel);
}
resolve(result);
}
calcFactOneLevel();
}
}
The code in second example will not block the node process. You can send the response when returned promise resolves.

switching from Jquery post requests to modern Promises

I'm working on a web application project with Flask+Python on the back-end, and Javascript on the front-end. I'd like to take advantage of some of the more modern (ES6/7) styles of things, such as Promises.
I've currently been writing all my javascript using Jquery 3+. Most of the time I'm making single Ajax requests to the server at a time. I've been specifically writing my Ajax requests using $.post and .done() and .fail(), which I know is already promise-based, or promise-like. Most of my code is in the style of
do function setup stuff and checks
make single ajax request
on success
good status, run several success code bits
bad status, run failure code
on failure - run failure code
I always seem to have to account for cases of server failures + cases of server success but it returned the wrong thing, which I usually control with a status argument. I've been looking into the straight Promise syntax with then, catch, resolve, reject, and I have some questions.
Is there any advantage to me switching to this format, from what I currently have, given my simple Ajax requests?
Can it be used to simplify the way I currently write my requests and handle my failure cases?
Here is a simple login example that I have, with a function that is called when a login button is clicked.
$('#loginsubmit').on('click', this, this.login);
// Login function
login() {
const form = $('#loginform').serialize();
$.post(Flask.url_for('index_page.login'), form, 'json')
.done((data)=>{
if (data.result.status < 0) {
// bad submit
this.resetLogin();
} else {
// good submit
if (data.result.message !== ''){
const stat = (data.result.status === 0) ? 'danger' : 'success';
const htmlstr = `<div class='alert alert-${stat}' role='alert'><h4>${data.result.message}</h4></div>`;
$('#loginmessage').html(htmlstr);
}
if (data.result.status === 1){
location.reload(true);
}
}
})
.fail((data)=>{ alert('Bad login attempt'); });
}
And a typical more complex example that I have. In this case, some interactive elements are initialized when a button is toggled on and off.
this.togglediv.on('change', this, this.initDynamic);
// Initialize the Dynamic Interaction upon toggle - makes loading an AJAX request
initDynamic(event) {
let _this = event.data;
if (!_this.togglediv.prop('checked')){
// Turning Off
_this.toggleOff();
} else {
// Turning On
_this.toggleOn();
// check for empty divs
let specempty = _this.graphdiv.is(':empty');
let imageempty = _this.imagediv.is(':empty');
let mapempty = _this.mapdiv.is(':empty');
// send the request if the dynamic divs are empty
if (imageempty) {
// make the form
let keys = ['plateifu', 'toggleon'];
let form = m.utils.buildForm(keys, _this.plateifu, _this.toggleon);
_this.toggleload.show();
$.post(Flask.url_for('galaxy_page.initdynamic'), form, 'json')
.done(function(data) {
let image = data.result.image;
let spaxel = data.result.spectra;
let spectitle = data.result.specmsg;
let maps = data.result.maps;
let mapmsg = data.result.mapmsg;
// Load the Image
_this.initOpenLayers(image);
_this.toggleload.hide();
// Try to load the spaxel
if (data.result.specstatus !== -1) {
_this.loadSpaxel(spaxel, spectitle);
} else {
_this.updateSpecMsg(`Error: ${spectitle}`, data.result.specstatus);
}
// Try to load the Maps
if (data.result.mapstatus !== -1) {
_this.initHeatmap(maps);
} else {
_this.updateMapMsg(`Error: ${mapmsg}`, data.result.mapstatus);
}
})
.fail(function(data) {
_this.updateSpecMsg(`Error: ${data.result.specmsg}`, data.result.specstatus);
_this.updateMapMsg(`Error: ${data.result.mapmsg}`, data.result.mapstatus);
_this.toggleload.hide();
});
}
}
}
I know this is already roughly using promises, but can I make improvements to my code flow by switching to the Promise then catch syntax? As you can see, I end up repeating a lot of the failure case code for real failures and successful failures. Most of my code looks like this, but I've been having a bit of trouble trying to convert these into something that's like
promise_ajax_call
.then(do real success)
.catch(all failure cases)

I always use Bluebird Promises. They have a Promise.resolve function that you can utilize with ajax. One thing to know about Promises, if you throw an error in a then, it will be caught in a chained catch. One way to clean this up a bit might be something like this (keep in mind, this is pseudo)
Promise.resolve($.ajax(...some properties..))
.then((data)=>{
if(data.result.status < 0){
//throw some error
}
// process the data how you need it
})
.catch((error){
// either the ajax failed, or you threw an error in your then. either way, it will end up in this catch
});

Control the async loop

I have two different Node.js programs.
One is an Express.js Server (PROGRAM1), which is to provide the user interface and RESTful APIs.
The other is a crawler (PROGRAM2), which keeps on reading item, download it from web and store everything into the database. By the way, I am using the Array.prototype.reduce() and Promise to iterate the files and handle I/Os one by one orderly.
One thing I would like to do here, is to monitor and control the progress of the crawler(PROGRAM2) from the PROGRAM1.
But I found it very complicate.
// Control the loop by this `flag`, the value can be assigned from outside
var flag = "IDLE";
// The outside can read this `index`, and monitor the progress
var current_index = -1;
var PAGE_SIZE = 100;
function handleBatch(index){
var defer = q.defer();
// Mongoose statement to find documents...
Book.find()
.skip(index*PAGE_SIZE).limit(PAGE_SIZE).then(function(books){
var finished = 0;
for(var i=0; i<books.length; i++){
var book = books[i];
downloadInfo(book).then(function(bookInfo){
if(flag === "STOP")
defer.reject(new Error("The loop should stop!"));
//store the info...
finished ++;
if(finished === PAGE_SIZE)
defer.resolve();
});
}
});
return defer.promise;
}
var promiseHandler;
function main(){
while(true){
if(flag === "IDLE")
continue;
else if(flag === "START"){
var [0,1,2,3,4,5...,2500].reduce(function(lastPromise, nextIndex){
promiseHandler = lastPromise.then(function(){
currentIndex = nextIndex;
});
}, q());
}else if(flag === "STOP"){
promiseHandler.then(null, function(err){
flag = "IDLE";
});
}
}
}
main() is just an example(e.g. Actually it is a server, and the state can be changed by the requests from PROGRAM1). By setting the flag as STOP, the loop in handleBatch() will discover the change and throw an Exception, then the program will be paused.
However, I just don't like this way, because it looks too ugly and control the process by throwing Errors. So I am searching for a better way to control and monitor the loop. Any one any idea?

You should look in the documentation for node.js' process.
And to answer your question about stopping the execution, here ->
process.exit(0);
Just as a tip: don't control your program using a loop. It's bad.

By the sound of it you are after a way to implement inter-process communication in node js. This is a broad programming topic that goes way beyond node js. There's many patterns and means to achieve this, but one of my favourites is to use a message queue to loosely couple the two processes.
On node js we have things like Redis and node-redis which can be used to implement a publish-subscribe pattern. Of course there's many messaging libraries that would work too.
In your case the Express API might publish a "pause" event, and the crawler could subscribe to that event and take some action. Your node apps then remain asynchronous (no while(true) nonsense!).

Recursion - Node out of memory

When I run the following code 9999999+ times, Node returns with:
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
Aborted (core dumped)
Whats the best solution to get around this issue, other than increasing the max alloc size or any command line arguments?
I'd like to improve the code quality rather than hack a solution.
The following is the main bulk of recursion within the application.
The application is a load testing tool.
a.prototype.createClients = function(){
for(var i = 0; i < 999999999; i++){
this.recursiveRequest();
}
}
a.prototype.recursiveRequest = function(){
var self = this;
self.hrtime = process.hrtime();
if(!this.halt){
self.reqMade++;
this.http.get(this.options, function(resp){
resp.on('data', function(){})
.on("connection", function(){
})
.on("end", function(){
self.onSuccess();
});
})
.on("error", function(e){
self.onError();
});
}
}
a.prototype.onSuccess = function(){
var elapsed = process.hrtime(this.hrtime),
ms = elapsed[0] * 1000000 + elapsed[1] / 1000
this.times.push(ms);
this.successful++;
this.recursiveRequest();
}

Looks like you should really be using a queue instead of recursive calls. async.queue offers a fantastic mechanism for processing asynchronous queues. You should also consider using the request module to make your http client connections simpler.
var async = require('async');
var request = require('request');
var load_test_url = 'http://www.testdomain.com/';
var parallel_requests = 1000;
function requestOne(task, callback) {
request.get(task.url, function(err, connection, body) {
if(err) return callback(err);
q.push({url:load_test_url});
callback();
});
}
var q = async.queue(requestOne, parallel_requests);
for(var i = 0; i < parallel_requests; i++){
q.push({url:load_test_url});
}
You can set the parallel_requests variable according to how many simultaneous requests you want to hit the test server with.

You are launching 1 billion "clients" in parallel, and having each of them perform an http get request recursively in an endless recursion.
Few remarks:
while your question mentions 10 million clients, your code creates 1 billion clients.
You should replace the for loop by a recursive function, to get rid of the out-of-memory error.
Something in these lines:
a.prototype.createClients = function(i){
if (i < 999999999) {
this.recursiveRequest();
this.createClients(i+1);
}
}
Then, you probably want to include some delay between the clients creations, or between the calls to recursiveRequest. Use setTimeout.
You should have a way to get the recursions stopping (onSuccess and recursiveRequest keep calling each other)
A flow control library like async node.js module may help.

10 million is very large... Assuming that the stack supports any number of calls, it should work, but you are likely asking the JavaScript interpreter to load 10 million x quite a bit of memory... and the result is Out of Memory.
Also I personally do no see why you'd want to have so many requests at the same time (testing a heavy load on a server?) one way to optimize is to NOT create "floating functions" which you are doing a lot. "Floating functions" use their own set of memory on each instantiation.
this.http.get(this.options, function(resp){ ... });
^^^^
++++--- allocates memory x 10 million
Here the function(resp)... declaration allocates more memory on each call. What you want to do is:
# either global scope:
function r(resp) {...}
this.http.get(this.options, r ...);
# or as a static member:
a.r = function(resp) {...};
this.http.get(this.options, a.r ...);
At least you'll save on all that function memory. That goes for all the functions you declare within the r function, of course. Especially if their are quite large.
If you want to use the this pointer (make r a prototype function) then you can do that:
a.prototype.r = function(resp) {...};
// note that we have to have a small function to use 'that'... probably not a good idea
var that = this;
this.http.get(this.options, function(){that.r();});
To avoid the that reference, you may use an instance saved in a global. That defeats the use of an object as such though:
a.instance = new a;
// r() is static, but can access the object as follow:
a.r = function(resp) { a.instance.<func>(); }
Using the instance you can access the object's functions from the static r function. That could be the actual implementation which could make full use of the this reference:
a.r = function(resp) { a.instance.r_impl(); }

According to a comment by Daniel, your problem is that you misuse a for() to count the total number of requests you want to send. This means you can apply a very simple fix to your code as follow:
a.prototype.createClients = function(){
this.recursiveRequest();
};
a.prototype.recursiveRequest = function(){
var self = this;
self.hrtime = process.hrtime();
if(!this.halt && this.successful < 10000000){
...
Your recursivity is enough to run the test any number of times.
What you do is never quit, though. You have a halt variable, but it does not look like you ever set that to true. However, to test 10 million times, you want to check the number of requests you already sent.
My "fix" supposes that onError() fails (is no recursive). You could also change the code to make use of the halt flag as in:
a.prototype.onSuccess = function(){
var elapsed = process.hrtime(this.hrtime),
ms = elapsed[0] * 1000000 + elapsed[1] / 1000
this.times.push(ms);
this.successful++;
if(this.successful >= 10000000)
{
this.halt = true;
}
this.recursiveRequest();
}
Note here that you will be pushing ms in the times buffer 10 million times. That's a big table! You may want to have a total instead and compute an average at the end:
this.time += ms;
// at the end:
this.average_time = this.time / this.successful;