We Keep Coding

Angular/Javascript - Custom User Scripts (Eval)

Hi we are building a web app platform where users can make their own smart forms using drag and drop features. We are looking for a way for admin users to create their own custom scripts to run some logic using pre-defined functions for the app. Currently the solution we have come up is using eval().
Knowing Eval is 'evil' we have implemented a function to check if the script is safe before it is executed. Essentially it breaks up the code into tokens and then runs those tokens against a blacklist. Stuff like new, eval, window, require, fetch,browser will show an error. Also the function is executed via a angular service so we try to limit what is injected.
Below is the basic high-level code. We have custom async functions so the solution needs to handle this.
My question is there a better(ie faster) and safer way to run custom scripts?
async runScript(script,callback) {
var updateForm=(id,value)=>{
return this.updateForm(id,value);
}
var getForm=(id)=>{
return this.getForm(id);
}
if (this.checkScriptSafe(script)) {
try {
return eval("(async () => {" + script + "})()");
} catch (e) {
if (e instanceof SyntaxError) {
alert(e.message);
} else {
console.log('Error',e);
alert("Error in script");
}
}
} else {
alert("Script not safe")
}
}
Example script:
"var value = 1 +4; await updateForm("11",value);alert("Success!");"

Function constructor would be a better approach. Function constructor creates a new function that will execute in the global scope. Your eval script (because of the arrow function) will run in the same context as your runScript method. They would access/modify your internals, or override your class methods. They can even override the runScript method itself and remove the checkScriptSafe check.
Using the function constructor is similar to typing in the dev tools console. If your application is not vulnerable to the dev tools console, then you wouldn't have any issues using the function constructor.
Here is an example:
const script = `
var value = 1 +4;\n
await updateForm("11",value);\n
alert("Success!");
`;
// we have to get a handle of the async function constructor
// in order to create an async function
const dummyFunction = async function() {}
const AsyncFunction = dummyFunction.constructor;
// create an async function which will run in the global scope
// the function will have an `updateForm` parameter
const userFunction = new AsyncFunction('updateForm', script);
// now userFunction is equavalent of:
// const userFunction = async function(updateForm) {
// var value = 1 +4;
// await updateForm("11",value);
// alert("Success!");
// }
// bind the current context 'this' to update form and pass it
// to user's function as parameter. The user's function
// will be able to execute it.
userFunction(this.updateForm.bind(this));
I'm not an expert in browser internals. But I assume tokenizing and interpreting the function on your own would be much slower than the Function constructor method. Even if you do everything in the most efficient way, you would still be in the JavaScript domain; v8 (or any other JS engine) would perform the actual interpretation after you. Why not directly give the script to the JS engine then? If the same custom script is going to run frequently, then with the right design, v8 will optimize the custom functions by compiling them into machine code. This wouldn't be the case with eval.

Require multiple times

File my_script.js:
(function() {
console.log("IMPORTED");
})();
Calling this file (run_me.js) should cause IMPORTED to print twice:
require("./my_script");
require("./my_script");
However it only prints once.
How can I change run_me.js so that IMPORTED is printed to the console twice.
Assume for this question, no changes can be made to my_script.js

require() caches its results. So, the first time a module is required, then its initialization code runs. After that, the cache just returns the value of module.exports without running the initialization code again. This is a very desirable feature of node.js modules.
If you want code to be run each time, then you should export a function that you can call after you require it like this:
Your module:
module.exports = function() {
console.log("IMPORTED");
}
Requiring it and running the code each time
require("./my_script")();
require("./my_script")();
Also, please note that there is no reason to use an IIFE in a module. The node.js module is automatically wrapped in a private function already so you don't need to do it again.
As you now say in a comment (but your question does not directly say), if you don't want to edit my_script at all (which is simply the wrong way to solve this issue), then you have to delete the module from the node.js cache before requiring it again which can be done like this:
delete require.cache[require.resolve('./my_script')];
I would not recommend this as a solution. It's not the proper way to code in node.js. It's a hack work-around. And, it is not compatible with ESM modules.

If you use jest and want code to be run each time for testing, you can use jest.isolateModules:
jest.isolateModules(() => {
require("./my_script");
});
jest.isolateModules(() => {
require("./my_script");
});

I don't think it is possible without modifying the myscript.js file. Especially since as you show it, it doesn't export anything.
It will execute the first time you require it (which is why you see "Imported" once), but then nothing will happen on future calls to require because the "cached" value (ie. module.exports) which is returned is empty.
See below for an example of what I think you want (except that myscript.js has been modified). The biggest difference is that in your original myscript.js file the function was actually executed, while in the example below it is just defined, and then actually executed in the require call in the run_me.js file.
File myscript.js:
module.exports = () => console.log("Imported");
File run_me.js:
require('myscript.js')(); // note the () at the end which actually calls the function
require('myscript.js')(); // note the () at the end which actually calls the function

You can use this package, it is an npm module that will clear the cache and load a module from source fresh each time.
https://www.npmjs.com/package/require-uncached
const requireUncached = require('require-uncached');
require('./foo')();
//=> 1
require('./foo')();
//=> 2
requireUncached('./foo')();
//=> 1
requireUncached('./foo')();
//=> 1

How to properly stub a function return value?

Premise: JS ES6, NodeJS
Testing Framework: TAP
Mocking Library: testdouble.js
I am attempting to mock the return value for the method of my class and keep receiving this error:
not ok Unsatisfied verification on test double. Wanted: - called with (true). But there were no invocations of the test double.
Here is my testing code:
// Imports for unit testing
const tap = require('tap');
const Subject = require('../src/iTunesClient.js');
const td = require('testdouble');
let reqJson;
// Ensure the iTunes class methods are called
tap.test('iTunesClient class methods function as intended', (t) => {
t.beforeEach((ready) => {
reqJson = td.replace('../src/reqJson.js');
ready();
});
t.afterEach((ready) => {
td.reset();
ready();
});
t.test('iTunesClient.getData', (assert) => {
const callback = td.function();
const subject = new Subject();
subject.setTerm('abc 123');
subject.setURL();
td.when(reqJson.get(td.callback)).thenCallback(true);
subject.getData(callback);
td.verify(callback(true));
assert.end();
});
t.end();
});
Specifically, this line is related to my issue:
td.verify(callback(true));
How can I fake the callback value of true for reqJson.get()? Right now, Subject.geData() is a method of the iTunesClient class which calls another file, reqJson.js, to use its exported get() method.

It's a little hard to tell from your example, but it looks like you're requiring iTunesClient before you call td.replace. In this case, the real reqJson module will be required and cached on line 3.
You need to call td.replace early enough to avoid this, e.g. in between requiring tap and iTunesClient.

I wanted to update this question, as I recently solved this issue. Essentially, I had two issues:
Account for both reqJson function parameters
Account for all callback return values
Per testdouble documentation for item 1:
When passed td.matchers.anything(), any invocation of that test double function will ignore that parameter when determining whether an invocation satisfies the stubbing.
Hence, I adjusted my line of code as follows:
Before: td.when(reqJson.get(td.callback)).thenCallback(true);
After: td.when(reqJson.get(td.matchers.anything(), td.callback)).thenCallback(null, null, null);

Very unusual scope behaviour when I call modernizr.build in Node JS

I'm trying to build several jsons using Modernizr at once, but it appears to break the scope of my function.
It's very hard to explain so have a look at this example, give it a go if you don't believe me:
[1,2,3,4,5].forEach(function(i){
require("modernizr").build({}, function (result) {
console.log(i);
});
})
outputs:
5
5
5
5
5
Instead of the expected 1, 2, 3, 4, 5, as would any similar function.
I have not come across this behaviour before in all my years of coding in ECMAScript like languages, and have built my project (and previous projects) around the idea that you cannot break a function's scope like that.
It breaks any system based on promises or even just simple callbacks.
It's baffled me all day, and I can't find an appropriate fix for it.
I'm have a very hard time even conceptualizing what it is that's causing this to happen.
Please help.
EDIT:
OK, it appears you're all hung up on the forEach...
Here's another example that will make it a little clearer:
function asd(i){
require("modernizr").build({}, function (result) {
console.log(i);
});
}
asd(1);
asd(2);
asd(3);
asd(4);
outputs
4
4
4
4
What on earth is happening?

The issue specific to Modernizr had to to with a global variable being clobbered.
the build command is basically a large requirejs configuration function, all powered by a large config object. There is some basic things that are true, always, that are established at the top of the function
{
optimize: 'none',
generateSourceMaps: false,
optimizeCss: 'none',
useStrict: true,
include: ['modernizr-init'],
fileExclusionRegExp: /^(.git|node_modules|modulizr|media|test)$/,
wrap: {
start: '\n;(function(window, document, undefined){',
end: '})(window, document);'
}
}
Then, since Modernizr works in both the browser and in node without changes, there needs to be a way for it to know if it should be loading its dependencies via the filesystem or via http. So we add some more options like basePath inside of a environment check
if (inBrowser) {
baseRequireConfig.baseUrl = '/i/js/modernizr-git/src';
} else {
baseRequireConfig.baseUrl = __dirname + '/../src';
}
At this point, the config object gets passed into requirejs.config, which wires up require and allows us to start calling build.
Finally, after all of that has been created, we have a build function that also ends up modifying the config object yet again for build specific settings (the actual detects in your build, regex to strip out some AMD crud, etc).
So here is a super simplified pseudocode version of what is ended up happening
var config = {
name: 'modernizr'
}
if (inBrowser) {
config.env = 'browser';
} else {
config.env = 'node';
}
requirejs.config(config);
module.exports = function(config, callback) {
config.out = function (output) {
//code to strip out AMD ceremony, add classPrefix, version, etc
callback(output)
}
requirejs.optimize(config)
}
spot the problem?
Since we are touching the .out method of the config object (whose scope is the entire module, and therefore its context is saved between build() calls) right before we run the asynchronous require.optimize function, the callback you were passing was rewriting the .out method every time build is called.
This should be fixed in a couple hours in Modernizr

The function block is called asynchronously, so this behavior is expected because this call is much slower than the walk of your foreach, so when you reach the function (result) {} block iis already five
Quite the same problem as described in Node.JS: How to pass variables to asynchronous callbacks? here and you should be able to use the same solution
[1,2,3,4,5].forEach(function(i){
(function(i) {
require("modernizr").build({}, function (result) {
console.log(i);
});
})(i);
})
untested but somethign like that should work

How to execute method just one time for application

I have a NodeJS application and I want to execute some method for file validations but just one time (this method will validate some files under the node application).
Is there an elegant way to do this? Events?

The NodeJS documentation on modules states that:
Modules are cached after the first time they are loaded.
which you can take advantage of by adding static code to the module. Regardless of how many times the module is loaded, the static code will retain its state(/value).
You can use that state from a method to implement a method that can be called whenever you like -- at the best time during initialization -- but only ever be called once. This is pretty simple:
var called = false;
function checkFiles() {
if (called) return;
// Perform your validation
called = true;
}
module.exports = {
checkFiles: checkFiles
};
Because of the module caching, you can require this file in as many places as you need and it will still only execute once.
To invoke this function, you have a few options:
For a simple application, you can call the function from your main module (or function) and it will be invoked at that time. If the validation should be asynchronous, you can wrap your main method in a function and pass that to the validator as a callback.
//#! /bin/env node
var express = require('express');
var validator = require('./validator');
validator.checkFiles();
var app = express();
var server = app.listen(3000, function () {
...
});
For a more complicated application, you should call this function during your existing initialization routine (again, using callbacks as necessary).
If you have a nice modern promise-based initializer, you could simply add the validator as the first step of the chain.