We Keep Coding

Angular/Javascript - Custom User Scripts (Eval)

Hi we are building a web app platform where users can make their own smart forms using drag and drop features. We are looking for a way for admin users to create their own custom scripts to run some logic using pre-defined functions for the app. Currently the solution we have come up is using eval().
Knowing Eval is 'evil' we have implemented a function to check if the script is safe before it is executed. Essentially it breaks up the code into tokens and then runs those tokens against a blacklist. Stuff like new, eval, window, require, fetch,browser will show an error. Also the function is executed via a angular service so we try to limit what is injected.
Below is the basic high-level code. We have custom async functions so the solution needs to handle this.
My question is there a better(ie faster) and safer way to run custom scripts?
async runScript(script,callback) {
var updateForm=(id,value)=>{
return this.updateForm(id,value);
}
var getForm=(id)=>{
return this.getForm(id);
}
if (this.checkScriptSafe(script)) {
try {
return eval("(async () => {" + script + "})()");
} catch (e) {
if (e instanceof SyntaxError) {
alert(e.message);
} else {
console.log('Error',e);
alert("Error in script");
}
}
} else {
alert("Script not safe")
}
}
Example script:
"var value = 1 +4; await updateForm("11",value);alert("Success!");"

Function constructor would be a better approach. Function constructor creates a new function that will execute in the global scope. Your eval script (because of the arrow function) will run in the same context as your runScript method. They would access/modify your internals, or override your class methods. They can even override the runScript method itself and remove the checkScriptSafe check.
Using the function constructor is similar to typing in the dev tools console. If your application is not vulnerable to the dev tools console, then you wouldn't have any issues using the function constructor.
Here is an example:
const script = `
var value = 1 +4;\n
await updateForm("11",value);\n
alert("Success!");
`;
// we have to get a handle of the async function constructor
// in order to create an async function
const dummyFunction = async function() {}
const AsyncFunction = dummyFunction.constructor;
// create an async function which will run in the global scope
// the function will have an `updateForm` parameter
const userFunction = new AsyncFunction('updateForm', script);
// now userFunction is equavalent of:
// const userFunction = async function(updateForm) {
// var value = 1 +4;
// await updateForm("11",value);
// alert("Success!");
// }
// bind the current context 'this' to update form and pass it
// to user's function as parameter. The user's function
// will be able to execute it.
userFunction(this.updateForm.bind(this));
I'm not an expert in browser internals. But I assume tokenizing and interpreting the function on your own would be much slower than the Function constructor method. Even if you do everything in the most efficient way, you would still be in the JavaScript domain; v8 (or any other JS engine) would perform the actual interpretation after you. Why not directly give the script to the JS engine then? If the same custom script is going to run frequently, then with the right design, v8 will optimize the custom functions by compiling them into machine code. This wouldn't be the case with eval.

in an `async` function are the input parameters preserved over an `await`

I have some browser code which has been showing some mysterious errors when running over a slow link. I am coming to the conclusion that the event.detail parameter is getting corrupted.
So I have an async function (as a method on a class) a bit like the following:
async doUpdate(e) {
e.stopPropagation();
const response = await readSomethingFromRemoteDatabase(this.id, e.detail.anotherid);
const myentity = this.entities.find(x => x.id === e.detail.anotherid)
if (typeof myentity !== 'undefined') {
//do something to update local data
}
}
I expect myentity to be found, but occasionally it isn't.
I have since changed the code to become the following:
async doUpdate(e) {
e.stopPropagation();
const data = e.detail
const response = await readSomethingFromRemoteDatabase(this.id, e.detail.anotherid);
const myentity = this.entities.find(x => x.id === data.anotherid)
if (typeof myentity !== 'undefined') {
//do something to update local data
}
}
and the issue has gone away.
I'm coming to the conclusion that with the slow link, its possible for doUpdate to get called twice, the second time before the readSomthingFromRemoteDatabase has responded, and the second call has come in with different data.
What that implies is that the const data is within the scope of the called function for the entire scope of its execution, but that the parameter variable e is not.
However, I can find no documented information about whether my assumption is true. Can someone point me to where I can find out if my assumption is true?
UPDATE in response to some of the comments.
Firstly, doUpdate is an event listener which is called via lit-element. Elsewhere in this class is a Lit render function, a bit like this
render() {
return html`
${this.entities.map(entity => html`
<entity-editor .entity=${entity} #entity-changed=${this.doUpdate}></entity-editor>
`)}
`;
}
The event detail is an object constructed from parts, so not a reference elsewhere. Normally made something like this (inside the entity-editor custom element.
this.dispatchEvent(new CustomEvent('entity-changed', {
bubbles: true,
composed: true,
detail: {
anotherid: this.id,
moreinfo: this.moreinfo
)
}));
readSomethingFromRemoteDatabase does little more than translate the object passed into a fetch POST request to the server, where the object passed is JSON.stringify'ed into the body of the request.
Sorry that I can't point you at real code - but this is a massive PUA I personally have been building since 2016 advancing through Polymer and now Lit to replace a Microsoft Access Application. It's private work for a company so is in a private repository.
At this stage it's 100s of elements in a client, and 100s of components in a Node.js api-server (proxied by NGINX)

Excel.run, pass and return context as a variable, where does context come from?

I have a big Google Apps Script project working on Google Spreadsheets and I am trying to convert it to an office-js add-in that works on Excel Workbooks. I understand that it is good practice to put everything that will be calling excel specific functions (directly interacting with the workbook) into an Excel.run() function so it does a proper clean-up and no memory leaks occur. I also understand I should do context.sync() as little as possible to optimize performance.
Here are my questions, (I think some of them come from my incomplete grasp of how js works; these things GAS handled without me needing to question them) :
1a) When we put our block of code in
Excel.run(context => {
//code that does stuff with context;
context.sync();
});
where does the context come from? Is this equivalent to
Excel.run(()=> {
let context = new Excel.RequestContext;
//code that does stuff with context;
context.sync();
});
1b) Also, if context gets generated with every new function why would I ever return context.sync() and not just context.sync()?
Is a new, second context generated in this case and what happens to it?
function handle_error(e){//second context generated in case of error
let context=New Excel.RequestContext;
context.workbook.load('name');
await context.sync();
some_logging_function(context.workbook.name, e);
}
function data_func(some_data: Non-Excel-Interface): Other-Non-Excel-Interface{
//manipulate data
//in case of error
handle_error(e);
//continue with data massaging
return altered_data;
}
Excel.run(context=>{ //first context
context.workbook.worksheets.getItem('Sheet1').getUsedRange().load('values');
context.sync();
let values = context.workbook.worksheets.getItem('Sheet1').getUsedRange().values;
let some_data: Non-Excel-Interface = {sheetName: 'Sheet1', data: values};
let new_vals = data_func(some_data);
context.workbook.worksheets.getItem('Sheet1').getUsedRange().values = new_vals.new_data;
context.sync();
});
If I put my main code inside Excel.run, then pass and return context: Excel.RequestContext and range: Excel.Range in other functions do I need Excel.run() in those functions, too? In other words, should the code inside functions a() and b() be inside Excel.run()?
function a(rng: Excel.Range, values:string[][]):Excel.Range{
rng.values = values;
return rng;
}
function b(context: Excel.RequestContext): Excel.RequestContext{
context.workbook.load('name');//load name property, but don't context.sync()
return context;
}
Excel.run(async context=>{
context = b(context);
let rng = context.workbook.worksheets.getItem('Sheet1').getUsedRange();
rng.load('values');
await context.sync();//values property and workbook name property must be available now
rng = a(rng, [['aa', 'bb', 'cc']]);
await context.sync();//new values must be available now
console.log(context.workbook.name, rng.values);//should show the title of the workbook and the newly assigned values of the range
});
Also, what is the advantage of asynchronous functions if I have to explicitly wait every time I need a value? I mean, if I am going to use context.sync() sparingly, that means I use it only when I desperately need it, so it must always come with await. So why not make context.sync() synchronous by default?

I'll try to answer some of these questions and try to get some help for the others. I also recommend the book Building Office Add-ins for an understanding of the Office JavaScript library. See this too, if you haven't already: Application specific API model.
1a. Yes. That's essentially correct. Under the hood, Excel.run creates an Office.RequestContext object and passes it to the batch function parameter. (But your two code blocks are not literally equivalent. You would not call Excel.run AND explicitly create a RequestContext object.)
1b. From skimming the book I linked to, I think that you have to return what the book calls the meta-promise so that the Excel.run can resolve the Promise that it returns. Here's an example from the book:
Excel.run(function (context) {
var selectionRange = context.workbook.getSelectedRange();
selectionRange.format.fill.clear();
selectionRange.load("values");
return context.sync()
.then(function () {
var rowCount = selectionRange.values.length;
var columnCount = selectionRange.values[0].length;
for (var row = 0; row < rowCount; row++) {
for (var column = 0; column < columnCount; column ++) {
if (selectionRange.values[row][column] > 50) {
selectionRange.getCell(row, column)
.format.fill.color = "yellow";
}
}
}
})
.then(context.sync);
}).catch(OfficeHelpers.Utilities.log);
From skimming the book I linked to, I think that the answer is yes; the Excel.run always creates a new context object and passes it to the batch function. There are techniques and an override of Excel.run that enable you to pass an object created in one context to another call of Excel.run, but these are intended for use with independent calls of Excel.run, not nested calls, as in your case.
No. You should not call Excel.run inside a or b.
I think there are scenarios in which you would not need to await context.sync. For example, when all the code in the parent function that comes after the context.sync only affects the UI of a task pane and does not depend on reading any data from the current Office document. The good practice of minimizing calls of context.sync is because it requires a round-trip between the document and JavaScript runtime in which the add-in code is running (on the user's computer). This would be true regardless of whether context.sync is synchronous or not.

For 1a, here's one of the descriptions of how the Run function works which I'm getting from ScriptLab's intellisense:
A function that takes in a RequestContext and returns a promise (typically, just the result of "context.sync()"). The context parameter facilitates requests to the Excel application. Since the Office add-in and the Excel application run in two different processes, the RequestContext is required to get access to the Excel object model from the add-in.
In terms of 1b, I don't think so. RequestContext does not seem to be an object you can instantiate yourself.
EDIT: actually it does look like this is possible. Please see below:
$("#run").click(() => tryCatch(run));
async function run() {
await Excel.run(async () => {
let ctx:Excel.RequestContext = new Excel.RequestContext();
let wb: Excel.Workbook = ctx.workbook
let rang: Excel.Range = wb.getSelectedRange()
rang.load("address")
await ctx.sync()
console.log(rang.address)
});
}
/** Default helper for invoking an action and handling errors. */
async function tryCatch(callback) {
try {
await callback();
} catch (error) {
// Note: In a production add-in, you'd want to notify the user through your add-in's UI.
console.error(error);
}
}
I'd recommend against using this approach. Passing in anonymous functions, like in the original example, is very common in JavaScript. So this would likely be considered a bad practice.

Trying to export function without success with JavaScript, Node.js

I'm trying to learn Node.js and I have a pretty good app now, but I wanted to implement a library (venom-bot) to send some text messages. My problem is that I'm having trouble trying to use functions outside the primary file. Here is my code:
// Dependencies
const server = require("./lib/server")
const venom = require("venom-bot")
// Declare the app
const app = {}
// Init function
app.init = () => {
// Start the server
server.init()
}
// Init venom-bot
venom.create().then((client) => start(client))
// A function to test if I can send the message
async function testing(msg) {
await msg.sendText(...some code here...)
}
function start(client) {
app.msg = client
// Here, if I pass app.msg as an argument, works
// My problem is that I can't use app.msg outside of here,
// even with the module.exports down there
// (I'm trying to use it on a helpers.js file).
testing(app.msg)
// Execute the application
app.init()
}
// Export the app
module.exports = app
On the helpers.js file I'm requiring it this way:
// Dependencies
const app = require("./index.js")
// A function to test
async function sendMsg(msg) {
await msg.sendText(...some code here...)
}
helpers.send = () => {
sendMsg(app.msg)
}
module.exports = helpers
Whenever helpers.send gets invoked, it should correctly use the async function right above the helpers.send passing the app.msg as argument, at least I think it should. What I'm missing here?
The error I got is:
(node:18148) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'sendText' of undefined
On testing purpose, I'm trying to simple call that helper when I receive a get request to a specified route.

HTML5 Call to global function from Worker

I have a global function in Utils.js called "sendAndWaitCommand".
When I'm try to call this function form the Worker (name 'uploadToDevice.js'), the Worker crashes.
Worker Initialization
var worker = new Worker('uploadToDevice.js');
worker.postMessage(SplitedFile);
The Worker (uploadToDevice.js)
self.addEventListener('message', function (e)
{
var SplitedFile = e.data;
sendAndWaitCommand(SplitedFile[0].substring(1));//crash here
}, false);
Utils.js
function sendAndWaitCommand(commandToSend)
{
...//Some heavy stuff to do.
}
Is there any way to call a global function without the worker will crash?
If not, is there a solution how I can call objects from outside the worker?

The JavaScript model isn't, naturally, thread based but event based. There's no facility to lock data and ensure their integrity in a multi-thread context.
That's why multi-threading schemes (among them the webworkers) don't allow data sharing (a "global" function is some data and usually points to data through the closure). You can't call a "global" function from your webworker. You communicate with messages.
Instead of a global function, you set an event listener :
var myWorker = new Worker("my_task.js");
myWorker.onmessage = function (oEvent) {
// the implementation of your "global function", for example :
sendAndWaitCommand(oEvent.data.commandToSend);
};

If you want to use the code in util.js you would have to import it using the importScripts command:
Example from MDF:
importScripts(); /* imports nothing */
importScripts('foo.js'); /* imports just "foo.js" */
importScripts('foo.js', 'bar.js'); /* imports two scripts */
Note that if util.js interacts with the global scope this won't work, since after being imported it will still be in the same scope as your worker. If util.js contains all the code needed to perform the sendAndWaitCommand then that might be a good choice.
If sendAndWaitCommand interacts with your other code, (jquery.ajax, your error handling code, etc.) it would probably be better to setup a message and event system, like so:
var worker = new Worker('uploadToDevice.js');
worker.onMessage = function (message) {
if (// Test here to see if message is result
message && message[0] == "!")
myApp.onDone(message.substr(1)) // Removes the ! and calls a function to handel the worker being done
else // assume the message is a sendAndWaitCommand request
sendAndWaitCommand(message).then(function (a) {
worker.postMessage(a); // Send the worker the result
});
worker.postMessage(SplitedFile); // Start the worker
The second approach will probably involve more refactoring of your code but may be necessary to use workers correctly.