We Keep Coding

Excel.run, pass and return context as a variable, where does context come from?

I have a big Google Apps Script project working on Google Spreadsheets and I am trying to convert it to an office-js add-in that works on Excel Workbooks. I understand that it is good practice to put everything that will be calling excel specific functions (directly interacting with the workbook) into an Excel.run() function so it does a proper clean-up and no memory leaks occur. I also understand I should do context.sync() as little as possible to optimize performance.
Here are my questions, (I think some of them come from my incomplete grasp of how js works; these things GAS handled without me needing to question them) :
1a) When we put our block of code in
Excel.run(context => {
//code that does stuff with context;
context.sync();
});
where does the context come from? Is this equivalent to
Excel.run(()=> {
let context = new Excel.RequestContext;
//code that does stuff with context;
context.sync();
});
1b) Also, if context gets generated with every new function why would I ever return context.sync() and not just context.sync()?
Is a new, second context generated in this case and what happens to it?
function handle_error(e){//second context generated in case of error
let context=New Excel.RequestContext;
context.workbook.load('name');
await context.sync();
some_logging_function(context.workbook.name, e);
}
function data_func(some_data: Non-Excel-Interface): Other-Non-Excel-Interface{
//manipulate data
//in case of error
handle_error(e);
//continue with data massaging
return altered_data;
}
Excel.run(context=>{ //first context
context.workbook.worksheets.getItem('Sheet1').getUsedRange().load('values');
context.sync();
let values = context.workbook.worksheets.getItem('Sheet1').getUsedRange().values;
let some_data: Non-Excel-Interface = {sheetName: 'Sheet1', data: values};
let new_vals = data_func(some_data);
context.workbook.worksheets.getItem('Sheet1').getUsedRange().values = new_vals.new_data;
context.sync();
});
If I put my main code inside Excel.run, then pass and return context: Excel.RequestContext and range: Excel.Range in other functions do I need Excel.run() in those functions, too? In other words, should the code inside functions a() and b() be inside Excel.run()?
function a(rng: Excel.Range, values:string[][]):Excel.Range{
rng.values = values;
return rng;
}
function b(context: Excel.RequestContext): Excel.RequestContext{
context.workbook.load('name');//load name property, but don't context.sync()
return context;
}
Excel.run(async context=>{
context = b(context);
let rng = context.workbook.worksheets.getItem('Sheet1').getUsedRange();
rng.load('values');
await context.sync();//values property and workbook name property must be available now
rng = a(rng, [['aa', 'bb', 'cc']]);
await context.sync();//new values must be available now
console.log(context.workbook.name, rng.values);//should show the title of the workbook and the newly assigned values of the range
});
Also, what is the advantage of asynchronous functions if I have to explicitly wait every time I need a value? I mean, if I am going to use context.sync() sparingly, that means I use it only when I desperately need it, so it must always come with await. So why not make context.sync() synchronous by default?

I'll try to answer some of these questions and try to get some help for the others. I also recommend the book Building Office Add-ins for an understanding of the Office JavaScript library. See this too, if you haven't already: Application specific API model.
1a. Yes. That's essentially correct. Under the hood, Excel.run creates an Office.RequestContext object and passes it to the batch function parameter. (But your two code blocks are not literally equivalent. You would not call Excel.run AND explicitly create a RequestContext object.)
1b. From skimming the book I linked to, I think that you have to return what the book calls the meta-promise so that the Excel.run can resolve the Promise that it returns. Here's an example from the book:
Excel.run(function (context) {
var selectionRange = context.workbook.getSelectedRange();
selectionRange.format.fill.clear();
selectionRange.load("values");
return context.sync()
.then(function () {
var rowCount = selectionRange.values.length;
var columnCount = selectionRange.values[0].length;
for (var row = 0; row < rowCount; row++) {
for (var column = 0; column < columnCount; column ++) {
if (selectionRange.values[row][column] > 50) {
selectionRange.getCell(row, column)
.format.fill.color = "yellow";
}
}
}
})
.then(context.sync);
}).catch(OfficeHelpers.Utilities.log);
From skimming the book I linked to, I think that the answer is yes; the Excel.run always creates a new context object and passes it to the batch function. There are techniques and an override of Excel.run that enable you to pass an object created in one context to another call of Excel.run, but these are intended for use with independent calls of Excel.run, not nested calls, as in your case.
No. You should not call Excel.run inside a or b.
I think there are scenarios in which you would not need to await context.sync. For example, when all the code in the parent function that comes after the context.sync only affects the UI of a task pane and does not depend on reading any data from the current Office document. The good practice of minimizing calls of context.sync is because it requires a round-trip between the document and JavaScript runtime in which the add-in code is running (on the user's computer). This would be true regardless of whether context.sync is synchronous or not.

For 1a, here's one of the descriptions of how the Run function works which I'm getting from ScriptLab's intellisense:
A function that takes in a RequestContext and returns a promise (typically, just the result of "context.sync()"). The context parameter facilitates requests to the Excel application. Since the Office add-in and the Excel application run in two different processes, the RequestContext is required to get access to the Excel object model from the add-in.
In terms of 1b, I don't think so. RequestContext does not seem to be an object you can instantiate yourself.
EDIT: actually it does look like this is possible. Please see below:
$("#run").click(() => tryCatch(run));
async function run() {
await Excel.run(async () => {
let ctx:Excel.RequestContext = new Excel.RequestContext();
let wb: Excel.Workbook = ctx.workbook
let rang: Excel.Range = wb.getSelectedRange()
rang.load("address")
await ctx.sync()
console.log(rang.address)
});
}
/** Default helper for invoking an action and handling errors. */
async function tryCatch(callback) {
try {
await callback();
} catch (error) {
// Note: In a production add-in, you'd want to notify the user through your add-in's UI.
console.error(error);
}
}
I'd recommend against using this approach. Passing in anonymous functions, like in the original example, is very common in JavaScript. So this would likely be considered a bad practice.

TypeError: Cannot read property 'latestTimestamp' of undefined [duplicate]

This question already has answers here:
Calling an asynchronous function within a for loop in JavaScript
(10 answers)
Closed 8 years ago.
I found lots of similar questions, but I still don't know what's wrong with my code. It seems that I cannot read global variable value (urls) in the callback function: I want to update the urls latestTimestamp value in the callback function(err, articles). Here is the code that went wrong:
var urls=[
{"url": "http://www.economist.com/feeds/print-sections/77/business.xml", "latestTimestamp": new Number(0)},
{"url": "http://news.sky.com/feeds/rss/home.xml", "latestTimestamp": new Number(0)},
]; // Example RSS Feeds;
// parse RssFeeds from given websites and write them into databse
function parseRssFeeds(collection){
var feed = require('feed-read'); // require the feed-read module
// loop through our list of RSS feed urls
for (var j = 0; j < urls.length; j++)
{
console.log('Original url timestamp is: '+ urls[j].latestTimestamp.toString());
// fetch rss feed for the url:
feed(urls[j], function(err, articles)
{
// loop through the list of articles returned
for (var i = 0; i < articles.length; i++)
{
var message =
{"title": articles[i].title,
"link": articles[i].link,
"content": articles[i].content,
"published": articles[i].published.getTime()};
collection.insert(message, {safe:true}, function(err, docs) {
if (err) {
console.log('Insert error: '+err);
}else{
console.log('This item timestamp is: '+ message.published);
// get the latest timestamp
if (message.published >urls[j].latestTimestamp) {
console.log('update timestamp to be: '+ message.published);
urls[j].latestTimestamp = message.published;
}
}
});// end collection insert
} // end inner for loop
}) // end call to feed method
} // end urls for loop
}
Thanks for any help. The error is:
TypeError: Cannot read property 'latestTimestamp' of undefined
at /Users/Laura/Documents/IBM/project/TestList/app.js:244:37
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/collection/core.js:123:9
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/db.js:1131:7
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/db.js:1847:9
at Server.Base._callHandler (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/server.js:478:18
at MongoReply.parseBody (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
at null.<anonymous> (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/server.js:436:20)
at emit (events.js:95:17)
at null.<anonymous> (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:201:13)

This should probably be closed as a duplicate, but I'll put an answer here because the relationships between all the duplicate questions are often so hard to grasp for JavaScript programmers that haven't understood the fundamental problem.
There are two ways to solve this. One way is to change the way that you create the callback. Instead of using an inline anonymous function:
feed(urls[j], function(err, articles)
{
// loop through the list of articles returned
// ...
you'd create another function that returns the callback. You'd pass that function the URL, and that's what the returned function would use:
function makeFeedResultHandler(url) {
return function(err, articles) {
// loop through the list of articles returned
// ... code as before up to this line:
if (message.published > url.latestTimestamp) {
console.log('update timestamp to be: '+ message.published);
url.latestTimestamp = message.published;
}
// ... etc
};
}
Then you'd call "feed" like this:
feed(urls[j], makeFeedResultHandler(urls[j]));
The key difference is that each function passed to "feed" will have its own private copy of the object (well, a copy of the reference to be picky) from the "urls" array, so it won't need to refer to the variable "j" at all. That's the crux of the problem: "j" is shared by all the callbacks in your code. By the time the callbacks are invoked, the value of "j" is equal to the length of the "urls" array, so urls[j] is undefined.
The other approach would be to use the .forEach method, available in newer JavaScript implementations. That approach would get rid of "j" altogether:
urls.forEach(function(url) {
console.log('Original url timestamp is: '+ url.latestTimestamp.toString());
// fetch rss feed for the url:
feed(url, function(err, articles)
{
// loop through the list of articles returned
// ... code as previously, substituting "url" for "urls[j]" everywhere
});
});
Again, that makes sure that every callback sent to the "feed" function has its own copy of the "urls" element.

Expanding on what #Pointy said in his comment under your post:
The insert function you are using with MongoDB is async, but you are treating the callback like it is synchronous. What is essentially happening in your loop, is everything works as planned until you hit collection.insert. From there, the process breaks off and essentially says "I'm going to tell mongo to insert a record now.. and eventually I'll expect a response." Meanwhile, the loop continues on to the next index and doesn't synchronously wait until the callback fires.
By the time your callback fires, your loop is already done, and J doesn't represent the index anymore, which is why its coming up undefined. You also run the risk of getting a different index than what you plan also with this current method.
I would recommend reworking your loop to support the async nature of node. There is a great library called - oddly enough - async that makes this process super simple. The async.each() function should help you accomplish what you are trying to do.

Trying to iterate through an array and add a get function for each

I want to iterate through the style tags and write a GET function for each. The problem is the GET function is being written with a reference to 'styleTags[i]' instead of converting 'styleTags[i]' to the appropriate tag.
var styleTags = ['cont', 'ecce'];
for (var i = 0; i < styleTags.length; i++) {
app.get('/photos-' + styleTags[i], selectNav, function(req, res) {
getDynPhotos(req, res, styleTags[i]);
});
}

I'm not entirely clear what problem you are asking about, but I do see an issue with your use of i in the callback. Because app.get() is presumably asynchronous and will complete some time later, the value of i will not still be valid. You need to create a closure that will capture the value of i. There are several ways to do that. Here's one way using an immediately invoked function expression (often abbreviated IIFE):
var styleTags = ['cont', 'ecce'];
for (var i = 0; i < styleTags.length; i++) {
(function(index) {
app.get('/photos-' + styleTags[index], selectNav, function(req, res) {
getDynPhotos(req, res, styleTags[index]);
});
})(i);
}
This will freeze the value of i in the function argument that I've named index so it will still have the right value at the later time when the callback is called.
If this isn't what you were asking about (though it is still something that needs to be fixed), then please describe in more detail what you were asking about.

Don't use a for loop in Node because it doesn't create a closure. Instead, I recommend async.each:
var async = require('async');
var styleTags = ['cont', 'ecce'];
async.each(styleTags, function(styleTag, callback) {
app.get('/photos-' + styleTag, selectNav, function(req, res) {
getDynPhotos(req, res, styleTag);
});
callback();
}

I see you're trying to build a Node.js Express route.
I'm somewhat surprised this didn't work, as I thought the declaration of these routes would be synchronous.
Could you look at app.routes to see what Node is putting in place for you? I'm guessing you may have done this already, but I thought I'd mention it.
I have two suggestions if that doesn't work: use regular expressions in your route to isolate the category section of your route (then validate the route at request time), or insert your route directly into the app.routes object structure.

queue.js: difference between array of functions & array of closures?

So, I am trying to make sure that a series of HTTP GET requests happen before I try to render the data gotten into a visualization. Typical deal, right?
I'm using queue.js, and seeing this on the queue.js github page (https://github.com/mbostock/queue):
Or, if you wanted to run a bazillion asynchronous tasks (here represented as an array of closures) serially:
var q = queue(1);
tasks.forEach(function(t) { q.defer(t); });
q.awaitAll(function(error, results) { console.log("all done!"); });
Queue.js can be run inside Node.js or in a browser.
So, what I did was made an array of functions, each of which contained a Meteor.http.get call (as I'm using Meteor.js) and then followed this line by line.
It seems like what is happening is that while my array -- which has 8 functions in it, all with what looks like the right function in each slot -- gets populated (and then passed as in the code exceprt to defer,) only one actually runs.
Here's what I'm wondering. Well, overall it's, why is only one function running though 8 are passed in to defer? But specifically it's -- having a hazy understanding of closures, I really have an array of functions. Is there something I missed there, since the documentation specifically says closures, which is why all the functions aren't executing?
Thank you for looking at this!

Here is perhaps the literal part of the statement you quoted, found in the test suite:
"queue of asynchronous closures, processed serially": {
topic: function() {
var tasks = [],
task = asynchronousTask(),
n = 10,
q = queue(1);
while (--n >= 0) tasks.push(task);
tasks.forEach(function(t) { q.defer(t); });
q.awaitAll(this.callback)
},
"does not fail": function(error, results) {
assert.isNull(error);
},
"executes all tasks in series": function(error, results) {
assert.deepEqual(results, [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]);
}
},
https://github.com/mbostock/queue/blob/master/test/queue-test.js#L103
Which runs task = asynchronousTask(), which is what is pulled off the queue and invoked
function asynchronousTask() {
var active = 0;
return function(callback) {
++active;
process.nextTick(function() {
try {
callback(null, active);
} finally {
--active;
}
});
};
}
The above, inner return function() {...} is what I believe is being referenced as a closure which retains it's reference in scope to the outer active variable as each asynchronous function is called off the queue.
This is, of course, fairly powerful in terms of callbacks and handlers, since it gives you the means to maintain and manipulate a locally shared variable, for instance if you want to know how many functions were returned, and when the list has been exhausted. In other words, a queue.
The following is not used in the example up top, but use it as a reference point to see how it differs from the synchronousTask function above.
function synchronousTask() {
var active = 0;
return function(callback) {
try {
callback(null, ++active);
} finally {
--active;
}
};
}
https://github.com/mbostock/queue/blob/master/test/queue-test.js#L265

Properly way to modify an object property inside an anonymous function callback on Javascript

I'm using Node.js + Express + nodejs-sqlite3 to make a form that when submited will insert a new row on an slite3 database.
On query sucess I want to write certain response.
So the small big problem is just: Modify a string that will be storing the html to be shown, inside the callback function of sqlite3.run()
I read about closures, and passing an object with methods to modify its own attributes. But it seems it's not working. It will pass the object attributes and methods, but no change will remain when the callback function ends. I read that objects will be passed as reference, not copies.
This is the code:
app.post("/insert.html", function(req, res){
function TheBody(){
this.html = "";
this.msg = "";
this.num = "";
}
TheBody.prototype.add = function(string){
this.html = this.html + string;
}
var body = new TheBody();
body.msg = req.body.message;
body.num = req.body.number;
var insertCallback = function(data){
return function(err){
if( err != null){
console.log("Can't insert new msg: " + err.message);
data.add("ERROR-DB");
} else {
console.log("Ok. Inserted: " + data.msg);
console.log(data.html);
data.add("OK - MSG: "+data.msg+" NUM: "+data.num);
console.log(data.html);
}
};
};
var db = new lite.Database('database.db');
var query = "INSERT INTO outbox (message, number) VALUES (?, ?)";
db.run(query, [body.msg, body.num], insertCallback(body) );
res.setHeader('Content-Type', 'text/html');
res.setHeader('Content-Length', body.html.length);
res.end(body.html);
}
On server side I'll see
Ok. Inserted: TestString
[Blank space since data.html still has no information]
OK - MSG: TestString NUM: TestNumber [Showing that indeed was modified inside the function]
But on the client side res.end(body.html); will send an empty string.
The object is not being passed as reference.
What's missing in the code, and what simpler alternatives I have to change a string variable inside a callback anonymous function?.
I already know I could use response.write() to write directly on the function if it were more simpler. But I discovered it would only work if I use response.end() inside the callback, otherwise (being outside as it is now) it will meet a race condition where the buffer will be closed before sqlite3.run() be able to use response.write().
-------- Answered --------
As hinted by Justin Bicknell and confirmed by George P. Nodejs-sqlite3 functions are run asynchronously. So I was ending the stream to the client before the callback would be called, thus nothing was being printed.
This was a problem more about "This is SPART- nodejs, so write your stuff according to events'" rather than a logic one. I found this kind of programming kind of convoluted but nobody else than me told me to use nodejs. For those wondering about how one could put some order over the order of queries on the database, nodejs-sqlite3 functions returns a database object that is used to chain the next query.
Since I was printing the information to the client just once in every handled event, the resulting object ended like this:
function TheBody(response){
this.response = response;
}
TheBody.prototype.printAll = function(string){
this.response.setHeader('Content-Type', 'text/html');
this.response.setHeader('Content-Length', string.length);
this.response.end(string);
}
Preferring that to clutter all the code a lot of res.setHeader() lines.

node-sqlite3 methods are, by default, run in parallel (asynchronously). That means that your code is going through this timeline:
Your code calls db.run(...)
Your code calls res.end(...)
db.run completes and calls your callback.
This is the source of a huge number of questions here on SO, so you can almost certainly find a better answer than anything that I could write here in a reasonable amount of time.
I would start here: How does Asynchronous Javascript Execution happen? and when not to use return statement?