I tried finding the solution to my problem, but couldnt find it, and was looking for some "best practice examples". I have a nodejs express application and my functions are split in files. For example I have this controller (oktacontroller.js):
var okta_api_key = <here some await data, getting from db>;
const OKTA_ORG_URL = '<here too>';
exports.createUser = async (req, res) => {
console.log(okta_api_key);
}
exports.getGroups = async (req, res) => {
console.log(okta_api_key);
}
In both exported functions (which are express routes) I need the var okta_api_key. I know I can get them by querying them in both functions, because they are async functions and I can use await there, but it feels dumb to query this every time (because it wont ever change).
How can I manage this? I know I can do this:
var okta_api_key;
(async () => {
okta_api_key = await <getting key async>
})()
But this feels off as well..
Is there any way too make some sort of large function, which is async, and exports the two functions? In other words: How do I use await on the top level of a file (module). It doesnt need to be on top level (its impossible), but some sort of method to have my "top level" variables exposed to my exported functions.
EDIT: Some other usecase, because I got the suggestion of putting it in a config file. Yes, for this one it is possible, but for example: I have some other api key which gets his access token from the service itself, on every startup (because it expires). That token cannot be stored in the config file, so I need some async work to get the value. I know top-level-await is not working (or even not desirable), but I just want an example of how you guys would do this if it were your project :)
You are close:
var okta_api_key = (async () => {
return await <getting key async>
})();
Create a promise, then await that promise whenever you want to use it.
How do I use await on the top level of a file (module).
Top level await might look great: You just add one await and then you can access the variable synchronously. But that simplifies things too much: It will stop all modules depending on that module from executing. In most cases you don't want that¹. Instead create a promise of the async task, then await it when needed. That way you limit the asynchronous execution to the code pieces that actually need it.
¹ Those rare cases are:
1) Loading some global config, that you have to access everywhere in your code, so it makes no sense to start the service if the config isn't ready.
2) awaiting in the top level file of your service: As no module depends on it, this won't cause any problems.
Side note: Top level await is not yet specified, and the NodeJS support is also not there yet. To use it in production you have to wait a few months (?).
Related
I have a web app which I am writing automation for, using WebDriver and Jasmine. This app can be thought of as an online library, where it can load different products, each of which have different content. This automation is supposed to loop over all the locations within the loaded product and run tests on each one.
Since I am running the automation on different products, the automation takes as input an identifier for the product being tested. I use that identifier in the beforeAll function to load information from a web service about the product being tested.
const identifier = process.argv[3];
describe('product check', function() {
var appData;
beforeAll(async function() {
// this comes from a web service, hence being async
appData = await getAppData(identifier);
})
}
The automation should then loop over the appData data structure and generate expectations based on its contents.
Now, my understanding of using loops within Jasmine is that you need to need to put your expectations in a function and call that repeatedly within the loop:
// this won't work
for(var i = 0; i<appData.numInstances; i++) {
it('is within a for loop', async function() {
expect(...);
})
}
// it has to be done like this instead
function containsIt(i) {
it('is within a function', async function() {
expect(...);
})
}
for(var i = 0; i<appData.numInstances; i++) {
containsIt(i)
}
What I am finding is that, if I have my expectations within a function as shown above, the automation doesn't wait for the beforeAll function to finish before the function containing it() is called, so I get an error:
TypeError: Cannot read property 'numInstances' of undefined
I have confirmed that getAppData() works correctly, and that appData is being populated within the beforeAll() function.
I realize that I could put the loop over appData within an it(), but that would mean that all my expect() statements are within the same it() block, and I would lose the ability to have meaningful descriptions from it() in my reporting.
What I would like is the ability to have a loop in which the different functions containing it()s are called repeatedly, while still loading my application data in the beforeAll function. Is this possible? If so, how?
I have faced this exact same situation before and I think with newer versions of Jasmine you won't face this issue although I am not 100% certain.
Looking at the docs they have an async/await in the beforeEach so I am thinking in newer versions it waits for everything to finish in the beforeEach (or beforeAll) before proceeding to its.
To get around your issue, you can use the done callback provided by Jasmine.
beforeAll(async function(done) { // add done callback to the function
// this comes from a web service, hence being async
appData = await getAppData(identifier);
done(); // call done here to let Jasmine know this beforeAll is done
})
With the done callback, the order should now be respected where the beforeAll will run and complete before the first it.
I've been using #zzzzBov's fantastic loadScript() function to load JS libraries using Promises.
However, it becomes tedious and messy to chain together Promises for all of the loaded libraries. I'd like to switch to an async/await solution, but—since loadScript() function doesn't return the usable library objects I need—this isn't as straightforward as using .then() at a first glance.
The function returns a load Event, which I await. So my initial approach would be to just define some useless constants as the Promise returns, and then reference them in an if statement:
let loadedGlobalObject: any;
const lib1Loaded: Event = await loadScript(lib1URL);
const lib2Loaded: Event = await loadScript(lib2URL);
if (lib1Loaded && lib2Loaded) { // Makes this block dependent upon the await statements
loadedGlobalObject = (window as any).globalObject;
console.log(loadedGlobalObject);
}
Is this correct form? I don't know how to "invoke" the await statements; an if statement was my first guess. So I'm wondering if this is the best solution, or if there is a more efficient/accepted technique (I know that I could substitute the libXLoaded constants into the if statement, but I'm going for readability).
If I understand what you are trying to do - you want to await the loading of your libraries, and access the global objects only once they are loaded. What about something like this:
try {
async function loadLibs () {
const lib1Event: Event = await loadScript(lib1Url);
let loadedGlobalObject: any;
loadedGlobalObject = (window as any).globalObject;
console.log(loadedGlobalObject);
}
loadLibs();
catch (err) {
console.log('Error loading script: ', err);
}
The loadScript() function rejects if there was an error while loading the script, so you can wrap your code in a try...catch. Otherwise, it resolves and you can access it's global object.
Note that you can only call await within a function marked as async.
So it turns out I didn't understand async/await at all. I had thought that the awaited constants were reactive, and lines referencing them were run only after their Promises resolved; instead, it's just the rest of the function that waits.
For error-handling, as #dwosk answered, you can use a try-catch, although I wasn't concerned about this. As #Rezaa91 commented, I could have used an await Promise.all([]), except for that I'm loading dependencies in order. And, now that I understand await statements better and as #dwosk and #Bergi noted, I don't need to define those constants.
In summary, my modified code (with Promise.all() added for fun) looks like this:
let loadedGlobalObject: any;
await Promise.all([loadScript(dependency1), loadScript(dependency2)]);
await loadScript(library1);
// Will have been loaded by now.
loadedGlobalObject = (window as any).globalObject;
console.log(loadedGlobalObject);
I'm struggling a little bit with the concepts of asynchronous programming and I would appreciate some help/guidance from someone.
Basically, I'm developing a node.js web server that's connected to a mongodb database.
I'm using EJS to generate HTML files as you can see below.
app.get("/", function(req, res){
res.render('home', {date: getData.todaysDate(), n: getData.todaysIncidents(), nTot: getData.totalIncidents()});
});
Most of these values ('n' and 'nTot') are obtained by querying my database and then doing some other operations, as you can see in the following example code sample.
//------getData.js------//
exports.todaysIncidents = function() {
let server_n = 0;
Incident.find({dia: {$gt:y}}, function(err, result) {
if (err) {
console.log(err);
} else{
//do some stuff...
server_n = 12345
}
}
});
return server_n;
};
Here is the problem: The values printed in the HTML file are always those used for variable initialization, such as 0 for variable 'server_n'. After doing some research, I understand that this happens because .find(...) is an asynchronous function so the program executes right away the instruction "return server_n;", which means that on the HTML file the value showed will be 0 and not 12345.
I already looked at other questions here in StackOverflow, but I'm struggling to understand a possible fix to this issue, I mean I can't be the only one going through this, right?
Could you please provide some basic explanation of how I could go around this issue? I'm still learning and a lot of these concepts remain hard to understand.
Thank you very much.
Yes, you are right that the problem is as a result of improper handling of asynchronous operations like querying the DB. So how do you fix it?
Using async/await:
There are multiple options to handle asynchronous operations in NodeJS, however, I would strongly recommend using async/await, it's syntactically clean and easy to understand.
To put it simply, async/await is a way of specifying and dealing with asynchronous operations. You use the async keyword to specify that a function is asynchronous, and you use the await keyword to wait for an asynchronous operation. One key thing to note is that you can only use the await keyword inside an async function. You can read more about async/await here.
If your nodeJS version is 7.6 or higher, async/await is supported out of the box, however, if you are using a lower version and can't upgrade, you can set up build tools like Babel to be able to use javascript features supported in newer ECMAScript spec.
When using async/await, your code should be something like this.:
//------getData.js------//
// NOTE: the typeof todaysIncidents is not more the regular function,
// it's now an AsyncFunction because of the async keyword
exports.todaysIncidents = async function () {
let server_n = 0;
try {
// In simple terms, the await keyword here would ensure that the DB query
// resolves or reject before moving on to the next statement
const incident = await Incident.find({ dia: { $gt: y } });
// The incident variable stores the result of the query
server_n = 12345
} catch (err) {
// Handle error from the DB query
console.log(err);
}
return server_n;
};
.
//------The router------//
// NOTE: You also need to make the route handler an AsyncFunction
app.get("/", async function (req, res) {
// You can await the differeint DB queries in here to ensure they resolve(or reject) before rendering the view
const todaysDate = await getData.todaysDate();
const todaysIncidents = await getData.todaysIncidents();
const totalIncidents = await getData.totalIncidents();
res.render('home', { date: todaysDate, n: todaysIncidents, nTot: totalIncidents });
});
Coming from C++ and Python, I still struggle with Javascript asynchronous ubiquity. Sometimes it's very useful, but other times I just don't see how to fit it in without writing terrible code.
I have a Node.js + Express CRUD setup, and I have to do some basic check-ups before continuing with the request. I want to check that http POST fields match with database fields before running the final query. I cannot declare it an async function and use await as it must match a given Interface.
showColumns(dbTable) returns a db premise with a SHOW COLUMNS query.
The only solution that I found is:
database.showColumns(dbTable).then((columns)=>{
//But do I really need to put all my logic inside a then???
let row = Object.keys(req.body).filter({}.hasOwnProperty.bind(columns));
//... all the rest of the logic goes here
});
In your opinion, what is the cleanest/most elegant way to solve that?
database.showColumns(dbTable)
.then(columns => this.handleColumns(columns))
.then(parsedData => this.doSthElse(parsedData);
You can extract your logic to a seperate method. However, it must be called inside then as it is the callback triggered once your async operation is finished.
Alternatively, you might consider using generators, async/await functions or promises.
You can use async/await for this.
(async function() {
try {
var columns = await database.showColumns(dbTable)
let row = Object.keys(req.body).filter({}.hasOwnProperty.bind(columns));
} catch (e) {
console.log("Promise error");
}
})
I have created a web server i node.js using express and passport. It authenticates using an oauth 2.0 strategy (https://www.npmjs.com/package/passport-canvas). When authenticated, I want to make a call such as:
app.get("/api/courses/:courseId", function(req, res) {
// pass req.user.accessToken implicitly like
// through an IIFE
createExcelToResponseStream(req.params.courseId, res).catch(err => {
console.log(err);
res.status(500).send("Ops!");
});
});
My issue is that i would like, in all subsequent calls from createExcelToResponseStream, to have access to my accessToken. I need to do a ton of api calls later in my business layer. I will call a method that looks like this:
const rq = require("request");
const request = url => {
return new Promise(resolve => {
rq.get(
url,
{
auth: {
bearer: CANVASTOKEN // should be req.user.accessToken
}
},
(error, response) => {
if (error) {
throw new Error(error);
}
resolve(response);
}
);
});
};
If i try to create a global access to the access token, i will risk
race conditions (i think) - i.e. that people get responses in the context of another persons access token.
If i pass the context as a variable i have to refactor a
lof of my code base and a lot of business layer functions have to
know about something they don't need to know about
Is there any way in javascript where i can pass the context, accross functions, modules and files, through the entire callstack (by scope, apply, bind, this...). A bit the same way you could do in a multithreaded environment where you have one user context per thread.
The only thing you could do would be
.bind(req);
But that has has to be chained into every inner function call
somefunc.call(this);
Or you use inline arrow functions only
(function (){
inner=()=>alert(this);
inner();
}).bind("Hi!")();
Alternatively, you could apply all functions onto an Object, and then create a new Instance:
var reqAuthFunctions={
example:()=>alert(this.accessToken),
accessToken:null
};
instance=Object.assign(Object.create(reqAuthFunctions),{accessToken:1234});
instance.example();
You could use a Promise to avoid Race conditions.
Let's have this module:
// ContextStorage.js
let gotContext;
let failedGettingContext;
const getContext = new Promise((resolve,reject)=>{
gotContext = resolve;
failedGettingContext = reject;
}
export {getContext,gotContext, failedGettingContext};
And this inititalization:
// init.js
import {gotContext} from './ContextStorage';
fetch(context).then(contextIGot => gotContext(contextIGot));
And this thing that needs the context:
// contextNeeded.js
import {getContext} from './ContextStorage';
getContext.then(context => {
// Do stuff with context
}
This is obviously not very usable code, since it all executes on load, but I hope it gives you a framework of how to think about this issue with portals... I mean Promises...
The thing that happens when you call the imported 'gotContext', you actually resolve the promise returned by 'getContext'. Hence no matter the order of operations, you either resolve the promise after the context has been requested setting the dependent operation into motion, or your singleton has already a resolved promise, and the dependent operation will continue synchronously.
On another note, you could easily fetch the context in the 'body' of the promise in the 'ContextStorage' singleton. However that's not very modular, now is it. A better approach would be to inject the initializing function into the singleton in order to invert control, but that would obfuscate the code a bit I feel hindering the purpose of the demonstration.