We Keep Coding

Bull queue not processing simple job

I'm completely new to Bull queue and I have a decent understanding of JavaScript. I am trying to get the following example to work:
const Queue = require('bull')
const myFirstQueue = new Queue('my-first-queue');
myFirstQueue.process(async (job, done) => {
await doSomething(job.data);
done();
});
const doSomething = data => {
return new Promise((resolve, reject) => {
return resolve(data);
});
};
myFirstQueue.on('completed', (job, result) => {
log.debug(`Job completed with result ${job}`);
});
(async function ad() {
const job = await myFirstQueue.add({
foo: 'bar',
});
})();
The error I'm getting is:
\node_modules\ioredis\built\redis\event_handler.js:177
self.flushQueue(new errors_1.MaxRetriesPerRequestError(maxRetriesPerRequest));
^
MaxRetriesPerRequestError: Reached the max retries per request limit (which is 20). Refer to "maxRetriesPerRequest" option for details.
at Socket.<anonymous> (...\node_modules\ioredis\built\redis\event_handler.js:177:37)
at Object.onceWrapper (node:events:652:26)
at Socket.emit (node:events:537:28)
at TCP.<anonymous> (node:net:747:14)
Based off the error I'm getting it seems like the queue cannot process this job at all and I'm not sure why I'm getting this error. I am running the script with the command node fileName.js in the correct directory. I can run other JavaScript files fine that are even more complex than this.
EDIT: Edited code to reflect changes from comments, still have the same error.

Azure function typescript doesn't correctly await for a method call

I'm working on an azure function that receives requests to update an azure cosmos database. So, after receiving the request, the function read the item from database, if the item exist then it is updated accordingly. But it doesn't work as I intended, reading through the log when the function run, it seemed that the function doesn't wait until the read from database is finished.
This is the snippet of the code:
const SaasWebhook: AzureFunction = function (context: Context, req: HttpRequest): Promise<void> {
return new Promise(async (resolve, reject) => {
// Initiate azure cosmos db
// Get the payload information
const subscriptionId: string = req.body?.subscriptionId;
const action: string = req.body?.action;
const planId: string = req.body?.planId;
const quantity: string = req.body?.quantity;
let patchedItem: any;
context.log(req.body);
try {
// Read subscription info from the storage
const item: Item = container.item(subscriptionId, subscriptionId);
const { resource: readItem } = await item.read();
context.log("Item from read: ");
context.log(readItem);
// Check the result, data stored in the container is formatted as
if (readItem.id) {
// UPDATE ITEM here
try {
// Try to write the update to the database
const { resource: updatedItem } = await container
.item(id, id)
.replace(patchedItem);
// Give success response
context.log("Operation success. Updated Item: ");
context.log(updatedItem);
context.res = {
status: 200, /* Return error status */
body: "Operation success."
};
resolve();
} catch (error) {
context.res = {
status: 500, /* Return error status */
body: "Operation failure: Item replace failed."
};
reject(error);
}
}
} catch (error) {
context.log("Internal error: failure on accessing Database");
context.log(error);
context.res = {
status: 500, /* Return error status */
body: "Internal error: failure on accessing Database"
};
reject(error);
}
});
}
On the logger, I would get these messages printed:
2021-02-05T08:14:57.938 [Information] {THE ITEM FROM DATABASE}
2021-02-05T08:14:58.118 [Information] Item from read:
2021-02-05T08:14:58.118 [Information] undefined
2021-02-05T08:14:58.118 [Information] Internal error: failure on accessing Database
2021-02-05T08:14:58.118 [Information] TypeError: Cannot read property 'id' of undefinedat Object.<anonymous> (D:\home\site\wwwroot\dist\SaasWebhook\index.js:43:26)at Generator.next (<anonymous>)at fulfilled (D:\home\site\wwwroot\dist\SaasWebhook\index.js:5:58)at processTicksAndRejections (internal/process/task_queues.js:97:5)
2021-02-05T08:14:58.120 [Error] Executed 'Functions.SaasWebhook' (Failed, Id=ec15bf1b-d9d5-4ebc-900f-73cbc0976b41, Duration=188ms)Result: FailureException: TypeError: Cannot read property 'id' of undefinedStack: TypeError: Cannot read property 'id' of undefinedat Object.<anonymous> (D:\home\site\wwwroot\dist\SaasWebhook\index.js:43:26)at Generator.next (<anonymous>)at fulfilled (D:\home\site\wwwroot\dist\SaasWebhook\index.js:5:58)at processTicksAndRejections (internal/process/task_queues.js:97:5)
For context, this function is used as a webhook required by Microsoft marketplace SaaS fulfillment API, so the function is called again for a certain time if the call failed, the {THE ITEM FROM DATABASE} log was probably from a previous call of the function.
The function is written in typescript and uploaded/deployed to azure function through Azure Function extension on Visual Studio Code. I have read that Typescript function actually got compiled into a javascript function prior to deployment, and I noticed that on the Javascript function the 'await' call was changed to 'yield', is this somehow related? And here's part of the js code that I meant:
try {
// Read subscription info from the storage
const item = container.item(id, id);
const { resource: readItem } = yield item.read();
context.log("Item from read: ");
context.log(readItem);
...
// The rest doesn't have much difference.
...
I've also tried using item.read().then(...) instead of await and got the same result. I'm thinking to rewrite the function in plain Javascript, but I would like any of your opinions first.

In the end, I rewrite the function in javascript and it worked as intended. My guess was this is due to the function's compilation to javascript, the change from 'await' to 'yield' messed with the flow of the function.

NodeJS - How to deal with UnhandledPromiseRejectionWarning?

I am fairly new to Node JS and so I am struggling a bit.
I am trying to read files for google drive using their API, from my Node Js code.
I am having the following code
router.get('/', function (req, res, next) {
var pageToken = null;
// Using the NPM module 'async'
async.doWhilst(function (callback) {
drive.files.list({
q: "mimeType='image/jpeg'",
fields: 'nextPageToken, files(id, name)',
spaces: 'drive',
pageToken: pageToken
}, function (err, res) {
if (err) {
// Handle error
console.error(err);
callback(err)
} else {
res.files.forEach(function (file) {
console.log('Found file: ', file.name, file.id);
});
pageToken = res.nextPageToken;
callback();
}
});
}, function () {
return !!pageToken;
}, function (err) {
if (err) {
// Handle error
console.error(err);
} else {
// All pages fetched
}
})
res.render('index', { title: 'Express' });
});
The above code is giving me the following error when i send the get request
(node:13884) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'forEach' of undefined
at E:\nodejs\newcelebapi\routes\index.js:49:17
at E:\nodejs\newcelebapi\node_modules\googleapis-common\build\src\apirequest.js:43:53
at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:13884) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:13884) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
The issue is in the following line
res.files.forEach(function (file) {
I've tried everything I could and gave up understanding the problem.
Could you please help me with this?
Thanks!

Per this example in the doc, you want to be using:
res.data.files.forEach(...)
not:
res.files.forEach(...)
And, it looks like as soon as you get by that problem, you will have another problem because you are calling res.render() in the wrong place. And, when you fix that, you will have an issue with redefining res in your Google callback which will hide the higher level res that you need for res.render().
I would strongly recommend that you not use the async library here. It doesn't seem like it's needed here and it just complicates things. And, if you did need help doing coordination of asynchronous operations, promises is the modern way to do so.
You don't show what you're trying to do with the resulting files (other than logging them), but here's a simple implementation that does that:
router.get('/', function (req, res, next) {
var pageToken = null;
drive.files.list({
q: "mimeType='image/jpeg'",
fields: 'nextPageToken, files(id, name)',
spaces: 'drive',
pageToken: pageToken
}).then(response => {
let files = response.data.files;
files.forEach(file => console.log('Found file: ', file.name, file.id))
res.render('index', { title: 'Express' });
}).catch(err => {
console.log(err);
res.sendStatus(500);
});
});
Note, that I named the parameter from drive.files.list() to be named response instead of res so I can access both res from the router.get() callback and response from the drive.files.list() callback. You gave them both the same name res which means you can't access the router.get() parameter of the same name.

Mocha give too long error message when testing node.js

I am learning node.js and how to test functions. I have a problem when using mocha: when functions are passing test, everything is completely fine, I get a nice looking message.
But if whichever function which doesnt pass a test - for example the result in the test is 0 but intentionally I wrote the asswertion to expect 1 - it gives me a mile long error massage in the bash-cli-console:
Async functions
(node:6001) UnhandledPromiseRejectionWarning: AssertionError [ERR_ASSERTION]: 0 == 1
at utils.requestWikiPage.then.resBody (/home/sandor/Documents/learning-curve-master/node-dev-course/testing-tut/utils/utils.test.js:10:20)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:188:7)
(node:6001) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:6001) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
1) it should return a html page
0 passing (2s)
1 failing
1) Async functions
it should return a html page:
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/sandor/Documents/learning-curve-master/node-dev-course/testing-tut/utils/utils.test.js)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! dev-course#1.0.0 test: `mocha ./testing-tut/**/*.test.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the dev-course#1.0.0 test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/sandor/.npm/_logs/2018-07-04T11_31_53_292Z-debug.log
[nodemon] app crashed - waiting for file changes before starting...
I dont know why do I get this part: UnhandledPromiseRejectionWarning...
and why do I get this part: npm ERR! code ELIFECYCLE
the function I am testing: (it makes a request to wikipedia for the wiki page of george washington and collects the html page from the response. On 'end' of the responses readstream it resolves the html page. The function works just fine)
// utils.js
function requestWikiPage() {
const reqOpts = {
hostname : 'en.wikipedia.org',
port : 443,
path : '/wiki/George_Washington',
method : "GET"
}
return new Promise(resolve => {
let req = https.request(reqOpts, (res) => {
let resBody = "";
res.setEncoding('utf-8');
res.on('data', (chunk) => {
resBody += chunk;
});
res.on('end', () => {
resolve(resBody);
});
});
req.on('err', (err) => {
console.log(err);
});
req.end();
});
}
module.exports.requestWikiPage = requestWikiPage;
Mocha code: (The 'resBody' variable is a string, containing a html page, where '' stays on the index of 0. In the assertion I test it for 1 to create an error message)
const utils = require('./utils');
var assert = require('assert');
describe('Async functions', function() {
it('it should return a html page', (done) => {
utils.requestWikiPage().then(resBody => {
assert.equal(resBody.indexOf('<!DOCTYPE html>'), 1);
done();
});
});
});
So I dont understand why do I get that long error message just because I expect to be not on the 0 index than on the first? (Actually I get that error message whith every function not just with this)
How can I set up mocha that it gives me a more minimal and intuitive error message.
Thanks a million for your answers

You need to properly reject the promise in #requestWikiPage if it doesn't resolve or there is an error, and then handle that rejection in your test. The following changes will likely solve the issue in your question (i.e. having mocha correctly handle a failed test without all the extra output), but the next step would obviously be getting your test to pass.
Notice we add the reject callback to our new Promise() and instead of console.log(err); below in your req.on('error'... callback, we now use reject as our error callback.
// utils.js
function requestWikiPage() {
const reqOpts = {
hostname : 'en.wikipedia.org',
port : 443,
path : '/wiki/George_Washington',
method : "GET"
}
return new Promise((resolve, reject) => {
let req = https.request(reqOpts, (res) => {
let resBody = "";
res.setEncoding('utf-8');
res.on('data', (chunk) => {
resBody += chunk;
});
res.on('end', () => {
resolve(resBody);
});
});
req.on('err', reject);
req.end();
});
}
module.exports.requestWikiPage = requestWikiPage;
And now handle if the promise is rejected via a catch block by using done as the catch callback (which would effectively pass the error to done which mocha requires).
const utils = require('./utils');
var assert = require('assert');
describe('Async functions', function() {
it('it should return a html page', (done) => {
utils.requestWikiPage().then(resBody => {
assert.equal(resBody.indexOf('<!DOCTYPE html>'), 1);
done();
}).catch(done);
});
});

Express route hanging async await

I am converting my application to use async/await instead of callbacks for query request made on the backend. It's been going good so far, but I am on a bit of a snag. My page is hanging on getting a get route and not rendering the ejs page. The console from the server also displays,
(node:7036)
UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): ReferenceError: qryFindLocalID is not defined
(node:7036) .
[DEP0018] DeprecationWarning: Unhandled promise rejections are depreca
ted. In the future, promise rejections that are not handled will terminate the Nod
e.js process with a non-zero exit code.
Any help would be appreciated.
The code so far is,
router.get("/secure", async (req, res) => {
let user = req.session.passport.user;
try {
if (user.chkUserStatus) {
if (user.lWaterLabID == 0 || user.lWaterlabID == -9999) {
// in both cases sLabID is set to 0
// assign the correct value for sLabName
user.sLabName = user.lWaterLabID == 0 ? "Site Admin" : "Uber Admin";
} else {
let labName = await request.query(
"Some Query"
);
user.sLabName = labName[0].sLabName;
}
} else {
// Houston we've got a problem ...
// user.chkUserStatus is not truthy
req.flash("loginMessage", "User Status is invalid.");
res.redirect("/");
}
const qryFindLocalID = await request.query(
`Some Query`
);
if (user.lWaterLabID == 0) {
const listRecentReports = await request.query(Some Query);
} else {
const listRecentReports = await request.query(Some Query);
}
} catch (err) {
// ... error checks
}
res.render("secure/index", {
qryFindLocalID: qryFindLocalID,
user: user,
listRecentReports: listRecentReports
});
});

The error message talks about an unhandled promise, but that's just wrapping the actual error, which is: ReferenceError: qryFindLocalID is not defined.
So where are you using qryFindLocalID? Ah, right at the bottom in the res.render call.
res.render("secure/index", {
qryFindLocalID: qryFindLocalID,
user: user,
listRecentReports: listRecentReports
});
Now why is qryFindLocalID undefined here? You defined it above in the try-catch block. But there's your problem -- You used const, so qryFindLocalID only exists in the try-catch. Outside of that, it doesn't exist.
You can fix that by using var in the try-catch (var is scoped to the function), or define qryFindLocalID using let ABOVE the try-catch.