We Keep Coding

Output Iterator in SuiteScript Map/Reduce Summarize Function

I'm working on a map/reduce script to handle some automated billing processing. I run a search for invoices in the GetInput stage, group them by customer in the Map stage, and then create the payments in the Reduce stage. However, in the Summarize stage, only one key/value pair ever exists. So, I created a dummy test script to play with the functionality and figure it out, and kept running into the same problem.
Here's what I have for the testing script:
define(['N/search'], (search) => {
const getInputData = (inputContext) => {
let filt = [
["trandate","on","3/29/2022"]
, "and"
, ["mainline","is","T"]
];
let cols = [
search.createColumn({ name : "tranid" })
, search.createColumn({ name : "entity" })
, search.createColumn({ name : "total" })
];
let results;
try {
// custom search wrapper
results = getSearchResults("invoice", filt, cols);
}
catch (err) {
log.error({ title : "Error encountered retrieving invoices" , details : err });
}
return results;
}
const map = (mapContext) => {
try {
let data = JSON.parse(mapContext.value);
let output = { key : "" , value : data };
let rand = randomInt(0, 1); // custom random number generator
if (rand === 0) {
output.key = "FAILURE";
}
else {
output.key = "SUCCESS";
}
mapContext.write(output);
}
catch (err) {
log.error({ title : "Map Stage Error" , details : err });
}
}
const reduce = (reduceContext) => {
reduceContext.write({
key : reduceContext.key
, value : JSON.stringify(reduceContext.values)
});
}
const summarize = (summaryContext) => {
summaryContext.output.iterator().each((key, value) => {
log.audit({ title : `summary -- ${key} -- ${typeof value}` , details : value });
});
}
return {getInputData, map, reduce, summarize}
});
By all accounts, the summary logging should have two key entries to report, but only ever has one. In the test I've tried it both by marking values as either SUCCESS or FAILURE in the Map stage and then just passing it through the Reduce stage, and also by passing the values through the Map stage to be marked as SUCCESS or FAILURE in the Reduce stage and then passed on with the invoice record ID as the key. No matter what, the output iterator in the Summarize stage only ever reports back a single key. I've had this work correctly in one particular situation, but for the life of me I can't figure out what's different/wrong.
Any insights? Otherwise the only way I can think of to be able to propagate necessary data is to utilize the 'N/cache' module, which does work pretty well but feels like it should be unnecessary.

My understanding is that you need to return true; from the each() callback function, or it will stop processing.
const summarize = (summaryContext) => {
summaryContext.output.iterator().each((key, value) => {
log.audit({ title : `summary -- ${key} -- ${typeof value}` , details : value });
return true;
});
}

I handle errors in MP script by this way
summarize: function summarize(summarizeContext) {
function handleErrorIfAny(summary)
{
var mapSummary = summary.mapSummary;
var reduceSummary = summary.reduceSummary;
handleErrorInStage('map', mapSummary);
handleErrorInStage('reduce', reduceSummary);
}
function handleErrorInStage(stage, summary)
{
summary.errors.iterator().each(function(key, value){
nLog.error('Failure in key ' + key, value);
return true;
});
}
handleErrorIfAny(summarizeContext);
}

Javascript async function flow

My function should assign an employee on a seat if the seat is available. I do not understand why the program doesn't act as synchronous even though I used "await".
In the first lines of the function, the program acts as expected. it waits to get "seats"from the database, then performs the "if(seats.length > 0)"
check and initialises an empty array.
async function AssignSeat(req, res) {
var seats = await connection.SeatEmployees.findAll({
where: {
SeatId: req.body.seat.SeatId
}
})
.catch(err => {
res.status(err.status).json(err)
});
if(seats.length > 0){
var isShared = true;
var employees = [];
await seats.forEach(async function(seat){
var employee = await connection.EmployeesGraph.findAll({
where: {
id: seat.EmployeeGraphId
}
})
.catch(err => {
res.status(err.status).json(err)
});
employees.push(employee);
})
.catch(err => {
res.status(err.status).json(err)
});
employees.forEach(employee => {
if(employee.frequent == true)
isShared = false;
})
if(isShared == true){
//assign user to seat;
}
}
}
My problem is at the 13th line of code, at " await seats.forEach(async function(seat)".
What I want to do is go through each element of "seats", Get the employee assigned to that seat, and push it into the "employees" array.
Only after iterating from all the seats and filling the employees array, I want proceed with the "employees.forEach(employee => {" line.
Instead, what happens is that after calling
-----"var employee = await connection.EmployeesGraph.findAll({ "---- ,the program doesn't wait for sequelize to get the employee from the database and then go to ----"employees.push(employee);"---- , as intended.
It goes to the paranthesis on the line after ----"employees.push(employee);"---- , then I get the error "TypeError: Cannot read property 'catch' of undefined".
Could you please explain why this happens?

The easiest solution is to use an actual for loop instead of forEach for this task. forEach() won't wait to iterate over everything.
try {
for (const seat of seats) {
var employee = await connection.EmployeesGraph.findAll({
where: {
id: seat.EmployeeGraphId
}
})
.catch(err => {
res.status(err.status).json(err)
});
employees.push(employee);
}
} catch (err) {
res.status(err.status).json(err)
}

How to query SQL in a loop in node js

SCENARIO:
Using mssql I'm connecting to sql and retrieving a list of ids, then based on those id I want to run stored procedures. What I'm currently doing is running the first stored proc, storing the id's in an array, then I'm running a for loop calling another module, where I pass the id to run a stored proc. This works fine when I've got a single id, but fails with 'Global connection already exists. Call sql.close() first.' when I try to run multiple ones.
How do I create connect to sql, run my query, then run the next one? What's the best approach?
The code below runs the stored proc with ids and causes the above error.
exports.runStoredProc = function (query,id) {
sql.connect(config.config).then(()=>{
return sql.query`${query} ${id}`
}).then(res=> {
do something with the response
}).catch(error => {
console.log(error)
})
}
Looks like the connection still exists when the below bit of code runs it using next id. I thought that creating a Promise will force to await execution before it runs the above bit of code again?
let toRun = result.recordset.length
let gen = async num => {
for(let i=0;i<num;i++) {
var resp = result.recordset[i].id
console.log(i, resp)
var sp = report
var reportId = await new Promise(() => db.runStoredProc(sp,resp))
}
}
gen(toRun).then(() => console.log("done!"))

You need to return Promise from runStoredProc
exports.runStoredProc = function (query,id) {
return sql.connect(config.config).then(()=>{
return sql.query`${query} ${id}`
}).then(res=> {
do something with the response
}).catch(error => {
console.log(error)
})
}
and no need to wrap db.runStoredProc in loop
let toRun = result.recordset.length
let gen = async num => {
for(let i=0;i<num;i++) {
var resp = result.recordset[i].id
console.log(i, resp)
var sp = report
var reportId = await db.runStoredProc(sp,resp)
}
}
gen(toRun).then(() => console.log("done!"))

callback failed to run after sqlite statment.all query

In the following function, I have attempted to run an statment.all query in sqlite3.
however the function ends without ever running the callback function.
I cannot pinpoint the problem even after a thorough debug and few experiments.
The strangest thing about this is that I have another similar query working perfectly fine within the code. It's included as well in the question.
here's the problematic code:
function gameSetup (questions_num){
var questions = [];
getQuestions.all(questions_num, function (e, rows) {
console.log("the callback is working!"); // this line never runs
if (e) {
return new Promise( (resolve, reject) => {
reject(e);
});
}
else{
var answers, question;
for (let row of rows)
{
answers = [row.correct_ans, row.ans2, row.ans3, row.ans4];
answers = shuffle(answers);
question = {"question":row.question, "ans1":answers[0] , "ans2":answers[1] , "ans3":answers[2] , "ans4":answers[3]};
questions.push(question);
console.log(questions);
}
}
return new Promise( (resolve, reject) => {
resolve(questions);
});
});}
this is the sql statement:var getQuestions = db.prepare("select * from t_questions order by random() limit ?");
and I have another similar code that works :
app.get("/rooms", checkAuth, (req, res) => {
getRooms.all((e, data) => {
if(e) {
return res.status(500).json(e);
}
else if (data.length == 0) {
res.render('rooms', {items: data, error: "no rooms found"});
}
else
{
res.render('rooms', {items: data, error: "false"});
}
});
});
with the following statement:
var getRooms = db.prepare("select * from t_games where start_time is null");
I might have given too much info, or too little. feedback on the question would be appreciated since this is my first question on the site.

I think calling random() in the statement might be where the problem is.
What happens if you try something like:
var getQuestions = db.prepare("select * from t_questions order by ? limit ?")
and
getQuestions.all([random(), questions_num], function (e, rows) { ...
Also, is random() a function you defined somewhere that returns the column to be ordered by? Or maybe Math.random()?

node.js resolve promise and return value

I use the Microsoft bot framework to come up with a "simple" PoC bot. I used a tutorial as a basis and extend it.
I've a couple of basic functions for differet intents (ie. greetings, goodbye, etc) and one with some more logic in it (reqstatus).
The simple ones (ie greeting.js) return the answer nicely but the more complex one doesn't (reqstatus.js). Running the main code of reqstatus.js (without the first "const getReqStatus = (entity) => {") in a standalone script works.
server.js (main) -> see call in "if (intent) {"...
const getFeelings = require('./intents/feelings.js')
const getGoodbyes = require('./intents/goodbyes.js')
const getGreetings = require('./intents/greetings.js')
const getHelp = require('./intents/help.js')
const getReqStatus = require('./intents/reqstatus.js')
...
const bot = new builder.UniversalBot(connector)
// Intents based on definitions on recast
const INTENTS = {
feelings: getFeelings,
goodbyes: getGoodbyes,
greetings: getGreetings,
help: getHelp,
reqstatus: getReqStatus,
}
// Event when Message received
bot.dialog('/', (session) => {
recastClient.textRequest(session.message.text)
.then(res => {
const intent = res.intent()
const entity = res.get('request_number')
console.log(`UserName: ${session.message.user.name}`)
console.log(`Msg: ${session.message.text}`)
console.log(`Intent: ${intent.slug}`)
if (intent) {
INTENTS[intent.slug](entity)
.then(res => session.send(res))
.catch(err => session.send(err))
}
})
.catch(() => session.send('Sorry I didn\'t get that. '))
})
...
greetings.js -> Returns the string ok
const getGreetings = () => {
const answers = ['Hi, my name is SuperBot. Nice to meet you!', ]
return Promise.resolve((answers))
}
module.exports = getGreetings
reqstatus.js -> Does not return anything
const getReqStatus = (entity) => {
var request = require('request');
var request_number = entity.toLowerCase()
var output = [];
// Processing
var lineReader = require('readline').createInterface({
input: fs.createReadStream('netreqs.csv')
});
lineReader.on('line', function (line) {
var jsonFromLine = {};
var lineSplit = line.split(';');
jsonFromLine.req = lineSplit[0];
jsonFromLine.req_count = lineSplit[1];
jsonFromLine.req_type = lineSplit[2];
//...
var req_lowever = jsonFromLine.req.toLowerCase()
if (req_lowever == request_number) {
output.push( `Your request ${jsonFromLine.req} was received`);
// simplified
}
});
// Output
lineReader.on('close', function (line) {
if (output == '') {
output.push( `I was not able to find a request like ${request_number}.`);
}
console.log(output); // list output
return Promise.resolve(output);
});
}
module.exports = getReqStatus
I also tried to put getReqStatus in a function but that also didn't work.
After a lot of trying and googling I'm still stuck and wanted to ask the experts here. Thanks a lot in advance.

I think that the problem is that your getReqStatus isn't really returning anything. In your example getGreetings function you're actually returning Promise.resolve(answers) as the return value of that function.
However, in your getReqStatus function, you just set up a listener lineReader close event:
lineReader.on('close', function (line) {
if (output == '') {
output.push( `I was not able to find a request like ${request_number}.`);
}
console.log(output); // list output
return Promise.resolve(output);
});
You're returning a Promise resolved inside the anonymous callback function you're passing to lineReader.on() as second parameter. That is not the return value from the getReqStatus function itself, so that getReqStatus is not returning anything, as expected.
The code of that function runs correctly as standalone code, as you say, just because it sets the listener properly and it does what it has to do. However, that code just doesn't return a Promise when wrapped in a function.
What you would need is to return a Promise that wraps the lineReader.on close handler, like:
function getReqStatus(){
//...code
return new Promise( function(resolve , reject ){
lineReader.on('close', function (line) {
if (output == '') {
output.push( `I was not able to find a request like ${request_number}.`);
}
console.log(output); // list output
return resolve(output);
});
});
}
I say would because I really don't know if this code will work, I don't have any kind of experience with the Microsoft Bot framework and not used at all with the readline module. However, even if this doesn't solve your problem, I hope it will help you a bit understanding why your function doesn't return a Promise and how could you fix it.