We Keep Coding

Loop to create or update nodes in Neo4j

I am new to Neo4j so I quite stuck with looping through some values.
I have a list of skill to skill strings
let data = [
'big_data, business_intelligence',
'big_data, data_collection',
'big_data, economic_growth',
'big_data, economy'
]
And I want to create or update the relation between left side with right side
for (let item of data) {
CreateSkillToSkillRelation(item);
}
const CreateSkillToSkillRelation = async (relation) => {
let mainSkill = relation.split(",")[0];
let secundarySkill = relation.split(",")[1];
try {
// Check if the relationship exists
let { records } = await session.run(
"MATCH(main:SKILL {name:$mainSkill}) -[relation:SKILL_TO_SKILL]-> (secundary:SKILL {name:$secundarySkill}) RETURN relation",
{ mainSkill, secundarySkill }
);
let count =
records[0]?._fields[0].properties.count > 0
? records[0]._fields[0].properties.count + 1
: 1;
// If count is greater then 1 then lets update the counter
if (count > 1) {
await session.run(
"MATCH(main:SKILL {name:$mainSkill}) -[relation:SKILL_TO_SKILL]-> (secundary:SKILL {name:$secundarySkill}) SET relation.count = $count RETURN main, secundary",
{
mainSkill,
secundarySkill,
count,
}
);
}
// Otherwise the skill relation is not created so lets create one
else {
await session.run(
"CREATE(main:SKILL {name:$mainSkill}) -[:SKILL_TO_SKILL {count:$count}]-> (secundary:SKILL {name:$secundarySkill}) RETURN main, secundary",
{
mainSkill,
secundarySkill,
count,
}
);
}
await session.close();
} catch (error) {
console.log(error);
}
};
But every time when I run this I get the following error Neo4jError: Queries cannot be run directly on a session with an open transaction; either run from within the transaction or use a different session.
Any idea how can I solve this?

for (let item of data) {
CreateSkillToSkillRelation(item);
}
Is not awaiting the promises you create and so you are basically trying to run all of these promises concurrently against a single session which only supports a single concurrent transaction.
You should create a session in each call of CreateSkillToSkillRelation or await each call to it using a single session.
Though note you close the session at the end of CreateSkillToSkillRelation but only on success, might I suggest moving await session.close(); into a finally block.

The answer of the colleague #just_another_dotnet_dev is absolutely correct: you run asynchronous functions in a loop, and close the session in one of them.
The Cipher language is very rich, and you can use it to do everything that you tried to do with a loop in Javascript. Something like this, using UNWIND and MERGE:
const CreateSkillToSkillRelations = async (data) => {
const session = driver.session();
try {
let { records } = await session.run(
`WITH split(row, ',') as rels
WITH trim(rels[0]) as mainSkill,
trim(rels[1]) as secundarySkill
MERGE (main:SKILL {name: mainSkill})
-[relation:SKILL_TO_SKILL]->
(secundary:SKILL {name: secundarySkill})
ON CREATE SET relation.count = 1
ON MATCH SET relation.count = relation.count + 1
RETURN main, relation, secundary`,
{ data }
);
} catch (error) {
console.log(error);
} finally {
await session.close()
}
};

Await Async when looping through large json file

The json file is large around 20mb.
I want to wait until a result is returned or the entire file is looped through, before sending back the age. Currently it returns 0 even if the age is not 0
const app = express()
const genesis = require('./people.json');
app.get('/', function (req, res) {
res.setHeader('Content-Type', 'application/json');
let age = getAge(req.query.name)
res.json({
“name”: req.query.name,
“age”: age, // this is always 0
});
});
function getAge(name) {
genesis.balances.forEach(element => {
if (element.name == name) {
// console here shows correct age
return element.person[0].age;
}
});
return 0;
}
app.listen(3000)

As I said in the comment, the problem was in your getAge method, it was always returning 0.
The return inside the forEach doesn't return the value off of the loop.
Please have a look at the following approach
function getAge(name) {
const person = genesis.balances.find((elm)=> elm.name === name);
return person ? person.age : 0;
}

See code comment below
function getAge(name) {
genesis.balances.forEach(element => { // forEach doesn't return anything
if (element.name == name) {
// console here shows correct age
return element.person[0].age;
}
});
return 0;
}
You probably want instead something like:
function getAge(name) {
const res = genesis.balances.filter(element => element.name == name);
if (res.length === 0) return 0; // not found
return res[0].person[0].age;
}
read more about forEach
Comment: having a person-array under element with "name" is a weird choice, why should a single person-name be mapped to multiple persons?

NodeJS processing for API calls sequentially, but resulting in stack overflow

OK, so I have a situation where I cannot just fire thousands of requests to an API server.
I have a Node process (no UI) that I need to have process each API response/update sequentially, waiting for completion before sending the next request.
I may be making this more complicated than I think - not sure. I can only figure out how to do this with recursive calls, but this results in a stack overflow as there can be thousands of records. The general process is this:
get rows from SQL table with ID's (result)
formulate and send of an API call to retrieve ID's info
if returned data has image data, write it back to SQL table
wait on this process so not to bombard API server with thousands of requests all at once
repeat until last ID is processed (can be thousands, more than stack space)
Here's sample code (not actual so ignore syntax errors if any)...
UPDATED: actual running code with sensitive items removed
var g_con = null; //...yeah I know, globals are bad
//
// [ found updating ]
//
function getSetImage(result, row, found) {
if(row >= result.length) { //...exit on no row or last row processed
con.end();
return;
}
item = result[row]; //...next SQL row
if((item !== undefined) && (item.autoid !== undefined)) {
//...assemble API and send request
//
let url = 'https://...API header...'
+ item.autoid
+ '...API params...';
request(url, (error, response, body) => {
if(response.statusCode !== 200)
throw('Server is not responding\n' + response.statusMessage);
let imageData = JSON.parse(body);
if((imageData.value[0] !== undefined) &&
(imageData.value[0].DETAIL !== undefined) &&
(imageData.value[0].DETAIL.Value.length) ) {
//...post back to SQL
//
found++;
console.log('\n' + item.autoid + '/['+ item.descr + '], ' + 'Found:' + found);
qry = 'update inventory set image = "'+imageData.value[0].DETAIL.Value+'" where autoid = "'+item.autoid+'";';
g_con.query(qry, (err) => {
if (err) {
console.log('ERROR:',err.message, '\nSQL:['+err.sql+']\n');
throw err.message;
}
});
row++;
setTimeout(()=>{getSetImage(result, row, found)}, 0); //...nested call after SQL
} else {
row++;
process.stdout.write('.'); //...show '.' for record, but no image
setTimeout(()=>{getSetImage(result, row, found)}, 0); //...nested call after SQL
}
}); //...request callback
}
// } else {
// throw '\nERROR! result['+row+'] undefined? Images found: '+found;
// }
}
//
// [ main lines ]
//
(() => {
let params = null;
try {
params = JSON.parse(fs.readFileSync('./config.json'));
//...load autoids array from SQL inventory table - saving autoids
// autoids in INVENTRY join on par_aid's in INVENTRYIMAGES
//
g_con = mysql.createConnection(params.SQLConnection);
g_con.connect((err) => { if(err) {
console.log('ERROR:',err.message);
throw err.message;
}
});
//...do requested query and return data or an error
//
let qry = 'select autoid, descr from inventory order by autoid;';
g_con.query(qry, (err, results, flds) => {
if (err || flds === undefined) {
console.log('ERROR:',err.message, '\nSQL:['+err.sql+']\n');
throw err.message;
}
console.log('Results length:',results.length);
let row = 0;
let found = 0;
getSetImage(results, row, found);
});
}
catch (err) {
console.log('Error parsing config parameters!');
console.log(err);
}
})();
So here's the answer using Promises (except for MySQL):
//
// [ found updating ]
//
async function getSetImage(data) {
for(let item of data) {
if(item && item.autoid) {
//...assemble API and send request
//
let url = g_URLHeader + g_URLPartA + item.autoid + g_URLPartB;
let image = await got(url).json().catch(err => {
console.log(err);
err.message = 'API server is not responding';
throw err;
});
if(image && image.value[0] && image.value[0].DETAIL &&
image.value[0].DETAIL.Value.length ) {
console.log('\nFound: ['+item.autoid+' - '+item.descr
+ '] a total of ' + g_found + ' in ' + g_count + ' rows');
g_found++;
//...post back to SQL
//
let qry = 'update inventory set image = "'
+ image.value[0].DETAIL.Value
+ '" where autoid = "'
+ item.autoid+'";';
await g_con.query(qry, (err) => {
if (err) {
console.log('ERROR:',err.message, '\nSQL:['+err.sql+']\n');
throw err.message;
}
});
} else {
process.stdout.write('.'); //...show '.' for record, but no image
} //...if/else image.value
g_count++;
} //...if item
} //...for()
}

As I've said in all my comments, this would be a ton simpler using promises and async/await. To do that, you need to switch all your asynchronous operations over to equivalents that use promises.
Here's a general outline based on the original pseudo-code you posted:
// use got() for promise version of request
const got = require('got');
// use require("mysql2/promise" for promise version of mysql
async function getSetImage(data) {
for (let item of data) {
if (item && item.id) {
let url = uriHeader + uriPartA + item.id + uriPartB;
let image = await got(url).json().catch(err => {
// log and modify error, then rethrow
console.log(err);
err.msg = 'API Server is not responding\n';
throw err;
});
if (image.value && image.value.length) {
console.log('\nFound image for ' + item.id + '\n');
let qry = 'update inventory set image = "' + image.value + '" where id = "' + item.id + '";';
await con.query(qry).catch(err => {
console.log('ERROR:', err.message, '\nSQL:[' + err.sql + ']\n');
throw err;
});
}
} else {
// no image data found
process.stdout.write('.'); //...show '.' for record, but no image
}
}
}
//...sql query is done, returning "result" - data rows
getSetImage(result).then(() => {
console.log("all done");
}).catch(err => {
console.log(err);
});
Some notes about this code:
The request() library is no longer getting new features and is in maintenance mode and you need to change to a different library to get built-in promise support. You could use request-promise (also in maintenance mode), but I recommend one of the newer libraries such as got() that is more actively being developed. It has some nice features (automatically checks status for you to be 2xx, built-in JSON parsing, etc...) which I've used above to save code.
mysql2/promise has built-in promise support which you get with const mysql = require('mysql2/promise');. I'd recommend you switch to it.
Because of the user of async/await here, you can just loop through your data in a regular for loop. And, no recursion required. And, no stack build-up.
The way promises work by default, any rejected promises will automatically terminate the flow here. The only reason I'm using .catch() in a couple places is just for custom logging and tweaking of the error object. I then rethrow which propagates the error back to the caller for you.
You can tweak the error handling to your desire. The usual convention with promises is to throw an Error object (not a string) and that's often what callers are expecting to see if the promise rejects.
This code can be easily customized to log errors and continue on to subsequent items in the array. Your original code did not appear to do that so I wrote it to abort if it got an error.

Is it invalid to write re.send() function inside mydb.collection().find().toArray callback?

I'am setting up a login page for my app. I want to send a file after verifing if the login page is provided with proper username and password.
I have a handler for a post request which checks if the user entered correct username and password.
app.post('/login',function(req,res){
var data="";
var flag_isthere=0,wrongpass=0;
console.log('login-done');
req.setEncoding('UTF-8')
req.on('data',function(chunk){
data+=chunk;
});
req.on('end',function()
{
MongoClient.connect("mongodb://localhost:27017/userdetails",{useNewUrlParser: true ,useUnifiedTopology: true },function(err,db)
{
if(err) throw err;
var q = JSON.parse(data)
const mydb=db.db('userdetails')
var c=mydb.collection('signup').find().toArray(
function(err,res)
{
for(var i=0;i<res.length;i++)
if( (res[i].email==q['email']) ) //check if the account exists
{
flag_isthere=1;
if( (res[i].pass != q['pass'] ) )
wrongpass=1;
break;
}
if(flag_isthere==0)
{
console.log(q['email'], ' is not registered')
}
else
{
console.log('Already exists!!!');
}
if( wrongpass==1)
{
console.log('password entered is wrong')
}
if(flag_isthere==1 && wrongpass==0)
{
console.log('Congratulations,username and password is correct');
res.send( { login:'OK', error:'' } ); //this statement is giving an error in node JS part
}
});//var c
})//mongoclient.connect
})//req.on
res.send({ login:'OK', error:'' }); //this works properly in node JS
console.log(flag_isthere , wrongpass ) //but here the flag_isthere==0 and wrongpass==0 , so it won't get validated
});
It gives the error as
TypeError: res.send is not a function
at E:\ITT_project_shiva\loginserver_new.js:112:25
at result (E:\ITT_project_shiva\node_modules\mongodb\lib\operations\execute_operation.js:75:17)
at executeCallback (E:\ITT_project_shiva\node_modules\mongodb\lib\operations\execute_operation.js:68:9)
at handleCallback (E:\ITT_project_shiva\node_modules\mongodb\lib\utils.js:129:55)
at cursor.close (E:\ITT_project_shiva\node_modules\mongodb\lib\operations\to_array.js:36:13)
at handleCallback (E:\ITT_project_shiva\node_modules\mongodb\lib\utils.js:129:55)
at completeClose (E:\ITT_project_shiva\node_modules\mongodb\lib\cursor.js:859:16)
at Cursor.close (E:\ITT_project_shiva\node_modules\mongodb\lib\cursor.js:878:12)
at cursor._next (E:\ITT_project_shiva\node_modules\mongodb\lib\operations\to_array.js:35:25)
at handleCallback (E:\ITT_project_shiva\node_modules\mongodb\lib\core\cursor.js:32:5)
[nodemon] app crashed - waiting for file changes before starting...
How do I send the response to the user after proper validation?

It's not that you're doing it from the callback that's the problem. There are two different problems:
You're shadowing res by redefining it in the callback's parameter list
(Once you fix that) You're calling res.send twice:
Once at the end of your posthandler
Once within the callback
send implicitly completes the response, so you can only call it once.
In your case, you want to call it from within your callback, once you've determined that none of the records matches.
See *** comments for a rough guideline (but keep reading):
app.post('/login', function(req, res) {
var data = "";
var flag_isthere = 0,
wrongpass = 0;
console.log('login-done');
req.setEncoding('UTF-8')
req.on('data', function(chunk) {
data += chunk;
});
req.on('end', function() {
MongoClient.connect("mongodb://localhost:27017/userdetails", {
useNewUrlParser: true,
useUnifiedTopology: true
}, function(err, db) {
if (err) throw err;
var q = JSON.parse(data)
const mydb = db.db('userdetails')
var c = mydb.collection('signup').find().toArray(
function(err, array) { // *** Renamed `res` to `array
for (var i = 0; i < array.length; i++)
if ((array[i].email == q['email'])) //check if the account exists
{
flag_isthere = 1;
if ((array[i].pass != q['pass']))
wrongpass = 1;
break;
}
if (flag_isthere == 0) {
console.log(q['email'], ' is not registered')
} else {
console.log('Already exists!!!');
}
// *** Handle result here
if (flag_isthere == 1 && wrongpass == 0) {
console.log('Congratulations,username and password is correct');
res.send({ login: 'OK', error: '' }); //this statement is giving an error in node JS part
} else if (wrongpass == 1) {
console.log('password entered is wrong')
// *** res.send(/*...*/)
} else {
// Handle the issue that there was no match
// *** res.send(/*...*/)
}
}
); //var c
}) //mongoclient.connect
}) //req.on
// *** Don't try to send a response here, you don't know the answer yet
});
but, it seems like you should be able to find just the one user (via findOne? I don't do MongoDB), rather than finding all of them and then looping through the resulting array.
See also the answers to these two questions, which may help you with asynchronous code issues:
How do I return the response from an asynchronous call?
Why is my variable unaltered after I modify it inside of a function?
A couple of other notes:
I strongly recommend using booleans for flags, not numbers.
NEVER store actual passwords in your database!! Store a strong hash, and then compare hashes.
You might find async/await syntax more convenient to work with. I think recent MongoDB clients support promises (which you need for async/await).

How to make sync call in forEach loop Angular 6

I am trying to check my all 4 images is uploaded to server without any error, then redirect to another page so i am trying to perform some sync checking in my code (I have total 4 images in my imgResultAfterCompress array). below is my code:
if(Boolean(this.updateImage(data.AddId))===true)
{
this.router.navigate(['/job-in-hotels-india-abroad']);
}
updateImage(AddId:number):Observable<boolean>
{
this.cnt=0;
this.uploadingMsg='Uploading Images...';
this.imgResultAfterCompress.forEach( (value, key) => {
if(value!=='')
{
this.itemService.updateImage(this.employer.ID,AddId,key,value).subscribe(data=>{
if(data && data.status == 'success') {
this.uploadingMsg=this.uploadingMsg+'<br>Image No - '+(key+1)+' Uploaded.';
this.cnt++;
}
else
this.alertService.error(data.message);
});
}
if(this.cnt==4)
this.uploadingDone= true;
else
this.uploadingDone= false
});
return this.uploadingDone;
}
Every time i am getting cnt value is 0, i want its value = 4 (completely uploaded all images) then redirection will occurred.

The easier way is to wrap your observables into a single one, using zip operator
https://rxjs-dev.firebaseapp.com/api/index/function/zip
Thus once every request is finished successfully your zipped Observable will be fulfilled.
UPDATE:
This is how I think it should look like. I could miss something specific, but the global idea should be clear
redirect() {
this.updateImages(data.AddId).subscribe(
() => this.router.navigate(['/job-in-hotels-india-abroad']),
error => this.alertService.error(error.message)
)
}
updateImages(AddId: number): Observable<boolean[]> {
this.uploadingMsg = 'Uploading Images...';
const requests: Observable<boolean>[] = [];
this.imgResultAfterCompress.forEach((value, key) => {
if (!value) {
return;
}
requests.push(
this.itemService.updateImage(this.employer.ID, AddId, key, value)
.pipe(
tap(() => this.uploadingMsg = this.uploadingMsg + '<br>Image No - ' + (key + 1) + ' Uploaded.'),
switchMap((data) => {
if (data && data.status == 'success') {
return of(true)
} else {
throwError(new Error('Failed to upload image'));
}
})
)
)
});
return zip(...requests);
}

Finally got the desire result by using forkJoin
Service.ts:
public requestDataFromMultipleSources(EmpId: number,AddId:number,myFiles:any): Observable<any[]> {
let response: any[] = [];
myFile.forEach(( value, key ) => {
response.push(this.http.post<any>(this.baseUrl + 'furniture.php', {EmpId: EmpId, AddId:AddId,ImgIndex:key,option: 'updateAdImg', myFile:value}));
});
// Observable.forkJoin (RxJS 5) changes to just forkJoin() in RxJS 6
return forkJoin(response);
}
my.component.ts
let resCnt=0;
this.itemService.requestDataFromMultipleSources(this.employer.ID,AddId,this.imgResultAfterCompress).subscribe(responseList => {
responseList.forEach( value => {
if(value.status=='success')
{
resCnt++;
this.uploadingMsg=this.uploadingMsg+'<br>Image No - '+(value.ImgIndex+1)+' Uploaded.';
}
else
this.uploadingMsg=this.uploadingMsg+'<br>Problem In Uploading Image No - '+(value.ImgIndex+1)+', Please choose another one.';
});
if(resCnt === this.imgResultAfterCompress.length)
{
this.alertService.success('Add Posted Successfully');
this.router.navigate(['/job-in-hotels-india-abroad']);
}
else
this.alertService.error('Problem In Uploading Your Images');
});

You shouldn't try to make sync call within a loop. It is possible using async/await, but it's bad for app performance, and it is a common anti-pattern.
Look into Promise.all(). You could wrap each call into promise and redirect when all promises are resolved.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all