I know that node.js is asynchronous, but what I don't really understand is how much.
Example:
If a need to do 3 things in sequence, and I need every thing is done before the other begins, Do i must need to use Callback?
var myContainer;
for(var i=0;i<10000;i++)
myContainer.push(i.toString());
for(var j=0;j<myContainer.length;j++)
console.log(myContainer[j]);
for(var x=0;x<myContainer.length;x++)
myModuleForEmails.sendEmailsTo(myContainer[x]);
Ok, suppose for one moment I have a module like Imap ready and calling myModuleForEmails.sendEmailsTo(myContainer[x]) I really send the email.
I know also that this program is absolutely useless, but it's just to understand.
Suppose I must push all 10000 string in myContainer, only THEN log all the string that are in myContainer in the console, and only AFTER BOTH I need to send the emails.
Is this version reliable or do I need 2 callback? And Does the number of iteration I do matter? i.e, if i had used 10 instead of 10000, could I have used this syntax because it takes so few to do 10 operation that finishes the first for cycle before starting the second?
Related
I'm working on a 2019 Advent of Code challenge (day 7), so bonus if you've done it before. Basically part of the challenge requires you to be running 5 "computers" which are processing a made-up programming language. Each computer gives an output that needs to be passed to the next computer to process, and it'll keep doing this in a cycle until a certain stop condition is met.
I have made a class that generates a "computer" and have 5 instances of it running concurrently. Each instance needs to await receiving output from the the computer it's connected to. Ignore what it's being assigned to, but here's the code for the relevant line:
this.code[this.code[i + 1]] = await ampA.output()
Here's the output method if you're curious:
output(output, value1) {
output = value1
return output
}
The problem I'm having is that this is calling ampA.output(), so basically it's calling the method without giving it any arguments, which is not correct. What I want it to do is wait for ampA to arrive at it's next output, and set the variable equal to that output. Is there a way to do this? Am I even in the ballpark of doing this correct? I'm relatively new to coding, and async/await is a bit out of my depth, so it's possible this is a very incorrect approach. Any help is appreciated, and let me know if more clarification about the situation is needed!
I'm making a function on a node.js server which reads a CSV file, I need to read all lines and execute several promised operations (MySQL queries) for each one (update or insert, then set an specified column which identifies this item as "modified in this execution") and once this finishes change another column on those not updated or inserted to identify this items as "deleted"
At first, the problem I had was that this CSV has millions of lines (literally) and a hundred of columns, so I run out of memory quite easily, and this number of lines can grow or decrease so I cannot know the amount of lines I will have to process every time I receive it.
I made a simple program that allows me to separate this CSV in some others with a readable amount of lines so my server can work with each one of them without dying, thus making an unknown amount of files each new file is processed, so now I have a different problem.
I want to read all of those CSVs, make those operations, and, once those operations are finished, execute the final one which will change those not updated/inserted. The only issue is that I need to read all of them and I cannot do this simultaneously, I have to make it sequentially, no matter how many they are (as said, after separating the main CSV, I may have 1 million lines divided into 3 files, or 2 millions into 6 files).
At first I though about using a forEach loop, but the problem is that, foreach doesn't respects the promisifying, so it will launch all of them, server will run out of memory when loading all those CSVs and then die. Honestly, using a while(boolean) on each iteration of the foreach to wait for the resolve of each promisified function seems pretty.... smelly for me, plus I feel like that solution will stop the server from working properly so I'm looking for a different solution.
Let me give you a quick explanation of what I want:
const arrayReader=(([arrayOfCSVs])=>{
initialFunction();
functions(arrayOfCSVs[0])
.then((result)=>{
functions(arrayOfCSVs[1])
.then((result2)=>{
functions(arrayOfCSVs[2])
(OnAndOnAndOnAndOn...)
.then((resultX)=>{
executeFinalFunction();
});
});
});
You can use Array.reduce to get the previous promise and queue new promise, without the need for waiting.
const arrayReader = ([arrayOfCSVs]) => {
initialFunction();
return arrayOfCSVs.reduce((prom, csv) => {
return prom.then(() => functions(csv));
}, Promise.resolve()).then(resultX => executeFinalFunction());
}
I am trying to adapt a script I already have to run using .csv data input. When the script is ran without the .csv, it runs perfectly for any configurations I choose to use. When it runs using the .csv, whatever scenario is in the first row will run perfect, but everything from there on will fail. The reason for the failure is because some of my variables are being reused from the first thread and I don't know how to stop this from happening.
This is what my script looks like:
HTTP Request - GET ${url} (url is declared in the CSV data input, and changes each run)
-> postprocessor that extracts Variable_1, Variable_2 and Variable_3
Sampler1
-> JSR223 preprocessor: creates payloadSampler1 using javascript, example:
var payloadSampler1 = { };
payloadSampler1.age = vars.get("Variable_2");
payloadSampler1.birthDate = "1980-01-01";
payloadSampler1.phone = {};
payloadSampler1.phone.number = "555-555-5555";
vars.put("payloadSampler1", JSON.stringify(payloadSampler1));
Sampler2
-> JSR223 preprocessor: creates payloadSampler1 using javascript (same as above but for different values)
Sampler3
-> JSR223 preprocessor: creates payloadSampler1 using javascript (same as above but for different values)
Sampler4
-> JSR223 preprocessor: creates payloadSampler1 using javascript (same as above but for different values)
HTTP Request - POST ${url}/${Variable_1}/submit
-> JSR223 preprocessor: creates payloadSubmit using javascript, and mix and matching the results from the above samplers - like so:
var payloadSubmit = { };
if (vars.get("someVar") != "value" && vars.get("someVar") != "value2" && vars.get("differentVar") != "true") {
payloadSubmit.ageInfo = [${payloadSampler1}];
}
if (vars.get("someVar2") != "true") {
payloadSubmit.paymentInfo = [${payloadSampler2}];
}
payloadSubmit.emailInfo = [${payloadSampler3"}];
payloadSubmit.country = vars.get("Variable_3");
vars.put("payloadSubmit", JSON.stringify(payloadSubmit));
-> BodyData as shown in the screenshot:
request
I have a Debug PostProcessor to see the values of all these variables I am creating. For the first iteration of my script, everything is perfect. For the second one, however, the Debug PostProcessor shows the values for all payloadSamplers and all the Variables correctly changed to match the new row data (from the csv), but, the final variable, payloadSubmit just reuses whatever the values where for the first thread iteration.
Example:
Debug PostProcessor at the end of first iteration shows:
Variable_1=ABC
Variable_2=DEF
Variable_3=GHI
payloadSampler1={"age":"18","email":null,"name":{"firstName":"Charles"}},{"age":"38","email":null}}
payloadSampler2={"paymentChoice":{"cardType":"CreditCard","cardSubType":"VI"}},"amount":"9.99","currency":"USD"}
payloadSampler3={"email":"tes#email.com"}
payloadSubmit={"ageInfo":[{"age":"18","email":null,"name":{"firstName":"Charles"}},{"age":"38","email":null}],"paymentInfo":[{"paymentChoice":{"cardType":"CreditCard","cardSubType":"VI"}},"amount":"9.99","currency":"USD"],"emailInfo":[{"email":"tes#email.com"}],"country":"GHI"}
But at the end of the 2nd iteration it shows:
Variable_1=123
Variable_2=456
Variable_3=789
payloadSampler1={"age":"95","email":null,"name":{"firstName":"Sam"}},{"age":"12","email":null}}
payloadSampler2={"paymentChoice":{"cardType":"CreditCard","cardSubType":"DC"}},"amount":"19.99","currency":"USD"}
payloadSampler3={"email":"tes2#email.com"}
payloadSubmit={"ageInfo":[{"age":"18","email":null,"name":{"firstName":"Charles"}},{"age":"38","email":null}],"paymentInfo":[{"paymentChoice":{"cardType":"CreditCard","cardSubType":"VI"}},"amount":"9.99","currency":"USD"],"emailInfo":[{"email":"tes#email.com"}],"country":"USA"}
I can also see that the final HTTP Request is indeed sending the old values.
My very limited understanding is that because I am invoking the variables like so "${payloadSampler1}" it will use the value that was set for that the first time the sampler was ran (back in the 1st thread iteration). These are the things I have tried:
If I use vars.get("payloadSubmit") on the body of an HTTP Sampler, I get an error, so that is not an option. If I use vars.get("payloadSampler1") on the Samplers that create the variables, extra escape characters are added, which breaks my JSON. I have tried adding a counter to the end of the variable name and having that counter increase on each thread iteration, but the results is the same. All the variables and samplers other than the last one have updated values, but the last one will always reuse the variables from the first thread iteration.
I also tried to use ${__javaScript(vars.get("payloadSubmit_"+vars.get("ThreadIteration")))}, but the results are always the same.
And I have also tried using the ${__counter(,)} element, but if I set it to TRUE, it will always be 1 for each thread iteration, and if I set it to FALSE, it starts at 2 (I am assuming it is because I use counter in another sampler within this thread - but even after removing that counter this still happens).
I am obviously doing something (or many things) wrong.
If anyone can spot what my mistakes are, I would really appreciate hearing your thoughts. Or even being pointed to some resource I can read for an approach I can use for this. My knowledge of both javascript and jmeter is not great, so I am always open to learn more and correct my mistakes.
Finally, thanks a lot for reading through this wall of text and trying to make sense of it.
It's hard to tell where exactly your problem is without seeing the values of these someVar and payload, most probably something cannot be parsed as a valid JSON therefore on 2nd iteration your last JSR223 PostProcessor fails to run till the end and as the result your payloadSubmit variable value doesn't get updated. Take a closer look at JMeter GUI, there is an yellow triangle with exclamation sign there which indicates the number of errors in your scripts. Also it opens JMeter Log Viewer on click
if there is a red number next to the triangle - obviously you have a problem and you will need to see the jmeter.log file for the details.
Since JMeter 3.1 it is recommended to use Groovy language for any form of scripting mainly due to the fact that Groovy has higher performance comparing to other scripting options. Check out Parsing and producing JSON guide to learn more on how to work with JSON data in Groovy.
The mongo shell defaults to safe writes, which from my understanding happens at the end of every carriage return. What if you have code in a loop like this:
db.coll1.find().forEach(function(doc){
db.coll2.update({"blah": doc._id}, {$set: {"blahblah": doc.value}});
});
Does the db.getLastError() occur for every single update or only at the very end of the for loop on the last update? Or does it happen at the end of the for loop for every updated document, all at one time?
The shell actually has w:1 (safe writes) when in interactive mode, when running in a loop it will not call getLastError until the end.
As reference you can actually see this comment by #Asya who works for MongoDB Inc.
the shell uses safe in that it called getLastError after every "command" (i.e. carriage return). If you are writing data, say, in a loop then GLE will only be called once at the end. Provide more details about how you plan to populate collection from the shell - maybe the right thing will already happen
Setting MongoDB's write concern in shell / shell script
Adding to that answer from Sammaye, if you wanted to call GLE after each update call you would do something like this:
db.coll1.find().forEach(function(doc){
db.coll2.update({"blah": doc._id}, {$set: {"blahblah": doc.value}}, true);
if(db.getLastError(1)){
printjson(db.getLastError(1));
printjson("failed to update ID: "+doc._id)
};
});
You'll get a dupe error at the end of the loop because of the carriage return if the last op fails, but otherwise it's pretty much what you would expect. If you want an easy test to reproduce just set the value field to be a string, then use $inc instead of $set - that will fail on non-numerics.
I might just be being a bit dim here, or it might be my relative weakness with javascript, but I'm kinda stuck.
I have a justgage dial, showing number of people checked in an office, out of a total number.
I have a setInterval call just after the gauge is initiallised. But I want to update BOTH the number present, AND the total number, because both might potentially change. I can get the numbers via an ajax call to my own api - but because the ajax call is asynchronous, if I use
setInterval (g.refresh(myfunc()), 1000)
how do I get multiple returns out of myfunc.
Alternatively, if I do
setInterval( myfunc(g), 1000)
function myfunc(g){
var foo=g;
$.ajax(blah)
.done(function(json){
// no g here
})
I have no reference to g on successful ajax call. (Though it is visible at var foo=g)
How do I get around this (multiple returns, or visible object)
Maybe there is a better solution but try to create the foo var as global and then you can use this in all the places you want.