Coming from a c# background, I'm probably looking at JavaScript from a completely wrong perspective, so please bear with me.
Leaving the advantages of async aside for a minute,
let's say I simply want to retreive a value from an SQLite database in an HTML5 page.
What I want to see is something like
var something = db.getPicture(1);
Now consider a (perhaps very naive) implementation of this:
this.getPicture(id)
{
this.database.transaction(function(tx)
{
tx.executeSql('SELECT ......', null, function(tx, results)
{
if (results.rows.length == 1)
return results.rows.items(0).Url; //This of course does not resturn
//anything to the caller of .getPicture(id)
}
},
function(error)
{
//do some error handling
},
function(tx)
{
//no error
});
}
First off, it's one big mess of nested functions and second... there's no way for me to return the result I got from the database as the value of the .getPicture() function.
And this is the easy version, what if I wanted to retreive an index from a table first,
then use that index in the next query and so on...
Is this normal for JavaScript developers, am I doing it completely wrong, is there a solution, etc...
The basic pattern to follow in JavaScript (in asynchronous environments like a web browser or Node.js) is that the work you need to do when an operation is finished should happen in the "success" callback that the API provides. In your case, that'd be the function passed in to your "executeSql()" method.
this.getPicture = function(id, whenFinished)
{
this.database.transaction(function(tx)
{
tx.executeSql('SELECT ......', null, function(tx, results)
{
if (results.rows.length == 1)
whenFinished(results.rows.items(0).Url);
}
},
In that setup, the result of the database operation is passed as a parameter to the function provided when "getPicture()" was invoked.
Because JavaScript functions form closures, they have access to the local variables in the calling context. That is, the function you pass in to "getPicture()" as the "whenFinished" parameters will have access to the local variables that were live at the point "getPicture()" is called.
Related
I come from a C/C++ background and I'm having real trouble trying to wrap my head around the syntax of node.js. Anyway, I found some code online to explain the difference between blocking and non-blocking code, and it's had me stumped for hours. I've tried searching and reading books but just can't find an answer to this. The example code retrieves a user ID from a database.
Blocking Version:
function getUser(id) {
var user = db.query(id);
return user;
}
console.log('Name: ' + getUser(432).name);
Non-Blocking Version (Node.js):
function getUser(id, callback) {
db.query(id, callback);
}
getUser(432, function (user) {
console.log(user.name);
});
I'm fine with the Blocking version because in that instance, the user ID is assigned to the variable user. What I just can't seem to understand is the user argument in the anonymous function. It seems that user just appears out of nowhere and then has instructions acted upon it, without any relationship to an existing variable.
How does the program know what user is? How does it even make any connection with the user's ID? I honestly can't tell if it's my lack of knowledge of JavaScript/Node, or whether the person who wrote this code didn't bother to complete it. All I know is, this makes no sense in C/C++.
Well, you've asked the program to fetch you a user, and supplied with a function that accepts an argument (or more, depending on the library). After the operation is complete, getUser will invoke the callback you passed with the result of the operation.
Here's a dummy getUser implementation for you:
function getUser(id, callback) {
setTimeout(function() {
var result = {id: id, name: "Madara"};
callback(result);
}, 1000); // Wait a second before doing it. Asynchronous!
}
getUser(42, function(user) { console.log(user); });
That function will wait one second, and then invoke the callback you pass with one argument, in this case, an object with the ID you passed, and "Madara" as the name.
Notice that there's nothing blocking with getUser, so getUser returns immediately, and without waiting for the callback to be called.
Maybe it will help you if I translate your example into C.
I haven't used C in a long time but I think it would look like this.
void getUser(int id, void (*callback)(User)) {
db.query(id, callback);
}
void printUserName(User user) {
printf(user.name);
}
getUser(432, &printUserName);
Coming from the Java Programming Language where all information that can be used in a method must either be part of a class, or passed by parameter, I'm very confused on to where JavaScript magically generates it's very important data.
In this case I'm using Socket.IO and trying to call a login-method after connection, now I could just use an anonymous function, but I personally believe they're really ugly.
Here's my code:
socketIO.sockets.on('connection', function(socket) {
console.log("Connection maid.");
socket.on('login', login);
});
function login(json) {
sqlConnection.query('select * FROM Accounts', function(err, rows, fields) {
if(err) throw err;
for(var row in rows) {
if(rows[row].Username == json.username) {
console.log("Exists");
socket.emit('login', {nuel: ''});
}
}
});
}
As you can see the login function is called whenever the socket receives a login message, and the json object is magically passed to it. I have no idea where it comes from or what generates it, nor do I have a clue how to pass additional arguments to the method, because it breaks/overwrites the magic data.
The error I'm running into is where socket doesn't exist in the current context, because it was not passed to the login function, however when I try to pass it, the object saved as json is completely eradicated from existence.... AND WHERE IS THE DATA THAT'S HERE COMING FROM.
I don't understand this at all, How can I pass this information to call the method, without completely breaking everything.
This question already has answers here:
Calling an asynchronous function within a for loop in JavaScript
(10 answers)
Closed 8 years ago.
I found lots of similar questions, but I still don't know what's wrong with my code. It seems that I cannot read global variable value (urls) in the callback function: I want to update the urls latestTimestamp value in the callback function(err, articles). Here is the code that went wrong:
var urls=[
{"url": "http://www.economist.com/feeds/print-sections/77/business.xml", "latestTimestamp": new Number(0)},
{"url": "http://news.sky.com/feeds/rss/home.xml", "latestTimestamp": new Number(0)},
]; // Example RSS Feeds;
// parse RssFeeds from given websites and write them into databse
function parseRssFeeds(collection){
var feed = require('feed-read'); // require the feed-read module
// loop through our list of RSS feed urls
for (var j = 0; j < urls.length; j++)
{
console.log('Original url timestamp is: '+ urls[j].latestTimestamp.toString());
// fetch rss feed for the url:
feed(urls[j], function(err, articles)
{
// loop through the list of articles returned
for (var i = 0; i < articles.length; i++)
{
var message =
{"title": articles[i].title,
"link": articles[i].link,
"content": articles[i].content,
"published": articles[i].published.getTime()};
collection.insert(message, {safe:true}, function(err, docs) {
if (err) {
console.log('Insert error: '+err);
}else{
console.log('This item timestamp is: '+ message.published);
// get the latest timestamp
if (message.published >urls[j].latestTimestamp) {
console.log('update timestamp to be: '+ message.published);
urls[j].latestTimestamp = message.published;
}
}
});// end collection insert
} // end inner for loop
}) // end call to feed method
} // end urls for loop
}
Thanks for any help. The error is:
TypeError: Cannot read property 'latestTimestamp' of undefined
at /Users/Laura/Documents/IBM/project/TestList/app.js:244:37
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/collection/core.js:123:9
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/db.js:1131:7
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/db.js:1847:9
at Server.Base._callHandler (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)
at /Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/server.js:478:18
at MongoReply.parseBody (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
at null.<anonymous> (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/server.js:436:20)
at emit (events.js:95:17)
at null.<anonymous> (/Users/Laura/Documents/IBM/project/TestList/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:201:13)
This should probably be closed as a duplicate, but I'll put an answer here because the relationships between all the duplicate questions are often so hard to grasp for JavaScript programmers that haven't understood the fundamental problem.
There are two ways to solve this. One way is to change the way that you create the callback. Instead of using an inline anonymous function:
feed(urls[j], function(err, articles)
{
// loop through the list of articles returned
// ...
you'd create another function that returns the callback. You'd pass that function the URL, and that's what the returned function would use:
function makeFeedResultHandler(url) {
return function(err, articles) {
// loop through the list of articles returned
// ... code as before up to this line:
if (message.published > url.latestTimestamp) {
console.log('update timestamp to be: '+ message.published);
url.latestTimestamp = message.published;
}
// ... etc
};
}
Then you'd call "feed" like this:
feed(urls[j], makeFeedResultHandler(urls[j]));
The key difference is that each function passed to "feed" will have its own private copy of the object (well, a copy of the reference to be picky) from the "urls" array, so it won't need to refer to the variable "j" at all. That's the crux of the problem: "j" is shared by all the callbacks in your code. By the time the callbacks are invoked, the value of "j" is equal to the length of the "urls" array, so urls[j] is undefined.
The other approach would be to use the .forEach method, available in newer JavaScript implementations. That approach would get rid of "j" altogether:
urls.forEach(function(url) {
console.log('Original url timestamp is: '+ url.latestTimestamp.toString());
// fetch rss feed for the url:
feed(url, function(err, articles)
{
// loop through the list of articles returned
// ... code as previously, substituting "url" for "urls[j]" everywhere
});
});
Again, that makes sure that every callback sent to the "feed" function has its own copy of the "urls" element.
Expanding on what #Pointy said in his comment under your post:
The insert function you are using with MongoDB is async, but you are treating the callback like it is synchronous. What is essentially happening in your loop, is everything works as planned until you hit collection.insert. From there, the process breaks off and essentially says "I'm going to tell mongo to insert a record now.. and eventually I'll expect a response." Meanwhile, the loop continues on to the next index and doesn't synchronously wait until the callback fires.
By the time your callback fires, your loop is already done, and J doesn't represent the index anymore, which is why its coming up undefined. You also run the risk of getting a different index than what you plan also with this current method.
I would recommend reworking your loop to support the async nature of node. There is a great library called - oddly enough - async that makes this process super simple. The async.each() function should help you accomplish what you are trying to do.
I'm using Node.js + Express + nodejs-sqlite3 to make a form that when submited will insert a new row on an slite3 database.
On query sucess I want to write certain response.
So the small big problem is just: Modify a string that will be storing the html to be shown, inside the callback function of sqlite3.run()
I read about closures, and passing an object with methods to modify its own attributes. But it seems it's not working. It will pass the object attributes and methods, but no change will remain when the callback function ends. I read that objects will be passed as reference, not copies.
This is the code:
app.post("/insert.html", function(req, res){
function TheBody(){
this.html = "";
this.msg = "";
this.num = "";
}
TheBody.prototype.add = function(string){
this.html = this.html + string;
}
var body = new TheBody();
body.msg = req.body.message;
body.num = req.body.number;
var insertCallback = function(data){
return function(err){
if( err != null){
console.log("Can't insert new msg: " + err.message);
data.add("ERROR-DB");
} else {
console.log("Ok. Inserted: " + data.msg);
console.log(data.html);
data.add("OK - MSG: "+data.msg+" NUM: "+data.num);
console.log(data.html);
}
};
};
var db = new lite.Database('database.db');
var query = "INSERT INTO outbox (message, number) VALUES (?, ?)";
db.run(query, [body.msg, body.num], insertCallback(body) );
res.setHeader('Content-Type', 'text/html');
res.setHeader('Content-Length', body.html.length);
res.end(body.html);
}
On server side I'll see
Ok. Inserted: TestString
[Blank space since data.html still has no information]
OK - MSG: TestString NUM: TestNumber [Showing that indeed was modified inside the function]
But on the client side res.end(body.html); will send an empty string.
The object is not being passed as reference.
What's missing in the code, and what simpler alternatives I have to change a string variable inside a callback anonymous function?.
I already know I could use response.write() to write directly on the function if it were more simpler. But I discovered it would only work if I use response.end() inside the callback, otherwise (being outside as it is now) it will meet a race condition where the buffer will be closed before sqlite3.run() be able to use response.write().
-------- Answered --------
As hinted by Justin Bicknell and confirmed by George P. Nodejs-sqlite3 functions are run asynchronously. So I was ending the stream to the client before the callback would be called, thus nothing was being printed.
This was a problem more about "This is SPART- nodejs, so write your stuff according to events'" rather than a logic one. I found this kind of programming kind of convoluted but nobody else than me told me to use nodejs. For those wondering about how one could put some order over the order of queries on the database, nodejs-sqlite3 functions returns a database object that is used to chain the next query.
Since I was printing the information to the client just once in every handled event, the resulting object ended like this:
function TheBody(response){
this.response = response;
}
TheBody.prototype.printAll = function(string){
this.response.setHeader('Content-Type', 'text/html');
this.response.setHeader('Content-Length', string.length);
this.response.end(string);
}
Preferring that to clutter all the code a lot of res.setHeader() lines.
node-sqlite3 methods are, by default, run in parallel (asynchronously). That means that your code is going through this timeline:
Your code calls db.run(...)
Your code calls res.end(...)
db.run completes and calls your callback.
This is the source of a huge number of questions here on SO, so you can almost certainly find a better answer than anything that I could write here in a reasonable amount of time.
I would start here: How does Asynchronous Javascript Execution happen? and when not to use return statement?
I'm trying to create a little plugin that will make working with SQLite databases a little easier. I've got the basic CRUD functions except for selecting data from the database.
I would like to return the selected data from the executeSql function, but it seems all I can do is call another method and not just return the data.
So what I'd like to do is:
var data = $(window).dbPlugin( 'select', '*', 'tableName', 'conditions' );
Then I would like to work with the "data" variable as the SQLResultSet from the query. However, the transaction seems to only allow me to call a data handler method and not just return the result.
testDB.transaction(
function ( transaction ) {
transaction.executeSql( sql, [], dataHandler, errorHandler);
}
);
function dataHandler( transaction, results ) {
// Now I can work with the "results" as the SQLResultSet
}
Any ideas how to just return the selected data or if it's even possible?
SQLlite is asynchronous, and there isn't any way to avoid that. You could pass in the function you want to be called as your callback into your .dbPlugin method. After conditions pass another value which is a function to be called after the query is run. Then set data in your callback function.
It's pretty annoying, I recently wrote something like you are writing on my last project. I used a jQuery plugin called blockUI that would essentially block the interface during SQL calls which made it appear a little more 'synchronous'
Good luck.