I'm using the pg postgres node module to interact with my database. The code works without an issue when I use pg's evented API. Below is an example of the code:
Migration.js
exports.up = function(logger){
var pg = require("pg")
, connectionString = //process.env.CONNECTIONSTRING
, client = new pg.Client(connectionString);
client.connect();
var cmd = "CREATE TABLE users( "
+ "id SERIAL NOT NULL, "
+ "firstName VARCHAR(50) NOT NULL, "
+ "lastName VARCHAR(50) NOT NULL, "
+ "CONSTRAINT pk_userid PRIMARY KEY (id) "
+ ")";
var query = client.query(cmd);
query.on("end", function(){
client.end();
logger.log("complete");
});
};
I'm using commander.js to write a command line utility around this migration script; however, the non-blocking call of the pg snippet is freeing up the command line interface script to finish before the database updates have been done. Below is an example commander snippet:
MyCliTool
var program = require('commander');
program
.version('1.0.2');
program
.command("up")
.description("Migrates up")
.action(function(){
require("../src/migration").up();
});
// Additional code removed for brevity
Is there a way that I can change the migration (or the commander.js app) script to ensure that the migration's up() function will finish prior to my cli script finishing? I have tried using callbacks but that appears to not be working.
UPDATE
Another example to illustrate this point. Below is a unit test (written with mocha) concerning the issue.
Up-test.js
describe("feature", function(){
it("should finish", function(){
var logger = {};
var finished = false;
logger.log = function(){finished = true;};
var migration = require("../src/migration.js");
migration.up(logger);
assert(finished, "function didn't finish"); // finished is false when this gets called.
});
});
Looks like your original script has a pain point around program.action(function () {});:
program
.command("up")
.description("Migrates up")
.action(function(){
require("../src/migration").up();
// nothing preventing this function from exiting immediately
});
I poked around a bit in commander.js docs and couldn't find anything regarding supplying a callback to your .action() program. If I were writing it, the code would look something like this:
program
.command("up")
.description("Migrates up")
.action(function (completedCallback) {
require("../src/migration").up(function () {
// migration is complete
completedCallback();
});
});
Your postgre-related script looks fine, though. A somewhat unrelated note might be to have require("../src/migration").up(); return an instance of require("events").EventEmitter and emit events you'd like to have visibility into, instead of using a logger variable. But that's just preference.
Try setting timers (http://nodejs.org/api/timers.html). We do this within our Azure CLI tool to have a spinner that runs when you are doing long running operations.
You can drill into the code here to see how we do it: https://github.com/WindowsAzure/azure-sdk-for-node/blob/master/lib/cli/cli.js. Search for the progress() function.
Related
This question is different from the one linked as already having an answer. It is specifically this piece of code adapted from node.js documentation regarding the use of fs and readfile and looking for an end of file flag, which I've learned is readfile.close method. Thanks for the answer.
I wrote a small utility locally to try to convert a text file of key:value pairs with a blank line separating programs into a JSON file to use in a React project.
I got the foundation of the readline function directly from node.js documentation. I'm using node 6.9.0 on my mac
here is the full script:
const readline = require('readline');
const fs = require('fs');
const rl = readline.createInterface({
input: fs.createReadStream('usat-ncaa-programs.txt')
});
var newPairs=["test"];
rl.on('line',
function (line) {
if (line===null){
newPairs.push("}], [ {")
} else if (line) {
var keyValue = line.match(/^(.*):(.*)/)
var newKeyValuePair = "'" + keyValue[1].trim() + "': '" + keyValue[2].trim() + "'"
newPairs.push(newKeyValuePair)
//console.log (newKeyValuePair)
}
})
console.log(newPairs)
The input file looks like this (there are about 12 programs), i've only included 2 1/2 so you can see the format:
University: Arizona State University
Division: I
University Home Page: http://www.asu.edu/
Recruiting Link: https://questionnaire.acsathletics.com/Questionnaire/Questionnaire.aspx?&SPSID=1061112&SPID=177408&DB_LANG=C&DB_OEM_ID=30300&q=2015&s=159130&o=143
Team Homepage: http://www.thesundevils.com/index.aspx?path=triathlon
Head Coach: Cliff English
w: 480.965.0546
e: cliff.endlish#asu.edu
bg-color: #990033
color: #FFB310
University: Belmont Abby College
Division: II
University Home Page: http://belmontabbeycollege.edu/
Recruiting Link: https://coach.scoutforce.com/p/414f3219dd
Team Homepage: http://abbeyathletics.com/sports/wtri/index
Head Coach: Nick Radkewich
w: 704.461.5010
e: NicholasRadewich#bac.edu
Twitter: https://twitter.com/AbbeyTri
bg-color: #FFFDD0
color: #DC143C
University:Black Hills State University
Division: II
University Home Page: http://www.bhsu.edu/
...
My problem is that while I can read the text file line by line and parse some information that looks like a JSON file, I am unable to access that data outside of the callback function.
I don't know how to save this data into a new file, or even just output the object to my console for a cut & paste and manually edit.
In teh above script the logged output of the variable newPairs is ["test"] rather than the line by line parsing that I'm trying to acccomplish.
If I place the console.log inside the callback, I get information logged with every iteration of the file read. I only want to work with the data when the file is done.
I did not find an EOF or similar flag in the node documentation for either fs or readline.
Additionally, if there is an easier way to get the data I am inputting into JSON format, I'd love to hear. Thanks in advance.
You have to understand that the callback function is executed asynchronously. This means that console.log(newPairs) gets executed before your callback, therefore it only results in "test".
You should listen to Readline's close event, like so:
rl.on('close', function() {
console.log(newPairs);
});
As the documentation states:
The 'close' event is emitted when one of the following occur:
The rl.close() method is called and the readline.Interface instance
has relinquished control over the input and output streams;
The input stream receives its 'end' event; The input stream receives -D to signal end-of-transmission (EOT);
The input stream receives -C to signal SIGINT and there is no SIGINT event listener registered on the readline.Interface instance.
The listener function is called without passing any arguments.
The readline.Interface instance should be considered to be "finished"
once the 'close' event is emitted.
So this would be the 'EOF' you're looking for :-)
i know its an old question but the answer didn't help me. You can use an promisse like this:
function readLineAsync(){
var data = [];
var i = 0;
return new Promise((resolve,reject)=>{
var lineRead = readLine.createInterface({
input:fs.createReadStream('path.to.file')
})
lineRead.on('line',(line)=>{
data[i] = line;
i++;
})
lineRead.on('close',()=>{
resolve(dataCsv);
})
})
}
(async function(){ console.log(await readLineAsync())}())
I´m starting with IndexedDB and to not reinvent the wheel I´m using Dexie.js https://github.com/dfahlander/Dexie.js
I created the database, I added data and now I´m creating a generic function that get a CSV and populate the database in anothers tables.
So, more or less my code is
// Creation and populate database and first table
var db = new Dexie("database");
db.version(1).stores({table1: '++id, name'});
db.table1.add({name: 'hello'});
Until here all is OK
Now, in success of ajax request
db.close();
db.version(2).stores({table2: '++id, name'});
db.open();
db.table2.add({name: 'hello'});
First time this code run everything is OK, but next time I get this error
VersionError The operation failed because the stored database is a
higher version than the version requested.
If I delete database and run code again only first time works OK.
Any idea? I don´t like too much IndexedDB version way, it´s looks frustrating and I don't get lot of help in the Net
Thanks.
Edit:
I discover the ¿problem/bug/procedure?. If I don´t add nothing before any version modification I haven't this issue, but does somebody know if is this the normal procedure?
So.. if this is the procedure I can't add any table dinamycally with a generic method. First all declarations and then add values. Any possibility to add a table after add values?
Edit again... I just realized that I could create another database. I'll post results. But any information about this issue is welcome :)
Edit again... I created dinamycally another database and everybody is happy!!
That is because the second time the code runs, your database is on version 2, but your main code still tries to open it at version 1.
If not knowing the current version installed, try opening dexie in dynamic mode. This is done by not specifying any version:
var db = new Dexie('database');
db.open().then(function (db) {
console.log("Database is at version: " + db.verno);
db.tables.forEach(function (table) {
console.log("Found a table with name: " + table.name);
});
});
And to dynamically add a new table:
function addTable (tableName, tableSchema) {
var currentVersion = db.verno;
db.close();
var newSchema = {};
newSchema[tableName] = tableSchema;
// Now use statically opening to add table:
var upgraderDB = new Dexie('database');
upgraderDB.version(currentVersion + 1).stores(newSchema);
return upgraderDB.open().then(function() {
upgraderDB.close();
return db.open(); // Open the dynamic Dexie again.
});
}
The latter function returns a promise to wait until it's done before using the new table.
If your app resides in several browsers, the other windows will get their db connection closed as well so they can never trust the db instance to be open at any time. You might want to listen for db.on('versionchange') (https://github.com/dfahlander/Dexie.js/wiki/Dexie.on.versionchange) to override the default behavior for that:
db.on("versionchange", function() {
db.close(); // Allow other page to upgrade schema.
db.open() // Reopen the db again.
.then(()=> {
// New table can be accessed from now on.
}).catch(err => {
// Failed to open. Log or show!
});
return false; // Tell Dexie's default implementation not to run.
};
I am using gulp-esdoc to generate my documentation (and it already works).
Today I added a git pre-commit hook, it launches multiples gulp tasks and prevent any commit if one of the tasks returns an error.
I works for most of my tasks, but for gulp-esdoc, I don't know how to return such an error to prevent the commit.
I configured my esdoc this way :
gulp.task('documentation', function() {
gulp.src("./client/")
.pipe(esdoc({
destination: "./docs",
"title": "SOMETHING",
index: "./README.md",
"plugins": [{
"name": "./README/docplugins/on-complete.js"
}]
}));
});
As you can see, I am using an esdoc plugin to do some code once the documentation has been generated, here is the code for the plugin :
var colors = require('colors'),
gutil = require('gulp-util');
exports.onComplete = function(ev) {
var output = require('../../docs/coverage.json');
var expectedCoverage = 100; // todo : get from option instead
var percentCovered = (output.actualCount / output.expectCount) * 100;
if (percentCovered < expectedCoverage) {
console.log('Code coverage is not sufficient (expected ' + expectedCoverage + '%, received only ' + percentCovered + '%) !!! Write some documentation or get out !'.red);
// return false; // todo
} else {
console.log('Code coverage is complete, well done :)'.green);
// return true;
}
};
The main goal here is to return an error if my code documentation is not 100% covered (the code logic is working, if a comment is missing in my JS code, I go into the line with the "todo".
Finally, here is my (simplified) git-hook (in my gulpfile) :
gulp.task('pre-commit', function () {
gulp.start('lint');
gulp.start('documentation');
});
The 'lint' part works, it prevent commit if I have an error in my code. But the 'documentation' code, using gulp-esdoc, doesn't return any error, so the commit is done :( I don't know what to do to return an error and prevent the git commit.
I need to do this because I want the junior developers joining my team to be forced to document their code :p
Thanks for helping !
One option is to throw a PluginError:
throw new gutil.PluginError('esdoc', 'Something broke');
This will cause your gulp process to exit with a non-zero exit code. It will however leave a rather ugly stack trace on STDERR (especially since gulp-esdoc rethrows errors from plugins).
Another option then is to explicitly exit the process with a specific exit code:
process.exit(1);
The process object in Node.js is a global object and can be accessed from anywhere without having to require() it.
I'm working on a Google Docs Add-On based on Google's Quickstart tutorial. I'm trying to change the workflow of the Add On in the tutorial to append a new page and then insert a translation on that new page rather than the line-by-line workflow.
I have a script working in single documents but I'm having a hard time moving it to the Add On architecture. I think it's something to do with passing selections from client-side JS to the server-side script doing the translation.
Here's the translate script
function translate(origin, dest, savePrefs) {
Logger.log('Starting the script');
if (savePrefs == true) {
var userProperties = PropertiesService.getUserProperties();
userProperties.setProperty('originLang', origin);
userProperties.setProperty('destLang', dest);
Logger.log(origin,dest);
}
var doc = DocumentApp.getActiveDocument();
var body = doc.getBody();
// Add a page break for the translated material.
body.appendPageBreak();
// Get the number of elements in the document
var elements = body.getNumChildren();
Logger.log('Got the page elements');
// Use the number to loop through each element in the document.
for( var i=0;i<elements;i++) {
var element = body.getChild(i).copy();
var type = element.getType();
Logger.log('Element Types were successful. Starting tests.');
// Test each type for a child element and run script based on the result
// Images are nested in a paragraph as a child, so the second `if` makes
// sure there is no image present before moving to the next paragraph.
if( type == DocumentApp.ElementType.PARAGRAPH ){
if(element.asParagraph().getNumChildren() != 0 && element.asParagraph().getChild(0).getType() == DocumentApp.ElementType.INLINE_IMAGE) {
var img = element.asParagraph().getChild(0).asInlineImage().getBlob();
body.appendImage(img);
} else if(element.asParagraph().getNumChildren() !=0 && element.asParagraph().getChild(0).getType() == DocumentApp.ElementType.INLINE_DRAWING) {
var drawing = element.asParagraph().copy();
body.appendParagraph(drawing);
} else {
var text = element.asParagraph().getText();
Logger.log(text);
var spn = LanguageApp.translate(text, origin, dest);
body.appendParagraph(spn);
}
} else if(type == DocumentApp.ElementType.TABLE) {
element.asTable().copy();
body.appendTable(element);
} else if(type == DocumentApp.ElementType.LIST_ITEM) {
var list = element.asListItem().getText();
body.appendListItem(LanguageApp.translate(list, origin, dest));
}
}
The client-side JS is:
$(function() {
$('#run-translation').click(loadPreferences);
google.script.run(runTranslation)
});
function runTranslation() {
this.disabled = true;
var origin = $('input[name=origin]:checked').val();
var dest = $('input[name=dest]:checked').val();
var savePrefs = $('#save-prefs').is(':checked');
google.script.run
.runTranslation(origin, dest, savePrefs);
}
If I hard-code the languages to use in translation into the server-side script, it works. But as soon as I try to use variables from the radio buttons, it doesn't run. I don't see any errors in the console and I can't run scripts from the editor to check the logs. How can I debug this code?
Calling server-side functions from client Javascript
You've got a minor syntax error with google.run:
google.script.run(runTranslation)
It should be:
google.script.run
.withFailureHander(failFunc) // Optional
.withSuccessHander(succFunc) // Optional
.serverFunction(optionalParams...);
The two Handler methods assign callback functions from your client-side JavaScript to be invoked in the case of success or failure of the server-side function. In both cases, the client-side function is provided as the only parameter.
The server-side function you want to communicate with is presented as if it is a method itself, with optional parameters to be passed across the divide.
The simplest case for you is:
google.script.run
.translate(origin, dest, savePrefs);
(You had runTranslation in your posted code, but the server-side function is named translate()... I assume that's the right one.)
Now, this might will not take care of all your problems, so you wisely asked about debugging...
Debugging asynchronous client / server code in Google Apps Script
The provided debug environment isn't enough for debugging this sort of client / server exchange. There are also weaknesses in the Debugger for use with asynchronous execution - you can read more about that in Not seeing logs from onEdit trigger.
The simplest tool to get you debugging in this case would be to write logs to a spreadsheet
/**
* Write message and timestamp to log spreadsheet.
* From: https://stackoverflow.com/a/32212124/1677912
*/
function myLog( message ) {
var ss = SpreadsheetApp.openById( logSpreadsheetId );
var logSheet = ss.getSheetByName("Log") || ss.insertSheet("Log");
logSheet.appendRow([ new Date(), message );
}
You can call this from your client-side Javascript thusly:
google.script.run.myLog( "CLIENT: " + message );
That's a basic approach, but you can extend it more through use of utility functions and the BetterLog library. See more about that in my blog entry Did you know? (You can log to a spreadsheet from client JavaScript!)
I am trying to run the following nodejs code, based on the duino module found here: https://github.com/ecto/duino
bleep.js:
var arduino = require("duino");
var board = new arduino.Board();
var led = new arduino.Led({
board: board,
pin: 13
});
var bleep = function() {
led.on();
setTimeout(function() {
led.off();
}, 100);
}
setInterval( bleep, 1000 );
EDIT
Based on zmo's suggestion, I am now running du.ino (found under duino/src/) on my arduino board and bleep.js on my laptop.
Whilst compiling bleep.js, I get the following error:
Error: Cannot open /dev/
I was trying to trace back to where the path is being set.
I have traced this error to duino/node_modules/serialport/serialport.js
May I know how the following line (found in serialport.js) works?
function SerialPort(path, options, openImmediately, callback) {
Before this function is called on line 49, the 'path' variable is [object Object].
But after, it becomes /dev/
where does the change take place?
Thank you.
uhuh... where to start.
It looks like you're trying to compile your js code as an arduino sketch .ino files using the arduino ide. That can't work, it should be C++ code.