How to execute functions synchronously to get the right output - javascript

I am making an application which reads a file and extracts the data line by line. Each line is then stored as an array element and that array is required as the final output.
const fs = require('fs');
const readline = require('readline');
var output_array=[];
const readInterface = readline.createInterface({
input: fs.createReadStream(inp_file),
console: false
});
readInterface.on('line', function(line) {
//code for pushing lines into output_array after doing conditional formatting
}
callback (null, output_array);
With this I am getting an empty array.
Though if I use 'setTimeout' then it's working fine
setTimeout(() => {
callback (null, output_array);
}, 2000);
How can I execute this synchronously without having to use 'setTimeout'?

You cannot execute asynchronous functions synchronously. But readline supports the close event, which is triggered when the inputfile is completely read, so you can call the callback there.
readInterface.on('close', () => {
callback(null, output_array);
});

Of course you could output the content on the close event, but you could also use a promise like this:
function readFile(fileName) {
return new Promise((resolve, reject) => {
const readInterface = readline.createInterface({
input: fs.createReadStream(fileName).on('error', error => {
// Failed to create stream so reject.
reject(error);
}),
});
const output = [];
readInterface.on('error', error => {
// Some error occurred on stream so reject.
reject(error);
});
readInterface.on('line', line => {
output.push(line);
});
readInterface.on('close', () => {
// Resolve the promise with the output.
resolve(output);
});
});
}
readFile(inp_file)
.then(response => {
callback(null, response);
})
.catch(error => {
// Take care of any errors.
});
I removed the console setting when calling createInterface() since it does not seem to be a valid option.

Make your callback from readline's close event handler. That way your callback will happen after all lines are read.
readInterface.on('close', function(line) {
callback (null, output_array);
}

Related

Promise within a function

This code does not execute the promise of testAuthentication or pinFileToIPFS and i am curious if this is a node concept i am not familiar of.
function uploadToPinata(filename) {
const pinata = pinataSDK(process.env.PINATA_KEY, process.env.PINATA_SECRET_KEY);
pinata.testAuthentication().then((result) => {
//handle successful authentication here
console.log(result);
}).catch((err) => {
//handle error here
console.log(err);
});
const readableStreamForFile = fs.createReadStream(filename);
const options = {
pinataMetadata: {
name: "NYC_NFT_TESTING",
},
pinataOptions: {
cidVersion: 0
}
};
pinata.pinFileToIPFS(readableStreamForFile, options).then((result) => {
//handle results here
console.log(result);
}).catch((err) => {
//handle error here
console.log(err);
});
}
is there a problem with this code using a promise within a function? I attemped to make the function async but that did not help. This code works just fine outside of a function but not within one.
"does not execute the promise": this is a strange phrase. Promises are objects. They don't execute -- they are created. And your function really creates them. However, it does not do much when these promises are resolved.
The problem is that uploadToPinata will execute all of the synchronous code and return. But then there are still a promises pending. Although you have console.log that will be executed once the relevant promise has resolved, there is no signal to the caller of uploadToPinata that these promises have resolved.
It is probably easiest to use async and await:
async function uploadToPinata(filename) {
const pinata = pinataSDK(process.env.PINATA_KEY, process.env.PINATA_SECRET_KEY);
const result = await pinata.testAuthentication();
//handle successful authentication here
console.log("authentication result", result);
const readableStreamForFile = fs.createReadStream(filename);
const options = {
pinataMetadata: { name: "NYC_NFT_TESTING" },
pinataOptions: { cidVersion: 0 }
};
const result2 = await pinata.pinFileToIPFS(readableStreamForFile, options);
//handle results here
console.log("pinFileToIPFS result", result2);
}
The caller of this function will now receive a promise, so it should probably want to do something when that promise has resolved:
uploadToPinata(filename).then(() => {
console.log("authentication and file operation done")
}).catch(error => console.log("error", error));

sync function that await an event (once more

Desperately trying to write a sync version of https://www.npmjs.com/package/node-firebird#reading-blobs-aasynchronous
Basically I need to (a)wait twice:
for the callback function to execute so that the eventEmitter is available
for the "end" event to occur
and then return the Buffer.
my code (JS/TS mix for now) currently does 2, but not 1 : readBlob returns undefined, then Buffer.concat(buffers) is called later ... :
function readBLOB(callback: any): Buffer {
return callback(async (err, _, eventEmitter) => {
let buffers = []
if (err)
throw err
eventEmitter.on('data', chunk => {
buffers.push(chunk);
});
return await eventEmitter.once('end', function (e) {
return Buffer.concat(buffers)
})
})
}
Sorry to ask one more time (yes, I checked a lot of other questions and tried a lot of things...), but how to make this work (simply...) ?
(the function that calls the callback is fetch_blob_async in https://github.com/hgourvest/node-firebird/blob/master/lib/index.js#L4261 , just in case...)
There are few mistakes here like returning an callback function, witch returns, i guess, undefined or returning something IN an callback function that makes no sense.
Also async / await makes no sense here it has no effect. async / await is only useful if you want to await till some Promise resolves. But you have no Promise in your code at all.
What you need is new Promise
function readBLOB(callback) {
return new Promise((resolve, reject) => {
callback((err, _, eventEmitter) => {
let buffers = [];
if (err) reject(err);
eventEmitter.on("data", chunk => {
buffers.push(chunk);
});
eventEmitter.once("end", function(e) {
resolve(Buffer.concat(buffers));
});
});
});
}
Simple like that. You resolve your Buffer and reject if some error occurs
Now you can use it like:
readBLOB(cb).then(data => {
console.log(data);
})

jest jumps out of function with promise

I'm trying to test a function which calls another module's function which returns a promise,
The problem is that jest does not wait for completion of the myFunction and jumps out of it and treat it as a promise, as result section shows the "done" message is printed before "resolve" message. I have work around using setImmediate but I rather not to use it and want to understand the reason.
the simplified version of the code is following:
The module which is mocked
// repo.js
const getItems = () => {
console.log('real');
return new Promise((resolve, reject) => {
setTimeout(
() => resolve('result'), 1000);
}
);
}
module.exports = {
getItems
};
Unit under test:
// sample.js
const repo = require('./repo');
const myFunction = (req, res) => {
console.log('myFunction');
repo.getItems()
.then(goals => {
console.log('resolve');
res.val = 'OK';
}).catch(err => {
console.log('reject');
res.val = 'Failed';
});
return;
};
module.exports = {myFunction};
Test file:
// sample.test.js
const repo = require('./repo');
const sample = require('./sample');
const result = {
'message': 'done'
};
describe('Tests for receiving items', () => {
it('should call and be successful. ', () => {
repo.getItems = jest.fn(() => {
console.log('mocking');
return new Promise((resolve) => ( resolve(result) ));
});
const response = {val: 'test'};
const request = {};
sample.myFunction(request, response);
console.log('done');
expect(response.val).toBe('OK');
})
}
);
The result is:
console.log MySample\sample.js:5
myFunction
console.log MySample\sampel.test.js:11
mocking
console.log MySample\sampel.test.js:17
done
console.log MySample\sample.js:9
resolve
Error: expect(received).toBe(expected)
Expected value to be (using ===):
"OK"
Received:
"test"
Expected :OK
Actual :test
The test you wrote reflects the correct usage, and you might say it fulfilled its purpose, because it uncovered a bug in your implementation.
To show what exactly went wrong, I will get rid of everything that is not needed, which leads to an even more minimal example. The following test file can be run by Jest and it reproduces your problem.
const myFunction = (res) => {
Promise.resolve()
.then(goals => {
res.val = 'OK';
}).catch(err => {
res.val = 'Failed';
});
return;
};
it('should call and be successful. ', () => {
const response = {val: 'test'};
myFunction(response);
expect(response.val).toBe('OK');
})
myFunction starts a promise (which resolves immediately here with no value) and returns nothing (undefined). You can also test the error part by using Promise.reject instead of Promise.resolve. When you call myFunction(response) the next line is executed when myFunction finishes. This is not when the promise actually finishes, but the function itself. The promise could take any amount of time and there is no way for you tell when it actually finished.
To be able to know when the promise finished, you need to return it, so you can use a .then() on it to execute something after the promise has been resolved. Both .then() and .catch() return a new promise which resolves with the returned value, which in this case is again undefined. That means you need to do your assertion in the .then() callback. Similarly, Jest thinks that the test ends as you exit the function even though it should wait for the promise to be settled. To achieve this you can return the promise from the test and Jest will wait for its completion.
const myFunction = (res) => {
// Return the promise from the function, so whoever calls myFunction can
// wait for the promise to finish.
return Promise.resolve()
.then(goals => {
res.val = 'OK';
}).catch(err => {
res.val = 'Failed';
});
};
it('should call and be successful. ', () => {
const response = {val: 'test'};
// Return the promise, so Jest waits for its completion.
return myFunction(response).then(() => {
expect(response.val).toBe('OK');
});
})
You can also use async/await, but keep in mind that you still need to understand how promises work, as it uses promises underneath. An async function always returns a promise, so Jest knows to wait for its completion.
it('async/await version', async () => {
const response = {val: 'test'};
// Wait for the promise to finish
await myFunction(response);
expect(response.val).toBe('OK');
})
Usually you would also return a value from the promise (in .then() or .catch()) instead of mutating an outer variable (res). Because if you use the same res for multiple promises, you will have a data race and the outcome depends on which promises finished first, unless you run them in sequence.

How to assert stubbed fetch more than once

Using proxyquire, sinon, and mocha.
I am able to stub fetch on the first call of fetch. But on the second fetch call, which is recursive, I am not able to assert it. From the output, it looks like the assertion may run before the test finishes. You will see this with second fetch console out after assertion.
index.js
var fetch = require('node-fetch');
function a() {
console.log('function a runs');
fetch('https://www.google.com')
.then((e) => {
console.log('first fetch');
b();
})
.catch((e)=> {
console.log('error')
});
}
function b() {
fetch('https://www.google.com')
.then((e) => {
console.log('second fetch');
})
.catch((e)=> {
console.log('error')
});
}
a()
test:
describe('fetch test demo', ()=> {
it('fetch should of called twice', (done)=> {
fetchStub = sinon.stub();
fetchStub2 = sinon.stub();
fetch = sinon.stub();
fetchStub.returns(Promise.resolve('hello'));
fetchStub2.returns(Promise.resolve('hi'));
var promises = [ fetchStub, fetchStub2 ]
fetch.returns(Promise.all(promises));
proxy('../index', {
'node-fetch': fetch
});
fetch.should.have.been.callCount(2);
done()
});
});
fetch test demo
function a runs
1) fetch should of called twice
first fetch
second fetch
lifx alert test
- fetch should of called three times
when rain change is over 50%
- should run fetch twice
0 passing (78ms)
2 pending
1 failing
1) fetch test demo fetch should of called twice:
expected stub to have been called exactly twice, but it was called once
stub(https://www.google.com) => [Promise] { } at a (/home/one/github/lifx-weather/foobar.js:5:3)
AssertionError: expected stub to have been called exactly twice, but it was called once
stub(https://www.google.com) => [Promise] { } at a (foobar.js:5:3)
at Context.it (test/bar.js:22:28)
Updated version
#dman, since you updated your test case I owe you an updated answer. Although rephrased, the scenario is still unorthodox - it seems like you want to ignore in a sense the 'law of gravity' even though you know it's right there in front of you.
I'll try to be as descriptive as possible. You have two functions which are doing async stuff by design. a() calls b() sequentially - by the way this is not recursion. Both functions do not notify their callers upon completion / failure, i.e. they are treated as fire-and-forget.
Now, let's have a look at your test scenario. You create 3 stubs. Two of them resolve to a string and one combining their execution using Promise.all(). Next, you proxy the 'node-fetch' module
proxy('./updated', {
'node-fetch': fetch
});
using the stub that returns the combined execution of stubs 1 & 2. Now, if you print out the resolved value of fetch in either function, you will see that instead of a string it's an array of stubs.
function a () {
console.log('function a runs');
fetch('http://localhost')
.then((e) => {
console.log('first fetch', e);
b();
})
.catch((e) => {
console.log('error');
});
}
Which I guess is not the intended output. But let's move over as this is not killing your test anyway. Next, you have added the assertion together with the done() statement.
fetch.should.have.been.callCount(2);
done();
The issue here is that whether you are using done() or not, the effect would be exactly the same. You are executing your scenario in sync mode. Of course in this case, the assertion will always fail. But the important thing here is to understand why.
So, let's rewrite your scenario to mimic the async nature of the behavior you want to validate.
'use strict';
const chai = require('chai');
const sinon = require('sinon');
const SinonChai = require('sinon-chai');
chai.use(SinonChai);
chai.should();
const proxy = require('proxyquire');
describe('fetch test demo', () => {
it('fetch should of called twice', (done) => {
var fetchStub = sinon.stub();
var fetchStub2 = sinon.stub();
var fetch = sinon.stub();
fetchStub.returns(Promise.resolve('hello'));
fetchStub2.returns(Promise.resolve('hi'));
var promises = [fetchStub, fetchStub2];
fetch.returns(Promise.all(promises));
proxy('./updated', {
'node-fetch': fetch
});
setTimeout(() => {
fetch.should.have.been.callCount(2);
done();
}, 10);
});
});
As you can see, the only change made was wrapping the assertion within a timer block. Nothing much - just wait for 10ms and then assert. Now the test passes as expected. Why?
Well, to me it's pretty straightforward. You want to test 2 sequentially executed async functions and still run your assertions in sync mode. That sounds cool, but it's not gonna happen :) So you have 2 options:
Have your functions notify callers upon completion and then run your assertions in truly async mode
Mimic the async nature of things using unorthodox techniques
Reply based on original test scenario
It can be done. I've re-factored your provided files a bit so that
can be executed.
index.js
const fetch = require('node-fetch');
const sendAlert = require('./alerts').sendAlert;
module.exports.init = function () {
return new Promise((resolve, reject) => {
fetch('https://localhost')
.then(function () {
sendAlert().then(() => {
resolve();
}).catch(
e => reject(e)
);
})
.catch(e => {
reject(e);
});
});
};
alerts.js
const fetch = require('node-fetch');
module.exports.sendAlert = function () {
return new Promise((resolve, reject) => {
fetch('https://localhost')
.then(function () {
resolve();
}).catch((e) => {
reject(e);
});
});
};
test.js
'use strict';
const chai = require('chai');
const sinon = require('sinon');
const SinonChai = require('sinon-chai');
chai.use(SinonChai);
chai.should();
const proxy = require('proxyquire');
describe.only('lifx alert test', () => {
it('fetch should of called twice', (done) => {
var body = {
'hourly': {
data: [{
time: 1493413200,
icon: 'clear-day',
precipIntensity: 0,
precipProbability: 0,
ozone: 297.17
}]
}
};
var response = {
json: () => {
return body;
}
};
const fetchStub = sinon.stub();
fetchStub.returns(Promise.resolve(response));
fetchStub['#global'] = true;
var stubs = {
'node-fetch': fetchStub
};
const p1 = proxy('./index', stubs);
p1.init().then(() => {
try {
fetchStub.should.have.been.calledTwice;
done();
} catch (e) {
done(e);
}
}).catch((e) => done(e));
});
});
What you're trying to do though is a bit unorthodox when it comes to
good unit testing practices. Although proxyquire supports this
mode of stubbing through a feature called global overrides, it is
explained here why should anyone think twice before going down
this path.
In order to make your example pass the test, you just need to add an
extra attribute to the Sinon stub called #global and set it to
true. This flag overrides the require() caching mechanism and
uses the provided stub no matter which module is called from.
So, although what you're asking can be done I will have to agree with
the users that commented your question, that this should not be
adopted as a proper way of structuring your tests.
Here is also a alternative way to do this using Promise.all().
Note: this won't work if using fetch's json method and you need to pass data in the resolve() for logic on data. It will only pass in the stubs when resolved. However, it will assert the number of times called.
describe('fetch test demo', () => {
it('fetch should of called twice', () => {
let fetchStub = sinon.stub();
let fetchStub2 = sinon.stub();
let fetch = sinon.stub();
fetchStub.returns(Promise.resolve('hello'));
fetchStub2.returns(Promise.resolve('hi'));
var promises = [ fetchStub, fetchStub2 ]
var promise = Promise.all(promises);
fetch.returns(promise);
proxy('../foobar', { 'node-fetch': fetch });
return promise.then(() => {
fetch.should.have.callCount(2);
});
});
});
I have found another way to get things done.
May be this could work for someone.
describe('Parent', () => {
let array: any = [];
before(async () => {
array = await someAsyncDataFetchFunction();
asyncTests();
});
it('Dummy test to run before()',async () => {
expect(0).to.equal(0); // You can use this test to getting confirm whether data fetch is completed or not.
});
function asyncTests() {
array.forEach((currentValue: any) => {
describe('Child', async () => {
it('Test '+ currentValue ,() => {
expect(currentValue).to.equal(true);
})
})
});
}
});
That's how I achieved the assertion on every element of the array. (Array data is being fetch asynchronously).

Chai 'expect' doesn't execute inside a callback

I'm writing unit-tests for my function that fetches info from some REST API. I am using ramda Future type (source).
The following test works weird:
it('should return Maybe of Nothing', done => {
let response = {
status: 200,
json: () => {
return {
results: []
}
}
}
let fakeFetch = {
fetch: () => {
return new Promise((resolve, reject) => {
resolve(response)
})
}
}
// String -> Future Error Maybe
let result = Utils.fetchGiantBomb(faker.random.word(), fakeFetch.fetch);
result.fork(err => {
assert.fail(err, 'expected to return Maybe of Nothing');
done();
}, data => {
expect(Maybe.isJust(data)).to.be.true;
done();
})
})
data should be of type Maybe.Nothing. If I expect Maybe.isNothing the test passes, but I want to see what happens when the test fails, so I set it to Maybe.isJust, which return false. After looking at this for a while, I noticed that when the expect fail it jump up to the error handling (err callback), which then just stop executing any assertion (which result in a 2000ms timeout).
In the Future sources I saw that when the success callback fails, it executes the failure callback. How can I complete this test so it display that the data is not what I expect?
I think the problem is that, when your REST call fails, done() is never called.
Not sure if expect has a .catch method when it fails, but you can try to add
.catch(done);
at the end of your expect function.
Hope that helps.
Calling future.fork(errorHandler, successHandler) will currently ensure that any exceptions thrown in the successHandler will propagate through to the errorHandler.
One way around this (though perhaps not ideal as it is undocumented) is to call future._fork(errorHandler, successHandler) rather than future.fork where errors thrown in the successHandler will not be captured.
Alternatively, a number of test frameworks support passing an error to the done callback such as:
result.fork(err => {
done('Expected to return Maybe of Nothing: ' + err);
}, data => {
expect(Maybe.isJust(data)).to.be.true;
done();
})
I think Ramda shouldn't catch the exception there. But i don't know what's they are trying to do.
It look like you are using Mocha. It maybe good idea to convert your Future to Promise first, then observe the Promise. ie:
const futureToPromise = future => {
return new Promise((resolve, reject) => future.fork(reject, resolve))
}
it('should return Maybe of Nothing', () => {
let response = {
status: 200,
json: () => {
return {
results: []
}
}
}
let fakeFetch = {
fetch: () => {
return new Promise((resolve, reject) => {
resolve(response)
})
}
}
// String -> Future Error Maybe
let result = Utils.fetchGiantBomb(faker.random.word(), fakeFetch.fetch);
// return it because Mocha can handle this
return futureToPromise(result).then(data => {
expect(Maybe.isJust(data)).to.be.true;
}, () => {
// fail
assert.fail(err, 'expected to return Maybe of Nothing');
})
})

Categories