I'm trying to use the async/await functionality to build a node JS script. I currently have a file called repo.js as a helper file to get data from Github's API and return it to a variable for me to access elsewhere in different JS files of my node application, repo.js is as such:
const axios = require('axios')
const repo = async () => {
const data = await axios.get('https://api.github.com/repos/OWNER/REPO/releases', {
headers: {
'Authorization': 'token MYTOKEN'
}
})
return data
}
exports.repo = repo
And then in my main.js file I'm trying to do...
const repo = require('./src/utils/repo')
program
.option('-d, --debug', 'output extra debugging')
.option('-s, --small', 'small pizza size')
.option('-p, --pizza-type <type>', 'flavour of pizza')
const repoData = repo.repo
console.log(repoData)
Unfortunately, this just returns [AsyncFunction: repo] to the console which isn't the intended behaviour. Why can't I access the contents here?
UPDATE
Based on some responses I've been given, I'm aware of the fact I need my code inside of a async function or to use .then(). The issue is, I don't want to put all of my application's code inside of a .then() just to rely on one thing from an API.
Example:
var version = ''
repo.getRepoDetails().then((res) => {
version = res.data[0].body.tag_name
})
Now I have access to version everywhere.
Every async/await function is a promise, meaning that you need to wait for it to finish in order to read it's result.
repo.repo().then(res => console.log(res))
If you application is a simple nodejs script(or single file) then you can wrap your code inside an IIFE like this:
const repo = require('./src/utils/repo')
(async () => {
program
.option('-d, --debug', 'output extra debugging')
.option('-s, --small', 'small pizza size')
.option('-p, --pizza-type <type>', 'flavour of pizza')
const repoData = await repo.repo() <--- You can use await now instead of then()
console.log(repoData)
})()
Async function always return promise object so you can access the result using promise.then() like
repo.repo().then(result => result)
Related
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Use async await with Array.map
(9 answers)
Closed 27 days ago.
In an async IIFE at the bottom of this javascript, you'll see that I'm trying to: 1) read a JSON file, 2) get multiple RSS feed URLs from that data, 3) pull and parse the data from those feeds, and create an object with that data, so I can 4) write that pulled RSS data object to a JSON file. Everything for #1 and #2 is fine. I'm able to pull data from multiple RSS feeds in #3 (and log it to console), and I'm comfortable handling #4 when I get to that point later.
My problem is that, at the end of step #3, within the const parseFeed function, I am trying to create and push an object for that iteration of rssJSONValsArr.map() in the IIFE and it's not working. The rssFeedDataArr result is empty. Even though I am able to console.log those values, I can't create and push the new object I need in order to reach step #4. My creating of a similar object in #2 works fine, so I think it's the map I have to use within const parseFeed to pull the RSS data (using the rss-parser npm package) which is making object creation not work in step #3. How do I get rssFeedOject to work with the map data?
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import Parser from 'rss-parser';
const parser = new Parser();
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const feedsJSON = path.join(__dirname, 'rss-feeds-test.json');
const rssJSONValsArr = [];
const rssFeedDataArr = [];
const pullValues = (feedObject, i) => {
const url = feedObject.feed.url;
const jsonValsObject = {
url: url,
};
rssJSONValsArr.push(jsonValsObject);
};
const parseFeed = async (url) => {
try {
const feed = await parser.parseURL(url);
feed.items.forEach((item) => {
console.log(`title: ${item.title}`); // correct
});
const rssFeedOject = {
title: item.title,
};
rssFeedDataArr.push(rssFeedOject);
} catch (err) {
console.log(`parseFeed() ERROR 💥: ${err}`);
}
};
(async () => {
try {
console.log('1: read feeds JSON file');
const feedsFileArr = await fs.promises.readFile(feedsJSON, {
encoding: 'utf-8',
});
const jsonObj = JSON.parse(feedsFileArr);
console.log('2: get feed URLs');
jsonObj.slice(0, 30).map(async (feedObject, i) => {
await pullValues(feedObject, i);
});
console.log('rssJSONValsArr: ', rssJSONValsArr); // correct
console.log('3: pull data from rss feeds');
rssJSONValsArr.map(async (feedItem, i) => {
await parseFeed(feedItem.url, i);
});
console.log('rssFeedDataArr: ', rssFeedDataArr); // empty !!!
// console.log('4: write rss data to JSON file');
// await fs.promises.writeFile(
// `${__dirname}/rss-bulk.json`,
// JSON.stringify(rssFeedDataArr)
// );
console.log('5: Done!');
} catch (err) {
console.log(`IIFE CATCH ERROR 💥: ${err}`);
}
})();
Example JSON file with two RSS feed URLs:
[
{
"feed": {
"details": {
"name": "nodejs"
},
"url": "https://news.google.com/rss/search?q=nodejs"
}
},
{
"feed": {
"details": {
"name": "rss-parser"
},
"url": "https://news.google.com/rss/search?q=rss-parser"
}
}
]
Any and all help appreciated. Thanks
The problem is you are printing rssFeedDataArr right after the .map call, which, like stated on the comments, is being incorrectly used, since you are not using the returned value, forEach would be the way to go here. For every value in rssJSONValsArr you are calling an anonymous and async function which in turn awaits for parseFeed, so you are basically creating a Promise in each iteration, but obviously those promises are resolved after your print statement is executed. You need to wait for all of those promises to be resolved before printing rssFeedDataArr. One way to do that, since you are creating a bunch of promises which can be run in parallel is to use Promise.all, like this:
await Promise.all(
rssJSONValsArr.map(async (feedItem, i) => {
await parseFeed(feedItem.url, i);
});
)
and you we can simplify it even more and return the promise created by parseFeed directly:
await Promise.all(
rssJSONValsArr.map((feedItem, i) => parseFeed(feedItem.url, i))
)
And in this case the right method is map and not forEach
In the case of rssJSONValsArr it works because the call to pullValues is being resolved instantly, it doesnt run asynchronously, even when its declared as async, there is not await inside the function definition.
I am developing an application using Node.js. I have been using callbacks but I recently started migrating to promises. My problem is that on the latest electron (12.0.0) and Node 14.15 promises just sometimes don't work. No error or anything, it just doesn't work or takes several seconds to. Check the following snippet
const fs=require('fs/promises')
test()
function test() {
fs.readFile('views/database_connection_status.html', 'utf-8').then((data) => {
console.log(data)
}).catch((error) => {
console.log(error)
})
}
The file does exist. Sometimes it loads, and displays correctly and sometimes just nothing.
const co = require('co');
const fs = require('fs');
test().then(res => {
callback(null,res)
}).catch(error => callback(error));
const test = () =>{
return co(function* (){
return fs.readFile('views/database_connection_status.html','utf-8)
}
}
Hey, Joseph, you can check out the "co" package for functions with promises, your code will look like the above.
Can read more about it https://www.npmjs.com/package/co
I have a file that basically looks like this(shortened)
const octokit = new (require("#octokit/rest"))();
function buildRepo(name) {
fs.promises
.readFile("data/settings.json")
.then(data => JSON.parse(data))
.then(settings => settings.repositories.find(repo => repo.name === name))
.then(repo => {
let repoName = repo.url
.substring(repo.url.lastIndexOf("/") + 1)
.slice(0, -4);
let jobName = repo.name;
return octokit.repos
.get({
owner: "munhunger",
repo: repoName
})
.then(({ data }) => {
...
});
});
}
module.exports = { buildRepo };
And so I want to write a test for what it does with the data that it gets from the octokit.repos.get function. But since that function will go out to the internet and look at GitHub repositories, I want to mock it.
I have a few tests running with jasmine, and I read up slightly on it and it seems as if jasmine should be able to mock this for me.
However, the test that I have written seems to fail.
const builder = require("./index");
describe("spyOn", () => {
it("spies", () => {
spyOnProperty(builder, "octokit");
builder.buildRepo("blitzbauen");
});
});
With the error octokit property does not exist. What am I doing wrong here? would I need to add octokit to module.exports?(which seems rather insane)
Yes, you'd need to add Octokit to module.exports since you now only export buildRepo.
Anything from a module that is not exported can't be accessed directly by other modules, so if it should be accessible it should be exported.
Alternatively, you may be able to mock the entire Octokit module with Jasmine so any calls by any script are made to the mocked version, but I'm not sure how you'd go about doing that since my experience with Jasmine is limited
In the following code, I'm reading some files and getting their filename and text. After that, I'm storing data in an option variable to generate an epub file:
const Epub = require("epub-gen")
const folder = './files/'
const fs = require('fs')
let content = []
fs.readdir(folder, (err, files) => {
files.forEach(filename => {
const title = filename.split('.').slice(0, -1).join('.')
const data = fs.readFileSync(`${folder}${filename}`).toString('utf-8')
content.push({ title, data })
})
})
const option = {
title: "Alice's Adventures in Wonderland", // *Required, title of the book.
content
}
new Epub(option, "./text.epub")
The problem is, new Epub runs before the files are read, before content is ready. I think Promise.all is the right candidate here. I checked the Mozilla docs. But it shows various promises as example, but I have none. So, I'm not very sure how to use Promise.all here.
Any advice?
Your problem is with readdir, which is asynchronous so new Epub, like you already figured out, is called before it's callback parameter.
Switch to using readdirSync or move const option ... new Epub... inside the callback parameter of readdir, after files.forEach.
At the moment you can do everything synchronous since you use readFileSync.
So you can place the Epub creation after the forEach loop.
If you want to go async, my first question would be:
Does your node.js version support util.promisify ( node version 8.x or higher iirc )?
If so, that can be used to turn the callback functions like readFile and such into promises. If not, you can use the same logic, but then with nested callbacks like the other solutions show.
const FS = require( 'fs' );
const { promisify } = require( 'util' );
const readFile = promisify( FS.readFile );
const readFolder = promisify( FS.readFolder );
readFolder( folder )
// extract the file paths. Maybe using `/${filename}` suffices here.
.then( files => files.map( filename => `${folder}${filename}`))
// map the paths with readFile so we get an array with promises.
.then( file_paths => file_paths.map( path => readFile( path )))
// fecth all the promises using Promise.all() .
.then( file_promises => Promise.all( file_promises ))
.then( data => {
// do something with the data array that is returned, like extracting the title.
// create the Epub objects by mapping the data values with their titles
})
// error handling.
.catch( err => console.error( err ));
Add promises to an array. Each promise should resolve with the value you were pushing into content
When all promises resolve, the returned value will be the array previously known as content.
Also, you can, and should, use all async fs calls. So readFileSync can be replaced with readFile (async). I did not replace your code with this async call however, so you can clearly see what was required to answer your original question.
Not sure if I got the nesting right in snippet.
const Epub = require("epub-gen")
const folder = './files/'
const fs = require('fs')
let promises = []
fs.readdir(folder, (err, files) => {
files.forEach(filename => {
promises.push(new Promise((resolve, reject) => {
const title = filename.split('.').slice(0, -1).join('.')
const data = fs.readFile(`${folder}${filename}`).toString('utf-8')
resolve({
title,
data
})
}))
})
})
const option = {
title: "Alice's Adventures in Wonderland", // *Required, title of the book.
content
}
new Epub(option, "./text.epub")
Promise.all(promises).then((content) => {
//done
})
I have a API script in a file
const ApiCall = {
fetchData: async (url) => {
const result = await fetch(url);
if (!result.ok) {
const body = await result.text(); // uncovered line
throw new Error(`Error fetching ${url}: ${result.status} ${result.statusText} - ${body}`); // uncovered line
}
return result.json();
},
};
export default ApiCall;
When I mock the call, I have two uncovered lines in code coverage.
Any idea how can I make them cover as well.
Here is what I have tried so far which is not working
it('test', async () => {
ApiCall.fetchData = jest.fn();
ApiCall.fetchData.result = { ok: false };
});
I am kind of new into Jest, so any help would be great.
You need to provide a stubb response in your test spec so that the if statement is triggered. https://www.npmjs.com/package/jest-fetch-mock will allow you to do just that. The example on their npm page should give you what you need https://www.npmjs.com/package/jest-fetch-mock#example-1---mocking-all-fetches
Basically the result is stored in state(redux) and is called from there. jest-fetch-mock overrides your api call/route and returns the stored result in redux all within the framework.
Assuming that what you want to test is the ApiCall then you would need to mock fetch. You are mocking the entire ApiCall so those lines will never execute.
Also, you have an issue, because if you find an error or promise rejection, the json() won't be available so that line will trigger an error.
Try this (haven't test it):
it('test error', (done) => {
let promise = Promise.reject(new Error("test"));
global.fetch = jest.fn(() => promise); //You might need to store the original fetch before swapping this
ApiCall.fetchData()
.catch(err => );
expect(err.message).toEqual("test");
done();
});
it('test OK', (done) => {
let promise = Promise.resolve({
json: jest.fn(() => {data: "data"})
});
global.fetch = jest.fn(() => promise);
ApiCall.fetchData()
.then(response => );
expect(response.data).toEqual("data");
done();
});
That probably won't work right away but hopefully you will get the idea. In this case, you already are working with a promise so see that I added the done() callback in the test, so you can tell jest you finished processing. There is another way to also make jest wait for the promise which is something like "return promise.then()".
Plese post back