I'm doing my first ever react website and I need help to write an asynchronous JavaScript function.
Here I'm uploading the user input files to a firebase storage and then making a post request to the API to store the data on the database. However, since the firebase upload takes some time to upload the data to its storage, the API request happens before the upload finishes, therefore the data does not get uploaded to the db. Now I know I should use promises of async await keywords here, but I can't figure out how to. I'd appreciate if someone could help. Thanks in advance!
Here's the relevant code snippet:
const save = (items) => {
items.forEach((item) => {
const fileName = new Date().getTime() + item.label + item.file.name;
const uploadTask = storage.ref(`/items/${fileName}`).put(item.file);
uploadTask.on(
"state_changed",
(snapshot) => {
const progress = (snapshot.bytesTransferred / snapshot.totalBytes) * 100;
console.log("Upload is" + progress + "% done.");
},
(err) => {
console.log(err)
},
() => {
storage.ref("items").child(fileName).getDownloadURL().then((url) => {
setSong((prev) => {
return { ...prev, [item.label]: url };
});
});
}
);
})
console.log(song)
axios.post("songs/create", song);
}
PS: Here, items is the array of input files from the user, each file is with a label and it is how the attributes are named on the json document.
setSong is a useState function. Here The JSON document already contains the other user inputs(which are not files), and the setSong method is used to append the firebase URLs of the files to it.
You have to wait for all files to get uploaded then you can call your API, in order to do that you should use Promise.all to wait to resolve all files :
const save = items => {
Promise.all(
items.map(item => {
return new Promise(resolve => {
const fileName = new Date().getTime() + item.label + item.file.name
const uploadTask = storage.ref(`/items/${fileName}`).put(item.file)
uploadTask.on(
'state_changed',
snapshot => {
const progress =
(snapshot.bytesTransferred / snapshot.totalBytes) * 100
console.log('Upload is' + progress + '% done.')
},
err => {
console.log(err)
},
() => {
storage
.ref('items')
.child(fileName)
.getDownloadURL()
.then(url => {
setSong(prev => {
return { ...prev, [item.label]: url }
})
resolve({[item.label]: url})
})
}
)
})
})
).then((res) => {
const song = {}
res.forEach(item => {
return {
...song,
...item
}
})
axios.post('songs/create', song)
})
}
Explanation
Functions and Async
Async/Await can be implemented wherever a function starts. Functions can be written in following forms:
function name(){};
function name() => {};
To write an async function, you would do the following:
async function name(){};
All of these functions are called functions though, to make functions run without calling them, we need to turn them into IIFE's, or Immediately Invoked Function Execution. If you want to create a function and execute it immediately you would surround the function in ()'s and end it with an ();.
(function () {})();
If we simplify this:
(() => {})();
Implementing async would go like this:
(async () => {})();
Await
The await operator is used to wait for a Promise, puting await in front of an expression that uses promises makes it wait for the promise. If it is used in front of an expression that doesn't use promises, it is redundant and your code editor/IDE will say so.
(async () => {
const str = await 'some string';
console.log(str);
})();
await is redundant here since the expression 'some string' does not relate to a promise, but a string.
(async () => {
const myPromise = new Promise((resolve, reject) =>
resolve('some string')
);
const str = await myPromise.then(x => console.log(x));
})();
await is properly used here since the expression myPromise is related to a promise that outputs a string.
Implementation
I'm not 100% sure how the api works within promises, I recommend you figure it out yourself, but this is my educated guess:
const save = (items) => {
Promise.all(
items.map((item) => {
return new Promise(async (resolve) => {
const fileName = new Date().getTime() + item.label + item.file.name;
const uploadTask = await storage
.ref(`/items/${fileName}`)
.put(item.file);
await uploadTask.on(
"state_changed",
(snapshot) => {
const progress =
(snapshot.bytesTransferred / snapshot.totalBytes) * 100;
console.log("Upload is" + progress + "% done.");
},
(err) => {
console.log(err);
},
async () => {
await storage
.ref("items")
.child(fileName)
.getDownloadURL()
.then((url) => {
setSong((prev) => {
return { ...prev, [item.label]: url };
});
resolve({ [item.label]: url });
});
}
);
});
})
).then(async (res) => {
const song = {};
res.forEach((item) => {
return {
...song,
...item,
};
});
await axios.post("songs/create", song);
});
};
Related
Overview
I am working in a react app using redux. I have an action that checks wether or not we have new image data. If so, it will upload it, receive the new URL and then use this data to update the object.
Problem
However, the order of operations within my function are working as expected, however code outside the function runs before it is completed.
Question
Why is my console log at the bottom executing before the contents of my async function are completed?
if(imageData) {
const imageGUID = guid();
const storageRef = projectStorage.ref(`${imageData.name}_${imageGUID}`);
// This function should complete before the console log at the bottom is called.
await storageRef.put(imageData).on('state_changed', snap => {
}, async (err) => {
console.log(err)
toastr.error("Uh Oh!", `Could not upload image`);
}, async () => {
imageURL = await storageRef.getDownloadURL();
console.log("NEW IMAGE URL: ", imageURL);
})
}
console.log("Done setting up new image: ", imageURL) // This is called before we get the IMAGE URL from Firestore.... why?
The .on function does not return a Promise so there is nothing to wait for the await. You have to convert the event base API of put to a Promise.
if (imageData) {
const imageGUID = guid();
const storageRef = projectStorage.ref(`${imageData.name}_${imageGUID}`);
// This function should complete before the console log at the bottom is called.
try {
await new Promise((resolve, reject) => {
storageRef.put(imageData)
.on('state_changed', snap => {},
reject, resolve)
})
imageURL = await storageRef.getDownloadURL();
console.log("NEW IMAGE URL: ", imageURL);
} catch (err) {
console.log(err)
toastr.error("Uh Oh!", `Could not upload image`);
}
}
console.log("Done setting up new image: ", imageURL)
storageRef.put(imageData).on - doesn't look like promise (you upload your image in callback), so await doesn't make sense
If you want to use promises, you should write it something like that
await new Promise((resolve, reject) => {
storageRef
.put(imageData)
.on('state_changed', snap => {
}, async (err) => {
console.log(err);
toastr.error('Uh Oh!', `Could not upload image`);
reject()
}, async () => {
imageURL = await storageRef.getDownloadURL();
console.log('NEW IMAGE URL: ', imageURL);
resolve()
});
})
This question already has answers here:
How to "await" for a callback to return?
(5 answers)
Closed 1 year ago.
I'm currently working with Firebase to upload an image to the storage.
I created an uploadImage function which takes an image and saves it to Firebase Storage then returns the url of that saved image. When the user clicks the Submit button, I use that uploadImage function to save that image and get back a url. The image is properly being saved, but the url that is being logged is undefined. I'm pretty sure it's an issue with the aysnc-await implementation I have. Any thoughts on this? Thank you!
UploadForm.js:
import { uploadImage } from "../firebase/functions";
const UploadForm = () => {
const [image1, setImage1] = useState(null);
const saveForm = async (file) => {
const url1 = await uploadImage(image1);
console.log("URL", url1); //the url here is UNDEFINED
};
return (
<ImageUploadForm setImageProp={setImage1} />
<button
onClick={(e) => {
e.preventDefault();
saveForm(image1);
}}
></button>
);
uploadImage function:
const uploadImage = async (file) => {
const storageRef = storageService.ref().child("posts/" + file.name);
storageRef.put(file).on(
"state_changed",
(snapshot) => {},
(error) => {
console.log(error);
},
async () => {
return await storageRef.getDownloadURL();
}
);
};
Your uploadImage function doesn't return anything, just return the promise
return storageRef.put(file).on(
or, (since I don't know how that function works / what it returns), possibly
const uploadImage = (file) => {
return new Promise((resolve, reject) => {
const storageRef = storageService.ref().child("posts/" + file.name);
storageRef.put(file).on(
"state_changed",
(snapshot) => {},
(error) => {
console.log(error);
reject(error);
},
() => {
const res = storageRef.getDownloadURL();
resolve(res);
}
);
};
}
I am trying to log the data of a promise to my console but it's not showing. i have tried defining then in then and on top of functions and tried with let and redefining the before executing the algorithm but. no response
sample
var trade;
const getTrades = async () => {
return await axios({
method: 'get',
url: bUrl + tradeQuery
})
}
const getSOrders = async () => {
return await axios({
method: 'get',
url: bUrl + mOrderQuery
})
}
const postOrder = async() => {
const binanceRest = new api.BinanceRest({
...
}
)
binanceRest.newOrder({
...
})
.then(async(data) => {
const trades = await getTrades()
const mOrders = await getSOrders()
console.log(data)
console.log(trades)
})
.catch((err) => {
console.error(err)
})
}
(
postOrder(),
async () => {
const trades = await getTrades()
const mOrders = await getSOrders()
const sells = mOrders.data.asks
const buys = mOrders.data.bids
while (true/*while order is in */) {
trade = trades.data[trades.data.length - 1]
console.log(sells)
}
}
)()
Your code is stuck in a an infinite while loop. Since node is single threaded, it will continue to execute the loop until broken and never execute the .then code in the original promise. You should be able to easily see this by commenting out the while loop.
Specifically:
binanceRest.newOrder is async call. It's true that postOrder is called first, but depending on the how nodejs decides to optimize, all of the awaits are hit at the same time. When the awaits resolve, if Nodejs decides that postOrders will resume your console.log will write, however if the while loop it hit first, then your program will be stuck there.
I'm sorry to say this, but I can see a lot of confusing things in this code.
while(true) {
trade = trades.data[trades.data.length - 1];
console.log(sells);
}
this will never exits. If you omitted part of your code is not a good idea as this makes us harder to give you the actual answer to your question. If this is your real code, I would change in
while(trades.data.length) {
trade = trades.data.pop();
console.log(sells);
}
postOrder is an async function which deals with Promise, and this is confusing: I would rewrite it as
const postOrder = async() => {
try {
const binanceRest = new api.BinanceRest({ ... });
const data = await binanceRest.newOrder({ ... });
const trades = await getTrades()
const mOrders = await getSOrders()
console.log(data)
console.log(trades)
}
catch(err) {
console.error(err)
}
}
Last postOrder is an async function which is called without await and this is source of confusion as well.
I would start cleaning your code, probably many problem will be solved as well.
Hope this helps.
Is not an "await" problem
I've added a bunch of console.log instructions at your code and there's no problem printing anything. Try adding the same to your original code, assuming your API's are working fine there's no reason for failure.
The following snippet mocks your API's, please take a look at the result. Your error must be at your logic, api's or syntax.
Also add a safe break inside your while, something like this: if (calls++ > 5) { break; }, to me it seems that the infinite loop is the problem.
/* MOCK API'S */
let test = 0, calls = 0;
const fakePromise = (arg, apiName) => { return { then: (f1, f2) => { console.log('mocked result for ' + apiName); return f1(arg); } }; };
const axios = (args) => { console.log('-> axios is called', args); return fakePromise({data: { asks: ['ask'], bids: ['bid'], length: 1, 0: 'fake index'} }, args.url); };
const api = { BinanceRest: function(){ return { newOrder: () => fakePromise(' -> newOrder result <- ', 'newOrder')}; } };
const bUrl = 'bUrl/', mOrderQuery = 'mOrderQuery', tradeQuery = 'tradeQuery';
/* YOUR CODE STARTS HERE */
var trade;
const getTrades = async () => {
console.log('-> getTrades is called');
return await axios({
method: 'get',
url: bUrl + tradeQuery
});
}
const getSOrders = async () => {
console.log('-> getSOrders is called');
return await axios({
method: 'get',
url: bUrl + mOrderQuery
})
}
const postOrder = async() => {
console.log('step 1');
const binanceRest = new api.BinanceRest({
// ...
});
console.log('step 2');
binanceRest.newOrder({
// ...
})
.then(async(data) => {
console.log('step 3');
const trades = await getTrades()
const mOrders = await getSOrders()
console.log('step 4.2');
console.log('data: ', data)
console.log('trades: ', trades)
console.log('mOrders', mOrders)
})
.catch((err) => {
console.error(err)
})
}
// test
(
postOrder(),
async () => {
console.log('step 4.1');
const trades = await getTrades()
const mOrders = await getSOrders()
console.log('step 5', mOrders);
const sells = mOrders.data.asks
const buys = mOrders.data.bids
console.log('step 5.0');
while (true/*while order is in */) {
console.log('step 5.' + (test++));
trade = trades.data[trades.data.length - 1]
console.log('sells: ', sells)
if (calls++ > 5) {
console.log('way too much calls, break! ');
break;
}
}
}
)()
Take a closer look at your postOrder func:
const postOrder = async() => {
const binanceRest = new api.BinanceRest({ ... })
binanceRest.newOrder({ ... })
.then(...)
.catch(...)
}
Now imagine the scenario when a typo has been made, f.e. it should be new api.FinanceRest(...) and as a result you end up with error:
Uncaught (in promise) TypeError: api.BinanceRest is not a function
Rest of code in postOrder is simply skipped, no console.logs. So the main point here: if in new api.BinanceRest({ ... }) something leads to an error (any error) than you'll end up with situation you have right now.
And also worth to mention that an async anonymous function with while loop still gets executed and that's because postOrder is an async func which means it returns a pending Promise.
Try running this in you browsers console to get more sense of what's goin' on:
function delay(sec) {
return new Promise((res, rej) => {
setTimeout(() => res("delay " + sec + "sec"), sec * 1000);
});
}
async function getTrades() {
return await delay(0.1);
};
async function getSOrders () {
return await delay(0.2);
};
async function postOrder() {
console.log("postOrder");
const api = {};
const binanceRest = api.BinanceRest()
delay(0.4)
.then(async (data) => {
console.log("postOrder result")
const trades = await getTrades()
const mOrders = await getSOrders()
console.log(trades)
console.log(mOrders)
})
.catch(err => {
console.error(err);
});
};
(postOrder(),
async () => {
const trades = await getTrades();
const mOrders = await getSOrders();
console.log("gfdfg");
})();
I need some help with this helper I'm writing. For some reason using reduction within an async on a readFile, when trying to write results to a file it won't advance to the next item of the array. However, if I use a console.log, it works just fine.
const neatCsv = require('neat-csv');
const fetch = require('node-fetch');
const fs = require('fs');
fs.readFile('./codes.csv', async (err, data) => {
if (err) { throw err; }
let baseUrl = 'https://hostname/orders?from=2019-10-21T00:00:00.001Z&to=2019-12-31T23:59:59.000Z&promo=';
const starterPromise = Promise.resolve(null);
const promos = await neatCsv(data);
const logger = (item, result) => console.log(item, result);
function write (item, result) {
return new Promise((resolve, reject) => {
fs.writeFile(`./output/${item.PROMO}.json`, JSON.stringify(result), (err) => {
if (err) { throw err; }
console.log(`Wrote file ${item.PROMO}`);
});
})
}
function asyncFetch(item) {
console.log(`runTask <---------${item.PROMO}---------`);
return fetch(`${baseUrl}${item.PROMO}`, { headers: { 'x-apikey': 'xyz' }})
.then(res => (res.json())
.then(json => json))
}
await promos.reduce(
(p, item) => p.then(() => asyncFetch(item).then(result => write(item, result))),
starterPromise
)
});
The csv file is just a basic layout like so..
PROMO
12345
56789
98765
...
The goal is to iterate over these, make a REST call to get the json results and write those to a file with the name of the current promo, then move to the next, making a new call and saving that one into a different file with its respective code.
In the reduce, if you call logger instead of write, it works fine. Calling write, it just makes the same call over and over and overwriting to the same file, forcing me to kill it. Please help, I'm losing my mind here...
You might have a better time using async functions everywhere, the fs promises API and a simple while loop to consume the CSV items. Dry-coded, naturally, since I don't have your CSV or API.
(Your original problem is probably due to the fact you don't resolve/reject in the write function, but the reduce hell isn't needed either...)
const neatCsv = require("neat-csv");
const fetch = require("node-fetch");
const fsp = require("fs").promises;
const logger = (item, result) => console.log(item, result);
const baseUrl = "https://hostname/orders?from=2019-10-21T00:00:00.001Z&to=2019-12-31T23:59:59.000Z&promo=";
async function asyncFetch(item) {
console.log(`runTask <---------${item.PROMO}---------`);
const res = await fetch(`${baseUrl}${item.PROMO}`, { headers: { "x-apikey": "xyz" } });
const json = await res.json();
return json;
}
async function write(item, result) {
await fsp.writeFile(`./output/${item.PROMO}.json`, JSON.stringify(result));
console.log(`Wrote file ${item.PROMO}`);
}
async function process() {
const data = await fsp.readFile("./codes.csv");
const promos = await neatCsv(data);
while (promos.length) {
const item = promos.shift();
const result = await asyncFetch(item);
await write(item, result);
}
}
process().then(() => {
console.log("done!");
});
A version that uses mock data and the JSON Placeholder service, works just fine:
const fetch = require("node-fetch");
const fsp = require("fs").promises;
const baseUrl = "https://jsonplaceholder.typicode.com/comments/";
async function asyncFetch(item) {
console.log(`runTask <---------${item.PROMO}---------`);
const res = await fetch(`${baseUrl}${item.PROMO}`);
return await res.json();
}
async function write(item, result) {
const data = JSON.stringify(result);
await fsp.writeFile(`./output/${item.PROMO}.json`, data);
console.log(`Wrote file ${item.PROMO}: ${data}`);
}
async function getItemList() {
return [
{PROMO: '193'},
{PROMO: '197'},
{PROMO: '256'},
];
}
async function process() {
const promos = await getItemList();
while (promos.length) {
const item = promos.shift();
const result = await asyncFetch(item);
await write(item, result);
}
}
process().then(() => {
console.log("done!");
});
I'm using through2 to generate multiple files from a Gulp stream. I'm using NodeJS 10.6.0 so thought I'd make full use of async/await, but am not fully understanding the mechanics yet. Currently the through2 done() callback is being fired before all files have been written.
Here's what I have (simplified) - note that I'm not returning the stream at the end as there is no need to.
async function createDirectory(pathDir) {
return new Promise((resolve, reject) => {
mkdirp(pathDir, (err) => {
if (err) reject(err);
else resolve();
});
});
}
async function writeFile(outputFilePath, outputFileContent) {
return new Promise((resolve, reject) => {
fs.writeFile(outputFilePath, outputFileContent, (err) => {
if (err) reject(err);
else resolve();
});
});
}
async function doWriteFile(outputFolderPath, outputFilePath, outputContent) {
await createDirectory(outputFolderPath);
await writeFile(outputFilePath, outputContent, outputContent);
}
async function doGenerateVariant(data, variantArr) {
for (const variant of variantArr) {
/* Do a load of stuff */
const variantOutputFolderPath = blah;
const variantOutputFilePath = blah;
const variantOutputContent = blah;
await doWriteFile(variantOutputFolderPath, variantOutputFilePath, variantOutputContent);
}
}
const generateVariant = () => {
return through.obj((file, enc, done) => {
const data = JSON.parse(file.contents.toString());
*/ Do a load of stuff */
const { variant } = data;
const variantArr = Object.values(variant);
doGenerateVariant(data, variantArr);
return done();
});
};
This doesn't work as done() gets returned before all files have been written. I'm guessing I'm missing a return or two but nothing I do seems to be working.
If I pass done() into doGenerateVariant and call it after doWriteFile everything works as expected but I know this isn't correct.
You need to wait for doGenerateVariant to do its job before calling done. Remember async function always returns a Promise. So you could do it this way
const generateVariant = () => {
return through.obj((file, enc, done) => {
const data = JSON.parse(file.contents.toString());
*/ Do a load of stuff */
const { variant } = data;
const variantArr = Object.values(variant);
doGenerateVariant(data, variantArr).then(() => done());
});
};
or using async/await
const generateVariant = () => {
return through.obj(async (file, enc, done) => {
const data = JSON.parse(file.contents.toString());
*/ Do a load of stuff */
const { variant } = data;
const variantArr = Object.values(variant);
await doGenerateVariant(data, variantArr);
done();
});
};