Function validateBusinessId results in void variable businessExists. I am trying to assign different database paths depending on whether the business exists in the database or if the uploaded file is a master file. Function validateBusiness produces void result. Not sure how to fix it. Where did I go wrong?
async function validateBusinessId(businessId: string) {
db.collection('business').doc(businessId).get()
.then((docSnapshot) => {
if (docSnapshot.exists) {
return true
} else {
return false
}
})
}
async function getDatabase(fileName: string, businessId: string) {
const businessExists = await validateBusinessId(businessId)
if ((fileName) && (fileName?.includes('masterFile'))) {
console.log('A new Master database file has been detected.')
return db.collection('products')
} else if ((businessId) && (businessExists)) {
// If the business exists, return business path
console.log('A new database file has been detected for business: ' + businessId)
return db.collection('business').doc(businessId).collection('products')
} else {
return null
}
}
Help is highly appreciated!
Ok, so I found that I handled the promise of a query result incorrectly. Here's the code that worked for me.
async function validateBusinessId(businessId: string) {
const databaseDocumentSnapshot = await db.collection('business').doc(businessId).get()
if (databaseDocumentSnapshot.exists) {
return true
} else {
return false
}
}
async function getDatabase(fileName: string, businessId: string) {
const businessExists = await validateBusinessId(businessId)
if ((fileName) && (fileName?.includes('masterFile'))) {
console.log('A new Master database file has been detected.')
return db.collection('products')
} else if ((businessId) && (businessExists)) {
return db.collection('business').doc(businessId).collection('products')
} else {
console.log('No business with such ID exists: ' + businessId);
return null
}
}
Related
I am tring to download around 20.000 images from a website by scraping it.
(I am authorized by the owner)
The path of the image is composed like this:
VolumeId/ChapterId/PageId.jpg
There are around 100 volumes, and every volume has x chapters, and every chapter has y pages.
In the database I have stored for every volume the number of chapters, but i don't have
the number of pages, so I have to navigate to the image url and check if it exists.
I know for sure that there are less than 30 pages per chapter so I did something like this:
let exists = true;
for (let i = 0; i < 30 && exists; i++) {
fetch(`imgUrl/${i}.jpg`)
.then(data => {
if (data.ok)
return data.arrayBuffer();
else
exists = false;
.then(arrayBuffer => {
if (exists) {
let buffer = Buffer.from(arrayBuffer );
if (!fs.existsSync(path.join(__dirname, imgPath))) {
fs.mkdirSync(path.join(__dirname, imgPath), {
recursive: true,
});
}
fs.writeFile(
path.join(__dirname, imgPath + "/" + img + ".jpg"),
buffer,
(err) => {
if (err) throw err;
}
);
}
});
}
The problem:
The problem is that the loop does not wait for the image to be fetched and saved locally.
I have tried with async/await and promises (I think I have implemented them wrong)
Is there a better way to download a large quantity of data? Maybe with streams?
It can be a little bit complicated to implement your code with just async/await and at the same time assure the "exists" condition between iterations, I suggest you use a class that implements an async iterator, refer to the official documentation for more details, the following code achieve what you are looking for (note: the code snippet you provided didn't show where "imgPath" is coming from so just fix my code accordingly) :
class FetchImages {
constructor(urls) {
this.urls = urls;
this.index = 0;
}
[Symbol.asyncIterator]() {
const urlsIterator = this.urls[Symbol.iterator]();
return {
async next() {
if (++index == 30) {
return {
done: true
};
}
const iteratorResult = urlsIterator.next();
if (iteratorResult.done) {
return { done: true };
}
const url = iteratorResult.value;
try {
let response = await fetch(url);
let data;
if (response.status == 200) {
data = await response.arrayBuffer();
} else {
// equivalent to exists == false, exit condition of the iterator
return {
done: true
};
};
let buffer = Buffer.from(data);
if (!fs.existsSync(path.join(__dirname, imgPath))) {
fs.mkdirSync(path.join(__dirname, imgPath), {
recursive: true,
});
}
fs.writeFileSync(
path.join(__dirname, imgPath),
buffer,
);
return {
done: false,
value: imgPath
};
} catch (err) {
return {
done: false,
value: err.message
};
}
}
}
}
}
(async function () {
const fetchImages = new FetchImages(urls);
for await (const fetchImage of fetchImages) {
// check status of each fetch
console.log(fetchImage);
}
})();
I have some code like so:
export async function handleRefresh() {
if (!existsSync('postr.toml')) fail('not a postr directory');
const posts = expandGlob('posts/*');
for await (const post of posts) {
if (!post.isDirectory) {
console.warn('warning: non-folder found in posts directory');
continue;
}
let {parsedFrontMatter, contents} = extractFrontMatter(await read(post.path + '/post.md'));
const adapters = parsedFrontMatter.adapters ?? [];
if (!parsedFrontMatter) {
fail('no frontmatter for ' + post.path);
continue;
}
if (!Array.isArray(adapters)) {
fail('adapters is not an array');
continue;
}
if (isValidFrontMatter(parsedFrontMatter)) {
fail('frontmatter is not valid');
continue;
}
adapters.forEach(async (adapter: string) => {
const adapterPlugins = parseToml(await read('postr.toml')).adapterPlugins ?? {};
if (!isObject(adapterPlugins)) {
fail('adapterPlugins in the configuration is not an object');
return;
}
const adapterPath = adapterPlugins[adapter];
if (!adapterPath) {
console.warn('warn: an adapter was set' +
'but the corresponding plugin was not configured in `postr.toml`. Skipping');
return;
}
if (!('path' in <any>adapterPath)) {
fail(`adapter ${adapter} does not have a path`);
return;
}
import((<any>adapterPath).path)
.then(async module => {
const action = getActionForPost(parsedFrontMatter);
if (module[action]) {
await module[action](contents, parsedFrontMatter, (<any>adapterPath).config, {
updateFrontMatter(newData: {[x: string]: any}) {
parsedFrontMatter = Object.assign(parsedFrontMatter, newData);
},
mapID(remote: string | number) {
addMapping(parsedFrontMatter.id as string, remote.toString(), adapter);
}
})
} else {
console.warn(`Adapter ${adapter} does not support action \`${action}\``);
return;
}
writeFinalContents(parsedFrontMatter, contents, post.path)
})
.catch(error => fail(`could not run adapter because of ${error.name}: ${error.message}`));
});
}
}
Huge function.
There are a lot of these necessary if checks. 3/4 of the function is if checks, you could say. I want some advice on how I could refactor these statements.
As you can see the checks are not always the same, there are some different checks going on there.
EDIT: I've added real code.
Please help me figure out the difference in return behaviour between the onCall and onRequest google functions below.
onCall, the problem: returns null on all returns, except at the first return (as commented below). The db entries and rest of the code works fine. Just no returns problem.
onRequest, returns perfectly fine on every return. The db entries and rest of the code also works fine.
Both as you will see compare the same, but I just can't seem to get it to work at all. Any advice on how to get my returns to work for the onCall (and structure it better) would be much appreciated.
I am keen on sticking with async await (as opposed to a promise). Using Node.js 12. I am calling the onCall in Flutter, don't know if that is relevant for the question.
The onCall:
exports.applyUserDiscount = functions.https.onCall(async (data, context) => {
if (!context.auth) return {message: "Authentication Required!", code: 401};
const uid = context.auth.uid;
const discountCode = data["discountCode"];
const cartTotal = data["cartTotal"];
try {
return await db.collection("discountCodes").where("itemID", "==", discountCode).limit(1).get()
.then(async (snapshot) => {
if (snapshot.empty) {
return "doesNotExist"; // The only return that works.
} else { // Everything else from here onwards returns null.
snapshot.forEach(async (doc) => {
if (doc.data().redeemed == true) {
return "codeUsed";
} else {
const newCartTotal = cartTotal - doc.data().discountAmount;
if (newCartTotal < 0) {
return "lessThanTotal";
} else {
doc.ref.update({
redeemed: true,
uid: uid,
redeemDate: fireDateTimeNow,
});
await db.collection("userdata").doc(uid).set({
cartDiscount: admin.firestore.FieldValue.increment(-doc.data().discountAmount),
}, {merge: true});
return doc.data().discountAmount.toString();
}
}
});
}
});
} catch (error) {
console.log("Error:" + error);
return "error";
}
});
The onRequest:
exports.applyUserDiscount = functions.https.onRequest(async (req, res) => {
const uid = req.body.uid;
const discountCode = req.body.discountCode;
const cartTotal = req.body.cartTotal;
try {
return await db.collection("discountCodes").where("itemID", "==", discountCode).limit(1).get()
.then(async (snapshot) => {
if (snapshot.isempty) {
res.send("doesNotExist");
} else {
snapshot.forEach(async (doc) => {
if (doc.data().redeemed == true) {
res.send("codeUsed");
} else {
const newCartTotal = cartTotal - doc.data().discountAmount;
if (newCartTotal < 0) {
res.send("lessThanTotal");
} else {
doc.ref.update({
redeemed: true,
uid: uid,
redeemDate: fireDateTimeNow,
});
await db.collection("userdata").doc(uid).set({
cartDiscount: admin.firestore.FieldValue.increment(-doc.data().discountAmount),
}, {merge: true});
res.send(doc.data().discountAmount.toString());
}
}
});
}
});
} catch (error) {
console.log(error);
res.send("error");
}
});
There are several points to be noted when looking at your code(s):
You should not use async/await within a forEach loop. The problem is that the callback passed to forEach() is not being awaited, see more explanations here or here. HOWEVER, in your case you don't need to loop over the QuerySnapshot since it contains only one doc. You can use the docs property which return an array of all the documents in the QuerySnapshot and take the first (and unique) element.
You mix-up then() with async/await, which is not recommended.
I would advise to throw exceptions for the "error" cases, like doesNotExist, codeUsed or lessThanTotal but it's up to you to choose. The fact that, for example, the lessThanTotal case is an error or a standard business case is debatable... So if you prefer to send a "text" response, I would advise to encapsulate this response in a Object with one property: in your front-end the response will always have the same format.
So, the following should do the trick. Note that I send back on object with a response element, including for the cases that could be considered as errors. As said above you could throw an exception in these cases.
exports.applyUserDiscount = functions.https.onCall(async (data, context) => {
if (!context.auth) ... //See https://firebase.google.com/docs/functions/callable#handle_errors
const uid = context.auth.uid;
const discountCode = data["discountCode"];
const cartTotal = data["cartTotal"];
try {
const snapshot = await db.collection("discountCodes").where("itemID", "==", discountCode).limit(1).get();
if (snapshot.empty) {
//See https://firebase.google.com/docs/functions/callable#handle_errors
} else {
const uniqueDoc = snapshot.docs[0];
if (uniqueDoc.data().redeemed == true) {
return { response: "codeUsed" };
} else {
const newCartTotal = cartTotal - uniqueDoc.data().discountAmount;
if (newCartTotal < 0) {
return { response: "lessThanTotal" };
} else {
await uniqueDoc.ref.update({ // See await here!!
redeemed: true,
uid: uid,
redeemDate: fireDateTimeNow,
});
await db.collection("userdata").doc(uid).set({
cartDiscount: admin.firestore.FieldValue.increment(-uniqueDoc.data().discountAmount),
}, { merge: true });
return {
response: uniqueDoc.data().discountAmount.toString()
}
}
}
}
} catch (error) {
console.log("Error:" + error);
return "error";
}
});
I have a cloud function that uses a transaction to updates the players in a game. When the /players is null, i am trying to return a Map, but i get "Maximum call stack size exceeded".
Here is my cloud function:
export const addUserToGame = functions.https.onCall((data, context) => {
// Expected inputs - game_id(from data) and UID(from context)
if (context.auth == null) {
return {
"status": 403,
"message": "You are not authorized to access this feature"
};
}
const uid = context.auth.uid;
const game_id = data.game_id;
let gameIDRef = gamesRef.child(game_id);
return gameIDRef.once("value", function (snapshot) {
let players: Map<String, Number> = snapshot.child("players").val();
let max_players: Number = snapshot.child("max_players").val();
if (players != null && players.has(uid)) {
return {
"status": 403,
"message": "Player already in the game"
}
} else if (players != null && players.size >= max_players) {
return {
"status": 403,
"message": "Game is already full"
}
} else {
let playersNodeRef = gamesRef.child(game_id).child("players");
return playersNodeRef.transaction(t => {
if (t === null) {
return new Map<String, Number>().set(uid, 1);//trying to set a map with the player data, when the /players is null
} else {
let playersData: Map<String, Number> = t;
if (playersData.size >= max_players) { // rechecking
return;
} else {
playersData.set(uid, 1);
return playersData;
}
}
}).then(result => {
if (result.committed) { // if true there is a commit and the transaction went through
return {
"status": 200,
"message": "User added to game successfully"
}
} else {
return {
"status": 403,
"message": "Unable to add user at this time. Please try again"
}
}
}).catch(error => {
return {
"status": 403,
"message": error
}
});
}
});
});
Here is the stack trace:
addUserToGame
Function execution took 1423 ms, finished with status code: 500
at /workspace/node_modules/lodash/lodash.js:13401:38
at encode (/workspace/node_modules/firebase-functions/lib/providers/https.js:179:18)
at Function.mapValues (/workspace/node_modules/lodash/lodash.js:13400:7)
at baseForOwn (/workspace/node_modules/lodash/lodash.js:2990:24)
at /workspace/node_modules/lodash/lodash.js:4900:21
at keys (/workspace/node_modules/lodash/lodash.js:13307:14)
at isArrayLike (/workspace/node_modules/lodash/lodash.js:11333:58)
at isFunction (/workspace/node_modules/lodash/lodash.js:11653:17)
at baseGetTag (/workspace/node_modules/lodash/lodash.js:3067:51)
at Object (<anonymous>)
Unhandled error RangeError: Maximum call stack size exceeded
How can i set a map to /players node?
There were more than one issues with the code and as #Renaud pointed out, i have changed the 'once' callback to use promises version.
Also i had issues sending back data in the transaction. The data that i sent was using complex JS objects like Map(), but after some struggle (with the syntax) i changed it to a normal JS object (json like structure). Please see my changes below:
if (t === null) {
return [{ [uid]: { "status": 1 } }]; // if null, create an array and add an object to it
} else {
let playersData = t;
if (playersData.size >= max_players) { // rechecking
return;
} else { // if not null create an object and add to the existing array
playersData.push({
[uid]: {
"status": 1
}
});
return playersData;
}
}
Your problem most probably comes from the fact you are returning a complex JavaScript object, see https://stackoverflow.com/a/52569728/3371862.
In addition, note that you should use the promise version of the once() method, since, in a Callable Cloud Function you must return a promise that resolves with the data object to send back to the client.
Instead of doing
return gameIDRef.once("value", function (snapshot) {...});
do
return gameIDRef.once("value").then(snapshot => {...});
With this you will be able to correctly build the promise chain to be returned. Also, when dealing with the different cases around the players value, instead of returning JavaScript objects that will be handle in the .then((result) => {...}) block (which is not necessary and not really logical), throw errors that will be handled in the catch() block.
Something along the following lines:
export const addUserToGame = functions.https.onCall((data, context) => {
// Expected inputs - game_id(from data) and UID(from context)
if (context.auth == null) {
return {
status: 403,
message: 'You are not authorized to access this feature',
};
// IMHO better to do throw new functions.https.HttpsError('...', ...);
}
const uid = context.auth.uid;
const game_id = data.game_id;
let gameIDRef = gamesRef.child(game_id);
return gameIDRef
.once('value')
.then((snapshot) => {
let players: Map<String, Number> = snapshot.child('players').val();
let max_players: Number = snapshot.child('max_players').val();
if (players != null && players.has(uid)) {
throw new Error('Player already in the game');
} else if (players != null && players.size >= max_players) {
throw new Error('Game is already full');
} else {
let playersNodeRef = gamesRef.child(game_id).child('players');
return playersNodeRef.transaction((t) => {
if (t === null) {
return new Map<String, Number>().set(uid, 1); //trying to set a map with the player data, when the /players is null
} else {
let playersData: Map<String, Number> = t;
if (playersData.size >= max_players) {
// rechecking
return;
} else {
playersData.set(uid, 1);
return playersData;
}
}
});
}
})
.then((result) => {
if (result.committed) {
// if true there is a commit and the transaction went through
return {
status: 200,
message: 'User added to game successfully',
};
} else {
// probably throw an error here
return {
status: 403,
message: 'Unable to add user at this time. Please try again',
};
}
})
.catch((error) => {
if (error.message === 'Player already in the game') {
throw new functions.https.HttpsError('...', error.message);
} else if (error.message === 'Game is already full') {
throw new functions.https.HttpsError('...', error.message);
} else {
throw new functions.https.HttpsError('internal', error.message);
}
});
});
See here for more details on how to handle errors in a Callable Cloud Function.
I have an Azure Function that inserts from JSON-LD into my GraphDB - however what I'm finding is my GraphDB keeps crashing, because the function is sending too many insert requests.
I've set "maxConcurrentRequests": 1 and "maxOutstandingRequests": 1 however it still doesn't seem to wait and process one at a time.
Could someone please explain why this so.
export async function onTrigger(context: Context, documents: Resource[] | null): Promise<void> {
if (documents == null) { return }
documents.forEach(async function (value: any) {
if ("metadata" in value) { } else { return; }
let song: MyDocument = value.metadata.songs.song;
// Create the JSON-LD object using the song object from above
let jsonld = ...
let nQuads = await jsonldParser.toRDF(jsonld, {format: 'application/n-quads'});
let insertQuery = "INSERT DATA {" + nQuads + "}";
try {
let res = await axios.post('http://localhost:7200/mygraphdb', "update="+encodeURIComponent(insertQuery))
if (res.status === 204) {
console.log(`All Done!`);
}
} catch (error) {
console.log(`Error! ${song.id}`);
}
});
}