We Keep Coding

Before finishing the map in Promise.all it executes next line in react js

Good Evening All,
Am just trying to create one collection with multiple data, that datas are nested with other collections am trying to call a function to get one id , that id used to call another connection and get some data and id's, that id is used to call other connection like wise i call all connection and gather the data
Here My code
firestore.collection("pods/").where("start_date", "==", dformat)
.where("pod_status", "==", "active").get()
.then((snap: any) => {
snap.docs.map((docsnap: any) => {
let details = docsnap.data();
let pod_docId = docsnap.id;
let pod_id = details.pod_id;
let course_id = details.course_id;
getDocDetails(course_id, firestore, "courses").then(
(courseData: any) => {
let course_data = courseData.data(); let levels = course_data.levels;
Promise.all(
levels.map((lev: any) => {
let level_id = lev;
getDocDetails(level_id, firestore, "levels")
.then((level_data) => {
let result: any = level_data; let level_details = result.data();
let modules = level_details.modules;
modules.map((module_id: any) => {
getDocDetails(module_id, firestore, "modules")
.then((module_data) => {
let response: any = module_data; let mod_det = response.data(); let sess = mod_det.sessions;
sess.map((sess_id: any) => {
sessions_id_list.push(sess_id);
logger.info("sessions_id_list", sessions_id_list);
});
})
.catch((err) => {
logger.info(
"Error while getting the Module details",
err
);
});
});
logger.info("sessions_id_list 2", sessions_id_list);
})
.catch((err) => {
logger.info("Error while getting the Level details", err);
});
})
);
logger.info("sessions_id_list 3", sessions_id_list);
let obj = {
pod_docId: pod_docId,
pod_id: pod_id,
sessions_id_list: sessions_id_list,
status: "created",
count: 0,
completed: 0,
};
addQueue(obj, firestore);
}
);
});
})
.catch((err: any) => {
reject(err);
});
Here before pushing the session list , it execute print session list 3 and call addQueue function. How can i overcome this, Help me guys, Thanks in Avance

How to initialize App Data in node js and access it without being undefined in jest test?

i am initializing a node js app with crucial data for the app to work from a database in index.js.
index.ts
import {getInitialData} from 'initData.ts';
export let APP_DATA: AppData;
export const initializeAppData = async () => {
try {
APP_DATA = (await getInitialData()) as AppData;
if (process.env.NODE_ENV !== 'test') {
initializeMongoose();
startServer();
}
} catch (error) {
console.log(error);
}
};
initData.ts
let dbName: string = 'initialData';
if (process.env.NODE_ENV === 'test') {
dbName = 'testDb';
}
const uri = `${process.env.MONGODB_URI}/?maxPoolSize=20&w=majority`;
export async function getInitialData() {
const client = new MongoClient(uri);
try {
await client.connect();
const database = client.db(dbName);
const configCursor = database
.collection('config')
.find({}, { projection: { _id: 0 } });
const config = await configCursor.toArray();
const aaoCursor = database
.collection('aao')
.find({}, { projection: { _id: 0 } });
const aao = await aaoCursor.toArray();
return { config, aao };
} catch {
(err: Error) => console.log(err);
} finally {
await client.close();
}
}
I'm using this array in another file and import it there.
missionCreateHandler
import { APP_DATA } from '../index';
export const addMissionResources = (
alarmKeyword: AlarmKeyword,
newMission: MissionDocument
) => {
const alarmKeywordObject = APP_DATA?.aao.find(
(el) => Object.keys(el)[0] === alarmKeyword
);
const resourceCommand = Object.values(alarmKeywordObject!);
resourceCommand.forEach((el) => {
Object.entries(el).forEach(([key, value]) => {
for (let ii = 1; ii <= value; ii++) {
newMission.resources?.push({
initialType: key,
status: 'unarranged',
});
}
});
});
};
I'm setting up a mongodb-memory-server in globalSetup.ts for Jest and copy the relevant data to the database from json-files.
globalSetup.ts
export = async function globalSetup() {
const instance = await MongoMemoryServer.create({
instance: { dbName: 'testDb' },
});
const uri = instance.getUri();
(global as any).__MONGOINSTANCE = instance;
process.env.MONGODB_URI = uri.slice(0, uri.lastIndexOf('/'));
process.env.JWT_SECRET = 'testSECRET';
const client = new MongoClient(
`${process.env.MONGODB_URI}/?maxPoolSize=20&w=majority`
);
try {
await client.connect();
const database = client.db('testDb');
database.createCollection('aao');
//#ts-ignore
await database.collection('aao').insertMany(aao['default']);
} catch (error) {
console.log(error);
} finally {
await client.close();
}
};
missionCreateHandler.test.ts
test('it adds the correct mission resources to the array', async () => {
const newMission = await Mission.create({
address: {
street: 'test',
houseNr: 23,
},
alarmKeyword: 'R1',
});
const expected = {
initialType: 'rtw',
status: 'unarranged',
};
addMissionResources('R1', newMission);
expect(newMission.resources[0].initialType).toEqual(expected.initialType);
expect(newMission.resources[0].status).toEqual(expected.status);
});
When runing the test, i get an 'TypeError: Cannot convert undefined or null to object at Function.values ()'. So it seems that the APP_DATA object is not set. I checked that the mongodb-memory-server is set up correctly and feed with the needed data.
When i hardcode the content of APP_DATA in index.ts, the test runs without problems.
So my questions are: How is the best practice to set up initial data in a node js app and where to store it (global object, simple variable and import it in the files where needed)? How can the test successfully run, or is my code just untestable?
Thank you!

How do I return each collection and its documents in firestore with cloud functions?

Would like to return an array that has [1year, 1month, etc] and each of those are arrays that contain each document.
Currently, this returns an empty array but when I print the size of the snapshots I get the correct values. Not sure if i'm using push() correctly or if this is an async issue. Thanks.
exports.getStockPrices = functions.https.onRequest((req, res) => {
cors(req, res, () => {
const currentUser = {
token: req.headers.authorization.split('Bearer ')[1]
};
// ! this is a post request
admin
.auth()
.verifyIdToken(currentUser.token)
.then(decodedToken => {
// keep this just in case we want to add anything to do with the user
const user = decodedToken;
// array of collections e.g [1year, 1mo, etc]
const data = [];
// array of documents e.g [18948901, 1984010471, etc]
const documents = [];
db.collection('historical')
.doc(`${req.body.ticker}`)
.listCollections()
.then(collections => {
// each collection is the 1year, 1mo, etc
collections.forEach(collection => {
collection.get().then(querySnapshot => {
console.log('number of documents: ' + querySnapshot.size);
querySnapshot.forEach(doc => {
// doc.data is each piece of stock data
documents.push(doc.data());
});
// each document e.g 1year, 1mo, etc
data.push(documents);
});
});
return data;
})
.then(data => {
return res.json({ data });
})
.catch(err => {
console.log(err);
return res.status(500).send({ error: 'error in getting data' });
});
})
.catch(err => {
console.log(err);
return res.status(500).send({
error: 'error authenticating user, please try logging in again'
});
});
});
});

Due the nature of async calls, your return occurs before your array is being filled.
You can try my example, my firebase function is defined as async this allows me to use await, this statement allows to add a kind of sync for your firestore operations by waiting for the promises.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.eaxmple = functions.https.onRequest(async (req, res) => {
var datax = []
var collections = await db.collection('collection').doc('docid').listCollections()
for (collection in collections) {
content = await collections[collection].get().then(querySnapshot => {
console.log('number of documents: ' + querySnapshot.size);
return querySnapshot.docs.map(doc => doc.data());
});
datax.push(content)
}
return res.json({datax});
});

Firestore cloud function to recursively update subcollection/collectionGroup

I have this cloud function:
import pLimit from "p-limit";
const syncNotificationsAvatar = async (
userId: string,
change: Change<DocumentSnapshot>
) => {
if (!change.before.get("published") || !change.after.exists) {
return;
}
const before: Profile = change.before.data() as any;
const after: Profile = change.after.data() as any;
const keysToCompare: (keyof Profile)[] = ["avatar"];
if (
arraysEqual(
keysToCompare.map((k) => before[k]),
keysToCompare.map((k) => after[k])
)
) {
return;
}
const limit = pLimit(1000);
const input = [
limit(async () => {
const notifications = await admin
.firestore()
.collectionGroup("notifications")
.where("userId", "==", userId)
.limit(1000)
.get()
await Promise.all(
chunk(notifications.docs, 500).map(
async (docs: admin.firestore.QueryDocumentSnapshot[]) => {
const batch = admin.firestore().batch();
for (const doc of docs) {
batch.update(doc.ref, {
avatar: after.avatar
});
}
await batch.commit();
}
)
);
})
];
return await Promise.all(input);
};
How can I recursively update the notifications collection but first limit the query to 1.000 documents (until there are not more documents) and then batch.update them? I'm afraid this query will timeout since collection could grow big over time.

Posting a solution I worked out, not following the context of the question though but it can easily be combined. Hope it helps someone else.
import * as admin from "firebase-admin";
const onResults = async (
query: admin.firestore.Query,
action: (batch: number, docs: admin.firestore.QueryDocumentSnapshot[]) => Promise<void>
) => {
let batch = 0;
const recursion = async (start?: admin.firestore.DocumentSnapshot) => {
const { docs, empty } = await (start == null
? query.get()
: query.startAfter(start).get());
if (empty) {
return;
}
batch++;
await action(
batch,
docs.filter((d) => d.exists)
).catch((e) => console.error(e));
await recursion(docs[docs.length - 1]);
};
await recursion();
};
const getMessages = async () => {
const query = admin
.firestore()
.collection("messages")
.where("createdAt", ">", new Date("2020-05-04T00:00:00Z"))
.limit(200);
const messages: FirebaseFirestore.DocumentData[] = [];
await onResults(query, async (batch, docs) => {
console.log(`Getting Message: ${batch * 200}`);
docs.forEach((doc) => {
messages.push(doc.data());
});
});
return messages;
};

How to output TreeModel Js model as JSON

So I'm triyng to update some ids from a categories tree using TreeModelJS.
after editing I would like to dump the tree to a file in JSON format.
but when outputing other keys from TreeModel gets outputed as well.
How could I output edited tree as JSON (model only)?
I managed to replace other keys values with null and so far I got this:
const axios = require('axios')
const TreeModel = require('tree-model')
const fs = require('fs')
const url = 'https://my-api-uri-for-categories'
const dumpPath = `${process.cwd()}/data/test/categories.json`
const getCategories = async () => {
try {
const response = await axios.get(url)
return response.data.categories
} catch (error) {
console.log('Error reading categories', error)
}
}
const dumpJsonTofile = data => {
try {
console.log('Dumping to file')
console.log(data)
fs.writeFileSync(
dumpPath,
JSON.stringify(data, (k, v) => {
if (k === 'parent' || k === 'config' || k === 'children') return null
else return v
}),
'utf8'
) // write it back
} catch (error) {
console.log('Error dumping categories', error)
}
}
const scraping = async category => {
try {
const response = await axios.get(category.url)
const document = response.data
const json = document.match(/{"searchTerm"(.*);/g)[0]
const data = JSON.parse(json.replace(';', ''))
return data
} catch (error) {
console.log(`Error while scraping category: ${category.name}`, error)
}
}
async function run() {
const categories = await getCategories()
const categoriesTree = new TreeModel({
childrenPropertyName: 'items',
})
const root = categoriesTree.parse({ id: 0, origin: {}, items: categories })
root.walk(async node => {
const category = node.model
console.log(`scraping category: ${category.name}...`)
if (!category.url) return console.log(`skipping (root?)...`)
const data = await scraping(category)
category.id = data.categoryId
})
dumpJsonTofile(root)
}
run()
but that still outputs a Node object like this:
{
"config":null,
"model":{},
"children":null
}
I need to output all the tree showing only the model key value for each item

Try JSON.stringify(root.model).