I have a collection of users which I'd like to update regularly using several different APIs (where each has its own rate limits etc).
So I have a few cron jobs with the following similar structure:
const cursor_i = db.collection('users').find();
while(await cursor_i.hasNext()){
let user = await cursor_i.next();
user.field_1 = 'Some New Stuff';
const write_result = await db.collection('newusers').save(user);
if(!write_result.result.ok){
console.log(write_result);
}
}
The problem is when a document is being updated at the same time, by more than one updater, only the last save() call would matter.
To clarify, consider the following code:
const cursor_1 = db.collection('users').find();
const cursor_2 = db.collection('users').find();
let user_cursor_1 = await cursor_1.next(); // user_cursor_1 has the first user in the collection
let user_cursor_2 = await cursor_2.next(); // user_cursor_2 has the first user in the collection
user_cursor_1.new_field_1 = 'ok';
const write_result = await db.collection('users').save(user_cursor_1);
if(!write_result.result.ok){
console.log(write_result);
}
// first user in collection now has a new field named new_field_1 with the value 'ok'
user_cursor_2.new_field_2 = 'ok';
const write_result_2 = await db.collection('newusers').save(user_cursor_2);
if(!write_result_2.result.ok){
console.log(write_result);
}
// first user in collection now has a new field named new_field_2 with the value 'ok' but DOES NOT have new_field_1 anymore
And so, the first user in the collection has been updated twice, but eventually will only have the effect of the second update.
I can think of a few ways to avoid it by implementing locks myself, but I'd guess MongoDB must have something to handle these cases.
you should find users after updating the first cursor, like this :
const cursor_1 = db.collection("users").find();
let user_cursor_1 = await cursor_1.next(); // user_cursor_1 has the first user in the collection
user_cursor_1.new_field_1 = "ok";
const write_result = await db.collection("users").save(user_cursor_1);
if (!write_result.result.ok) {
console.log(write_result);
}
const cursor_2 = db.collection("users").find();
let user_cursor_2 = await cursor_2.next(); // user_cursor_2 has the first user in the collection
user_cursor_2.new_field_2 = "ok";
const write_result_2 = await db.collection("newusers").save(user_cursor_2);
if (!write_result_2.result.ok) {
console.log(write_result);
}
Related
I am trying to make barbershop web app where costumer can see list of free appointments and when they reserve free appointment I want to delete that field from firebase.
I have a collection which represents one barber.
This is how it looks in firebase.
As you see radno_vrijeme is object or map in firebase which contains 6 arrays, and in each array there is list of free working hours.
In my function I am able to do everthing except last line where I need to update firebase collection.
const finishReservation = async () => {
try {
const freeTimeRef = collection(db, `${barber}`);
const q = query(freeTimeRef);
const querySnap = await getDoc(q);
querySnap.forEach(async (doc) => {
const radnoVrijeme = doc.data().radno_vrijeme;
// Find the index of the hour you want to delete
const index = radnoVrijeme["Mon"].indexOf(hour);
// Remove the hour from the array
radnoVrijeme["Mon"].splice(index, 1);
// Update the document in the collection
console.log(radnoVrijeme);
const radnoVrijemeMap = new Map(Object.entries(radnoVrijeme));
await freeTimeRef.update({ radno_vrijeme: radnoVrijemeMap });
});
} catch (error) {
console.log(error);
}
};
I tried to pass it as JSON stringified object, but it didn't work. I always get this error :
"FirebaseError: Expected type 'ya', but it was: a custom Ia object"
When you are trying to fetch multiple documents using a collection reference or query, then you must use getDocs():
const finishReservation = async () => {
try {
const freeTimeRef = collection(db, `${barber}`);
const q = query(freeTimeRef);
const querySnap = await getDocs(q);
const updates = [];
querySnap.forEach((d) => {
const radnoVrijeme = d.data().radno_vrijeme;
const index = radnoVrijeme["Mon"].indexOf(hour);
radnoVrijeme["Mon"].splice(index, 1);
const radnoVrijemeMap = new Map(Object.entries(radnoVrijeme));
updates.push(updateDoc(d.ref, { radno_vrijeme: radnoVrijemeMap }))
});
await Promise.all(updates);
console.log("Documents updated")
} catch (error) {
console.log(error);
}
};
getDoc() is used to fetch a single document using a document reference.
I want to do an update to a Sequelize object (with paranoid true) and immediately delete it. Below code works, but it invokes 2 update queries and a transaction block.
const object = await Car.findOne({});
object.wheels = 3;
object.canRun = false;
await sequelize.transaction({ isolationLevel: Transaction.ISOLATION_LEVELS.REPEATABLE_READ }, async (transaction) => {
await object.save({ transaction });
await object.destroy({ transaction });
});
I've tried to make it one single update, but below code doesn't work.
const object = await Car.findOne({});
object.wheels = 3;
object.canRun = false;
object.deletedAt = new Date();
await object.save({ paranoid: false });
Is there any way to do it with only 1 query?
So I'm working on a project where I'm making a call to a database to retrieve the data stored there. This data comes as an array. here is the code:
const allLogins = await Login.find().sort("name");
const token = req.header("x-auth-token");
const user = jwt.verify(token, config.get("jwtPrivateKey"));
const logins = allLogins
.filter((login) => login.userId === user._id)
.map((login) => {
login.password = decrypt(login.password);
});
If I call a console.log after the decrypt has been run I see that it has been completed correctly. The issue I have is if I console.log(logins) it says it is an array of two items that are both undefined. If instead I run it like this...
const allLogins = await Login.find().sort("name");
const token = req.header("x-auth-token");
const user = jwt.verify(token, config.get("jwtPrivateKey"));
let logins = allLogins.filter((login) => login.userId === user._id);
logins.map((login) => {
login.password = decrypt(login.password);
});
Then it works as it should. I'm not sure why the first set of code doesn't work and why the second set does work.
Any help would be appreciated!
Basic :
array. filter - accept a callback and call back return boolean (that match our criteria)
array.map - accept a callback and call back return transformed object
In the second working example:
logins.map((login) => {
// note: logins is iterated but not assigned to logins back
// so accessing login is working
login.password = decrypt(login.password); // map should return data
+ return login; // if we update all code will work
});
Now coming to first example:
const logins = allLogins
.filter((login) => login.userId === user._id)
.map((login) => {
login.password = decrypt(login.password);
+ return login; // this will fix the issue
});
I have this code:
const project = await Project.findOne({"code":currentUser.groupcode}); // this works
const card = await Card.findOne({"id":req.body.id}); // this works
card.panel = req.body.newSection;
card.save(); // this works
project.cards[`${req.body.id}`].panel = req.body.newSection;
project.save(); // this does not work
I'm trying to update the panel field in both the cards collection and projects collection.
When I log project.cards[`${req.body.id}`].panel, it is the correct value, so it is the correct routing.
What's wrong here?
Try putting await before your save statements and see if it resolves the issue.
If I have guessed your schemas correctly this is the answer:
const project = await Project.findOne({ code: currentUser.groupcode }).populate("cards");
const card = await Card.findOne({ id: req.body.id }); // this works
card.panel = req.body.newSection;
await card.save(); // this works
const cardIndex = project.cards.findIndex(card => card._id.toString() === req.body.id);
project.cards[cardIndex].panel = req.body.newSection;
await project.save(); // this does not work
Ok
I needed to add
project.markModified(`cards.${req.body.id}.panel`)
before project.save();
So I'm trying to connect to external server called Pexels to get some photos. I'm doing that from node.js but it is just a javascript issue. Pexels unfortunately lets user to download object with only 40 pictures per page.
https://api.pexels.com/v1/curated?per_page=40&page=1 // 40 is maximum
But actually I need more then that. I'd like to get 160 results, ie. to combine all first four pages. In order to do that I tried looping the request:
let pexelsData = [];
for(let i = 1; i < 5; i++) {
const randomPage = getRandomFromRange(1, 100); //pages should be randomized
const moreData = await axios.get(`https://api.pexels.com/v1/curated?per_page=40&page=${randomPage}`,
createHeaders('bearer ', keys.pexelsKey));
pexelsData = [ ...moreData.data.photos, ...pexelsData ];
}
Now I can use pexelsData but it work very unstable, sometimes it is able to get all combined data, sometimes it crashes. Is there a correct and stable way of looping requests?
You work with 3rd party API, which has rate limits. So you should add rate limits to your code. The simplest solution for you is using p-limit or similar approach form promise-fun
It will looks like that:
const pLimit = require('p-limit');
const limit = pLimit(1);
const input = [
limit(() => fetchSomething('foo')),
limit(() => fetchSomething('bar')),
limit(() => doSomething())
];
(async () => {
// Only one promise is run at once
const result = await Promise.all(input);
console.log(result);
})();
you can break it into functions like..
let images=[];
const getResponse = async i=> {
if(i<5)
return await axios.get(`https://api.pexels.com/v1/curated?per_page=40&page=${i}`)
}
const getImage = (i)=>{
if(i<5){
try {
const request = getResponse(i);
images = [...images,...request];
// here you will get all the images in an array
console.log(images)
getImage(++i)
} catch (error) {
console.log("catch error",error)
// getImage(i)
}
}
}
getImage(0); //call initail