We Keep Coding

Updating Multiple Records with varying conditions in a single query

With Typeorm + NestJS + Postgres is there a way to update multiple records with varying conditions and varying values in a single query. Normally I could do
await getConnection()
.createQueryBuilder()
.update(Entity)
.set({ columnName: "newValue" })
.where({ id: In(1,2,3,4,5,6) })
.execute();
and this will update all entries with the specified ID. But in the case of having the following data structure
const array = [{id: 1, value: 'New Value For Record 1'},..., {id: 1000, value: 'New Value For Record 1000'}]
I could use for loop to update each single entry as below:
array1.forEach(rec => {
usersRepo.update(
{ id: rec.id },
{
columnName: rec.value
}
);
})
but this does not seem to be efficient and won't give good performance. Is there a way to do achieve multiple update on varying conditions with query builder.

You can achieve that we two queries
Get all the rows you need, basically a find would be enough, like this.
after that map the results array to do your changes
Last thing to do is to use save method, from Typeorm docs:
save - Saves a given entity or array of entities. If the entity already exists in the database, then it's updated. If the entity does not exist in the database yet, it's inserted.
const array = await yourRepo.find()
const newArr = array.map(el=> ({...el, value: 'New Value For Record'+el.id})
await yourRepo.save(newArr)
Cheers.

How to update firestore collection based on other docs?

I am building an order form that limits how many items you can order based on the stock of the item. I have a menu collection which has items
// menu
{ id: "lasagna", name: "Lasagna", price: 10, stock: 15 }
{ id: "carrot-soup", name: "Carrot Soup", price: 10, stock: 15 }
{ id: "chicken-pot-pie", name: "Chicken Pot Pie", price: 10, stock: 15 }
And an orders collection
// orders
{ id: <auto>, name: "Sarah", cart: {lasagna: 1, carrot-soup: 3}, ... }
{ id: <auto>, name: "Wendy", cart: {chicken-pot-pie: 2, carrot-soup: 1}, ... }
{ id: <auto>, name: "Linda", cart: {lasagna: 3}, ... }
4 carrot-soup has been ordered so the stock should be updated
// updated stock
{ id: "carrot-soup", name: "Carrot Soup", stock: 11 }
Orders are inserted from my Form component
function Form(props) {
// ...
// send order to firestore
const onSubmit = async _event => {
try {
const order = { cart, name, email, phone, sms }
dispatch({ action: "order-add" })
const id = await addDocument(store, "orders", order)
dispatch({ action: "order-add-success", payload: { ...order, id } })
}
catch (err) {
dispatch({ action: "order-add-error", payload: err })
}
}
return <form>...</form>
}
This is my database addDocument function
import { addDoc, collection, serverTimeStamp } from "firebase/firestore"
async function addDocument(store, coll, data) {
const docRef = await addDoc(collection(store, coll), { ...data, timestamp: serverTimestamp() })
return docRef.id
}
How should I decrement the stock field in my menu collection?
Ideally the client should have only read access to menu but to update the stock the client would need write access.
Another possibility is to have the client query the orders, sum the items, and subtract them from the read-only menu. But giving the client read access to other people's orders seems wrong too.
I am new to firestore and don't see a good way to design this.

You should deffinitely use a cloud function to update the stock. Create a function onCreate and onDelete functions trigger. If users can change data you would also need to onWrite function trigger.
Depending on the amount of data you have you woould need to create a custom queue system to update the stock. Belive me! It took me almost 2 years to figure out to solve this. I have even spoken with the Firebase engeeners at the last Firebase Summit in Madrid.
Usualy you would use a transaction to update the state. I would recommend you to do so if you don't have to much data to store.
In my case the amount of data was so large that those transactions would randomly fail so the stock wasn't correct at all. You can see my StackOverflow answer here. The first time I tought I had an answer. You know it took me years to solve this because I asked the same question on a Firebase Summit in Amsterdam. I asked one of the Engeeners who worked on the Realtime Database before they went to Google.
There is a solution to store the stock in chunks but even that would cause random errors with our data. Each time we improved our solution the random errors reduced but still remained.
The solution we are still using is to have a custom queue and work each change one by one. The downside of this is that it takes some time to calculate a lot of data changes but it is 100% acurate.
Just in case we still have a "recalculator" who recalculates one day again and checks if everything worked as it should.
Sorry for the long aswer. For me it looks like you are building a similar system like we have. If you plan to create a warehouse management system like we did I would rather point you to the right direction.
In the end it depends on the amount of data you have and how often or fast you change it.

Here is a solution based on Tarik Huber's advice.
First I include functions and admin
const functions = require("firebase-functions")
const admin = require("firebase-admin")
admin.initializeApp()
Then I create increment and decrement helpers
const menuRef = admin.firestore().collection("menu")
const increment = ([ id, n ]) =>
menuRef.doc(id).update({
stock: admin.firestore.FieldValue.increment(n)
})
const decrement = ([ id, n ]) =>
increment([ id, n * -1 ])
Here is the onCreate and onDelete hooks
exports.updateStockOnCreate =
functions
.firestore
.document("orders/{orderid}")
.onCreate(snap => Promise.all(Object.entries(snap.get("cart") ?? {}).map(decrement)))
exports.updateStockOnDelete =
functions
.firestore
.document("orders/{orderid}")
.onDelete(snap => Promise.all(Object.entries(snap.get("cart") ?? {}).map(increment)))
To handle onUpdate I compare the cart before and after using a diff helper
exports.updateStockOnUpdate =
functions
.firestore
.document("orders/{orderid}")
.onUpdate(snap => Promise.all(diff(snap.before.get("cart"), snap.after.get("cart")).map(increment)))
Here is the diff helper
function diff (before = {}, after = {}) {
const changes = []
const keys = new Set(Object.keys(before).concat(Object.keys(after)))
for (const k of keys) {
const delta = (before[k] ?? 0) - (after[k] ?? 0)
if (delta !== 0)
changes.push([k, delta])
}
return changes
}

MongoDB select first result from documents meeting $in

I have this query in loop:
const currentDatas = await Promise.all(nearestStations.map(async (ns: any) => {
return await this.stationCurrentDataRepo.findOne({
where: { stationId: parseInt(ns[0], 10) },
order: { date: 'DESC' },
});
}));
I want to optimize that to don't make hundreds queries and get the data in one query.
What I need is to get newest record (sort by date) for every stationId from array of ids ($in array of ids). I need all data from every found document meeting what I specified above.

In MongoDB this is done with aggregation pipeline and $group operator.

i want to send unique records for every user who logs in from a mysql database

i want to send every user who logs in a list of unique records i.e not same records from the database,
for every user i want to skip the records that have already been sent to other signed users.bare with me ,am a beginner,how can i implement such?
here is the code that fetches the records from the database
phrases.findAll({
where: {
userId: user.id,
phraseStatus: 1
},
limit: 10,
offset,
10
})
.then((data) => {
userObj.phrases.push(...data);
return res.status(200).json(userObj);
});

You will need to keep track of what has been sent to other users. I suggest you keep a table like phrases_sent to record what has been sent so far. Add the phrase_id to this new table so you have a relationship.
Take a look at associations in the docs.
You can then query the phrases table by outer joining to your phrases_sent table to return phrases that have not been sent so far.
Something like:
const phrases = await sequelize.models.phrase.findAll({
where: {
'$phrasesSents.phrase_id$': {[Op.is]: null} // query the joined table
},
attributes: ['phrase.phrase'], // the unused phrases
include: [
{
attributes: [],
model: sequelize.models.phrasesSent,
required: false, // specify this is a left outer join
}
],
raw: true,
nest: true
});
Would yield:
SELECT "phrase"."phrase"
FROM "phrases" AS "phrase"
LEFT OUTER JOIN "phrases_sent" AS "phrasesSents" ON "phrase"."id" = "phrasesSents"."phrase_id"
WHERE "phrasesSents"."phrase_id" IS NULL;
I believe doing a LEFT OUTER JOIN has performance benefits over a NOT IN (SELECT ...) style query.

Find Mongoose Document and then Sort Randomly with Different Tiers?

I currently have a .find() associated with documents in a collection, and it's currently sorting the document based on their "status" and sorting them by their most recent. Is it possible to sort them by the status first, and then randomly sort those within their respective tiers?
For instance if I have users that are three tiers: Tier 1, Tier 2, and Tier 3. Currently, it's sorting by Tier 1's (most recently), then Tier 2's (most recently), then Tier 3's (most recently). I'd like it to instead sort like: Tier 1's (random), Tier 2's (random), Tier 3's (random). But still display the results like so: Tier 1, Tier 2, then Tier 3. Is this possible?
router.get('/users', (req, res) => {
const errors = {};
Users.find().sort({tierStatus: -1, tierStatus: -1, tierStatus: -1})
.then(users => {
if (!users) {
errors.nousers = 'There are no users';
return res.status(404).json(errors);
}
res.json(users);
})
.catch(err => res.status(404).json({
nousers: 'There are no users'
}));
});

Currently there's no way to generate a random value in MongoDB thus you cannot sort "randomly". The only operator you can consider is $sample which picks random documents but you cannot use it along with sort() or $sort
The only solution is to sort by deterministic field and then shuffle in your application logic.

you can use $sample sort operator in mongoose aggregation
for example I'd like select 6 random blogs:
const randomBlogs = await blogs.aggregate([
{ $sample: { size: 6 } },
]);