I need to update every document that has a certain field foo
const firestoreDB = getFirestore(app);
const q = query(collection(firestoreDB, 'myCollection'), where('foo', '!=', false));
But I dont need to read the documents, in order to update their values. But since batch(and document) syntax dictates that I need to have a documentSnapshotRef to update it, this seems like it leads to unnecessary reads.
(async () => {
const docsSnapshot = await getDocs(query);
const batch = writeBatch(firestoreDB);
const docsSnapshot.forEach( (docRef) => {
batch.update(docRef,{
foo: 123, // some number that I need to update
}
});
})()
So in this case, if there are a 100 documents with foo in it, then I would incur 100 reads and 100 writes, while in reality I only needed a 100 writes. Any solution to this? Some way to use query with batch?
As per ZAky's suggestion, I have created a new document called #metadata which contains the document id of each individual document which contains the foo field, as an array.
// fileName: #metadata
fooDocs:[
[0]: 234jhksd32, //document ID auto generated by firestore
//... other document IDs
]
This is specific to firebase, as in order to reference a document, you need to know its document ID. Otherwise you need to query the database, which will create a read on each document that satisfies the condition. You can draw a rough analogy to this with a JS object. To find an entry, either you need to know the key,
const requiredEntry = obj[key] // option 1
or you need to search the whole object to find the object that satisfies a condition(In this case: has the field foo), using a command such as
const requiredEntry = Object.values(obj).find(item=>item.foo) // option 2
So in order to use a method like option 1, in firebase, we read first read the metadata document(incurs 1 read)
(async () => {
const metadataDocSnapshot = await getDoc(doc(firestoreDB, 'myCollection', '#metadata'));
const { fooDocs } = metadataDocSnapshot.data(); // destructure the field that contains our cached document ID's
// to be continued ...
Now we can use these document ID's to directly update the doc, since we can create a documentRef using doc()
// ...
const batch = writeBatch(firestoreDB);
const fooDocs.forEach( (docID) => {
const docRef = doc(firestoreDB, 'myCollection' , docID);
batch.update(docRef,{
foo: 123, // some number that I need to update
}
});
await batch.commit();
})()
P.S. This is my own interpretation of an answer to my own question. Feel free to edit this answer, or post your own, if you think you can solve this better
Related
// Update the other user that current user has asked for revision
export async function updateOtherUserForContractRevision(contractID : string, comments : any) {
// getting userDetails
let currentUser : string | any = localStorage.getItem("userDetails");
currentUser = JSON.parse(currentUser);
// update fields in contractRecieved
const contractsRecievedRef : any = doc(db, "contractsRecieved", currentUser.uid);
// const queryContractsRecieved = query(contractsRecievedRef,
// where('contractDetails.contract.contractID','array-contains',contractID)
// );
// console.log(".////////updateOtherUserForContractRevision", queryContractsRecieved ,contractID)
onSnapshot(contractsRecievedRef, (doc: any) => {
doc.data().contract.map((contract: any) => {
if(contract.contractDetails.contract.contractID === contractID) {
// I reach my desired array Index and want to update the msg field to my comments parametre
console.log(".////////contract", contract);
}
})
})
}
I want to update my msg field as well as content field in contract object, which in turn is present in contractDetails object in the contract array(0th index). I have searched and reached to my desired array value using onSnapShot, how can I update these fields in the onSnapShot method? or I should use another approach for searching and updating fields in objects contained by the array.
JUST A THOUGHT: If I could get the reference to the array index and then use it to update the object, maybe It'll work
for example (I'll update sentForRevision)
onSnapshot(contractsRecievedRef, async (doc: any) => {
doc.data().contract.map(async (contract: any) => {
if(contract.contractDetails.contract.contractID === contractID) {
console.log(".////////contract", doc.ref);
// If this contract.ref exists and points to the index present in the document
await updateDoc(contract.ref,{
"contractDetails.sentForRevision": true,
});
}
})
})
There is no way you can query a Firestore collection based on a value that exists in an object that is contained in an array. This kind of filtering cannot be achieved using partial data. I have even written an article regarding this topic called:
How to update an array of objects in Firestore?
If you need that, you can duplicate the data on which you want to perform the filtering and add it to a separate array. In this way, you can query the collection using array-contains operator. Once you get the desired documents, you can get the array, perform the updates and then write the documents back to Firestore.
Enviroment: nodejs, firebase-admin, firestore.
Database scructure (space):
Database scructure (user):
Creating new space (example):
// init data
const userId = "someUserId";
// Create new space
const spaceRef = await db.collection("spaces").add({ name: "SomeName" });
// Get spaceId
spaceId = spaceRef.id;
// Get user Doc for upate their spaces
const userRef = await db.collection("users").doc(userId);
// Add "spaceId" to user spaces list
userRef.collection("spaces").doc(spaceId).set({ some: "data" });
// Create collection "members" in new space with "userId"
spaceRef.collection("members").doc(userId).set({role: "OWNER"})
Question: I want to execute this code inside single runTransaction, but as I see transactions support only one-time read and multiple update, this does not suit me, since I get the spaceId I need during the execution of the code.
Why I want to use transaction: In my data structure, the relationship between the created space and the presence of the ID of this space on the user is required. If we assume that an error occurred during the execution of this code, for example, the space was created, but this space was not added inside the user profile, then this will be a fatal problem in my database structure.
Similar to other databases, transactions solve this problem, but I can't figure out how to do it with firestore.
Maybe you know a better way to protect yourself from consistent data in this case?
Actually, you don't need a Transaction for that, since you are not reading documents.
With db.collection("users").doc(userId); you are actually not reading a document, just calling "locally" the doc() method to create a DocumentReference. This method is not asynchronous, therefore you don't need to use await. To read the doc, you would use the asynchronous get() method.
So, using a batched write, which atomically commits all pending write operations to the database, will do the trick:
const userId = 'someUserId';
const userRef = db.collection('users').doc(userId);
const spaceRef = firestore.collection('spaces').doc();
const spaceId = spaceRef.id;
const writeBatch = firestore.batch();
writeBatch.set(spaceRef, { name: "SomeName" });
writeBatch.set(userRef.collection("spaces").doc(spaceId), { some: "data" });
writeBatch.set(spaceRef.collection("members").doc(userId), {role: "OWNER"});
await writeBatch.commit();
You should include this code in a try/catch block and if the batch commit fails you will be able to handle this situation in the catch block, knowing that none of the write were committed.
Let's say I have the following code:
db.task(t => {
return t.none('set search_path to myschema').then(() => {
return t.any('select * from mytable').then(results => {
return t.none('set search_path to originalschema').then(() => {
return results
})
})
})
})
Could a query outside of db.task(), that happened to run in between of the two search_path changes inside db.task(), actually access the data in 'myschema' instead of 'originalschema'?
Could a query outside of db.task(), that happened to run in between of the two search_path changes inside db.task(), actually access the data in 'myschema' instead of 'originalschema'?
No.
SET search_path is a session-based operation, i.e. it applies only to the current connection, which the task allocates exclusively for the entire duration of its execution.
Once the task has finished, it releases the connection back to the pool. At that point, any query that gets that same connection will be working with the alternative schema, unless it is another task that sets the schema again. This gets tricky, if you are setting schema in just one task, and generally not recommended.
Here's how it should be instead:
If you want to access a special-case schema inside just one task, best is to specify the schema name explicitly in the query.
If you want to set custom schema(s) dynamically, for the entire app, best is to use option schema, of the Initialization Options. This will propagate the schema automatically through all new connections.
If you want to set schema statically, there are queries for setting schema permanently.
Addition:
And if you have a very special case, whereby you have a task that needs to run reusable queries inside an alternative schema, then you would set the schema in the beginning of the task, and then restore it to the default schema at the end, so any other query that picks up that connection later won't try to use the wrong schema.
Extra:
Example below creates your own task method (I called it taskEx), consistent across the entire protocol, which accepts new option schema, to set the optional schema inside the task:
const initOptions = {
extend(obj) {
obj.taskEx = function () {
const args = pgp.utils.taskArgs(arguments); // parse arguments
const {schema} = args.options;
delete args.options.schema; // to avoid error thrown
if (schema) {
return obj.task.call(this, args.options, t => {
return t.none('SET search_path to $1:name', [schema])
.then(args.cb.bind(t, t));
});
}
return obj.task.apply(this, args);
}
}
});
const pgp = require('pg-promise')(initOptions);
So you can use anywhere in your code:
const schema = 'public';
// or as an array: ['public', 'my_schema'];
db.taskEx({schema}, t => {
// schema set inside task already;
});
Note that taskEx implementation assumes that the schema is fully dynamic. If it is static, then there is no point re-issuing SET search_path on every task execution, and you would want to do it only for fresh connections, based on the following check:
const isFreshConnection = t.ctx.useCount === 0;
However, in that case you would be better off using initialization option schema instead, as explained earlier.
I have some code that looks like the following:
export const createTable = async (data) => {
const doc = db.collection("tables").doc();
const ref = db
.collection("tables")
.where("userId", "==", data.userId)
.orderBy("number", "desc").limit(1);
db.runTransaction(async transaction => {
const query = await transaction.get(ref);
let number = 1;
if (!query.empty) {
const snapshot = query.docs[0];
const data = snapshot.data();
const id = snapshot.id;
number = data.number + 1;
}
data = {number, ...data};
transaction.set(doc, data);
});
Basically I have a tables collection and each table has an auto generated number like #1, #2, #3
When creating new tables, I want to fetch the latest table number and create the new table with that number incremented by 1.
I wanted to wrap it in a transaction so that if a table created while running the transaction, it will restart so that I don't end up with duplicate numbers.
However, this errors out on the .get(), and from googling I've read that Firestore can't monitor a whole collection within transactions, but instead it requires a specific doc passed to it. Which I obviously can't do because I need to monitor for new docs created in that collection, not changes in a particular doc.
If so, what's the correct way to implement this?
The Firestore transaction API for client apps requires that you get() each individual document that you want to participate in the transaction. So, if you have a query whose results you want to transact with, you will need to:
Perform the query (outside of the transaction)
Collect document references for each document in the result set
In the transaction, get() them all individually.
You will be limited to 500 documents per transaction.
If you want to dynamically look for new documents to modify, you will probably much better off implementing that on the backend using a Firestore trigger in Cloud Functions to automatically handle each new document as they are created, without requiring any code on the client.
Because you're updating just one document, you probably don't need to use transactions for incrementing values.
You can use Firestore Increment to achieve this.
Here is an example taken from here:
const db = firebase.firestore();
const increment = firebase.firestore.FieldValue.increment(1);
// Document reference
const storyRef = db.collection('stories').doc('hello-world');
// Update read count
storyRef.update({ reads: increment });
This is the easiest way to increment values in Firestore.
I am trying to push an array element but am destroying all the content there and replacing with the pushed data:
db .collection('households')
.doc(householdId)
.set( { users: [uid], }, { merge: true }, )
.then(() => { resolve(); })
.catch(() => reject());
I thought the merge true doesn't destroy the data that is already there? Struggling a little with the firestore api docs.
This is the structure of my data:
households
2435djgnfk
users [
0: user1
1: user2
]
Thank you!
You should use Firestore Transaction for this.
const householdRef = db.collection('households').doc(householdId);
const newUid = '1234'; // whatever the uid is...
return db.runTransaction((t) => {
return t.get(householdRef).then((doc) => {
// doc doesn't exist; can't update
if (!doc.exists) return;
// update the users array after getting it from Firestore.
const newUserArray = doc.get('users').push(newUid);
t.set(householdRef, { users: newUserArray }, { merge: true });
});
}).catch(console.log);
Updating an array or a stored object without getting it first will always destroy the older values inside that array/object in firestore.
This is because they are fields and not actual document themselves. So, you have to first get the document and then update the value after that.
I think now you can do it better with the update command on document by using FieldValue.arrayUnion without destroying data that was added meanwhile. Like this:
const admin = require('firebase-admin');
let db = admin.firestore();
const FieldValue = admin.firestore.FieldValue;
let collectionRef = db.collection(collection);
let ref = collectionRef.doc(id);
let setWithOptions = ref.update(arrayFieldName, FieldValue.arrayUnion(value));
As described in https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.html
Arrays in Firestore don't work like this. According to the documentation:
Although Cloud Firestore can store arrays, it does not support querying array members or updating single array elements.
If you want to change any element in an array, you have to read the array values from the document first, make changes to it in the client, then write the entire array back out.
There are probably other ways to model your data that are better for your use case. That page of documentation linked above has some solutions.