I see a similar question here . However, I am very new to coding and I am trying to delete all the documents that are older than 1 month and not premium from the “users” collection. When deleting a document, the “user_online” filed need to be 30 days or more old and “user_premium” need to be “no”.
I am using the node js with adminsdk.
I would be very grateful if anyone can help me with a node js code to achieve above.
using some other post I came up with following.
var userdelete_query = db.collection('users').where('user_online', '<=', new Date(Date.now() - 2592000000) && 'user_premium', '==', 'no');
userdelete_query.get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
doc.ref.delete();
console.log(`deleted: ${doc.id}`);
});
});
above code shows no errors, but nothing happens. I think the following part is the problem
&& 'user_premium', '==', no); -
To combine two where clauses, you need to chain them, as explained in the doc.
var userdelete_query = db.collection('users')
.where('user_online', '<=', new Date(Date.now() - 2592000000))
.where('user_premium', '==', 'no');
userdelete_query.get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
doc.ref.delete();
});
});
Note that you will need to create a composite index for this query to work.
Note that with the above code you don't know when all the docs are deleted. If you want to monitor the execution of all the parallel calls to the delete() method, you can use Promise.all() as follows:
const userdelete_query = db.collection('users')
.where('user_online', '<=', new Date(Date.now() - 2592000000))
.where('user_premium', '==', 'no');
userdelete_query.get()
.then(function(querySnapshot) {
const promises = [];
querySnapshot.forEach(function(doc) {
promises.push(doc.ref.delete());
});
return Promise.all(promises);
})
.then(function() {
console.log("ALL DOCS ARE DELETED");
})
A third possible approach would be to use a batched write (or better, a batched deletion).
Related
I've been struggling for a while to find out how exactly to use the getAll function on the first param of the callback function on runTransation function.
The doc firebase doc only shows how to use the get function to retrieve a single doc, but I want to retrieve multiple docs based on multiple where statements from a collection.
the question is how to use getAll function bellow?
export const match = functions.firestore
.document('waitingList/{userId}').onCreate(async (snapshot, context) => {
app.firestore().runTransaction(async transaction => {
transaction.getAll( //?); // ???
});
});
UPDATE 1 | 23/4/2022
I figured out a way (but I'm not fully satisfied with it because of duplication in reading from firestore).
The solution is as follows
app.firestore().runTransaction(async transaction => {
const docRefs: FirebaseFirestore.DocumentReference<any>[] = [];
(await firestore
.collection('collectionName')
.limit(100)
.get())
.forEach((doc) => {
docRefs.push(doc.ref);
}); //This is reading the Database for 100 docs
const users = await transaction.getAll(...docRefs); // This also is reading the database for the same 100 docs (But in a transaction context)
});
if someone knows how to read the docs only once in the transaction please do provide the solution.
Something like that:
transaction.getAll(...docRefs).then((docs) => {
docs.forEach((doc) => {/*...*/}
}
As #Abobker already stated, I agree that for now this will be the best way to retrieve multiple docs based on multiple where statements from a collection:
app.firestore().runTransaction(async transaction => {
const docRefs: FirebaseFirestore.DocumentReference<any>[] = [];
(await firestore
.collection('collectionName')
.limit(100)
.get())
.forEach((doc) => {
docRefs.push(doc.ref);
}); //This is reading the Database for 100 docs
const users = await transaction.getAll(...docRefs); // This also is reading the database for the same 100 docs (But in a transaction context)
});
Although is not the most efficient solution, It does address the problem.
I have developed a game using Firestore, but I have noticed some problems in my scheduled cloud function which deletes rooms that were created 5 minutes ago and are not full OR are finished.
So for that, I am running the following code.
async function deleteExpiredRooms() {
// Delete all rooms that are expired and not full
deleteExpiredSingleRooms();
// Also, delete all rooms that are finished
deleteFinishedRooms();
}
Deleting finished rooms seems to work correctly with this:
async function deleteFinishedRooms() {
const query = firestore
.collection("gameRooms")
.where("finished", "==", true);
const querySnapshot = await query.get();
console.log(`Deleting ${querySnapshot.size} expired rooms`);
// Delete the matched documents
querySnapshot.forEach((doc) => {
doc.ref.delete();
});
}
But I am experiencing concurrency problems when deleting rooms created 5 minutes ago that are not full (one room is full when 2 users are in the room, so that the game can start).
async function deleteExpiredSingleRooms() {
const currentDate = new Date();
// Calculate the target date
const targetDate = // ... 5 minutes ago
const query = firestore
.collection("gameRooms")
.where("full", "==", false)
.where("createdAt", "<=", targetDate);
const querySnapshot = await query.get();
console.log(`Deleting ${querySnapshot.size} expired rooms`);
// Delete the matched documents
querySnapshot.forEach((doc) => {
doc.ref.delete();
});
}
Because during the deletion of a room, a user can enter it before it is completely deleted.
Any ideas?
Note: For searching rooms I am using a transaction
firestore.runTransaction(async (transaction) => {
...
const query = firestore
.collection("gameRooms")
.where("full", "==", false);
return transaction.get(query.limit(1));
});
You can use BatchWrites:
const query = firestore
.collection("gameRooms")
.where("full", "==", false)
.where("createdAt", "<=", targetDate);
const querySnapshot = await query.get();
console.log(`Deleting ${querySnapshot.size} expired rooms`);
const batch = db.batch();
querySnapshot.forEach((doc) => {
batch.delete(doc.ref);
});
// Commit the batch
batch.commit().then(() => {
// ...
});
A batched write can contain up to 500 operations. Each operation in
the batch counts separately towards your Cloud Firestore usage.
This should delete all the rooms matching that criteria at once. Using a loop to delete them might take a while as it'll happen one by one.
If you are concerned about the 500 docs limit in a batch write, consider using Promise.all as shown:
const deleteOps = []
querySnapshot.forEach((doc) => {
deleteOps.push(doc.ref.delete());
});
await Promise.all(deleteOps)
Now to prevent users from joining the rooms that are being delete, it's kind of harder in a Cloud Function to do so as all the instances run independently and there may be a race condition.
To avoid that, you many have to manually check if the room that user is trying to join is older than 5 minutes and has less number of players. This is just a check to make sure the room is being deleted or will be deleted in no time.
function joinRoom() {
// isOlderThanMin()
// hasLessNumOfPlayers()
// return 'Room suspended'
}
Because the logic to filter which rooms should be deleted is same, this should not be an issue.
Maybe you are looking for transactions check out the documentation out here: https://firebase.google.com/docs/firestore/manage-data/transactions
Or watch the YouTube video that explains the concurrency problem and the differences between batched writes and transactions: https://youtu.be/dOVSr0OsAoU
I have an angular app that is using Firestore. Whenever I query for docs in collection that meet a specific condition, the array that is returned contains every document in the collection. I do not understand why this is happening as I am following the documentation.
On call to the collection in the component
this.FirebaseService.getDocsByParam( 'versions', 'projectId', this.projectData.uid )
.then((snapshot) => {
var tempArray = [];
var docData;
snapshot.forEach((doc) => {
docData=doc.data();
docData.uid=doc.id;
tempArray.push(docData);
});
this.versionList = tempArray;
this.versionData = this.versionList[this.versionList.length-1];
this.initializeAll();
})
.catch((err) => {
console.log('Error getting documents', err);
});
Firebase service making the call
getDocsByParam( collection, getParam:string, paramValue:string ) {
var docRef = this.afs.collection(collection, ref => ref.where(getParam, '==', paramValue));
return docRef.ref.get();
}
Below is a screen shot of the versions collection. It shows one of the returned docs, which does not even have the required field.
When you call docRef.ref on a AngularFirestoreCollection it returns the underlying collection, not the query. So your return docRef.ref.get() is indeed getting the entire collection.
I think you can use docRef.query to get the query, but I don't even thing there's any reason to use an AngularFire call at all here. Since your code is already using the plain JavaScript API to process the documents, you might as well stick to that SDK in your getDocsByParam too:
getDocsByParam( collection, getParam:string, paramValue:string ) {
var docRef = this.afs.collection(collection).ref;
return docRef.where(getParam, '==', paramValue).get();
}
I'm new to firestore and I'm making a register page with vue.
Before a new user is made, it has to check if the given username already exists or not and if not, make a new user.
I can add a new user to the database, but I don't know how to check if the username already exists or not. I tried a lot of things and this is the closest I've gotten:
db.collection("Users")
.get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
if (this.username === doc.data().username) {
usernameExist = true;
}
});
});
Anyone got any ideas?
Link to documentation: https://firebase.google.com/docs/firestore/query-data/queries#simple_queries
You can where this query, which is beneficial to you in multiple ways:
1: Fewer docs pulled back = fewer reads = lower cost to you.
2: Less work on the client side = better performance.
So how do we where it? Easy.
db.collection("Users")
.where("username", "==", this.username)
.get()
.then(querySnapshot => {
//Change suggested by Frank van Puffelen (https://stackoverflow.com/users/209103/frank-van-puffelen)
//querySnapshot.forEach(doc => {
// if (this.username === doc.data().username) {
// usernameExist = true;
// }
//});
usernameExists = !querySnapshot.empty
});
I'd like to make a copy of a collection in Firestore upon an event using Cloud Functions
I already have this code that iterates over the collection and copies each document
const firestore = admin.firestore()
firestore.collection("products").get().then(query => {
query.forEach(function(doc) {
var promise = firestore.collection(uid).doc(doc.data().barcode).set(doc.data());
});
});
is there a shorter version? to just copy the whole collection at once?
I wrote a small nodejs snippet for this.
const firebaseAdmin = require('firebase-admin');
const serviceAccount = '../../firebase-service-account-key.json';
const firebaseUrl = 'https://my-app.firebaseio.com';
firebaseAdmin.initializeApp({
credential: firebaseAdmin.credential.cert(require(serviceAccount)),
databaseURL: firebaseUrl
});
const firestore = firebaseAdmin.firestore();
async function copyCollection(srcCollectionName, destCollectionName) {
const documents = await firestore.collection(srcCollectionName).get();
let writeBatch = firebaseAdmin.firestore().batch();
const destCollection = firestore.collection(destCollectionName);
let i = 0;
for (const doc of documents.docs) {
writeBatch.set(destCollection.doc(doc.id), doc.data());
i++;
if (i > 400) { // write batch only allows maximum 500 writes per batch
i = 0;
console.log('Intermediate committing of batch operation');
await writeBatch.commit();
writeBatch = firebaseAdmin.firestore().batch();
}
}
if (i > 0) {
console.log('Firebase batch operation completed. Doing final committing of batch operation.');
await writeBatch.commit();
} else {
console.log('Firebase batch operation completed.');
}
}
copyCollection('customers', 'customers_backup').then(() => console.log('copy complete')).catch(error => console.log('copy failed. ' + error));
Currently, no. Looping through each document using Cloud Functions and then setting a new document to a different collection with the specified data is the only way to do this. Perhaps this would make a good feature request.
How many documents are we talking about? For something like 10,000 it should only take a few minutes, tops.
This is the method i use to copy data to another collection, I used it to shift data (like sells or something) from an active collection to a 'sells feed' or 'sells history' collection.
At the top i reference the documents, at the bottom is the quite compact code.
You can simply add a for loop on top for more than 1 operation.
Hope it helps somebody :)
DocumentReference copyFrom = FirebaseFirestore.instance.collection('curSells').doc('0001');
DocumentReference copyTo = FirebaseFirestore.instance.collection('sellFeed').doc('0001');
copyFrom.get().then((value) => {
copyTo.set(value.data())
});
There is no fast way at the moment. I recommend you rewrite your code like this though:
import { firestore } from "firebase-admin";
async function copyCollection() {
const products = await firestore().collection("products").get();
products.forEach(async (doc)=> {
await firestore().collection(uid).doc(doc.get('barcode')).set(doc.data());
})
}