I need to fetch sub-set of documents in Firestore collection modified after some moment. I tried going theses ways:
It seems that native filtering can work only with some real fields in stored document - i.e. nevertheless Firestore API internally has DocumentSnapshot.getUpdateTime() I cannot use this information in my query.
I tried adding my _lastModifiedAt 'service field' via server-side firestore cloud function, but ... that updating of _lastModifiedAt causes recursive invocation of the onWrite() function. I.e. is does also not work as needed (recursion finally stops with Error: quota exceeded (Function invocations : per 100 seconds)).
Are there other ideas how to filter collection by 'lastModifiedTime'?
Here is my 'cloud function' for reference
It would work if I could identify who is modifying the document, i.e. ignore own updates of _lastModified field, but I see no way to check for this
_lastModifiedBy is set to null because of current inability of Firestore to provide auth information (see here)
exports.updateLastModifiedBy = functions.firestore.document('/{collId}/{documentId}').onWrite(event => {
console.log(event.data.data());
var lastModified = {
_lastModifiedBy: null,
_lastModifiedAt: now
}
return event.data.ref.set(lastModified, {merge: true});
});
I've found the way to prevent recursion while updating '_lastModifiedAt'.
Note: this will not work reliably if client can also update '_lastModifiedAt'. It does not matter much in my environment, but in general case I think writing to '_lastModifiedAt' should be allowed only to service accounts.
exports.updateLastModifiedBy = functions.firestore.document('/{collId}/{documentId}').onWrite(event => {
var doc = event.data.data();
var prevDoc = event.data.previous.data();
if( doc && prevDoc && (doc._lastModifiedAt != prevDoc._lastModifiedAt) )
// this is my own change
return 0;
var lastModified = getLastModified(event);
return event.data.ref.set(lastModified, {merge: true});
});
Update: Warning - updating lastModified in onWrite() event causes infinite recursion when trying to delete all documents in Firebase console. This happens because onWrite() is also triggered for delete and writing lastModified into deleted document actually resurrects it. That document propagates back into console and is tried to be deleted once again, indefinitely (until WEB page is closed).
To fix that issue above mentioned code has to be specified individually for onCreate() and onUpdate().
How about letting the client write the timestamp with FieldValue.serverTimestamp() and then validate that the value written is equal to time in security rules?
Also see Mike's answer here for an example: Firestore Security Rules: If timestamp (FieldValue.serverTimestamp) equals now
You could try the following function, which will not update the _lastModifiedAt if it has been marked as modified within the last 5 seconds. This should ensure that this function only runs once, per update (as long as you don't update more than once in 5 seconds).
exports.updateLastModifiedBy = functions.firestore.document('/{collId}/{documentId}').onWrite(event => {
console.log(event.data.data());
if ((Date.now() - 5000) < event.data.data()._lastModifiedAt) {return null};
var lastModified = {
_lastModifiedBy: null,
_lastModifiedAt: now
}
return event.data.ref.set(lastModified, {merge: true});
});
Related
I am new to Firestore and learning things out. On my learning path, I have reached the section on Events for meta data changes in the Firebase documentation.
This is looking very useful but I am unable to understand how to test it. This is the code in documentation
db.collection("cities").doc("SF")
.onSnapshot({
// Listen for document metadata changes
includeMetadataChanges: true
}, function(doc) {
// ...
});
I added my simple update command to see what happens, it is updating every second. I want to understand what it is trying to give me back ? and in what case I can use it ? Why is it updating every second ?
firebase.firestore().collection("cities").doc("DC")
.onSnapshot({
// Listen for document metadata changes
includeMetadataChanges: true
}, function(doc) {
// ...
var docRef = firebase.firestore().collection('cities').doc('DC');
var updateTimestamp = docRef.update({
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
});
So actually you created modern infinite loop.
You subscribed to snapshotChanges of DC document (for data and metadata), so it means any change in the document you will receive it. And as soon as you run it for the first time, the current data comes as first time run subscription.
Then in subscribe, you update same document, it means again your subscription will be run, and again and again.
firebase.firestore().collection("cities").doc("DC")
.onSnapshot({includeMetadataChanges: true}, (docSnapshot) => {
console.log(docSnapshot);
});
I have an android application I developed, that allows the sign up of users. I wrote a firebase cloud function that triggers when a User is created, to generate a 5-digit random integer value for the user who just signed up and it stores the generated code in firebase real time database in the following structure.
MainProject
|
|-Codes
|-UniqueUID_1
|-code:72834
|-UniqueUID_2
|-code:23784
The function that I deployed in order to make sure that the code generation is in the backend, is as seen below. There is a value "checker" which is initialised as 0. I use this value to determine when to exit the while loop. Basically I want the function to generate a 5-digit random value, then check the real time database if that generated value exists in all entries under "Codes", then if it does not exist, append it to the Codes under the relevant UID. If it exists, checker remains zero and the loop continues.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
var checker = 0;
exports.createUserCode = functions.auth.user().onCreate(event => {
while (checker == 0){
var newRand = getUserCode(89999,10000);
var userObject = {
uCode : newRand
};
//run a db query to strategically check value generated
return admin.database().ref("Codes/").orderByChild("uCode").equalTo(newRand).once("value",snapshot => {
if (!snapshot.exists()){
checker = 1;
//add uCode into respective uid slot under Codes
console.log(""+newRand+" : "+event.uid);
return admin.database().ref('Codes/' + event.uid).set(userObject);
}else{
checker = 0;
console.log("uCode "+newRand+" exists");
console.log("uCode generation failed for: "+event.uid);
}
});
}
});
function getUserCode(size, add){
return Math.floor(Math.random()*size+add);
}
I tested it and it worked fine. I thought the problem was solved. However, on the 7th to 11th trial, it gave me a Function returned undefined, expected Promise or value error. I tried it again after a while, and it generated the code fine. Some one else tested it and it brought the same error.
How can I fix this issue to ensure it always works? Thanks in advance.
It's really not clear to me what this function is supposed to do, and the top-level while loop doesn't make sense to me. However, I can see there are a few things wrong with what this code is doing.
First of all, it's depending on the global state checker too heavily. This value will not be the consistent for all function invocations, because they all won't be running on the same server instance. Each running server instance will see a different value of checker. Please watch this video series for more information about how Cloud Functions runs code.
Second of all, when checker has a value of 1 when the function starts, the function will do exactly what the error message says - it will return undefined. It should be pretty easy to see how this happens by reading the code.
To fix this, I suggest first coming up with a clear description of what this function is supposed to do when invoked. Also, I would strongly suggest eliminating dependency on global variables, unless you are absolutely certain you understand what you're doing and the effect they have.
I had the same problem a while ago. ESLint won't allow the function to complete because it evaluates whether every part of your code returns a promise.
From what i can see the first part of the if does return something. Try returning a boolean in the else block.
if (!snapshot.exists()){
checker = 1;
//add uCode into respective uid slot under Codes
console.log(""+newRand+" : "+event.uid);
return admin.database().ref('Codes/' + event.uid).set(userObject);
}else{
checker = 0;
console.log("uCode "+newRand+" exists");
console.log("uCode generation failed for: "+event.uid);
return false;
}
So I wanna set up a basic chat with Firestore and while implementing it I Noticed that the Listener actually registers 2 Events for 1 Document being added.
Here is the Code to add a Document:
const chatRef = this.$fireStore.collection('chats/global/messages');
chatRef.add({
displayName: 'Prof Dr Barcode',
content: this.newMsg,
timestamp: this.$fireStoreObj.FieldValue.serverTimestamp()
});
And my listener looks like this:
const chatRef = this.$fireStore.collection('chats/global/messages');
const initialRef = chatRef.limit(10).orderBy('timestamp');
initialRef.onSnapshot(querySnapshot => {
querySnapshot.docChanges().forEach(change => {
console.log(change.type);
if (change.type === 'added') {
this.messages.push(change.doc.data());
}
});
});
Now the change.type console log triggers twice for each document added. Once with added and once with modified, while investigating I saw that when I log myself the Document in the Listener it gets logged twice, at first with the added event the timestamp field is null, then instantly it triggers again with modified and the timestamp field is populated.
I dont think this is the correct way Firestore should behave?
What you're observing is the expected behavior. This is because the Firestore client SDK writes the new document to the local cache first, immediately, and all listeners are notified of the change as an "added" change. Because you wrote a server timestamp, that final value gets interpreted after the document in synchronized to the server (not on the client), which results in a change to the timestamp field. That change gets synchronized to the client, resulting in a "modified" change on the client.
If you remove the server timestamp, you should not see the additional modification, as the server now has nothing to add to the document that the client didn't already know about.
I am trying to remove an item from $firebaseArray (boxes).
The remove funcion:
function remove(boxJson) {
return boxes.$remove(boxJson);
}
It works, however it is immediately added back:
This is the method that brings the array:
function getBoxes(screenIndex) {
var boxesRef = screens
.child("s-" + screenIndex)
.child("boxes");
return $firebaseArray(boxesRef);
}
I thought perhaps I'm holding multiple references to the firebaseArray and when one deletes, the other adds, but then I thought firebase should handle it, no?
Anyway I'm lost on this, any idea?
UPDATE
When I hack it and delete twice (with a timeout) it seems to work:
function removeForce(screenIndex, boxId) {
setTimeout(function () {
API.removeBox(screenIndex, boxId);
}, 1000);
return API.removeBox(screenIndex, boxId);
}
and the API.removeBox:
function removeBox(screenIndex, boxId) {
var boxRef = screens
.child("s-" + screenIndex)
.child("boxes")
.child(boxId);
return boxRef.remove();
}
When you remove something from firebase it is asynchronous. Per the docs the proper way to remove an item is from firebase, using AngularFire is:
var obj = $firebaseObject(ref);
obj.$remove().then(function(ref) {
// data has been deleted locally and in the database
}, function(error) {
console.log("Error:", error);
});
$remove() ... Removes the entire object locally and from the database. This method returns a promise that will be fulfilled when the data has been removed from the server. The promise will be resolved with a Firebase reference for the exterminated record.
Link to docs: https://www.firebase.com/docs/web/libraries/angular/api.html#angularfire-firebaseobject-remove
The most likely cause is that you have a security rules that disallows the deletion.
When you call boxes.$remove Firebase immediately fires the child_removed event locally, to ensure the UI is updated quickly. It then sends the command to the Firebase servers to check it and update the database.
On the server there is a security rule that disallows this deletion. The servers send a "it failed" response back to the client, which then raises a child_added event to fix the UI.
Appearantly I was saving the items again after deleting them. Clearly my mistake:
function removeSelected(boxes) {
var selectedBoxes = Selector.getSelectedBoxes(boxes);
angular.forEach(selectedBoxes, function (box) {
BoxManager.remove(box);
});
Selector.clearSelection(boxes, true);
}
In the clearSelection method I was updating a field on the boxes and saved them again.
Besides the obvious mistake this is a lesson for me on how to work with Firebase. If some part of the system keeps a copy of your deleted item, saving it won't produce a bug but revive the deleted item.
For those, who have the similar issue, but didn't solve it yet.
There are two methods for listening events: .on() and .once(). In my case that was the cause of a problem.
I was working on a migration procedure, that should run once
writeRef
.orderByChild('text_hash')
.equalTo(addItem.text_hash)
.on('value', val => { // <--
if (!val.exists()) {
writeRef.push(addItem)
}
});
So the problem was exactly because of .on method. It fires each time after a data manipulation from FB's console.
Changing to .once solved that.
I have on a website facebook style notification in top right corner. I show there up to 5 latest notifications. I do initial pulling with child_added and also after same firebaseRef child_added listening for new notifications.
Now I'd like to play a sound on new notification and a little number of new notifications.
The only thing I can't figure is how to distinguish when was a new notification and when was it already seen, a.k.a page reload? Is there any other approach than making some new property read?
I was looking around and found some old answers from 2012 with suggestions limitToLast(1) which doesn't help in my case.
EDIT:
https://stackoverflow.com/a/27693310/633154 This #Kato answers recommends to listen only to new notifications which time is more than current Firebase time Firebase.ServerValue.TIMESTAMP. This seems the way to go, but I am creating a new notification with REST API and myself setting timestamp as my server's UTC. So there may be some minor inconsistencies. Shouldn't be a big deal
EDIT 2:
With this query, I'm getting correctly up to 5 last notifications on page load and no new notifications are coming afterwards
notifRef.limitToLast(5).once("value", function(snapshot) {
snapshot.forEach(function(data) {
addNotifications(data.val());
});
});
In the above linked other SO thread #Kato's answer doesn't work, notifRef.orderBy is not a function.
I have tried multiple other versions according to doc
https://www.firebase.com/docs/web/guide/retrieving-data.html#section-queries
My structure is same
{
"messages": {
"$messageid": { // firebase generated key 'JqcEWLFJrl1eaed5naN'
"sender": "kato",
"message": "hello world"
"timestamp": 1433036536108 // Firebase.ServerValue.TIMESTAMP
}
}
}
Here is what i tried to do and errors I'm getting:
var queryRef = notifRef.orderByKey().startAt(Firebase.ServerValue.TIMESTAMP);
Error:Query: When ordering by key, the argument passed to startAt(), endAt(),or equalTo() must be a string.
var queryRef = notifRef.orderByChild('timestamp').startAt(Firebase.ServerValue.TIMESTAMP);
Error: Query: First argument passed to startAt(), endAt(), or equalTo() cannot be an object.
In the documentation I have not seen that to startAt anything but the element position is passed (integer) but not a firebase timestamp object, that's why such error.
Only below compiles, just having startAt without ordering, but it's not shooting any new notifications!
var queryRef = notifRef.startAt(Firebase.ServerValue.TIMESTAMP);
queryRef.on('child_added', function(snap) {
console.log(snap.val());
addNotifications(snap.val());
// TODO clean up if more than 5 notifications
});
Any idea where could be the problem? What is the correct way to listen only to newer notifications than current timestamp?
EDIT 3
Here is my final solution
notifRef.limitToLast(5).once("value", function(snapshot) {
var lastKey = null; // at least 1 key is always present
var count = 0; // because startAt is inclusive, we have to ignore first child_added
snapshot.forEach(function(data) {
addNotifications(data.val());
lastKey = data.key();
});
checkNotifications();
notifRef.orderByKey().startAt(lastKey).on('child_added', function(snap) {
if (count > 0) {
console.log(snap.val());
addNotifications(snap.val());
// TODO clean up if more than 5 notifications
checkNotifications();
}
count++;
});
});
I don't trust browser time, so had to go first by querying last 5 existing keys, and after that passing to startAt the last key I received. notifRef.orderByKey().startAt(lastKey) can't be outside notifRef.limitToLast(5).once("value" because according to doc, once is queried last so the lastKey js variable passed to startAt would be always null.
Also need to have the count variable, because startAt is taking inclusive, but because it was already there, I need to ignore the first one.
Also with this solution when there are more than 5 notifications, I query my backend with checkNotifications only once at the end when notifications are received with once query. Otherwise on child_added it would do up to 5 times on every page load.
If there is anything that could be optimized, please tell
One solution would be to have your local client listen for the last 5 latest notifications via ref.limitToLast(5).on('child_added', ...) and then only render them to the user if some timestamp field on each of those notifications is newer than your local timestamp on the machine.
When writing those notifications from other clients, you could include a timestamp field as specified via Firebase.ServerValue.TIMESTAMP, which will use the server's notion of the Unix timestamp. Readers of that data could then compare that timestamp to their local clock to make the aforementioned determination.