I am experiencing some interesting behaviour regarding a subscription on a collection's valueChanges.
Immediately after creating a doc, the collection's subscription is invoked, but instead of an array of many documents I am only receiving an array of size one - the single, newly created document.
After having a read through this (Firestore Docs | Get Realtime Updates) I am still slightly confused.
Local writes in your app will invoke snapshot listeners immediately. This is because of an important feature called "latency compensation." When you perform a write, your listeners will be notified with the new data before the data is sent to the backend.
Does this explain the behaviour I am seeing?
Here is a stackblitz demonstrating the problem. Just uncomment out the commented line in ngOnInit() and reload to see what I believe is expected behaviour.
I can solve this by either having an empty subscription listening to this collection elsewhere, or duplicating the take(1) subscription code directly before
Thats a good catch. Pretty sure you are right - valueChanges() as docs states:
The current state of your collection. Returns an Observable of data as
a synchronized array of JSON objects.
And as you found yourself:
Local writes in your app will invoke snapshot listeners immediately.
So this is what happens:
Your addPizza() is an async function. It sends request to backed to add new pizza. But it doesnt wait for anything and jumps to your second function - this.getPizzasAsyncAwait(). And because that local write invokes listener immediately, your Observable emits that value and broadcasts it. And since you also use Rxjs's take(1) - after that it unsubscribes. That also explains why take(2) brings all other records. You can move your getPizzasNormal() method to OnInit() and you'll receive the whole collection.
The firebase js sdk handles optimistically the add, before getting the collection values, the collection is not defined yet. Once you add a value the collection contains 1 value and then is updated by the server side values.
If you want to avoid getting this intermediate state when the collection values are set only from local changes, you can subscribe before start changes to be done before subscribing :
this.piazzaRef.add({
name: name,
addedAt: new Date().toISOString()
}).then(() => {
this.getPizzasAsyncAwait();
this.getPizzasNormal();
});
I updated your example here
Related
My application stores in-memory copies of Firestore documents. When data is changed by a given client, it first updates it's own in-memory copy, then writes those same updates to firebase.
function updateDoc(docPath, changes) {
updateInMemoryDocWithChanges(changes);
app.firebase.db.doc(docPath).update(updates);
}
It's possible these documents will be changed by other clients at the same time. In order to capture these changes, we're using onSnapshot({ includeMetadataChanges: true}, ...) to listen for changes to those documents, only updates the in-memory copy when hasPendingWrites is false:
function handleSnapshot(doc) {
// local write just occurred
if (doc.metadata.hasPendingWrites) return;
updateInMemoryObject(doc.data())
}
The Problem
On slow connections, there can be a noticeable delay on receiving the onSnapshot event with hasPendingWrites: false. So the following order of operations happens, but because of the delay, the client is temporarily updated with stale data.
updateDoc()
snapshot, hasPendingWrites=true (ignored)
another change is made, updateDoc() called again
snapshot, hasPendingWrites=false from the update written in step 1. In memory copy is updated, overwriting changes made in step 3!
snapshot, hasPendingWrites=true with updates from step 3 (ignored)
then a few moments later..
snapshot, hasPendingWrites=false with updates from step 3. In memory copy is updated.
As a result the client sees some disconcerting loss of data before having it re-appear in 7.
I would like to simply ignore snapshot notifications that are the result of local client changes, but the only way I can see to do this is to match up snapshots that hasPendingWrites=true with hasPendingWrites=false, perhaps with the same doc? Seems ugly and error prone.
What is the right way of approaching this?
To be sure the DocumentSnapshot corresponds to the current state of the database on the server, check docSnapshot.metadata.fromCache. If this property is true, the value is coming from the local cache and may not be up to date. If it is false, the value is guaranteed to be up to date with the server.
These properties go hand in hand for me, and I (currently) explain them to myself as:
hasPendingWrites indicates if there are local changes to this document that haven't been sent to/handled by the server yet.
fromCache indicates that this document may be stale. I think the property name is a bit unfortunate here, as its value indicates data freshness more than where the snapshot was loaded from.
The solution in cases like this is to implement some form of data versioning. For example, add an integer version property that you increment each time you modify the object. Reject any updates with an older version than the in-memory copy. Your flow would become:
updateDoc() (in-memory=1)
snapshot, hasPendingWrites=true (ignored) (snapshot=1)
another change is made, updateDoc() called again (in-memory=2)
snapshot, `hasPendingWrites=false from the update written in step 1. (in-memory=2, shapshot=1) Update is rejected because memory version > snapshot version
snapshot, hasPendingWrites=true with updates from step 3 (ignored) (snapshot=2)
then a few moments later..
snapshot, hasPendingWrites=false with updates from step 3. (in-memory=2, snapshot=2) In memory copy is updated because memory version <= snapshot version
I'm working in a ReactJS and Meteor project and I found a strange behavior I'm gonna describe here:
There is a Tracker.autorun block with a Meteor.subscribe call inside. So far, so good. In the server side, there is a matching Meteor.publish which declares a callback.
As far as I understand, the Meteor.publish callback should fire once for each subscription received, but somehow this callback is firing 3~4 times for a single subscription.
In my last test the Tracker.autorun block executed 4 times, the subscribe only executed 1 single time and the callback fired 4 times.
The Meteor.subscribe only runs once, even the tracker runs several times. How could it cause the callback to fire more the once?
Does it make sense?
Do you know what could explain such behavior?
If you need any other information, just let me know.
Thanks in advance
Meteor.publish('current-user', function currentUser(credentials) {
return Users.find();
});
Tracker.autorun((c) => {
if (!currentUserHandler) {
currentUserHandler = Meteor.subscribe('current-user', this.credentials);
}
});
You should expect that the autorun will fire twice as a normal condition, once without data, and the second with some data.
That is to allow you to show a "loading" state before the data arrives.
You are subscribing to the users collection, which is a special collection. Meteor uses it for authentication, and also to record session activity. You are doing a Users.find(), which is an unfiltered query on the whole users collection, so any modification to any user will cause it to fire. You also won't be able to see all of the users records (for security reasons).
It's probable that you are storing additional data on the users record, hence the need for you to subscribe to it. I would recommend that you consider storing this data in another collection, such as 'members', 'visitors', 'profiles' or whatever name suits you. Things are likely to work better that way.
Something I noticed the other day. I was subscribing to the Actions stream inside #ngrx/effects. I noticed (an issue for me) where a component, which subscribes to Actions late, receives the last dispatched action. This I can see is because the dispatcher inside #ngrx/store is a BehaviourSubject and quoting the RxJS docs:
Rx.BehaviorSubject class
Represents a value that changes over time. Observers can subscribe to the subject to receive the last (or initial) value and all subsequent notifications.
Unfortunately, I'd like to subscribe to the actions stream without retrieving the last value. It creates problems for me like showing error messages when a user returns to a page.
The difference between BehaviorSubject and Subject (which does it the way I'd like) is demo'd here:
https://codepen.io/anon/pen/zwgype
Is there any way of achieving this?
You can skip the first value with the 'skip' operator:
someBehaviorSubject.skip(1)
I've been following lots of meteor examples and working through discover meteor, and now I'm left with lots of questions. I understand subscribe and fetch are ways to get "reactivity" to work properly, but I still feel unsure about the relationship between find operations and subscriptions/fetch. I'll try to ask some questions in order to probe for some holistic/conceptual answers.
Question Set 1:
In the following example we are fetching 1 object and we are subscribing to changes on it:
Meteor.subscribe('mycollection', someID);
Mycollection.findOne(someID);
Does order of operations matter here?
When does this subscription "expire"?
Question Set 2:
In some cases we want to foreign key lookup and use fetch like this:
MyCollection2.find({myCollection1Id: doc1Id}).fetch();
Do we need also need a MyColletion2.subscribe when using fetch?
How does subscribe work with "foreign keys"?
Is fetch ~= to a subscription?
Question Set 3:
What is an appropriate use of Tracker.autorun?
Why/when should I use it instead of subscribe or fetch?
what happens when you subscribe and find/fetch
The client calls subscribe which informs the server that the client wants to see a particular set of documents.
The server accepts or rejects the subscription request and publishes the matching set of documents.
Sometime later (after network delay) the documents arrive on the client. They are stored in a database in the browser called minimongo.
A subsequent fetch/find on the collection in which the aforementioned documents are stored will query minimongo (not the server).
If the subscribed document set changes, the server will publish a new copy to the client.
Recommended reading: understanding meteor publications and subscriptions.
question 1
The order matters. You can't see documents that you haven't subscribed for (assuming autopublish is off). However, as I point out in common mistakes, subscriptions don't block. So a subscription followed by an immediate fetch is should return undefined.
Subscriptions don't stop on their own. Here's the breakdown:
A global subscription (one made outside of your router or template) will never stop until you call its stop method.
A route subscription (iron router) will stop when the route changes (with a few caveats).
A template subscription will stop when the template is destroyed.
question 2
This should be mostly explained by the first section of my answer. You'll need both sets of documents in order to join them on the client. You may publish both sets at once from the server, or individually - this is a complex topic and depends on your use case.
question 3
These two are somewhat orthogonal. An autorun is a way for you to create a reactive computation (a function which runs whenever its reactive variables change) - see the section on reactivity from the docs. A find/fetch or a subscribe could happen inside of an autorun depending on your use case. This probably will become more clear once you learn more about how meteor works.
Essentially, when you subscribe to a dataset, it fills minimongo with that data, which is stored in the window's local storage. This is what populates the client's instance of that Mongo with data, otherwise, basically all queries will return undefined data or empty lists.
To summarize: Subscribe and Publish are used to give different data to different users. The most common example would be giving different data based on roles. Say, for instance, you have a web application where you can see a "public" and a "friend" profile.
Meteor.publish('user_profile', function (userId) {
if (Roles.userIsInRole(this.userId, 'can-view', userId)) {
return Meteor.users.find(userId, {
fields: {
public: 1,
profile: 1,
friends: 1,
interests: 1
}
});
} else {
return Meteor.users.find(userId, {
fields: { public: 1 }
});
}
});
Now if you logged in as a user who was not friends with this user, and did Meteor.subscribe('user_profile', 'theidofuser'), and did Meteor.users.findOne(), you would only see their public profile. If you added yourself to the can-view role of the user group, you would be able to see public, profile, friends, and interests. It's essentially for security.
Knowing that, here's how the answers to your questions breaks down:
Order of operations matters, in the sense that you will get undefined unless it's in a reactive block (like Tracker.autorun or Template.helpers).
you still need to use the subscribe when using fetch. All fetch really does is return an array instead of a cursor. To publish with foreign keys is a pretty advanced problem at times, I recommend using reywood:publish-composite, which handles the annoying details for you
Tracker.autorun watches reactive variables within the block and will rerun the function when one of them changes. You don't really ever use it instead of subscribing, you just use it to watch the variables in your scope.
I am trying to understand WHEN Firebase actually loads the data to the client vs. doing "lazy load" (only download the data when it's needed). The reason is that I am saving images (base64) in Firebase (please don't ask why as it's only few hundred MBs). So there are two choices:
// With typical Firebase
var imagesRef = Ref.child('images');
// With Angularfire
var imagesObj = $firebaseObject(Ref.child('images'));
Ref is just a reference to my Firebase URL.
I know with Angularfire, there is $loaded() which makes me think Angularfire actually loads all the data AT ONCE and makes it available when you call $firebaseObject() right away. Is it correct?
As for using child(), I don't see any load() event to catch based on the documentation. Maybe I missed it. But does it load all data from the server to client?
If I have like 500MB of images, I definitely don't want this load-all-at-once happening.
firebase retrieve the data when you call .on on a ref
As not widely know, all the data are retrieved in one piece (wether you call .on 'value' or .on 'child_added'), so you better paginate your result using orderByFirst / Last, or using firebase util
What angular fire does when you instanciate a firebaseObject / array is calling on 'value' / 'child_added' from within the constructor function of the instance, so yes , the data are retrieved almost instantly (but in a defer hence the $loaded() function).
Check out the source code of the Object manager and the constructor of $firebaseObject for instance, it's pretty clear