I have an app where I save its state as often as possible to a Json file. When multiple events occur at the same time, they may overwrite information over each other. The state is held by a singleton, and my current solution is to add small delays before saving to the file, and it works for the time being but I feel like its waiting to go wrong at some point.
Related
In my app, I have a list that requires an "or" condition. But, as the docs say:
In this case, you should create a separate query for each OR condition and merge the query results in your app.
As a result, in my service, I'm managing two queries and surfacing them as a single observable list to consumers.
The problem comes in with updating. I have the choice of doing extra work to match up the item needing update to the correct collection so I can do the following:
myCollection.doc(item.id).update(item);
or I can make this much more simple and just:
angularFirestore.doc(`path/to/${item.id}`).update(item);
I'm operating under the assumption that the first method will result in faster updates as I'm using the same reference that it would optimistically update instantly. And that the latter will be slower in that it would be more round about by updating the persistence layer and then the collection referencing getting notified about later (probably still a small time).
All of the above is assumption, however. I back this just with a few random instances where I've seen it take a second or two for an update or delete to show up in an other part of the view, but I haven't been able to actually inspect the process.
Does anyone know if the above is correct? Should I be doing the extra work to write through the collection references or does angularfire(and/or firestore) handle this and make them effectively the same operation under the hood?
AngularFire2 is a thin wrapper around RxFire, which itself is a relatively thin wrapper around the Firebase JavaScript SDK.
There should be no significant performance difference between updating a document through AngularFire or updating it directly through the JavaScript SDK. In both cases the majority of the time is spent in the JavaScript SDK, and on the wire between the client and server. For this reason I typically update directly through the JavaScript SDK, since it's often a bit more direct and the AngularFire abstraction has little advantage for me in write operations. Given that AngularFire is built on top of this SDK, it picks up the changes instantly even when they're not made through AngularFire.
If you have an instance where this does not seem to be the case, I recommend creating a question with the minimal, complete/standalone code that reproduces that problem.
I created an app that stores, compares, filters and takes statistics out of a collection of records. I've done it so it works offline, as in some user cases the user might not have constant (or at all) access to internet.
My problem is that after I've included ~60 records, the app starts to behave really slow. For instance, I list a collection of simple objects from LocalStorage into a ng-model (Select list), and after those ~60 records are in, to open the Select box will be seriously slowed down.
What could the problem be? I'm thinking, either some function is sucking more resources than necessary, or LocalStorage is not intended for such uses?
I'm starting to get into PouchDB, would you say that migrating all to Pouch instead of LocalStorage would be a good move?
I can't paste the whole controller here as it's huge, but I've put an online version for testing. You can see it here.
For you not to have to create 60 records just to see the effect, you can download this CSV and import it in the app.
In order to import, the pass for Edit Mode is: admin
Let's see if someone has a tip for this one!
I see you are storing all your records inside a single LocalStorage value (with the key being recordspax). So yeah, that will get quite slow, because your app has to 1) JSON parse/stringify and 2) store/retrieve the entire list every time you read/write data to the database.
Basically you are reading your entire database in and out of disk for every operation. Since both LocalStorage and JSON stringify/parse happen synchronously on the main thread, it can block DOM rendering and will thus slow down your app.
PouchDB could be a help here, but you could also benefit from something simpler like LocalForage, or simply changing your DB design so that every record has its own key/value rather than storing everything into a single key with a single value.
(Both LocalForage and PouchDB use IndexedDB/WebSQL rather than LocalStorage, meaning that database operations are not synchronous and do not block the DOM. However, you still don't want to stuff everything into a single document and therefore read the entire DB in and out of disk. :))
We are trying to pre-cache a large sum of data on load of our web application into indexed db. From my performance testing the speed is decent on a desktop browser (e.g. Internet Explorer) where I can insert 10,000 records in around 2 seconds. But comparing the exact same functionality on the iPad it drops to 30 seconds. That comparison just blew my mind.
Does anyone know of any hints or tricks to inserting large data sets into indexedDB. I dont know if it is possible at all but if we could build up a copy of an indexedDB server side with all the data prepopulated and then just shoot it over to the client and it just stores it down to the browser. Is anything along these lines doable?
Thanks
I had problems with massive bulk insert (100.000 - 200.000 records). I've solved all my IndexedDB performance problems using Dexie library. It has this important feature:
Dexie has a kick-ass performance. It's bulk methods take advantage of
a not well known feature in indexedDB that makes it possible to store
stuff without listening to every onsuccess event. This speeds up the
performance to a maximum.
Dexie: https://github.com/dfahlander/Dexie.js
Some pretty bad IndexedDB performance problems can be caused by a prolonged period of the browser just calling onsuccess callbacks and running into event loop overhead after the work is actually done. The performance pattern observed by my app which was doing this was that it did a bunch of work, then it just went answering thousands of callbacks very inefficiently:
The right hand part of this image is the callbacks on every request. The solution to doing that is, of course, to not put a callback on every request, but it was previously unclear to me how to do this.
The way that Dexie.js accomplishes this (for details, see src/dbcore/dbcore-indexeddb.ts) is that it saves the last request (e.g. IDBObjectStore.put, etc) sent and sets an onsuccess callback on that one, which then collects the results from the rest of the requests. Thus, it avoids the callback hell.
Another approach from this is to use the IDBTransaction.oncomplete event, and not worry about the callbacks on the individual requests at all.
(note: yes, I know how old this question is, I had this problem today and wanted to put something more useful for this question which is high in Google results)
How is your data stored in the indexeddb? Is everything in a single object store of do you use multiple objectstores. Do you need all the cached data immediatly?
If you only have a single object store you can start with storing all the data you initialy need, commit that transaction and start a new for all the rest. This way you can start retrieving the initial data while inserting the rest. IndexedDB is async so it should block you.
If you have multiple object stores you can use the same stratigy. First fill up the objectstore you need immediatly and delay the others.
Or maybe consider using the AppCache API instead of the indexeddb api. Using this you can just cache a javascriptfile containing all the json objects you want to cache. This is more the case when you don't need a lot of querying on the data.
imagine that we have loaded a complex website with lots of Javascript which loaded all sort of info via AJAX and by making computations based on user input. So, now suppose we want to archive it in such a way that we can reliably load it later on from file (maybe even without an internet connection) and study its behavior / debug it / etc. Is there a way to do this?
The browsers already do this to make the "Back" button work fast -- in Firefox it's called "bfcache". This cache lives only in memory, though. I don't know if it's possible to serialize it to a file, if yes, it would be very interesting.
I don't think there's a way to export the entire DOM state without manually looking at each piece, and storing it. There is a lot of information that goes in representing that DOM than what visible in the source.
For instance, you might want to save the window scrollbar position which is available in the window object as window.scrollX and window.scrollY. This is just one example but there's plenty of other state information to be saved including attached event handlers etc.
If you could identify the pieces that are relevant for you purposes while ignoring others, you could store it locally using Google Gears (now obsolete) or the new Local Storage introduced in HTML5 and if you are already serializing this information, you could pass it on to some server and restore it from there. The new storage mechanism in HTML5 is called DOM Storage but its a little misleading because it's just a key value pair storage where both the keys and values are strings.
Edit: This might be a different perspective on the problem but here it goes. Instead of storing the entire DOM state, you could store just the intial state, and the relevant actions that change it. To get to the final state, a replay mechanism would be used that runs each action in sequence. This is a popular design pattern known as the Command pattern. That's how multiplayer games keep each player up-to-date and in-sync by passing only the player actions like a keystroke, mouse movement, etc. instead of the entire view and the receiver applies those actions to update its state. It's a lot more complicated than that in practice but thats the crux of it.
Where would you want to store it?
Currently there's no way to store anything browser side (apart from new browser features, that very few people have installed). The only realistic solution would be in a Cookie if the DOM is small enough (this is because a Cookie can only hold a certain amount of data).
If you're looking at storing the DOM server-side, then you could use document.body.innerHTML to access the current DOM state and then send it to your server.
I'm getting in to a situation where I have several interacting widgets (on a web UI), all of whom can be in multiple different states, and whose behavior depends on others the others. I'm running in to situations where, for example, a set of data gets sorted twice, or the data gets displayed before it's sorted, rather than the other way around. It's a little bit of a wack-a-mole problem, where I think I've simplified things and gotten it working, only to find out I've broken things somewhere else.
I have functions that do things like:
widgetAFunction
load data into widget B
tell widget B to sort the data
tell widget B to display the data
My love of code reuse makes me want to do something like write a loadData function in widget A that goes something like this:
widgetBLoadDataFunction
update data
sort the data
refresh the view
So that all widgetA has to do is call one function on widgetB. But then there are cases where I just want to sort the data, without updating the data, so I write:
widgetBSortFunction
sort the data
refresh the view
And then maybe I want a filter function
widgetBFilterFunction
filter the data
refresh the view
And maybe I want to be update the data but not sort it, so I have
widgetBNoSortLoadDataFunction
update data
refresh the view
It doesn't seem that complex, but I wind up with these really long, very brittle chains of function calls, or a bunch of very similar calls. As Martin Fowler would say, the code is getting a little smelly.
So, what other alternatives do I have? I did something on a recent project where I did a state machine kind of thing, where I registered a bunch of functions with a set of conditions, or states which would trigger their execution. That worked somewhat well, and I'm thinking that approach might be good to use again.
Does anyone know what I'm talking about here, and even better, can anyone point me toward some patterns that will help me get my head around this better?
What you need is a finite state machine implementation. Basically every finite state machine needs:
Events that the program responds to
States where the program waits between events
Transitions between states in response to events
Actions taken during transitions
Variables that hold values needed by actions between events
A good article from IBM teachs you a way of implementing it by means of Javascript.
Edit: Here is a FSM builder, so you don't have to build your own.
Fernando already mentioned FSMs, and gave good info and links. :)
In addition, I'll add that your classes should already incorporate enough state so that you're not worried about sorting twice, etc. I.e., widgetB.sort() should check if it's been sorted since last update and just return if so. There's practically no downside to doing this, and it can improve performance (and also guard consistency).