I want to preLoad data from the server so that it will be immediately available when I call for it. The data is stored in a file called "data.json".
At first, I thought to save data.json to an object and reference it when I need it.
However, depending on the user's actions it may take a while before I need the data that this object has stored. So memory management becomes a bit of a concern as the object is quiet large (~5 mb).
My question is, when I call for data.json via ajax, does the browser internally "cache" this file for the duration of the website session.
Meaning, if I called for the file via ajax again after calling for it already, the browser would instantly get the file from its own internal memory instead of going back to the server?
If so it seems it would be wasteful saving an extra copy of this file in JavaScript. However, I can't find any information/standards about this online.
So in short, do I need to save the downloaded file to an object? Or is it safe to let browsers handle this internally?
There are several different types of "cache" in play here. It sounds like you're asking "How long does the browser's JavaScript engine keep an object in memory" and the answer is "As long as there's a reference to it."
The browser's (HTTP) cache, on the other hand, lives longer; an entry may live for days or weeks or years, depending on the available space, the freshness headers on the response, etc.
For a scenario like you've described, you probably want to prefetch the JSON to a local cache file, then load that cache file into JavaScript only as needed.
Related
I'm building a tool, which core structure is: make an AJAX request to Cloudflare worker, which fetches HTML data, and then returns it.
So the steps are:
Send request from client
Worker receives request and makes another, which returns a response as a typical HTML document.
Aaaand on the third step I have two options:
to return the obtained HTML back via AJAX response and then parse it on client
to parse HTML first, and then return processed data via AJAX response
The first one is straightforward: I receive the response from my worker, and insert it the returned HTML somewhere in a hidden <div> and then parse it.
The reason I would prefer to go with a second one, though, is not to waste the bandwidth while delivering HTML from Cloudflare Worker back to client, because original page has a lot of irrelevant bloat. I mean, for example, the original page looks something like this:
<div class="very-much-bloat" id="some-other-bloat" useful_parameter ="value">
<div id="some-other-irrelevant-info" id="really-great-id">
something that I need
</div>
</div>
And all that I need from this is, for example
{
"really-great-id" : "something that I need",
"useful_parameter" : "value"
}
If I go with the first step, it would be pretty straightforward to parse it in-browser, however I'll waste bandwidth for delivering a lot of information that is later disposed of.
However, if the second one would involve using complex libraries, it wouldn't be probably a way to go since max execution time per request is 10ms (that's a free plan on Cloudflare, which otherwise is plenty enough: 100,000 requests per day is more than I probably ever need with this app).
The question is: is there any efficient way to parse HTML on Cloudflare worker without breaking 10ms time limit? Page size obtained with worker is around 10-100 KB, parsed data size is around 1-10KB (10 times less than original roughly). While I understand that 100KB may not sound like a lot, it's still mostly garbage that's better to filter as soon as possible.
Cloudflare Workers currently does not support the DOM API. However, it supports an alternative HTML parsing API that might work for you: HTMLRewriter
https://developers.cloudflare.com/workers/runtime-apis/html-rewriter/
This API is different from DOM in that it operates in a streaming fashion: JavaScript callbacks are invoked as the HTML data streams in from the server, without ever holding the entire document in memory at one time. If it fits your use case, it may allow you to respond faster and use fewer resources than a DOM-based solution would. The CPU time used by HTMLRewriter itself does not even count against the 10ms limit -- only the time spent by your callbacks counts. So if you design your callbacks carefully, you should have no problem staying within the limit.
Note that HTMLRewriter is primarily designed to support modifying an HTML document as it streams through. However, it should not be too hard to have it consume the document and generate a completely different kind of data, like JSON. Essentially, you would set up the rewriter so that the "rewritten" HTML is discarded, and you'd have your callbacks separately populate some other data structure or write to some other stream that represents the final result.
I am currently exploring different processes for saving a javascript object generated by a user.
One option I am exploring is to require the user to save their data to a file, that they can later re-upload to the website to continue working on.
Problem is, I need a way to ensure that the save file cannot change between download and re-upload.
Making a save file with javascript is possible.
Freezing objects is possible, but I am not sure if this will prevent the user from manipulating the data before re-upload.
Is there a way to save a JSON object to a file and prevent the user from editing the file?
If you save an object locally onto a users HHD you cannot make it tamper-proof. They always have the ability to edit it no matter what.
You can however make it so you can test whether it's been tampered with using checksums and hashes and the like.
If your site knows when this has happened then it's easy - you can just check the file hash value against your value and if they're the same allow the user to pick up where they left off.
If it's being done anonymously then your file generation process must have its own encrypted/unknown checksum process attached to the file. Then on upload check again to see if tampering has occurred.
Again, I suggest you think in terms of DETECTING alteration rather then PREVENTING alteration.
Essentially what you want is a checksum to also be generated and verify against that. You can read more about methods to do so here: https://en.wikipedia.org/wiki/File_verification
If you truly wish to ensure the file has not been tampered, you will need to store the checksum results somewhere else that the user can't access.
I have some data that I want to display on a web page. There's quite a lot of data so I really need to figure out the most optimized way of loading and parsing it. In CSV format, the file size is 244K, and in JSON it's 819K. As I see it, I have three different options:
Load the web page and fetch the data in CSV format as an Ajax request. Then transform the data into a JS object in the browser (I'm using a built-in method of the D3.js library to accomplish this).
Load the web page and fetch the data in JSON format as an Ajax request. Data is ready to go as is.
Hard code the data in the main JS file as a JS object. No need for any async requests.
Method number one has the advantage of reduced file size, but the disadvantage of having to loop through all (2700) rows of data in the browser. Method number two gives us the data in the end-format so there's no need for heavy client-side operations. However, the size of the JSON file is huge. Method number three has the advantage of skipping additional requests to the server, with the disadvantage of a longer initial page load time.
What method is the best one in terms of optimization?
In my experience, data processing times in Javascript are usually dwarfed by transfer times and the time it takes to render the display. Based on this, I would recommend going with option 1.
However, what's best in your particular case really does depend on your particular case -- you'll have to try. It sounds like you have all the code/data you need to do that anyway, so why not run a simple experiment to see which one works best for you.
I'm coding a website that involves storing very simple data, just a very long list of names with no additional data, on the server. As this data is so simple, I don't really want to use MySQL (it would be a bit too clunky) so I'm asking what's the best way to store very simple data on the server.
I definitely would favour speed over anything else, and easy access to the data via javascript and AJAX would be very good too as the rest of the site is coded in javascript/jQuery. I don't really care if the data can be viewed freely (as it will be available anyway), as long as it can't be changed by unauthorised users.
There are a lot of things to think about with this.
Is the information the same for all users with just a single set that applies to all users out there? Or is there a separate set of data for each user?
How is the data going to be served to the client, my guess here is that you would be having a web service or otherwise that might return a JSON.
From a security standpoint, do you want someone to be able to just "grab" the data and run?
Personally I find that a database if often a better choice, but otherwise i would use an XML file. Keep in mind though that you have to be careful with loading/reading of XML files to serve web requests to prevent any potential file locking issues.
Use an XML file that is web-accessible. Then you can query the XML file from the browser if need be, and still parse/write it in PHP. You'll want to use the flock function in PHP to make sure that two instances of a page don't try to write to the file at the same time.
Write it to a file and save the data as a serialized object. This way when you read in the data it's instantly accessible as the variable type you need (array, obj, etc). This will be faster than XML parsing.
I would like to keep the contents of large UI lists cached on the client, and updated according to criterial or regularly. Client side code can then just fill the dropdowns locally, avoiding long page download times.
These lists can be close to 4k items, and dynamically filtering them without caching would result in several rather large round trips.
How can I go about this? I mean, what patterns and strategies would be suitable for this?
Aggressive caching of JSON would work for this, you just hash the JS file and throw it on the end of it's URL to update it when it changes. One revision might look like this:
/media/js/ac.js?1234ABCD
And when the file changes, the hash changes.
/media/js/ac.js?4321DCBA
This way, when a client loads the page, your server-side code links to the hashed URL, and the client will get a 304 Not Modified response on their next page load (assuming you have this enabled on your server). If you use this method you should set the files to never expire, as the "expiring" portion will be dealt with by the hash, i.e., when the JS file does expire, the hash will change and the client won't get a 304, but rather a 200.
ac.js might contain a list or other iterable that your autocomplete code can parse as it's completion pool and you'd access it just like any other JS variable.
Practically speaking, though, this shouldn't be necessary for most projects. Using something like memcached server-side and gzip compression will make the file both small and amazingly fast to load. If the list is HUGE (say thousands of thousands of items) you might want to consider this.
Combres is a good solution for this - it will track changes and have the browser cache the js forever until a change is made, in which case it changes the URL of the item.
http://combres.codeplex.com/
You might consider rather than storing the data locally using jQuery and AJAX to dynamically update the dropdown lists. Calls can be made whenever needed and the downloads would be pretty quick.
Just a thought.
This might be helpful:
http://think2loud.com/using-jquery-and-xml-to-populate-a-drop-down-box/
If its just textual data, you have compression enabled on the web server, and there are less than 100 items, then there may be no need to maintain lists in the client script.
Its usually best to put all your data (list items are data) in one place so you dont have to worry about synchronization.