I'm building a guestlist app for my company using PHP, Javascript/jQuery/Ajax, JSON and localstorage. It will be used on smartphones: primarily Iphone 4. I'm using localstorage as my cache since the search-part of the application has to work in offline mode.
I'm having performance issues while searching through a guestlist.
The app's flow looks like this (for this examaple I'm working with guestlist which contains 600 guests)
1. Retrieve all guests from the server with PHP encode with JSON and send back to js via AJAX. This works fine.
2. Parse the PHP responseText (called phpData) using JSON.Parse:
var parsedMysqlData = JSON.parse(phpData);
which gives us an JSON.Array containing 600 objects looking like this:
Object: {
Id: Int
EventId: int
guestInfo: string
guestlist: string
guestName: string
reference: string
total: int
used: int
}
3. Save the JSON.Array to the user's localstorage using JSON.Stringify:
localStorage.setItem(0, JSON.stringify(parsedMysqlData));
4. when the user starts searching we get his search string then retrieve our guestlist using JSON.parse in localstorage like this:
var currentGuestlist = JSON.parse(localStorage.getItem(0));
And then iterate through our objects using this for-loop trying to match his search string with our guests the array currentGuestlist:
for (i=0; i<currentGuestlist.length; i++) {
// match 'currentGuestList[i]['name']' with what the user typed in search form
}
For some reason this takes a really long time on an iPhone 4. Searching through 600 objects will freezes the iphone for about 4 seconds before returning the matches.
Before storing arrays containing JSON objects in localStorage and parsing it with JSON, I simply stored unordered strings in localStorage and it work a whole lot faster. The JSON objects ad structure to the data stored in localStorage which is crucial. So I guess the speed issue has to have something to do with the fact that I'm using JSON objects? How can i structure my data i localStorage in an organized way while still maintaining as good speed performance as before?
Lastly anykind of tips or advice on which techniques you would use to make this app as lightweight and fast as possible is greatly appreciated.
Are you fetching the list per each search from the local storage? Don't do that, instead store it in the local storage only as needed (whenever it changes), and keep it always as a data structure.
Simply having objects instead of plain strings cannot be the reason for slowness, as everything in JavaScript is an object already, and thus it should only be slowing by a constant factor.
Furthermore, if this is about autocomplete kind of behaviour, then I suggest you would slow down the search, and also consider that if the user types in the box "Ma", the list gets filtered, and the user adds "tt" for "Matt", only previously filtered matches need to be considered...
Perhaps your set up would me more suited to use the WebSQL-database instead of storing a JSON object in the local storage. WebSQL is now deprecated but it's pretty well supported in webkit-browsers. and i'm using it with good results on a couple of projects.
You can read more about it here: http://html5doctor.com/introducing-web-sql-databases/
Related
I have a frontend Application in Javascript with jQuery
I get a JSON Object every few seconds.
I have to incorporate this JSON Object into my local data.
I can either take the Object and completely replace a local Object with it, or I could check each entry and only replace the changed entries.
In case it makes a difference:
I don't know the structure of the object in advance
The object maximum depth is usually 2 ({ "a": { "b":"c"}})
Which one is faster in terms of processing time?
Is there even a straight answer or does it depend on the object and / or browser?
My real world problem is: users of my mobile app type their city and I have to make sure it really exists, and that it is correctly written (caseinsensitive, so these are correct: New York, NEW york, new york. This is not correct: newyork)
There are online apis that work quite well (Google Geocode API for example) but:
After a very little amount of requests, you have to pay (2.500/day right now)
Users must be connected to the internet
That's why I tought that an offline-local solution would be better. There are many websites (like Maxmind) where you can download a list containing every city in the world. I could embed this huge txt/csv right inside my application and do a string search locally (it's a big file, ok, but not that big. It's just a onetime download of something like 30-40MB of uncompressed .txt)
I'm trying to avoid jQuery at all costs and I don't want to use any PHP/MySQL solutions (even if fulltext indexes could be handy), that's why I'm trying to do all this just using javascript.
Given a string as input, let's say "city3", what's the best/fastest way to check if it's inside an (external) huge list like:
city1,
city2,
city3,
city4,
[...]
After solving this (big) problem: if there are no exact matches, is there a way to search for the correct city without freezing the device for 10 minutes?
In the example before, lets say the user types "cit y3" or "cyty3" or "cìty3": can any js function tell him that he might be looking for "city3"? Is this kind of search too slow in this scenario?
Thanks
If speed is an issue then I would recommend loading the data into a JavaScript object and performing an in-memory search rather than repeatedly scanning a big blob of text in a file.
Try formatting the data into JSON with the city names as keys, that will give you good search performance.
A Workaround is creating a Database either SQL either noSQL, and Query this database through your JavaScript Code, using jquery Json functions.
Using a SQL Database ideal would be either MySQL either MariaDB An enhanced, drop-in replacement for MySQL.
In this solution you will probably need a Backend such as PHP to fetch the data from your Database convert them to JSON Format, and then get them through your JavaScript using jQUery Library , with the $.getJSON function
Using a noSQL Database ideal would be MongoDB.
In this solution you can fetch your data directly from javascript, also with the $.getJSON function.
Example for MongoDB Provided Here
if you dont want to use database i think you can do this:
-first , instead use one big file split it into several files. (you can write a script for this and use it just one time for split the big file). in each file put cities that starts with (example) aa , second file cityes that starts with ab.
-then for each city check first letters and then search inside that file.
For example if you need to search for city "Ahmedabad" it will search only in the files with cities that starts with Ah. Probably this is not the best solution ,at the end you got 421 file instead 1 , but reasearch will be faster.
I'm ending up having to do hacks to convert 'true' to just true and it's creating code smell.
Is there a library like https://github.com/thephpleague/fractal that allows me to transform my response into the types I need?
In cases like this it's almost always better to fix the API to return data in a usable format rather than trying to post-process the result on the client.
In your case there are several routes you could take:
Store the list as a JSON string directly in the database.
This means you don't have to do any processing on the server and can just return it 'as is'. However you lose the ability to do queries on the data directly and need to resort to things like LIKE and string operations.
Store the data relationally, and process it on the server to turn it into JSON
Here you retain the ability to do queries on your data, but you may need to do several queries to get all the data you need and then connect it on the server. (eg. you would do one SELECT on the user table to get a user, and then you would need to do another SELECT on the friends table where the userid matches your first user. You would then need to merge these results to create your JSON.) This is usually the best way to do it.
You can also turn the result into JSON directly inside the database engine using a user defined function. For example using https://github.com/mysqludf/lib_mysqludf_json#readme
This is somewhat similar to 2, but it ties your stored procs to the JSON format.
I have a web application built in .NET
During a session, the user has to access a dictionary value, that is populated from the database. The user makes frequent calls to it.
I want to cut down the HTTP calls and want to bring the dictionary value to the client and access it from there.
I wanted to understand what would be the best practice to store this dictionary value locally in such a way that retrieving data is very quick?
What I'm trying to really do is similar to what FaceBook does with "#", so when you write #Name, it quickly makes a search in the database and replaces the text with the link. In my case, I have a fixed set of data to search from and usually it is not beyond 10-15 name-value pairs.
I contemplated of storing values in cookies, but don't really know if there would be any storage limit and am also unaware of how quick the retrieval would be.
Any help is much appreciated.
Thank You
If the dictionary is static then you can use JSON.stringify and localStorage to store it. This way the user only needs to load it the first time they ever visit your site. After it is stored in localStorage, I would suggest loading it into a JavaScript object using JSON.parse each time the page is loaded to speed up searching since a JavaScript object works like a hash table.
You could then add a simple check each time the page loads to see if the dictionary needs to be refreshed.
var globalDictionaryHash = null;
function loadDictionary()
{
if (localStorage.getItem("my_dict") == null)
{
localStorage.setItem("my_dict", JSON.stringify(myObjectFromAJAXCall));
}
globalDictionaryHash = JSON.parse(localStorage.getItem("my_dict"));
console.log(globalDictionaryHash['key']);
}
I am creating an offline mobile web app, and am looking at using JSON to replicate some of my database tables and storing that in localStorage. (I am aware of Web SQL Database but it doesn't look particularly future-proof.)
I started with a very basic JSON output from the database, which looks a bit like this:
{
"1": {"id":"1","name":"Hello","alias":"hello","category":"8"},
"2": {"id":"2","name":"World","alias":"world","category":"3"},
...
}
However, there is a lot of data in many tables and space could be an issue with the constant repeating of field names. Storing the data like this halves the size:
{
"1": ["1","Hello","hello","8"},
"2": ["2","World","world","3"},
...
}
But now I have to reference a piece of data with a numeric index, possibly filling my code with magic numbers. I thought of storing an array like ["id","name"...] in another variable but the extra lookups seem like they would get messy.
Are there any practical ways to avoid that, but also keeping the Javascript code fairly neat? Any other useful strategies for this kind of development?
would it be possible to convert it into a format like this:
{
id:{1:1, 2:2, ...},
name:{1:hello, 2:world},
alias:{1:hello, 2:world},
category{1:8,2:3}
}
This way, you only store each column once, but you can still easily find things by their id.
JSON is not a database. JSON is a data interchange format.
Not sure if it would work across all mobile platforms, but XML would be an option.