I'm writing a web application that displays a large number of rows of data (~2000 at present), each of which has a drop-down "select" element with ~100 options. Any of those options can be selected by default. I'm generating all the actual DOM elements client-side. My problem: rendering this beast takes ~4 seconds on my relatively recent machine, which is really suboptimal. I know the problem is specifically with all the select elements, because replacing them with a bit of static text or a single-option list causes render time to be nigh imperceptible.
The vanilla code, minus failed experiments (see below) is here.
Avoiding the suggestions of "paginate your data" and "don't have so many options in a select", what is the most efficient way I can write my append / render code, assuming I do have a legitimate reason to display that much data and have that many options? For my purposes, Firefox is the only platform I care about.
Things I have tried:
Using an async loop to append rows to the table (slower than a regular loop, and oddly didn't render the intermediate results)
Building up a string that represents the body of the table and inserting it into the DOM in a single call (almost identical performance)
Instead of inserting the entire options list, inserting a single-option "select" element, and then populating the entire list when the "select" element gains focus (presumably because someone is trying to change it). This was actually pretty high-performing for the initial render, but then populating the element with the full list caused some weird behavior, losing focus and never actually being able to "open" the select element.
Right now my default assumption is that the third option is the way to go, and I need to figure out how to do some bookkeeping about what has already been populated. But my suspicion is that there is a plainly better / faster / more idiomatic way to do this. Is there?
Yes, I would "lazily" generate and/or populate the dropdowns.
That is, only create and populate the dropdowns when the user clicks on them, as probably almost all of the dropdowns in the 2000 rows will never be used right?
Perhaps a select element might not be the best UI here too, but instead some kind of HTML menu like so: https://jqueryui.com/menu/ that is created, populated and displayed only when the user clicks on some kind of button to display it.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I've been working with tables and a huge amount of data. There are a tables in my website with 10 thousand rows. This table has dynamic search, filters, etc. I've been using pure JavaScript considering performance, but it gets laggy with this amount of rows.
Do you guys know any alternative for pure JavaScript with better performance?
EDIT> I REALLY need to load the 10 thousand rows at once. I can load them all in the browser in 5 seconds. The main problems are the filters and search...
EDIT2> The search is dynamic. I can search by name and filter it by first character.
I've been working on it for months...
SEARCH:
search when the field has more than 3 characters and only when its length and characters has changed (onchange event on input may trigger multiple times when a character changes, so I make sure it only triggers once using some verifications)
each row that matches the searched string is coppied to another table. The original table is hidden and the new one is displayed.
when the user changes the search field or cancel the dynamic search, the new table is erased.
Conclusion: it's faster to create a new table with desired elements than hide the undesired ones.
FITLER
The rows are actually in 25 tables (A to Z + non-alphabetical characters)
When you select a character, only that table is shown
Conclusion: it's faster to hide a whole table than hide the
undesired rows
Thanks for the repplies. I've edited with some extra info so we can narrow the possible solutions...
I'm asuming you get the contents from database and load with with something like PHP (I'm going to asume PHP for now)
You could make the Javascript make an ajaxcall to a php-file which does the filtering (actually you should make the database do it, a lot faster!) and place the resulting table back on screen.
A faster method combined with the above might be this: Get all id's on the initial rows in an array and save those (in a session might work pleasant).
When you have to filter, don't make PHP get the whole table, just apply the filter to only the stored id's and send javascript the matching rows.
Then make javascript do something like this:
- set all visible
- set resulting id's to hidden (hidden in favor of remove, because I think a user might perform multiple filter actions?)
Another idea just popped in my head: If you don't need to display it on load, you can start the initial load with all tables hidden and a message "please search to display".
A common technique to handle this case is to load the data in memory or a subset of the data, and recycle your table rows such that you aren't actually ever creating thousands upon thousands of rows. You can get creative with this and create a web interface that seemingly scrolls endlessly but in reality you are just reusing dom elements and shuffling them around.
Most well-built data grid widgets whether they are on the web, mobile or even a desktop interface will employ this technique to handle your particular problem.
In most cases a user will never actually find themselves benefiting from seeing 10's of thousands of rows of data at once anyway.
fetch from the server only the things to be seen by the user, Like everyone has pointed out 10,000 rows needn't be there on that page.
you can use the concept of pagination and for every page few rows are fetched and shown . JQuery's Ajax is capable of calling the server side function to fetch rows to add them to your page.
don't know any backend details here, but in struts framework there is display:table tag and I believe in .NET framework there is GridView for pagination in the client side that you can look into
I'm just a student, newly joined to the community. Take what I say with a grain of salt.
I'm not sure why everyone is so much as blinking at the ten thousand rows business when we're measuring modern personal computers' memory in gigabytes.
Alright. I'm going to assume that what you're doing needs to be done in the browser, and so you can't switch to doing native code. In that case, looking for an alternative to Javascript won't get you much of anywhere. In the context of a browser, you're looking at an interpreted language. In terms of number of instructions the program ultimately has to run, the difference between one language or another is negligible in the face of how long it takes to be interpreted. Besides, Javascript has gotten nicely polished over the years.
So never mind that. There's a much more important thing to consider here, and it applies no matter what you're programming in or on: The cache(s). Igor Ostrovsky explains it beautifully; read it until you grok it.
So I'm guessing you have objects that would stringify to something like, "obj1 = {field-1:'a', field-2:'b', ..., field-n:'n'}". And you can select a field-i to sort by. The trouble with this is that when you sort by field-i, you're loading the entirety of obj1 into the cache, even though you don't need it. What you really want to do is load the field-i's forobj1, obj2, obj3, ..., objm all at once. So you look at an array, stringifying to something like: "field1 = [refToRow1, field1inRow1, refToRow2, field1inRow2, ..., refToRowM, field1inRowM]".
You might not be able to do fit all M rows in the cache, after all M==10000! But you can group them together into chunks that you could reasonably expect a cache to manage. Anyone got a good number for this? Say, 64kB? So you figure for each i in M you've got a reference, and a field that's probably just a reference to a short string (it'd be better if you could have the string itself right there, but I don't think Javascript works that way). So that's 8B? 8B*1024 = 64kB? Hell, if that's right, you could fit it all into the cache in two chunks, which means you'd want to do it in 4.
So now you've got a collection of smaller arrays, and you want to sort them. This is a classic application for B-trees. And while having a separate B-tree for each and every column in the table may seem like a lot, it's not.
Okay, so that handles sorting. You tell it to sort by a column, and the truth is it's already sorted! You're just repopulating the visible table using a different b-tree. You still need to handle filtering, but that's fine. You do some cache juggling as you find something to display and follow the reference to get the other fields, but I'd still expect this to go fast since you're skipping over so many rows.
Normally, I would say if you want to speed things up, look into multiprocessing. But I think browsers are still working to make that a thing with their Javascript implementations. Plus, while it would be well-suited for sorting, it would be a lot of effort to make it useful for the filtering part, and I expect you can do fine without.
I hope this isn't too scatter-brained, and that it gives you some ideas. Good luck!
Let's say that I have a select box in an html page. And the values in that select box come from a db query that returns a not so small list of options.
Now if I need to add another 4 select boxes showing the same list of options, would I have to duplicate the code and so send back to the browser 4*(select_box_result_size) or this is normally done differently?
Regardless of how you are managing the server-side code, the answer is that each select box needs its own group of options. That said, if all options are the same then you should only make one query to your database, and generate the all the select boxes you need with a single function. Simply get the options from the database, store them in a variable and then create your boxes.
Do note that in your HTML each box will have the values hardcoded into it, so your client-side code will have duplicate data, but that doesn't really matter. What is important is that the code that you'll have to maintain is clean, and that you put as little stress as possible on your database. One query + one function is all you need.
If performance was really an issue, you always have the possibility of dynamically-generating the combo boxes using JavaScript. You could essentially embed one and then copy it four more times. However there are two things you need to consider for this:
Do you really have a performance issue that is being caused by these combo boxes? (Probably not)
Is the combo box so big that it will actually take longer to load the HTML across a network connection than it will to copy the box four times with JavaScript? (Again, probably not... remember that the longer the box the longer it will take to copy.)
Unless you have 1000+ options, I would recommend to simply stick to generating the box four times. If you do have that many then do two things:
Benchmark. Create two pages, one with a single combo box and one with four and compare the size. Then compare the load times.
Consider improving your UI. Perhaps if you have 1000 options it would be better for the end user to implement some sort of filtering process to reduce the number of necessary elements... i.e. If you have a box containing all the cities of the world, have one box to select the country and then populate the city box with only the applicable cities by way of a JSON request.
And the values in that select box come from a db query that returns a
not so small list of options.
For user friendliness you could try autosuggest, there are libraries for that but you could try writing your own.
If same data is displayed multiple times (4 times) depending on your server side script it might be best to fetch the data in memory and generate the selects instead of querying 4 times.
would I have to duplicate the code
It's best not to duplicate, at the very least you can write a helper function to generate the select list. If you're serious about removing code from html markup you can look into using one of the many MVC frameworks out there that use templates and allow you to bind data to it.
I have to hide some rows of a <table> which will contain specific words. I'm searching for these rows by using the :contains selector from jQuery and using .hide().
The problem is that it results in an unpleasant glitter showing those rows momentarily before they hide.
Is there an possibility of reducing that glitter, or bind some event to the document which will analyze each DOM element upon creation and not rendering them if the will meet some requirements?
Most templating engines will do what you want, allowing you to add some logic to decide whether or not to create an element in the first place.
Another option would be to have your table hidden from the start (in CSS display:none), run your jQuery to find the rows you want to hide (or even remove) and only then use jQuery to make your table visible (with .show()). The downside to this approach is that it doesn't degrade gracefully if you have a visitor without JavaScript enabled. They won't ever be able to see the table!
Of course, if you don't have JavaScript enabled, you won't be able to hide the rows on the client side either. You'd have to handle it on the server side.
I've been creating a calender box widget (one of those little div-popups that allows you to select a date).
All's pretty much done on it short of one feature: When you create the object, you send it a field (the field that will contain the date); because of a (really) weird set of requirements, this field cannot have an easily readable format (YYYYMMddhhmmss) and so my script hides the field, and drops a div with similar styling in its place. I haven't found a way to neatly drop a div in as a sibling to the field AND right next to it (as opposed to appended at the end of the parent).
How can I take a field by ID as an argument, hide it, and drop a div in it's place at the same location?
If I could drop it, in the HTML, directly after the field, I could copy it's CSS over to the DIV (or a new field, even) and no one would be the wiser; references to the old field would still be valid, and humans could easily read the new field, but as is, the best solution I've found is to have the object take two parameters, one for the target field and one for the target div. It's not ideal.
jsFiddle: Full Project (I'm so, so, SO incredibly sorry for the widget's name. It's bad even by pun standards.)
jsFiddle: Simplified Example (Includes chosen answer)
PS:
I only have one real goal in this project: to minimize dependencies. This widget is replacing an old one my company used for ages which, over time, accumulated a dozen and a half modifications (each in different files) and needed at least as many style sheets and existing plugins. No one really knew what was going on with it. As is this one needs only jQuery, no other scripts, no other style sheets...
...I'd like to keep it that way...
http://api.jquery.com/after/
$('yourelement').hide().after(newDiv)
I'm creating a form on a web page, but I want that form to change as a user inputs values.
Example: If we have a drop down menu: ObjectType
And someone chooses Cars when that selected I'd want new options below to show up. Say, Make, Model, Year, & Notes.
Whereas, if that person chooses Vegetables then maybe all I want is a checkbox for Root Vegetables.
My question is what is the best way to do this. My form is pretty intricate and is going to have perhaps 20 different possible set ups, but I want those set ups to be sorted. I used appendChild() with AJAX requests, but I just ended up with out of order fields. And on top of that, the occasional slow connection keeps an individual waiting for a field to show up.
Rather than loading HTML pages with these form objects (technically part of a table), should I be keeping all of this stuff in an array? Is it possible to just load one HTML page or XML object and use selectElementById() on that result after it has been saved?
The objects I'm loading are rows, here's an example of one and what it would need to be able to do:
<td>
<strong>Abilities:</strong>
</td>
<td>
<textarea
id ='abilities'
name="abilities"
style="width:100%" rows="4"
onchange='userCreation.updateAbilities()'
onkeyup='userCreation.updateAbilities()'>
</textarea>
</td>
I have a table in my page, I was creating a new tr element with JavaScript, setting this as the innerHTML and then appending the child. Is there a better way to do this?
I would just put all of the form elements in a page, and hide the ones you're not using with CSS. That way you maintain order, and don't have to wait for fields to show up. It will slightly slow down page load time, but make your life a lot simpler.
Otherwise, I'd store the name (or whatever) of each field in an array or object, and when it comes time to AJAX load it, look up its position first and put it there.
Just have the entire for in the page and just change the visibility of different sections, based on user interaction instead of dynamically loading elements via AJAX.