Filtering with large dataset - javascript

I have a large data-set with roughly 10,000 records. I want to be able to have a filter mechanism on this data-set. That basically performs a LIKE sql expression on a field and returns the matching results.
To do this, I've used JQuery to bind the "input" event on my filter textbox to my filter handler function.
The issue at the moment is, that If a load of keys are pressed at once in the textbox, then the filter function gets called many times, thus making many SQL calls to filter which is very inefficient.
Is there a way I can detect in my handler when the user has finished typing or when there's a gap of a certain period and only then performing the filtering? So i only make one database call when loads of characters get input at once. If the characters get input slowly though, I want to be able to filter each time.
Cheers.

Here is a way of doing it jsfiddle
var test = 0;
$('body').on('keyup','input',function(){
var val = $.trim($(this).val());
test++;
if(val !== ""){
doSomething(test);
}
});
function doSomething(t){
setTimeout(function(){
if(t === test){
//Place to Call the other Function or just do sql stuff
alert($.trim($('input').val()));
}
},500);
}
There is a global variable to test against, if the user typed another letter. And the setTimeout function waits 500ms to see if they did type another letter.
I'm sure there are better ways to do this, but that will probably get you started.

Related

Datatables: can you use each() inside an event listener?

I have a Datatable where I need to do two things:
Sum all the figures in a column.
Find out if the cells in another column have a certain value and, if so, blank them. (Reason: a MySQL table that has a default date of "1970-01-01" if you don't insert the date. I want to replace all the "1970-01-01" instances with an empty cell).
To do that, I attached an event listener to the draw event, and then:
For 1) I used the sum() Datatables plugin.
For 2), I followed this answer: Find and replace cells value with Datatables plugin, and used each().
var tabla=$('#list').DataTable();
tabla.on('draw.dt', function() {
var bla=tabla.column(10).data().sum();
tabla.column(9).nodes.each(function(node, index, dt) {
var ble=tabla.cell(node).data();
if (/^1970-01-01/.test(ble)) {
table.cell(node).data('');
}
});
});
The problem is that, when I execute this, I get a "tabla.column(...).nodes.each is not a function". Searching around, I find that this error is caused most of the times when jQuery or the datatable haven't been initialized yet... but in my case, they have most certainly been, because the first thing (the sum()) does get done; in other words, the "bla" variable does get calculated.
This makes me wonder whether you can use each() inside an event trigger; otherwise, what could be causing this?
You could use the $.each function to iterate over an arbitrary collection
However, and though I'm not sure what's your intention regarding the sum of the column, I'd recommend working with the render option for each column involved.
For instance, in the case of the date column you could have a render function like this:
render: function (data,type,fullRow){
if(data === '1970-01-01') {
return null;
}
else {
return data;
}
}
(Disclaimer: not tested!)
The drawCallback function (and the draw event, for that matter ) gets called every single time Datatables need to redraw the table: initial loading, filtering, sorting, new row added, etc. This might have some unforeseen collateral effects, or simply an impact in performance.
Hope it helps you!

Search & LazyLoad can't keep up w/ typing speed?

So, all of the code works quite well. A database a queried, the node/parent IDs are lazily passed to jsTree, and, if one types a term into the search bar—a similar process goes on, but the nodes passed to jsTree are those returned by another SQL query (using something like SELECT nodeID FROM table WHERE name LIKE %searchTerm%).
There's only one problem:
If I type too quickly into the search bar, the results get all mixed up with one another. If I type slowly (I'd estimate 2 letters a second max), everything works well. Any faster and everything is blurred together. (That is, if I'm searching for names which contain the term "test", and type this quickly, I'll get names that contain "t", "te", "tes", and "test" instead of just names that contain "test".)
Anyone have this problem before? Is there some "wait until ready" function I should be using?
I can suggest you to do the 'Timeout' workaround. Basically, you create a SetTimeout function with a delay of 200-400 miliseconds and launch your lazyload ajax there. Every inputbox.change event restarts the time. See example in pseudo-javascript:
$('#your-input').keydown(function(){
if(ajaxTimer != undefined) {
clearTimeout(ajaxTimer);
}
ajaxTimer = setTimeout(function(){
$.ajax({...}).done(function() {...}
},400);
})
Use a throttle on the typing input, basically guarantees that a function will be called a maxmimum of once every X seconds. Sure you could write your own however there is a great library out there already.
check it out if you're interested http://benalman.com/projects/jquery-throttle-debounce-plugin/

While loops in protractor

I have recently started using protractor for e2e tests. I have a combobox which lists events that match what the user types in - so the list is empty if no such events are found.
What I would like to do is enter 3 random characters, and if the combobox list is empty, clear the combobox and retry another 3 random characters. This should be repeated until a non-empty list is found.
I have tried using a simple while loop to do this, but the asynchronous nature of webdriverjs means that I become stuck in an infinite loop. Is it possible to somehow wait inside the loop for the combobox to be populated? Or is there some other, cleaner solution to this problem?
Code:
var query = element(by.model('searchStr'));
query.clear();
var letters = generateRandomLetters();
console.log(letters);
query.sendKeys(letters);
var eventList = element.all(by.repeater('result in results'));
eventList.count().then(function(count) {
if(count) {
//test continues here
}
});
Declare a function that calls itself recursively until your condition is met. That should to the trick.

Two drop down option lists in a form using javascript

I have a problem with two drop down lists ( options ) in a form. They both work separately but together they don't work at all. It's like they are canceling each other out or something. If anyone can help me then I'll be truly grateful. Here's the code with the problem... JS fiddle - http://jsfiddle.net/wemdragon/3Uz/
There are a couple problems. First off, .val() gets a string rather than an int, so if ('0') will return true. Therefore, options 2! is always emitted no matter what you have selected for the second option. You need to update that conditional to check if (second.val() != 0) (this will coerce '0' and 0 and return true -- you can also get more specific or simply use an empty string as the value instead, which is probably preferred).
Second of all, your calculation code is never reached since it is the else in the chain of ifs that cover all possible combination of selections. You need some other flag to check that the calculation should be done, or make it a function that is called when a valid selection is made.

Executing ajax code after a few seconds of inactivity from the user

I'm new here so please go easy on me. This is somewhat of a confusing situation. :)
I'm working on a search input in which a user enters a name of another user (or part of a name) in a search text box and then a list of users matching that search string is returned. The problem is that it is a bit slow when there are tens of thousands of users involved. Due to this slowness, when the user quickly enters a name into the search input, the screen will begin to flash search results for each key stroke (well after the user has already entered the search string in). It's like a severely delayed reaction.
For instance, if I enter in the name 'Wendy', the search results for the search string 'W' (the first character I entered) will not even be displayed yet. The search results for the letter 'W' will then be displayed, followed by 'We' and so on and so forth even though i've already typed the full name and just want to see the results for 'Wendy'.
What I want to do is only perform the search when the user has not entered anything for a certain period of time (i'm thinking two seconds). Otherwise, the word 'Searching' will be displayed. The code of the Javascript method is below. Just as a note, that code currently works for searching through the users, I just need to implement the delayed execution.
function updateSearchResults() {
if($F('searchString').length > 0){
Element.update($('searchResultsScrollBox'), '<h3>Searching...</h3>');
$('searching').style.display = 'block';
var url = '<c:url value='ajaxFrontGetUsers.html'/>';
var ajax = new Ajax.Updater({success: 'searchResults'}, url,
{method: 'post', parameters: $('searchForm').serialize(true)});
$('searching').style.display = 'none';
}
}
I hope this all makes sense. Thanks in advance for anyone that can help.
Try the following steps:
Every few milliseconds, check to see if the textbox has new data in it.
If the textbox has new text, execute your Ajax, and copy the text to a variable (for comparison in step 1).
If you want to improve performance from there, activate the timer whenever the user types something, and deactivate it when the Ajax call is made.
Hey, thanks for your answer.
I ended up setting 500 millisecond intervals in which a javascript function would continuously check to see if new characters were entered in the search string within that 500 millisecond time period. If they were, the search function would be called to search on the string the user had entered. Otherwise, it would wait another 500 milliseconds until the user had stopped typing. In the end, it's very similar to what you proposed.
Thanks again!
Or you could put an "onkeypress"event handler on the item that clears some global variable or cancels a timer to keep the AJAX event from firing. I know Prototype implements this for you via it's in-place editor and the "frequency" option. (I believe it sets a timeout timer that it cancels after every key press.)
I know this is an old question, but for others taking a look, I think your going about this the wrong way. I think you should date/time stamp each ajax call, and keep track of the data time stamps in your javascript. Then, when an ajax call is returned you can check the date/time stamp and make sure it is the result set for the most resent ajax call. This way you do make the ajax call for each keystroke immediately, but only display results if the ajax results catches up to the typing.
Also, are you sending over ALL matching results? Like thousands for just the letter W? To speed up the javascript side maybe you should rank the results on the database and return only the top 10-20. The doesn't want to see more than 10-20 results anyways.
Additionally, is your SQL query optimal? Seems like the query is taking too long. If your doing a one sided like search (ie. like input+'%') not a two sided like search (ie. like '%' + input + '%') then there are some really good indexes you can put on the table to help you out.

Categories