We Keep Coding

Use AJAX or similar technology to narrow down already existing list

Hopefully I explain this to where it makes sense, the most I could find by searching terms like I used in the title gave plenty of autocomplete examples, but nothing quite what I'm looking for. I have a list of buttons (they're coded as inputs right now) and I want to add a search field that will narrow down the buttons as the user types in a search field.
Say for example, I have 30 buttons with popular websites. If a person wanted to pull Google, they'd start typing it out which would start by including everything with the letter "G" in it, then "O", etc. Everything else would "disappear" from the page.
I can sort of think of a way to do this manually, but I think my code wouldn't be DRY. Possibly set an "on" and "off" ID, and use CSS to display:none or something to that effect.
I think the best way to do this would be via AJAX, but there may be some javascript voodoo more applicable.

To easy. At first, its unneccessary to filter the answers serverside, if all the data is already at the users. Also, you shouldnt write html and filter it with js, you should write it in js and generate an html output. Lets start with the structure:
var links=[
{
name:"google",
url:"http://google.com"
},
{nextone}
];
Now generate the links in html:
window.onload=function(){
var container=document.body;//change this to your needs
for(i=0;i<links.length;i++){
var link=links[i];
link.html=document.createElement("a");
link.html.innerHTML=link.name;
link.html.src=link.url;
container.appendChild(link.html);
}
};
If sth is inputed, hide the the unmatched ones:
function filter(string){
//loop trough links
for(i=0;i<links.length;i++){
var link=links[i];
//if string doesnt match name
if(!link.name.split(string)[1]){
link.html.style.display="none";
}else{
link.html.style.display="block";
}
}
}
Use like this:
filter("goo");
You could bind that to an input:
yourinput.addEventListener("onchange",function(){filter(this.value)},false);

Improving Twitter's typeahead.js performance with remote data using Django

I have a database with roughly 1.2M names. I'm using Twitter's typeahead.js to remotely fetch the autocomplete suggestions when you type someone's name. In my local environment this takes roughly 1-2 seconds for the results to appear after you stop typing (the autocomplete doesn't appear while you are typing), and 2-5+ seconds on the deployed app on Heroku (using only 1 dyno).
I'm wondering if the reason why it only shows the suggestions after you stop typing (and a few seconds delay) is because my code isn't as optimized?
The script on the page:
<script type="text/javascript">
$(document).ready(function() {
$("#navPersonSearch").typeahead({
name: 'people',
remote: 'name_autocomplete/?q=%QUERY'
})
.keydown(function(e) {
if (e.keyCode === 13) {
$("form").trigger('submit');
}
});
});
</script>
The keydown snippet is because without it my form doesn't submit for some reason when pushing enter.
my django view:
def name_autocomplete(request):
query = request.GET.get('q','')
if(len(query) > 0):
results = Person.objects.filter(short__istartswith=query)
result_list = []
for item in results:
result_list.append(item.short)
else:
result_list = []
response_text = json.dumps(result_list, separators=(',',':'))
return HttpResponse(response_text, content_type="application/json")
The short field in my Person model is also indexed. Is there a way to improve the performance of my typeahead?

I don't think this is directly related Django, but I may be wrong. I can offer some generic advice for this kind of situations:
(My money is on #4 or #5 below).
1) What is an average "ping" from your machine to Heroku? If it's far, that's a little bit extra overhead. Not much, though. Certainly not much when compared to then 8-9 seconds you are referring to. The penalty will be larger with https, mind you.
2) Check the value of waitLimitFn and rateLimitWait in your remote dataset. Are they the default?
3) In all likelyhood, the problem is database/dataset related. First thing to check is how long it takes you to establish a connection to the database (do you use a connection pool?).
4) Second thing: how long it takes to run the query. My bet is on this point or the next. Add debug prints, or use NewRelic (even the free plan is OK). Have a look at the generated query and make sure it is indexed. Have your DB "explain" the execution plan for such a query and make it is uses the index.
5) Third thing: are the results large? If, for example, you specify "J" as the query, I imagine there will be lots of answers. Just getting them and streaming them to the client will take time. In such cases:
5.1) Specify a minLength for your dataset. Make it at least 3, if not 4.
5.2) Limit the result set that your DB query returns. Make it return no more than 10, say.
6) I am no Django expert, but make sure the way you use your model in Django doesn't make it load the entire table into memory first. Just sayin'.
HTH.

results = Person.objects.filter(short__istartswith=query)
result_list = []
for item in results:
result_list.append(item.short)
Probably not the only cause of your slowness but this horrible from a performance point of view: never loop over a django queryset. To assemble a list from a django queryset you should always use values_list. In this specific case:
results = Person.objects.filter(short__istartswith=query)
result_list = results.values_list('short', flat=True)
This way you are getting the single field you need straight from the db instead of: getting all the table row, creating a Person instance from it and finally reading the single attribute from it.

Nitzan covered a lot of the main points that would improve performance, but unlike him I think this might be directly related to Django (at at least, sever side).
A quick way to test this would be to update your name_autocomplete method to simply return 10 random generated strings in the format that Typeahead expects. (The reason we want them random is so that Typeahead's caching doesn't skew any results).
What I suspect you will see is that Typeahead is now running pretty quick and you should start seeing results appear as soon as your minLength of string has been typed.
If that is the case then we will need to into what could be slowing the query up, my Python skills are non-existent so I can't help you there sorry!
If that isn't the case then I would maybe consider doing some logging of when $('#navPersonSearch') calls typeahead:initialized and typeahead:opened to see if they bring up anything odd.

You can use django haystack, and your server side code would be roughly like:
def autocomplete(request):
sqs = SearchQuerySet().filter(content_auto=request.GET.get('q', ''))[:5] # or how many names you need
suggestions = [result.first_name for result in sqs]
# you have to configure typeahead how to process returned data, this is a simple example
data = json.dumps({'q': suggestions})
return HttpResponse(data, content_type='application/json')

Filtering with large dataset

I have a large data-set with roughly 10,000 records. I want to be able to have a filter mechanism on this data-set. That basically performs a LIKE sql expression on a field and returns the matching results.
To do this, I've used JQuery to bind the "input" event on my filter textbox to my filter handler function.
The issue at the moment is, that If a load of keys are pressed at once in the textbox, then the filter function gets called many times, thus making many SQL calls to filter which is very inefficient.
Is there a way I can detect in my handler when the user has finished typing or when there's a gap of a certain period and only then performing the filtering? So i only make one database call when loads of characters get input at once. If the characters get input slowly though, I want to be able to filter each time.
Cheers.

Here is a way of doing it jsfiddle
var test = 0;
$('body').on('keyup','input',function(){
var val = $.trim($(this).val());
test++;
if(val !== ""){
doSomething(test);
}
});
function doSomething(t){
setTimeout(function(){
if(t === test){
//Place to Call the other Function or just do sql stuff
alert($.trim($('input').val()));
}
},500);
}
There is a global variable to test against, if the user typed another letter. And the setTimeout function waits 500ms to see if they did type another letter.
I'm sure there are better ways to do this, but that will probably get you started.

Adding more sense to jquery auto suggestions

I have implemented jquery auto suggestions in my application which has a bunch of cities in it. The problem is it showing every strings which has the entered query. if i type how it shows all the strings which has 'how' in it in between..
The following is the list of results i get if i type how but i prefer to get it in a different order which i descrided and i don't want to show other results. I dont want to show other results.
Bhowra Bh
Chalk howa <--------------------------------This should come third
Chowka Ghat
Chowrigacha
Howbagh Jabalpur <--------------------------This should come first
Howrah <------------------------------------This should come second
Khowang
Ladhowal
Majhowalia
MHOW
Mhow
Mount Howrah <------------------------------This should come fourth
Pehowa Road
Pipalwali Chowki
Saheed Ishwar Chowdhary H
how do is make this work. here is the code i use.. or is there any other way to achieve this in javascript without jquery..?
$(".selector").autocomplete(/*parameters*/);
NOTE: I use an array to store the city names and use them as source for auto complete. And i don't use AJAX

You can use a callback to do the filtering and sorting yourself.
$(".selector").autocomplete(
source: function(request, response) {
// filter parameters based on request.term
response(/*params*/, request.term);
}
);
It's also possible to alter the search term at that point, but that wouldn't work in this case.

Executing ajax code after a few seconds of inactivity from the user

I'm new here so please go easy on me. This is somewhat of a confusing situation. :)
I'm working on a search input in which a user enters a name of another user (or part of a name) in a search text box and then a list of users matching that search string is returned. The problem is that it is a bit slow when there are tens of thousands of users involved. Due to this slowness, when the user quickly enters a name into the search input, the screen will begin to flash search results for each key stroke (well after the user has already entered the search string in). It's like a severely delayed reaction.
For instance, if I enter in the name 'Wendy', the search results for the search string 'W' (the first character I entered) will not even be displayed yet. The search results for the letter 'W' will then be displayed, followed by 'We' and so on and so forth even though i've already typed the full name and just want to see the results for 'Wendy'.
What I want to do is only perform the search when the user has not entered anything for a certain period of time (i'm thinking two seconds). Otherwise, the word 'Searching' will be displayed. The code of the Javascript method is below. Just as a note, that code currently works for searching through the users, I just need to implement the delayed execution.
function updateSearchResults() {
if($F('searchString').length > 0){
Element.update($('searchResultsScrollBox'), '<h3>Searching...</h3>');
$('searching').style.display = 'block';
var url = '<c:url value='ajaxFrontGetUsers.html'/>';
var ajax = new Ajax.Updater({success: 'searchResults'}, url,
{method: 'post', parameters: $('searchForm').serialize(true)});
$('searching').style.display = 'none';
}
}
I hope this all makes sense. Thanks in advance for anyone that can help.

Try the following steps:
Every few milliseconds, check to see if the textbox has new data in it.
If the textbox has new text, execute your Ajax, and copy the text to a variable (for comparison in step 1).
If you want to improve performance from there, activate the timer whenever the user types something, and deactivate it when the Ajax call is made.

Hey, thanks for your answer.
I ended up setting 500 millisecond intervals in which a javascript function would continuously check to see if new characters were entered in the search string within that 500 millisecond time period. If they were, the search function would be called to search on the string the user had entered. Otherwise, it would wait another 500 milliseconds until the user had stopped typing. In the end, it's very similar to what you proposed.
Thanks again!

Or you could put an "onkeypress"event handler on the item that clears some global variable or cancels a timer to keep the AJAX event from firing. I know Prototype implements this for you via it's in-place editor and the "frequency" option. (I believe it sets a timeout timer that it cancels after every key press.)

I know this is an old question, but for others taking a look, I think your going about this the wrong way. I think you should date/time stamp each ajax call, and keep track of the data time stamps in your javascript. Then, when an ajax call is returned you can check the date/time stamp and make sure it is the result set for the most resent ajax call. This way you do make the ajax call for each keystroke immediately, but only display results if the ajax results catches up to the typing.
Also, are you sending over ALL matching results? Like thousands for just the letter W? To speed up the javascript side maybe you should rank the results on the database and return only the top 10-20. The doesn't want to see more than 10-20 results anyways.
Additionally, is your SQL query optimal? Seems like the query is taking too long. If your doing a one sided like search (ie. like input+'%') not a two sided like search (ie. like '%' + input + '%') then there are some really good indexes you can put on the table to help you out.