Efficient way to search for autocomplete suggestions in a large array - javascript

I have a webapp with an input field which should be filled in with user's region according to ISO3166-2. To improve user experience I wanna make this field an autocomplete - user starts typing, and gets suggestions. There's also a country field, which, if filled, would limit the amount of regions to suggest to regions from the chosen country.
The regions list is a pretty large amount of data, which includes region names and codes for all officially recognized countries in the world and it's footprint is about 250kB.
Those of you, who had similar experience, which of the following ways to implement it would you recommend to achieve the best performance and why?
1) don't send the whole regions list to the client and make a request to server instead every time user types in region field (debounce it, ofcourse), so we look for suggestions server-side but have additional roundtrips;
2) use a webworker to find suggestions;
3) smth else?

You can have an associative array with keys as the region name (or somewhat similar), then you can have the array populated from the backend according the the logic.
Binary search the array using the typed keyword
If data exists then populate the autocomplete or else populate the array from backend data.
You have to write code to maintain the array yourself.
Or else you can save your time by using something like Twitter typeahead with Bloodhound having prefetching, intelligent caching, fast lookups, and backfilling with remote data.
Here's the link: https://github.com/twitter/typeahead.js

Related

HTML input dynamic filtering from PHP array variable?

I have a score entry page on a PHP-based website. It uses a DB query to retrieve the player’s Name & DB ID into a PHP array variable, then loops around the returned data to create an HTML <select> dropdown list.
This has now become too long… especially when entering on a mobile device (on iOS its even worse with that scroll wheel implementation they use - I dont even know what it looks like on Android!)
So I have a PHP array with the Name & ID fields in it.
I would like to convert this to filter a dynamic dropdown list as characters are typed in by the user.
I am novice at Javascript and it's nuances, though understand the principles of the language. The DOM model and other things I am also not very familiar with. I expect to use an onChange() function on the <input> textbox(?).
But how do I either tie that back to my existing PHP array variable? Or copy this variable “across” to the JS function?
I have changed the underlying DB to a NoSQL version (MongoDB). The bulk of tutorials, blogs, etc, for “AJAX Live Search” or equivalent seem to be centred around MySQL (which I previously used…)
The esiest approach would be usage of some js library, which can do what you need.
In my experience this should work for you: https://selectize.dev/
It combines select with text input, where text input is only used to seach in the select and hides unmatching records from it.

Is there a tool/engine/api we can use so our users can make/edit algorithms/calculations based on list of variables

Our goal is to make sure our users can edit algorithms, see history & also give certain users the right to only see the algorithms.
We have a Workflow engine/API. Every workflow has an instance which has Variables. these can contains values with basic and complex types; Json, String, Int, Guid, DateTime etc.
Certain Steps in the workflow require calculations. these calculations can and should be shared in different workflows.
What I am looking for is something to handle the interface for the users that make the algorithms.
It would need to be able to accept a list of variables with values as input.
(do calculations which users have built algorithms for)
As output a list of Key,Value to be able to save the data in our DB
We could make our own interface I'm just hoping there might be something to save some time. I would prefer to use a tool. Is there anything out there I might not be looking good enough
PS:
I've also looked at the possibility of using Excel as the interface/ saving method. saving the excel as different versions allowing to upload an excel to a calculation (creating a history record.) This would work however there might be tools better suited than excel.

Best practice for using PHP DB to populate JQuery autocomplete

For my site, I have a list of cities in the U.S. (about 30,000) and I'm using that to autocomplete a freeform field, so users can start typing their city, and choose it from the list.
My question is, what is the best way to put the data into the autocomplete list?
Read all of the data from the DB via PHP, and put it into a large Javascript array.
Make a call to the DB every time the user updates the city field, and search for cities that match what they're typing.
I ask this because I'd like to know which is best for performance. It seems like #2 would have a lot of DB calls, and overhead if 100+ people are hitting the site at a time.
#1 may not be great becuase 30,000 elements in an array seems like a lot.
I'm currently using method #1 and it seems OK, but it does look kind of sloppy when you view the source code and it shows 30,000 lines for the array.
Any suggestions/Input is appreciated.
Both solutions seems to be expensive for performance. The first one may cause problems with slow computers while the second one is expensive for the server resources.
I would suggest you to use a full-text search engine like Sphinx because it supports complex search queries and is really much faster than DB thanks to caching.
I do always use #2
This is what my PHP can look like:
PHP
//retrieve the search term that autocomplete sends
$term = trim(strip_tags($_GET['term']));
//Prepare Query
for($i=1; $i<=strlen($term); $i++) {
$test_term = substr($term, 0, $i);
$query = "SELECT DISTINCT column as value FROM table WHERE column LIKE '".$test_term."%'";
}

What is the best way to do complicated string search on 5M records ? Application layer or DB layer?

I have a use case where I need to do complicated string matching on records of which there are about 5.1 Million of. When I say complicated string matching, I mean using library to do fuzzy string matching. (http://blog.bripkens.de/fuzzy.js/demo/)
The database we use at work is SAP Hana which is excellent for retrieving and querying because it's in memory so I would like to avoid pulling data out of there and re-populating it in memory on the application layer but at the same time I cannot take advantages of the libraries (there is an API for fuzzy matching in the DB but it's not comprehensive enough for us).
What is the middle ground here? If I do pre-processing and associate words in the DB with certain keywords the user might search for I can cut down the overhead but are there any best practises that are employed when It comes to this ?
If it matters. The list is a list of Billing Descriptors (that show up on CC statements) therefore, the user will search these descriptors to find out which companies the descriptor belongs too.
Assuming your "billing descriptor" is a single column, probably of type (N)VARCHAR I would start with a very simple SAP HANA fuzzy search, e.g.:
SELECT top 100 SCORE() AS score, <more fields>
FROM <billing_documents>
WHERE CONTAINS(<bill_descr_col>, <user_input>, FUZZY(0.7))
ORDER BY score DESC;
Maybe this is already good enough when you want to apply your js library on the result set. If not, I would start to experiment with the similarCalculationMode option, like 'similarcalculationmode=substringsearch' etc. And I would always have a look at the response times, they can be higher when using some of the options.
Only if response times are to high, or many active concurrent users are using your query, I would try to create a fuzzy search index on your search column. If you need more search options, you can also create a fullext index.
But that all really depends on you use case, the values you want to compare etc.
There is a very comprehensive set of features and options for different use cases, check help.sap.com/hana/SAP_HANA_Search_Developer_Guide_en.pdf.
In a project we did a free style search on several address columns (name, surname, company name, post code, street) and we got response times of 100-200ms on ca 6 Mio records WITHOUT using any special indexes.

How to autocomplete text field in javascript conditionally based on datasource?

I want to be able to perform "conditional autocompleting" based on the text that is typed in the query. For example:
A user can:
Type "f", which then pull up suggest names for all field names "field_name", "field_age" as suggestions.
If the user then types or selects "field_name" then types "=" in the box, then there would be a request for a specific list of data unique to"field_name" such as [Albert, Bob, Clarisse].
If the user then types " " (space) then the options would be remotely pulled in or accessed ["AND", "OR", "==", "!=". Once the user selects or types say "==" then based on the current string so far "field_name == ' ' ", the list [Albert, Bob, Clarisse] would be used.
If the user were to instead type field_age... then a request would go out specficially for a list or json object of ages [7,3,6] that would then pop up. A sample input in a search box would be:
field_name = 'Albert' AND field_age = '7'
I've seen a library typeahead.js, but it appears to just pull in all the data to be autocompleted as opposed to "data as needed" (in this case, names and ages. the reason being is that names could be gigantic, with many many names and I do not want to client to download all the values for every single possible field available as it would be a rather large download).
What is the best way to do this? Is typeahead.js support this or some alternate library better?
Doesn't dijit/form/ComboBox deliver what you're looking for here?
There are lots of Similar answers to your question posted on here - have a browse through them...
My suggestion would be to use two boxes, seems like the most logical approach to me. The first one narrows things down by name, and the second one narrows it down by age or some other field. This way you can then do an AND/OR SQL statement. You can also, of course, limit the number of responses that you send back from the server, as you are correct, sending back a thousand+ auto completes is of little use. As far as which library, at that point you should be able to use any autocomplete library you wish, there are probably a few dozen of them. jquery UI, dojo, or my favorite Select2.
You can also of course write you own method to utilize a single text box instead of two. In which case I would suggest doing it on the server-side instead of client-side. I don't know of any libraries that can do this automatically on the client or server-side.

Categories