We Keep Coding

Smart autocomplete with tensorflow.js

Is it possible to build some sort of autocomplete select which suggests values based on previous inputs?
Let’s say I have airport select with 5k items. I would like give user top 5 values to select based on his previous preferences.

Deep learning approach:
Basically what you want is an RNN powered word-level prediction model. Such models are used in text generation. You can refer to one example here.
These models specifically take in a word ( t ) and predict the next word ( t1 ). Then taking t1 it predicts t2. This cycle goes on until the max number of words is reached.
If I had to train such an RNN model on a single sentence like
Hello world from TensorFlow
Then the feature-label pairs would look like:
X, Y
Hello, World
World, from
from, TensorFlow
These sentences could be trained from the user's previous inputs.
Things you will require in building such a model:
Lots of user data.
A ranking algorithm to get the top N predictions
Probably, a GPU or TPU. CPUs might prove to be very slow and might take days.
Non-Deep learning approach:
You can Bayes Theorem or simply check which word has the highest probability after a specific word. The word "morning" will have the highest probability after the word "good".
Tip:
Mostly, you will also need to handle custom user words which you may not add in your dictionary. It could be the name of a person. So while collecting data, you need to tag such words into a common word like .

Javascript html canvas organise lines to prevent overlap

I'm working on some logic for a labelling exercise.
I want to connect every red dot to a blue dot, but without the lines overlapping. (not as shown below)
I have a jsfiddle that generates dots and connects them here
https://jsfiddle.net/s1u7okd5/
enter code here
Red dots can vary, obviously, blue dots are fixed. I don't need someone to do the work for me, but I could do with some direction.
Questions:
1: I assume it's always possible to find a solution where the lines don't overlap (ignoring thickness of drawn line). Is this true?
2: I hoping to avoid a brute force approach. Is this possible?

Assuming that the number of red dots (N) is equal to the number of blue dots (from pictures seems so), a quite naive solution can be:
calculate the length of all possible links between all pairs blueDot-redDot (time complexity O(N^2));
sort links by increasing distance (O(Nlog(N)));
until there are bluePoint still not selected:
extract the shorter link;
Check if the associated bluePoint and redPoint has already been involved in others links (O(1) if you store flags in arrays indexed by point IDs):
if yes: discard the link;
if no: connect the points and flag them as selected.
Probably there can be some optimization, but this is anyway an iterative O(N^2) solution: much better than a brute force solution that, for example, use backtracking to explore all possibilities and find the right one.

String similarity [duplicate]

I'm building a website that should collect various news feeds and would like the texts to be compared for similarity. What i need is some sort of a news text similarity algorithm.
I know that php has the similar_text function and am not sure how good it is + i need it for javascript.
So if anyone could point me to an example or a plugin or any instruction on how this is possible or at least where to look and start investigating.

There's a javascript implementation of the Levenshtein distance metric, which is often used for text comparisons. If you want to compare whole articles or headlines though you might be better off looking at intersections between the sets of words that make up the text (and frequencies of those words) rather than just string similarity measures.

The question whether two texts are similar is a philosophical one as long as you don't specify exactly what it should mean. Consider the Strings "house" and "mouse". Seen from a semantic level they are not very similar, but they are very similar regarding their "physical appearance", because only one letter is different (and in this case you could go by Levenshtein distance).
To decide about similarity you need an appropriate text representation. You could – for instance – extract and count all n-grams and compare the two resulting frequency-vectors using a similarity measure as e.g. cosine similarity. Or you could stem the words to their root form after having removed all stopwords, sum up their occurrences and use this as input for a similarity measure.
There are plenty approaches and papers about that topic, e.g. this one about short texts. In any case: The higher the abstraction level where you want to decide if two texts are similar the more difficult it will get. I think your question is a non-trivial one (and hence my answer rather abstract) ... ;-)

Grade Sudoku difficulty level

I am building a Sudoku game for fun, written in Javascript.
Everything works fine, board is generated completely with a single solution each time.
My only problem is, and this is what's keeping me from having my project released to public
is that I don't know how to grade my boards for difficulty levels. I've looked EVERYWHERE,
posted on forums, etc. I don't want to write the algorithms myself, thats not the point of this
project,and beside, they are too complex for me, as i am no mathematician.
The only thing i came close to was is this website that does grading via JSbut the problem is, the code is written in such a lousy undocumented, very ad-hoc manner,therefor cannot be borrowed...
I'll come to the point -Can anyone please point me to a place which offers a source code for Sudoku grading/rating?
Thanks
Update 22.6.11:
This is my Sudoku game, and I've implemented my own grading system which relies
on basic human logic solving techniques, so check it out.

I have considered this problem myself and the best I can do is to decide how difficult the puzzle is to solve by actually solving it and analyzing the game tree.
Initially:
Implement your solver using "human rules", not with algorithms unlikely to be used by human players. (An interesting problem in its own right.) Score each logical rule in your solver according to its difficulty for humans to use. Use values in the hundreds or larger so you have freedom to adjust the scores relative to each other.
Solve the puzzle. At each position:
Enumerate all new cells which can be logically deduced at the current game position.
The score of each deduction (completely solving one cell) is the score of the easiest rule that suffices to make that deduction.
EDIT: If more than one rule must be applied together, or one rule multiple times, to make a single deduction, track it as a single "compound" rule application. To score a compound, maybe use the minimum number of individual rule applications to solve a cell times the sum of the scores of each. (Considerably more mental effort is required for such deductions.) Calculating that minimum number of applications could be a CPU-intensive effort depending on your rules set. Any rule application that completely solves one or more cells should be rolled back before continuing to explore the position.
Exclude all deductions with a score higher than the minimum among all deductions. (The logic here is that the player will not perceive the harder ones, having perceived an easier one and taken it; and also, this promises to prune a lot of computation out of the decision process.)
The minimum score at the current position, divided by the number of "easiest" deductions (if many exist, finding one is easier) is the difficulty of that position. So if rule A is the easiest applicable rule with score 20 and can be applied in 4 cells, the position has score 5.
Choose one of the "easiest" deductions at random as your play and advance to the next game position. I suggest retaining only completely solved cells for the next position, passing no other state. This is wasteful of CPU of course, repeating computations already done, but the goal is to simulate human play.
The puzzle's overall difficulty is the sum of the scores of the positions in your path through the game tree.
EDIT: Alternative position score: Instead of completely excluding deductions using harder rules, calculate overall difficulty of each rule (or compound application) and choose the minimum. (The logic here is that if rule A has score 50 and rule B has score 400, and rule A can be applied in one cell but rule B can be applied in ten, then the position score is 40 because the player is more likely to spot one of the ten harder plays than the single easier one. But this would require you to compute all possibilities.)
EDIT: Alternative suggested by Briguy37: Include all deductions in the position score. Score each position as 1 / (1/d1 + 1/d2 + ...) where d1, d2, etc. are the individual deductions. (This basically computes "resistance to making any deduction" at a position given individual "deduction resistances" d1, d2, etc. But this would require you to compute all possibilities.)
Hopefully this scoring strategy will produce a metric for puzzles that increases as your subjective appraisal of difficulty increases. If it does not, then adjusting the scores of your rules (or your choice of heuristic from the above options) may achieve the desired correlation. Once you have achieved a consistent correlation between score and subjective experience, you should be able to judge what the numeric thresholds of "easy", "hard", etc. should be. And then you're done!

Donald Knuth studied the problem and came up with the Dancing Links algorithm for solving sudoku, and then rating the difficulty of them.
Google around, there are several implementations of the Dancing Links engine.

Perhaps you could grade the general "constrainedness" of a puzzle? Consider that a new puzzle (with only hints) might have a certain number of cells which can be determined simply by eliminating the values which it cannot contain. We could say these cells are "constrained" to a smaller number of possible values than the typical cell and the more highly constrained cells that exist the more progress one can make on the puzzle without guessing. (Here we consider the requirement for "guessing" to be what makes a puzzle hard.)
At some point, however, the player must start guessing and, again, the constrainedness of a cell is important because with fewer values to choose between for a given cell the easier it is to find the correct value (and increase the constrainedness of other cells).
Of course, I don't actually play Sudoku (I just enjoy writing games and solvers for it), so I have no idea if this is a valid metric, just thinking out loud =)

I have a simple solver that looks for only unique possibilities in rows, columns and squares. When it has solved the few cells solvable by this method, it then picks a remaining candidate tries it and sees if the simple solver then leads to either a solution or a cell empty of possibilities. In the first case the puzzle is solved, in the second, one possibility has shown to be infeasible and thus eliminated. In the third case, which leads to neither a final solution nor an infeasibility, no
deduction can be reached.
The primary result of cycling through this procedure is to eliminate possiblities until picking
a correct cell entry leads to a solution. So far this procedure has solved even the hardest
puzzles without fail. It solves without difficulty puzzles with multiple solutions. If the
trial candidates are picked a random, it will generate all possilbe solutions.
I then generate a difficulty for the puzzle based on the number of illegal candidates that must
be eliminated before the simple solver can find a solution.
I know that this is like guessing, but if simple logic can eliminated a possible candidate, then one
is closer to the final solution.
Mike

I've done this in the past.
The key is that you have to figure out which rules to use from a human logic perspective. The example you provide details a number of different human logic patterns as a list on the right-risde.
You actually need to solve the puzzle using these rules instead of computer rules (which can solve it in milliseconds using simple pattern replacement). Every time you change the board, you can start over from the 'easiest' pattern (say, single open boxes in a cell or row), and move down the chain until you find one the next logical 'rule' to use.
When scoring the sodoku, each methodology is assigned some point value, which you would add up for every field you needed to fill out. While 'single empty cell' might get a 0, 'XY Chain' might get 100. You tabulate all of the methods needed (and frequency) and you wind up with a final weighting. There are plenty of places that list expected values for those weightings, but they are all fairly empirical. You're trying to model human logic, so feel free to come up with your own weightings or enhance the system (if you really only use XY chains, the puzzle is probably easier than if it requires more advanced mechanisms).
You may also find that even though you have a unique sodoku, that it is unsolvable through human logic.
And also note that this is all far more CPU intensive than solving it in a standard, patterned way. Some years ago when I wrote my code, it was taking multiple (I forget exactly, but maybe even up to 15) seconds to solve some of the generated puzzles I'd created.

Assuming difficulty is directly proportional to the time it takes a user to solve the puzzle, here is an Artificially Intelligent solution that approaches the results of the ideal algorithm over time.
Randomly generate a fixed number of starting puzzle layouts, say 100.
Initially, offer a random difficulty section that let's a user play random puzzles from the available layouts.
Keep an average random solution time for each user. I would probably make a top 10/top X leaderboard for this to generate interest in playing random puzzles.
Keep an average solution time multiplier for each puzzle solution (if the user normally solves the puzzle in 5 minutes and solves it in 20 minutes, 4 should be figured in to the puzzles average solution time multiplier)
Once a puzzle has been played enough times to get a base difficulty for the puzzle, say 5 times, add that puzzle to your list of rated puzzles and add another randomly generated puzzle to your available puzzle layouts.
Note: You should keep the first puzzle in your random puzzles list so that you can get better and better statistics on it.
Once you have enough base-rated puzzles, say 50, allow users to access the "Rated Difficulty" portion of your application. The difficulty for each puzzle will be the average time multiplier for that puzzle.
Note: When users choose to play puzzles with rated difficulty, this should NOT affect the average random solution time or average solution time multiplier, unless you want to get into calculating weighted averages (otherwise if a user plays a lot of harder puzzles, their average time and time multipliers will be skewed).
Using the method above, a solution would be rated from 0 (already solved/no time to solve) to 1 (users will probably solve this puzzle in their average time) to 2 (users will probably take twice as long to solve this puzzle than their average time) to infinity (users will take forever to find a solution to this puzzle).

Algorithm problem: Packing rods into a row

Alright, this might be a tricky problem. It is actually an analogy for another similar problem relating to my actual application, but I've simplified it into this hypothetical problem for clarity. Here goes:
I have a line of rods I need to be sorted. Because it is a line, only 1 dimension needs to be of concern.
Rods are different lengths and different weights. There is no correlation between weight and length. A small rod can be extremely heavy, while a large rod can be very light.
The rods need to be sorted by weight.
The real catch is, however, some rods can only be placed no more than certain distances from the start of the line, regardless of their weight. Anywhere before that is fine, though.
No guarantee is given that constraints will be spaced enough away from each other to prevent the possibility of constrained rods being squeezed into overlapping. In this (hopefully rare) case, either the rods need to be re-arranged somehow within their constraints to create the needed space, or an ideal compromise solution may need to be found (such as violating a constraint of the least light rod, for example).
It is possible at a future date that additional constraints may be added *in addition to the length constraint to indicate specific (and even non-compromising) boundaries within the line where rods cannot overlap into.
My current solution does not account for the latter situations, and they sound like they'll involve some complex work to resolve them.
Note that this is for a client-side web application, so making the solution apply to Javascript would be helpful!

If it is possible I'd suggest formulating this as a mixed integer program. If you can encode the constraints in this was you can use a solver to satisfy the constraints.
See this page for some more info on this type of approach:
http://en.wikipedia.org/wiki/Linear_programming
If you can interface this to Javascript somehow then it might prove to be an elegant solution.

At first, I tried to approach this as a sorting problem. But I think it is better to think of it as an optimization problem. Let me try to formalize the problem. Given:
wi: weight of rod i
li: length of rod i
mi: maximum distance of rod i from origin. If there is no constraint, you can set this value to sum(i=1,n, li)
The problem is to find a permutation ai, such that the cost function:
J=sum(i=1,n, wai*sum(j=1,i-1, laj))
is minimized and the constraints:
sum(j=1,i-1, laj) <= mi, 1 <= i<n
are satisfied.
I am not sure this is a correct formulation, though. Without any constraints, the optimal solution is not always the rods sorted by weight. For example, let l={1,4}, and w={1,3}. If a={1,2}, then J is 1*0+3*1=3, and if a={2,1} (sorted by weight), J is 3*0+1*4=4. Clearly, the unsorted solution minimizes the cost function, but I am not sure this is what you want.
Also, I don't know how to solve the problem yet. You could try a heuristic search of some kind in the short term. I am writing this reformulation so that someone else can provide a solution while I think more about the solution. If it is correct, of course.
Another thing to note is that you don't have to find the complete solution to see if there is a solution. You can ignore the rods without position constraints, and try to solve the problem with only the constrained rods. If there is a solution to this, then the problem does have a solution (an obvious suboptimal solution is to sort the unconstrained rods, and append them to the solution of the reduced problem).
After saying all this, I think the algorithm below would do the trick. I will describe it a bit visually to make it easier to understand. The idea is to place rods on a line segment from left to right (origin is the leftmost point of the line segment) as per your problem description.
Separate out the rods with position constraints on them. Then, place them such that they are at the limit of their constrained positions.
If there are no overlapping rods, goto step 4
For each overlapping pair of rods, move the one closer to origin towards the origin so that they are no longer overlapping. This step may require other rods on the line to be shifted towards the origin to open up some space. You detect this by checking if the moved rod now overlaps with the one just to the left of it. If you cannot create enough space (moving the rod closest to origin to 0 still doesn't free up enough space), then there is no solution to the problem. Here, you have the opportunity to find a solution by relaxing the constraint on the rightmost rod of the original overlapping pair: just move it away from origin until there is no overlap (you may need to push preceding rods right until all overlaps are fixed before you do this).
Now, we have some rods placed, and some free spaces around them. Start filling up the free space with the heaviest rods (including the ones with constraints which are to the right of the free space) that would fit in it. If you cannot find any rods that would fit, simply shift the next rod on the right of the free space to close the gap.
Repeat step 4 until you reach the rightmost constrained rod. The remaining line segment is all free space.
Sort all left over rods by weight, and place them in the remaining free space.
A few notes about the algorithm:
It doesn't solve the problem I stated earlier. It tries to sort the rods according to their weights only.
I think there are some lost opportunities to do better, because we slide some rods towards the origin to make them all fit (in step 3), and sometimes pick the heavy rods from these "squeezed in" places, and put them closer to origin (in step 4). This frees up some room, but we don't slide the pushed away rods back to the limits of their constrained positions. It may be possible to do this, but I will have to revise the algorithm when my brain is working better.
It is not a terribly efficient algorithm. I have a feeling that it can be done in O(n^2), but anything better would require creative data structures. You need to be able to find the heaviest rod with length less than a given L faster than O(n) to do better.

I am not very good at solving algos. But here goes my attempt:
Relate this to a Knapsack problem
Instead of the return cost or value
of a box, let them be assigned the
higher value to the ones having
lesser limit of going farther.
Some thing like you are trying to
pack everything closer to the
starting point rather than into a
Knapsack as per the Knapsack problem.
As for the future date & modification
is concerned, I believe,using constraints which
are similar would require a modification in
the return value or cost of the box
only.

I'm 99% certain this can be cast as an integer knapsack problem with an extra constraint which, I think, can be accommodated by first considering the rods with the distance-from-start condition.
Here's a link to an explanation of the knapsack problem: http://www.g12.cs.mu.oz.au/wiki/doku.php?id=simple_knapsack