We Keep Coding

Reduce the complexity of matching the elements of two arrays

I wrote a code that extracts column headers (the first row in a sheet) from a google sheet and compares them with an array of objects. Each object in the objects array has 3 properties: "question", "answer", and "category". The code compares the header of each column, with the "question" property pf each object in the array.
If they're similar it should add the index of the column as a key to some dictionary and set its value to be an array that holds the answer and the category of that question. No need to much explain why I'm doing this, but briefly I built this logic to be able to grade applicants answers on some questions (hence linking the index of a question to its right answer and to its category). Here is the code:
for (i = 0; i<columnHeaders[0].length; i++){
for (j=0; j<questionsObjects.length; j++){
//Get the current question and convert it to lower case
question = questionsObjects[j].question.toString().toLowerCase();
//Get column header, remove any spaces and new lines from it, and convert it to lower case
columnHeader = columnHeaders[0][i].toString().toLowerCase();
if (isStringSimilar(columnHeader, question)){
//Link the column index to its corresponding question object
var catAndAnswer = [];
catAndAnswer.push (questionsObjects[j].category.toLowerCase());
catAndAnswer.push (questionsObjects[j].rightAnswer.toLowerCase());
columnsQuestionsDictionary[i] = catAndAnswer;
} else {
SpreadsheetApp.getActive().getSheetByName("log").appendRow(["no", columnHeader, question]);
}
}
}
The code runs well, my only problem is complexity, it's very high. In some cases this method takes almost 6 minutes to execute (for this case I had around 40 columns and 7 question objects)! To decouple the nested loops, I thought of concatenating the questions values (of all objects in the questions object array) into 1 single string where I precede each question with its index in the objects array.
For example:
var str = "";
for (j=0; j<questionsObjects.length; j++){
str = str + j + questionsObjects[j].question.toString.toLowerCase();
}
Then, I can have another separate loop through the columns headers, extract each header into a string, then use regex exec method to match that header in the long questions string (str), and if it's found I would get its index in str, then subtract 1 from it to know its index in the objects array. However, it turned out that the complexity of matching a regular expression is O(N) where N is the length of the string we search in (str in this example), given that this will be inside the columns loop, I see that we still get a high complexity that can go to O(N^2).
How can I optimize those nested loops so the code runs in the most efficient way possible?

OK, so I used the way suggested by Nina Schholz in the comments and I moved columnHeader = columnHeaders[0][i].toString().toLowerCase(); to be in the outer loop instead of being in the inner one since it's only needed in the outer one.
The time needed to run the code was reduced from ~295 seconds to ~208 seconds, which is good.
I also tried switching the loops order where I made the outer loop to be the inner one and the inner one to be the outer one and updated the usage of i and j accordingly. I did that because it's always recommended to have the outer loop with less iterations and the inner one with more iterations (according to this resource), and in my case, the loop that iterates over questions object array is always expected to have number of iterations <= the other loop.
This is because if we want to calculate the complexity of 2 nested loops, it'll be (ixj) + i, where i and j represents the number of iterations of the outer and the inner loops, respectively. Switching the loops order won't impact the multiplication part (ixj) but it'll impact the addition part. So, it's always better to have the outer number of iterations smaller than the inner ones.
After doing this the final time of the run became ~202 seconds.
Of course since the loops are switched now, I moved this line to the inner loop: columnHeader = columnHeaders[0][i].toString().toLowerCase();, but at the same time I moved this question = questionsObjects[j].question.toString().toLowerCase(); to be under the outer loop because it's only needed there.

undertanding javascript pagination math problem

I am trying to understand how to approach math problems such as the following excerpt, which was demonstrated in a pagination section of a tutorial I was following.
const renderResults = (arrayOfItems, pageNum = 1, resultsPerPage = 10) => {
const start = (pageNum - 1) * resultsPerPage;
const end = pageNum * resultsPerPage;
arrayOfItems.splice(start, end).forEach(renderToScreenFunction);
};
In the tutorial this solution was just typed out and not explained, which got me thinking, had I not seen the solution, I would not have been able to think of it in such a way.
I understood the goal of the problem, and how splice works to break the array into parts. But it was not obvious to me how to obtain the start and end values for using the splice method on an array of of indefinite length. How should have I gone about thinking to solve this problem?
Please understand, I am learning programming in my spare time and what might seem simple to most, I have always been afraid and struggle with math and I am posting this question in hopes to get better.
I would really appreciate if anyone could explain how does one go about solving such problems in theory. And what area of mathematics/programming should I study to get better at such problems. Any pointers would be a huge help. Many thanks.

OK, so what you're starting with is
a list of things to display that's, well, it's as long as it is.
a page number, such that the first page is page 1
a page size (number of items per page)
So to know which elements in the list to show, you need to think about what the page number and page size say about how many elements you have to skip. If you're on page 1, you don't need to skip any elements. What if you're on page 5?
Well, the first page skips nothing. The second page will have to skip the number of elements per page. The third page will have to skip twice the number of elements per page, and so on. We can generalize that and see that for page p, you need to skip p - 1 times the number of elements per page. Thus for page 5 you need to skip 4 times the number of elements per page.
To show that page after skipping over the previous pages is easy: just show the next elements-per-page elements.
Note that there are two details that the code you posted does not appear to address. These details are:
What if the actual length of the list is not evenly divisible by the page size?
What if a page far beyond the actual length of the list is requested?
For the first detail, you just need to test for that situation after you've figured out how far to skip forward.

Your function has an error, in the Splice method
arrayOfItems.splice(start, end).forEach(renderToScreenFunction);
The second argument must be the length to extract, not the final
index. You don't need to calculate the end index, but use the
resultsPerPage instead.
I've rewrite the code without errors, removing the function wrapper for better understanding, and adding some comments...
// set the initial variables
const arrayOfItems =['a','b','c','d','e','f','g','h','i','j','k','l','m'];
const pageNum = 2;
const resultsPerPage = 5;
// calculate start index
const start = (pageNum - 1) * resultsPerPage; // (2-1)*5=5
// generate a new array with elements from arrayOfItems from index 5 to 10
const itemsToShow = arrayOfItems.splice(start, resultsPerPage) ;
// done! output the results iterating the resulting array
itemsToShow.forEach( x=> console.log(x) )
Code explanation :
Sets the initial parameters
Calculate the start index of the array, corresponding to the page you try to get. ( (pageNum - 1) * resultsPerPage )
Generates a new array, extracting resultsPerPage items from arrayOfItems , starting in the start index (empty array is returned if the page does not exist)
Iterates the generated array (itemsToShow) to output the results.
The best way to understand code, is sometimes try to run it and observe the behavior and results.

Swapping elements in Angular JS

I was curious what the appropriate way is to animate and swap elements using angular. I have a two-dimensional array that has a nested ng-repeat to create the table. My end goal is to have two elements in the table swap places using an animation, so basically just hover over to eachother's positions.
I have this function that swaps the position of two jquery elements, but it doesn't update the array and ultimately the angular array will no longer be synced with what is displayed. I also find that the animation hangs a little when moving elements between rows whereas columns are smooth. ie. [0][0] -> [0][1] is smooth and [0][0] -> [1][0] hangs
jQuery.fn.swapWith = function (to, callback) {
animating = true;
thisPos = this.position();
toPos = to.position();
$.when(this.animate({
top: toPos.top,
left: toPos.left
}, 300),
to.animate({
top: thisPos.top,
left: thisPos.left
}, 300)).done(function () {
animating = false;
if (callback) {
callback();
}
});
};
I've also found this codepen which is pretty nice, but also not quite what I'm looking for. http://codepen.io/daleyjem/pen/xbZYpY Is there a standard way of handling something like this in angular?
To put in more simple terms: If I want array[0][0] to swap positions with array[0][1] and have the $scope object update appropriately, how should I do it? Same with array[0][0] to array[1][0]?
EDIT: I should mention, I have noticed that simply setting the elements equal to eachother will update the table appropriately. The problem being the lack of animation. example:
a = [0][1]
b = [0][0]
[0][1] = b
[0][0] = a

Is it possible to programmatically define a function from a string or some other technique without using eval()?

I am working on coding for a situation where I need to construct a function of nested callbacks of an unknown length. It is to create a sequenced animation queue to move an element across an unknown # of positions.
For example, output would look something like this with X 'complete' callbacks nested inside:
$('#element').animate(css, { complete: function () {
$('#element').animate(css, { complete: function () {
// more nested calls inside
}
});
Right now I am generating these functions as a string, and then once completed, feeding it to new Function():
myFunc = new Function(generatedFuncString);
The content is trusted but this still uses eval() which has negative performance implications. I was just wondering if there is another/better way?
edit: The reason I am doing it this way is because I have a very complicated set of animations to perform and am working outside of the jQuery animation queue. If anyone has a better suggestion for how to accomplish a situation like this that would be helpful...
Imagine a baseball diamond with a runner(A) on 1st and a runner(B) on 3rd. In one animation bundle, I want to animate runner A to 3rd (stopping at 2nd in the middle, 2 advances), and runner B to HOME (1 advance).
I have to fire-off the initial advance with 'queue: false' so that runner A and B move to their first base at the same time (runner A to 2nd, runner B to home).
When Runner A is done moving to 2nd, I want to then move him to 3rd (hence constructing a animate() call with nested callbacks pro grammatically to ensure this sequencing is preserved).
The reason I am constructing the function via string is because I know what the inner-most callback is going to be first, and then recursively constructed 1 or more outer-callbacks from there. I couldn't figure out a way to do this by working with functions as objects and keeping all of the references in tact.
Keep in mind this is a simple example. Imagine a situation where the bases are loaded, and I need to animate a grand slam (all 4 runners circle all bases, runner originating at home needs to make 3 stops before running back to home). Etc etc.

Answering the question you ask in your title: You can create functions from strings via eval, new Function, and by inserting a script element with the text you want. But it all comes to the same thing: Firing up the JavaScript parser and creating the function.
But rather than nesting, I think you want chaining. Build a list of the animations in an array, and use the animate callback to call the next animation in the array. Something like:
var animations = [
/* css for first animation */,
/* css for second animation */,
/* etc. */
];
var index = 0;
function runAnimation() {
if (index < animations.length) {
$("#element").animate(animations[index++], runAnimation);
}
}
You'd build up the array dynamically, of course.
gdoron points out in the comments that if all of your animations are on the same element, it can be even simpler:
var animations = [
/* css for first animation */,
/* css for second animation */,
/* etc. */
];
var index = 0;
for (index = 0; index < animations.length; ++index) {
$("#element").animate(animations[index]);
}
...because when you call animate multiple times on the same element, by default the animations queue up (the queue option defaults to true; sadly the docs don't seem to say that). My code example above doesn't rely on the queue, and so in theory each entry in the array could be an object with a property for the selector for the elements to animate and the css to apply. But if it's all one element, you can just use a straight loop.

KnockoutJS foreach blocks main thread

When I have a large dataset in my viewModel and I use foreach to loop over an Array of Objects to render each Object as a row within a table, KnockoutJS will block the main thread until it can render, which sometimes takes minutes (!).
Here is a jsFiddle example using a dataset containing 2000 Objects, containing a url and a code. Real data will have longer URLs in some cases and 4 other columns (only 2 in this example.) I also added some simple styles because adding styles also seems to slow things down a bit during the process.
Warning: your browser might break
http://jsfiddle.net/DESC3/7/

I suggest that if you have such large datasets you try an alternative solution. For example slickGrid renders large datasets in a much more efficient way, by only generating HTML elements for the data that is actually visible. We've used this for large datasets, and it performs well.

How about something like this. Say, you've got viewModel.items = ko.observableArray() that you'd like to render.
Have a separate non-observable array of all your data: var itemsToRender = functionThatReturnsLargeArray().
Put some portion of your data from itemsToRender into your observable array. Say, 50 elements only.
Keep adding elements into observable array in portions inside a setTimeout callback.
NOTE1: You can add some time-tracking into setTimeout callback and increase/reduce the number of items that you add at each iteration. Your goal is to keep each callback time below 50-100 milliseconds so your application still feels responsive.
var batchSize = 50; // default number of items rendered per iteration
var batchOffset = 0;
function render(items, itemsToRender, done) {
setTimeout(function () {
var startTime = new Date().getTime();
items.pushAll(itemsToRender.slice(batchOffset, batchSize));
batchOffset += batchSize;
// at this point Knockout rendered next batchSize items from itemsToRender
var endTime = new Date().getTime();
// update batchSize for next iteration
batchSize = batchSize * 50 / (endTime - startTime); // 50 milliseconds
batchSize = Math.min(itemsToRender.length, batchOffset + batchSize);
if (batchSize > 0) render() else done(); // callback if you need one
}, 0);
}
/* I haven't actually tested the code */
Another batch size updating strategy could be based on target FPS. Say you'd like to achieve 60 fps update rate and thus 60 calls to setTimeout per 1000 milliseconds. That would take somewhat longer to process the whole collection. You can also use requestAnimationFrame instead of setTimeout and see how that would work out.
EDIT: Build-in throttling was added into Knockout JS 1.3 (currently it's in beta but seems pretty stable).
NOTE2: If some other data on the view depends on viewModel.items you can still map it down to original array itemsToRender. Say, for example, that you'd like to show the number of items in collection. If you use viewModel.items().length you'll end up with changing size value in the UI while more items get renderred. To avoid that you can first define your size binding as a dependentObservable based on itemsToRender, not viewModel.items. After you've done rendering all items you can re-map it onto viewModel.items if you see fit.