Update DOM while iterating over Backbone Collection? - javascript

So I am trying to display a loading bar when rendering very large collections. I have a placeholder for the loading bar when the page initially loads, and I am trying to update it like this:
addAll:
#collection.each(((obj, index) ->
#addOne(obj, index)), this
)
addOne: (obj, index) ->
percent_complete = ((index / #collection.length) * 100)
$(".loading_bar").width("#{percent_complete}%")
# Proceed with rendering, create the view, etc
The problem here is that the DOM isnt updated until the addAll function completes. I have a feeling that this is me not understanding some basic JS fundamentals. Any help would be greatly appreciated!

Yes, you are missing something fundamental: the browser won't do any of its own work until your code returns control to the browser.
Consider some code like this:
collection = [1..1000]
addOne = (index) ->
$('#n').text(index + 1)
percent_complete = ((index + 1) / collection.length) * 100
$("#bar").width("#{percent_complete}%")
addOne(i) for e,i in collection
console.log('done')
You'll see a brief pause and then #bar and #n will be updated and done will appear in the console. Demo: http://jsfiddle.net/ambiguous/f5qKV/ (you might need to increase the 1000 to make things more obvious).
However, if you return control to the browser with setTimeout(..., 0) on each iteration:
collection = [1..1000]
addOne = (index) ->
$('#n').text(index + 1)
percent_complete = ((index + 1) / collection.length) * 100
$("#bar").width("#{percent_complete}%")
i = 0
timeOut = ->
if(i == collection.length)
console.log('done')
return
addOne(i++)
setTimeout(timeOut, 0)
setTimeout(timeOut, 0)
you'll be able to see #bar and #n changing and then you'll see done in the console when everything has finished. Demo: http://jsfiddle.net/ambiguous/UCbY8/1/
Note that the setTimeout version uses the setTimeout callback to trigger the next timeout, that ensures that everything happens in the same order as they would in a simple for or each loop.
The lesson is that you have to add some old-school pseudo cooperative multi-tasking hacks into the mix if you want to use that kind of progress indicator.
Handing control back to the browser also leaves you open to user interaction when you're not expecting it. You might want to add a general UI blocker to keep people from clicking on things while you're working if you go this route.

Related

How to only click a page element once in a for-loop that checks for a new HTML event?

I'm writing a userscript for a website where occasionally a coin drop will appear on-screen and only a limited number of people on the site can claim it. My script detects when a new coin drop appears based on the length of the page element "coindrop-status", and when a new drop is detected it auto-clicks the prompt to open the initial drop splash screen, then auto-clicks the actual grab button within that splash screen.
The problem is that because the first auto-click is within a for-loop, it continuously spam-clicks to open the splash screen until the drop has been fully claimed and the loop breaks, preventing stage 2 of the auto-click function from clicking the actual button to grab the drop within the splash screen.
I've tried to solve this problem many times now but because coin drops are so infrequent, it's a massive pain to debug - how can I change my script so that when a drop is detected, the splash screen is only clicked once (so that it stays open) before clicking the grab button within it repeatedly?
var newDrop = false;
function dropCheck() {
clearInterval(scanFreq);
var coinLength = document.getElementsByClassName("coindrop-status").length - 1;
for(var i = coinLength; i >= 0; i--) {
if(document.getElementsByClassName("coindrop-status")[i].innerText == "Grab") {
newDrop = true;
document.getElementsByClassName("coindrop-status")[i].click();
setTimeout(function() {document.elementFromPoint(1250, 840).click()},1000);
setTimeout(function() {document.elementFromPoint(1250, 840).click()},1000);
}
}
if(newDrop) {
newDrop = false;
setTimeout(dropCheck,800);
} else {
setTimeout(dropCheck,100);
}
}
var scanFreq = setInterval(dropCheck,800);
Admittedly, clicking the grab button multiple times is probably overkill, but I figure at least it guarantees that the coin drop actually gets grabbed.
Forgive any bad coding practice I may have integrated into this script; I'm still learning to program. I'm sure there are much, much more elegant ways to accomplish the goal of this userscript, so if you have any other suggestions please feel free to give some constructive criticism as well.
First, you don't need to loop like this. Like at all. There is a class called MutationObserver which will call a callback only when elements in the DOM changed. My approach when waiting for a specific element to be added is basically this:
/**
* Waits for a node to start to appear in DOM. Requires a CSS query to fetch it.
* #param {string} query CSS query
* #returns {Promise<HTMLElement>}
*/
function awaitElement(query) {
return new Promise((resolve, reject)=>{
const targetNode = document.body;
const testNow = targetNode.querySelector(query);
if(testNow != null) {
resolve(testNow);
return;
}
const observer = new MutationObserver((mutationList, observer) => {
const result = targetNode.querySelector(query);
if(result != null) {
observer.disconnect();
resolve(result)
}
});
observer.observe(targetNode, { attributes: false, childList: true, subtree: true });
});
}
Used as:
(async () => {
while(true) {
let myElm = await awaitElement(".coindrop-status");
// handle the drop here. Note that you must remove the element somehow, otherwise you will get an endless loop
}
})();
You will probably need to adjust my function a little, so that you can make it ignore coin drops you already handled. One way to handle this without any modification is add a custom class name to the handled coin divs, and then use the not selector when searching for more.
I also do not think it's really wise to use element from point. Doesn't the popup for claiming the coin have some selector as well?

How to make the Apify Crawler to scroll full page when web page have infinite scrolling?

I'm facing a problem that I unable to get all the product data as the website using a lazy load on product catalog page. meaning it needs to scroll until the whole page loaded.
I getting only first-page products data.
First, you should keep in mind that there are infinite ways that infinite scroll can be implemented. Sometimes you have to click buttons on the way or do any sort of transitions. I will cover only the most simple use-case here which is scrolling down with some interval and finishing when no new products are loaded.
If you build your own actor using Apify SDK, you can use infiniteScroll helper utility function. If it doesn't cover your use-case, ideally please give us feedback on Github.
If you are using generic Scrapers (Web Scraper or Puppeteer Scraper), the infinite scroll functionality is not currently built-in (but maybe if you read this in the future). On the other hand, it is not that complicated to implement it yourself, let me show you a simple solution for Web Scraper's pageFunction.
async function pageFunction(context) {
// few utilities
const { request, log, jQuery } = context;
const $ = jQuery;
// Here we define the infinite scroll function, it has to be defined inside pageFunction
const infiniteScroll = async (maxTime) => {
const startedAt = Date.now();
let itemCount = $('.my-class').length; // Update the selector
while (true) {
log.info(`INFINITE SCROLL --- ${itemCount} items loaded --- ${request.url}`)
// timeout to prevent infinite loop
if (Date.now() - startedAt > maxTime) {
return;
}
scrollBy(0, 9999);
await context.waitFor(5000); // This can be any number that works for your website
const currentItemCount = $('.my-class').length; // Update the selector
// We check if the number of items changed after the scroll, if not we finish
if (itemCount === currentItemCount) {
return;
}
itemCount = currentItemCount;
}
}
// Generally, you want to do the scrolling only on the category type page
if (request.userData.label === 'CATEGORY') {
await infiniteScroll(60000); // Let's try 60 seconds max
// ... Add your logic for categories
} else {
// Any logic for other types of pages
}
}
Of course, this is a really trivial example. Sometimes it can get much more complicated. I even once used Puppeteer to navigate my mouse directly and drag some scroll bar that was accessible programmatically.

JavaScript callback - multithreading

I want to develope a very simple strategic game.
You can find an example on http://wiesenberg.info/hope/ !
If you click on a button, that project will be created (progressbar). After it is finished, the project will show up on the playground. As you can see it works if you click one by one. But once you click a button twice (start 2 project of the same kind), the second project will not be finished because the first has not finished and increased the variable. Is there an easy way of doing multithreading or just solving my bug?
I also want to add a function that you can stop the project by clicking on the progressbar. I tried it out, it dissapears in the list, but the icon still shows up, because the function didnt get that another function was executed. so I also need a break function for the progressbar. I hope you undertood my problem! Thanks
Heres the important code:
function addSygehus() {
sygehusBarID++;
sygehusID++;
$("#addProject").append("<div id=sygehus" + sygehusID +
" class='progress progress-striped active'><div class='progress-bar progress-bar-success' id=sygehusbar"
+ sygehusBarID + " aria-valuetransitiongoal='100'>Sygehus</div></div>");
function countdown(callback) {
var bar = document.getElementById("sygehusbar"+sygehusBarID),
time = 0, max = firsttime,
int = setInterval(function() {
bar.style.width = Math.floor(100 * time++ / max) + '%';
if (time - 1 == max) {
clearInterval(int);
// 600ms - width animation time
callback && setTimeout(callback, 600);
}
}, 1000);
}
countdown(function() {
$("#sygehus" + sygehusID).remove();
$("#sygehusPic" + sygehusID).show(1000);
});
}
I would look at JQuery Deferred (http://api.jquery.com/jQuery.Deferred/) and Promises Documentation in general to manage what your trying to achieve. So you can apply some control to the order in which the asynchronous calls occur.
You've also got Q.js (https://github.com/kriskowal/q) is a nice library for managing these kinds of scenarios.
Otherwise you have web workers as already mentioned for emulating multithreaded behaviour but I don't think that's what your looking for.

In JavaScript, How to make make changes to document elements while complex functions are performed?

I'm processing a large amount of data with JavaScript. I put a circular gif and a progress div to show how much progress has been done in creating the model. However, when it gets to the bulky part of code processing, the loading gif stops spinning and the percentage updating stops working. (until the very end for a split second)
This is the block of code that freezes the gif.
// convert binary normals to ascii
for(i=0;i<norms.length;i++){ //<-- the array length is about 200,000, and could be larger
normals.push(toAscii(norms[i], mins[5], maxes[5])); //nx
normals.push(toAscii(norms[i+1], mins[6], maxes[6]));//ny
normals.push(toAscii(norms[i+2], mins[7], maxes[7]));//nz
i = i+2; //skip next 2 as they're already converted
percentComplete = (normals.length/norms.length)*100;
percentComplete = Math.round(percentComplete);
document.getElementById('loadingScrn').innerHTML = "Processing "
+percentComplete + "%" + " Complete"; //<-- The loading gif is right below this element on the webpage and neither update while this function is running
}
How can I get the browser update the display while JavaScript functions process large data? Is there a way to thread activities so that both updating the Document and JavaScript processing occur simultaneously?
JavaScript runs on the same thread as the browser GUI in most cases (or the tab's GUI, if each tab is given its own process). You will have to break the work into small pieces and schedule the next piece from the currently-executing one using setTimeout().
For example, this might work in your case:
var i = 0;
function doWork() {
do {
// One iteration here...
i++;
} while (i % 100 != 0 && i < norms.length);
// ^^^
// Break work into pieces of 100 elements each; adjust this
// number as needed.
if (i < norms.length) {
setTimeout(doWork, 1);
}
}
setTimeout(doWork, 1);
See this example jsfiddle.
You are correctly observing that JavaScript code runs in the same thread as the document's interface, blocking it when you perform large operations.
Web Workers are a JavaScript feature that is designed to help solve this problem. It allows you to spawn new threads that run along side the document, and communicate results asynchronously as they become available. Unfortunately this is not yet supported in Internet Explorer, but it is planned for IE10, and other browsers already support it.
As suggested by cdhowie and Jonathan M, another solution (inferior, but supported everywhere) is to use setTimeout to pause your code occasionally and let the browser respond to events. You would need to make your code somewhat more complicated to make this work. To give you an idea, to pause every 1000 items you would do something like this:
var workSliceSize = 1000;
var doWorkFromIndex = function(start) {
for (var i = start; i < norms.length; i++) {
if (i - start > workSliceSize) {
setTimeout(0, doWorkFromIndex, i + 1);
break;
}
normals.push... // your code here
}
}
doWorkFromIndex(0);
Try setTimeout(). It processes code asynchronously. I've used it to free up the screen processing by doing:
setTimeout(function() {
// the stuff I want to accomplish while keeping the gif going
},
0
);
Here I've set the timeout time period at zero milliseconds, but it can be whatever you want.
Try
var i = 0;
function processNormals() {
normals.push(toAscii(norms[i], mins[5], maxes[5])); //nx
normals.push(toAscii(norms[i+1], mins[6], maxes[6]));//ny
normals.push(toAscii(norms[i+2], mins[7], maxes[7]));//nz
i = i+3; //skip next 2 as they're already converted
percentComplete = (normals.length/norms.length)*100;
percentComplete = Math.round(percentComplete);
document.getElementById('loadingScrn').innerHTML = "Processing " +percentComplete + "%" + " Complete";
if(i < norms.length) setTimeout(processNormals, 20);
}

jQuery onClick execution

I have this bit of javascript written with jQuery 1.2.5. It's contained inside the main function() of a plugin that I wrote. The plugin is a horizontal gallery scroller very similar to jCarousel. It does alot of auto calculating of widths and determines how many to scroll based on that and the size of the images, which is what all the calculations are that are going on.
What my question is, how do I prevent this from firing off before a previous execution is finished. For instance, if I get a little click happy and just frantically mash down on .digi_next. Things don't go so well in the UI when that happens and I'd like to fix it :) I thought the answer might lie in queue, but all my attempts at using it haven't turned out anything worthwhile.
var self = this;
$(".digi_next", this.container).click(function(){
var curLeft = $(".digi_container", self.container).css("left").split("px")[0];
var newLeft = (curLeft*1) - (self.containerPad + self.containerWidth) * self.show_photos;
if (newLeft < ((self.digi_gal_width - (self.containerPad + self.containerWidth) * self.show_photos)) * -1) {
newLeft = ((self.digi_gal_width - (self.containerPad + self.containerWidth) * self.show_photos)) * -1;
}
$(".digi_container", self.container).animate({
left: newLeft + "px"
}, self.rotateSpeed);
});
Just use a global busy flag. When you enter your click handler, check it, and only proceed if it's false. Immediately set it to true, and then set it back to false when the animation ends. JavaScript is single-threaded, so there is no race condition to worry about.
var busy = false;
$("...").onclick(function() {
if (busy) return false;
busy = true;
$("...").animate(..., ..., ..., function() {
busy= false;
});
return false;
});
Take a look at jQuery UI. Specifically the effects-part of the plug in. I use the slide-effect on my personal website (click on the arrows at the sides of the boxes).
I prevent users triggering the effect more than once - before the effect has ended - with the one event-handler and a callback function.
Here's the source-code
As an alternative to the afformentioned global flag, you could assign the value to the DOM element, that way enabling multiple elements on the page to have the same behaviour:
$("...").onclick(function(el) {
var self = el;
if (self.busy) return false;
self.busy = true;
$("...").animate(..., ..., ..., function() {
self.busy= false;
});
return false;
});
Since JavaScript functions calls are asyncronus, you can pass as a in parameter a callback function that's called when the previous call ends (same for errors).
You can pass the function you wrote in this post as the callback for the function that fire before.
Hope this helps.
Regards

Categories