We Keep Coding

javascript functions wait for data availability or variable not capable of handling huge data

forgive the trivial question but I am more used to C++ and Python code than javascript.
I have the following code from the THREE JS PLY loader:
var geometry;
var scope = this;
if (data instanceof ArrayBuffer) {
geometry = isASCII(data) ? parseASCII(bin2str(data)) : parseBinary(data);
} else {
geometry = parseASCII(data);
}
parse: function (data) {
function isASCII(data) {
var header = parseHeader(bin2str(data));
return header.format === 'ascii';
}
function bin2str(buf) {
var array_buffer = new Uint8Array(buf);
var str = '';
for (var i = 0; i < buf.byteLength; i++) {
str += String.fromCharCode(array_buffer[i]); // implicitly assumes little-endian
}
return str;
}
It works fine if I load a small ply file but browser crashes on very large one. I believe there are two "possible" issues:
1) on a large file the string str returned by the function bin2str(buf) might not be able to handle the parsing process
2) in the function isASCII(data) the line
parseHeader(bin2str(data));
crashes the browser as the bin2str(data) cannot return a proper value in time as the process is very memory consuming
I am using the conditional as i am not totally sure of what the problem is. Any suggestion and/or possible solution?
Thank you,
Dino

In the end the solution I have adopted was to decimate my file using the free software MeshLab.
Hope this helps.
Dino

How do I trigger events using Meteor server side events

I am using Meteor, which uses Mongodb as its database. I have code that inserts several documents into a collection when users fill out a form. When these documents are inserted, I would like to fire some JavaScript code within the server side directories that sorts through the collection in question for documents with matching fields as the documents just inserted.
My problem is that I do not know how to fire code on the server when the new documents arrive. Would it make sense to Meteor.call a Meteor.method at the end of the code involved with inserting, with the Meteor.method called preforming the sorting code I need?
Edit:
As you can see, in the below code I'm not calling any Meteor methods as none exist yet. The vast majority of this code is simply lead up for the insert({}) at the end of the page, so I think it can be safely ignored. The only server side code I have is to declare the possibleGames mongo collection.
I am not sure what you mean by call a plain JavaScript function, my problem is getting any code firing at all.
possibleGames = new Mongo.Collection("possibleGames");
Template.meet_form.events({
"submit .meet_form": function(event, template){
event.preventDefault();
var user = Meteor.userId();
var where = event.target.where.value;
var checkedGames = [];
function gameCheck (game) {
if (game.checked === true){
checkedGames.push(game.value);
};
};
var checkedDays = [];
function dayCheck (day) {
if (day.checked === true){
checkedDays.push(day.value);
};
};
console.log(event.target.where.value)
gameCheck(event.target.dnd);
gameCheck(event.target.savageWorlds);
gameCheck(event.target.shadowRun);
console.log(checkedGames);
dayCheck(event.target.monday);
dayCheck(event.target.tuesday);
dayCheck(event.target.wednesday);
dayCheck(event.target.thursday);
dayCheck(event.target.friday);
dayCheck(event.target.saturday);
dayCheck(event.target.sunday);
console.log(checkedDays);
var whereWhat = [];
for (i = 0; i < checkedGames.length; i++) {
var prepareWhereWhat = where.concat(checkedGames[i]);
whereWhat.push(prepareWhereWhat);
};
console.log(whereWhat);
var whereWhatWhen = [];
for (a = 0; a < whereWhat.length; a++) {
var prepareWWW1 = whereWhat[a];
for (b = 0; b < checkedDays.length; b++) {
var prepareWWW2 = prepareWWW1.concat(checkedDays[b]);
whereWhatWhen.push(prepareWWW2);
};
};
console.log(whereWhatWhen);
for (i = 0; i < whereWhatWhen.length; i++) {
possibleGames.insert({
game: whereWhatWhen[i],
user: user,
created_on: new Date().getTime()
})
}
}
});

You don't need to do a meteor.call on the server because you're already on the server.
Just call a plain javascript function.
If what you want to call from your first Meteor.method is already in another Meteor.method, then refactor that function to extract out the common bit.
Some code would also help if this is still confusing.

Speeding up Ajax success DOM manipulation

I am using an AJAX call on my page which return table rows and on success adds them to a table in the page. Below is the code currently used:
function GetDPRecords(Perso) {
//alert(Perso);
$Records = $('#DPRecords');
//alert($PersoFileName.val()+" | "+$ProcFromDate.val()+" | "+$ProcToDate.val());
$.ajax({
type: "GET",
url: "SelectDPRecords.jsp",
data: $('form#Search_Form').serialize() + "&Personalized=" + Perso,
beforeSend: function () {
$Records.find("tr:gt(0)").remove();
$("<tr><td colspan='4'><h3 style='margin: 4px 10px'> Loading... </h3></td></tr>").hide().appendTo($Records).show(400);
},
success: function (data) {
$Records.find("tr:gt(0)").remove();
$(data).hide().appendTo($Records).show(400);
}
});
}
The issue is that I at times expect a large number of rows to be returned (1,000-5,000). I did a test run with 4,000 rows data returned and it caused the browser to be unresponsive for about 20 seconds.
Any way to optimize the code and reduce the loading time?

One possible solution is to use a paging system: instead of returning 1,000-5,000 rows, you break the results into pages of, say, 50 results each, and only return one page at a time. You would then give the user buttons to load other pages at the top/bottom of the table.
For an example of what I am talking about, see http://luis-almeida.github.io/jPages/defaults.html. It uses pictures instead of rows, but it is the same basic concept.

Just to add #aj_r's suggestion If you do not want to hit the server again to retrieve the data for next range then you can store the result to a javascript variable (JSON Object Array) and then use that locally with pagination.

I've been facing the same problem recently using huge datagrids in jqGrid and I've been able to find a tricky solution which turned out to be working very well.
Thing is you cannot render all this data at once - this is just too much, especially considering how slow DOM manipulations are. So you need some sort of queue. You can split this data into chunks and render them sequentiali using setTimeout() or setInterval() but those doesn't have well reputation in terms of performance as well.
I ended up using reuestAnimationFrame and splitting my huge data into pieces and rendering them whenever an animation frame was available. So you need to start with polyfill to make sure what you are going to do will actually work, I'm using a great one by Paul irish:
Heres an awsome jsFiddle: http://jsfiddle.net/cjw5eota/1/
And here's the JS:
var dataRows = 5000; // enter ammount of returned rows
var chunkSize = 200; // define single chunk size, optimize for best performance bu trying different values, if your DOM manipulation is heavy use less, if its lightweight you can use more, use 1 to see in slowmo how it works
// We are simulating big returned object here
var data = {}
for (var i = 0; i < dataRows; i++) {
data[i] = {
'id': i,
'name': 'I am data for row ' + i
};
}
// shim layer with setTimeout fallback
window.requestAnimFrame = (function () {
return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || function (callback) {
window.setTimeout(callback, 1000 / 60);
};
})();
function renderRecords(data) {
var dataLength = Object.keys(data).length;
var i = 0;
function renderInQueue() {
console.time('Rendering in queue');
for (t = 0; t < chunkSize; t++) {
if (i < dataLength) {
var row = '<tr><td>' + data[i].id +
'</td><td>' + data[i].name + '</td></tr>'
$('table').append(row);
}
i++;
}
if (i < dataLength) {
requestAnimationFrame(renderInQueue);
} else {
console.log('Done rendering');
console.timeEnd('Rendering in queue');
}
}
renderInQueue();
}
// run the script of rendering
renderRecords(data);
I have included a simple performance benchmark for you to see in console how much an entire rendering process takes. Play with chunkSize to see how it changes and try to find a value that will best suit your needs, that is give you decent rendering time without putting heavy load on the browser.
The more you try to render at once the fastest the rendering will be, but more resources will be required to handle each iteration of rendering.

Javascript run faster if console opened

i'm developing a phonegap app using a lot of javascript. Now i'm debugging it using Safari Developer Tool, in particular i'm focused on some button that on the device seems to be a bit luggy.
So I've added some console.timeEnd() to better understand where the code slow down, but the "problem" is that when i open the console the code start running faster without lag, if i close it again, the lag is back.
Maybe my question is silly but i can't figure it out
Thanks
EDIT: Added the code
function scriviNumeroTastiera(tasto){
console.time('Funzione ScriviNumeroTastiera');
contenutoInput = document.getElementById('artInserito').value;
if ($('#cursoreImg').css('display') == 'none'){
//$('#cursoreImg').show();
}
else if (tasto == 'cancella'){
//alert(contenutoInput.length);
if (contenutoInput.length == 0) {
}
else {
indicePerTaglioStringa = (contenutoInput.length)-1;
contenutoInput = contenutoInput.substr(0, indicePerTaglioStringa);
$('#artInserito').val(contenutoInput);
//alert('tastoCanc');
margineAttualeImg = $('#cursoreImg').css('margin-left');
indicePerTaglioStringa = margineAttualeImg.indexOf('p');
margineAttualeImg = margineAttualeImg.substr(0, indicePerTaglioStringa);
margineAggiornato = parseInt(margineAttualeImg)-20;
$('#cursoreImg').css('margin-left', margineAggiornato+'px');
}
}
else {
//contenutoInput = document.getElementById('artInserito').value;
contenutoAggiornato = contenutoInput+tasto;
margineAttualeImg = $('#cursoreImg').css('margin-left');
indicePerTaglioStringa = margineAttualeImg.indexOf('p');
margineAttualeImg = margineAttualeImg.substr(0, indicePerTaglioStringa);
margineAggiornato = parseInt(margineAttualeImg)+20;
$('#cursoreImg').css('margin-left', margineAggiornato+'px');
$('#artInserito').val(contenutoAggiornato);
}
console.timeEnd('Funzione ScriviNumeroTastiera');
}
The code is a bit crappy, but it's just a beginning ;)

This could happen because PhoneGap/Cordova creates its own console object (in cordova.js), and it gets overwritten when you open the Safari console (safari's might be faster than phonegap's, that could be why you notice it faster).
So, one way to measure the time properly, without opening the console, would be to go to the good old alert, so you'd first add this code anywhere in your app:
var TIMER = {
start: function(name, reset){
if(!name) { return; }
var time = new Date().getTime();
if(!TIMER.stimeCounters) { TIMER.stimeCounters = {} };
var key = "KEY" + name.toString();
if(!reset && TIMER.stimeCounters[key]) { return; }
TIMER.stimeCounters[key] = time;
},
end: function(name){
var time = new Date().getTime();
if(!TIMER.stimeCounters) { return; }
var key = "KEY" + name.toString();
var timeCounter = TIMER.stimeCounters[key];
if(timeCounter) {
var diff = time - timeCounter;
var label = name + ": " + diff + "ms";
console.info(label);
delete TIMER.stimeCounters[key];
}
return diff;
}
};
(This just mimics the console.time and console.timeEnd methods, but it returns the value so we can alert it).
Then, instead of calling:
console.time('Funzione ScriviNumeroTastiera');
you'd call:
TIMER.start('Funzione ScriviNumeroTastiera');
and instead of calling:
console.timeEnd('Funzione ScriviNumeroTastiera');
you'd call:
var timeScriviNumeroTastiera = TIMER.end('Funzione ScriviNumeroTastiera');
alert('Ellapsed time: ' + timeScriviNumeroTastiera);
This would give you the proper ellapsed time without opening the console, so it computes the real time in the phonegap app.
Hope this helps.
Cheers

This really isn't something you would normally expect - opening the console should not speed up anything. If anything, it will make things slower because of additional debugging hooks and status display. However, I've had a case like that myself. The reason turned out to be very simple: opening the console makes the displayed portion of the website smaller and the code efficiency was largely dependent on the viewport size. So if I am right, making the browser window smaller should have the same effect as opening the console.

conditionally load javascript (external and internal) and keep execution order

I'm looking for a way to conditionally load and keep the execution order of some javascript files (external and internal) without any library dependency. Basically, what I want to do is load them up only if the browser supports localStorage.
Here's basically my shell:
if (window.localStorage) {
//load up JS only if it's needed
var body = document.getElementsByTagName('body')[0],
js1 = document.createElement('script'),
js2 = document.createElement('script'),
js3 = document.createElement('script'),
js4 = document.createElement('script'),
js5 = document.createElement('script');
js1.src = 'http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js';
js2.src = 'http://www.google.com/jsapi';
js3.src = 'my_file1.js';
js4.src = 'my_file2.js';
js5.src = 'my_file3.js';
body.appendChild(js1);
body.appendChild(js2);
body.appendChild(js3);
body.appendChild(js4);
body.appendChild(js5);
} else {
//no localStorage support, display messaging
}
I've tried dynamically adding script nodes via createElement/body.appendChild but those don't seem to work.
Is there an easy way to achieve this? Right now everything works, but IE6 and IE7 folks download script they aren't even executing, which is what I want to fix.

Adding script nodes should work just fine. Because those scripts will execute asynchronously to the code adding them, you'll need to give them a callback to call to do the next thing in order. E.g.:
if (window.localStorage) {
// Load the local storage stuff; once loaded, it'll call
// `doTheNextThing`
var script = document.createElement('script');
script.type = "text/javascript";
script.src = /* ... the URL of the script ... */;
document.body.appendChild(script); // Or append it to `head`, doesn't matter
// and `document.body` is convenient
}
else {
// Skip loading it
setTimeout(doTheNextThing, 10);
}
function doTheNextThing() {
// ...
}
...where the dynamic script you're loading for the localStorage stuff call doTheNextThing after it loads — so in the case where there's localStorage, the dynamically-loaded script calls doTheNextThing but in the case where there isn't, the code above does. Note that I made the call from the code above asynchronous (via setTimeout) on purpose: Making it always asynchronous regardless of how it gets called reduces your odds of missing bugs (e.g., adding something that relies on it being called synchronously and then forgetting to test that minor change on IE).
Update: The above assumes you're in control of the script you're loading, but you've clarified that you're not. In that case, what you need to do is load the scripts one at a time and poll for the feature that they provide (usually a property on the window object, like window.jQuery), something like this (untested):
// Load the script designated by `src`, poll for the appearance
// of the symbol `name` on the `window` object. When it shows
// up, call `callback`. Timeout if the timeout is reached.
function loadAndWait(src, name, timeout, callback) {
var stop, script;
// Do nothing if the symbol is already defined
if (window[name]) {
setTimeout(function() {
callback("preexisting");
}, 10);
}
else {
// Load the script
script = document.createElement('script');
script.type = "text/javascript";
script.src = src;
document.body.appendChild(script);
// Remember when we should stop
stop = new Date().getTime() + timeout;
// Start polling, long-ish initial interval
setTimeout(poll, 150);
}
function poll() {
if (window[name]) {
// Got it
callback("loaded");
}
else if (new Date().getTime() > stop) {
// Time out
callback("timeout");
}
else {
// Keep waiting, shorter interval if desired
setTimeout(poll, 75);
}
}
}
...which you'd use like this for the jQuery load:
loadAndWait(
"http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js",
"jQuery",
10000, // ten seconds or whatever
function(result) {
// ...do the next one if result !== "timeout"
}
);
You can either nest calls to loadAndWait in each of the previous calls' callbacks, or use an array and counter:
loadThese(
[
{ src: "http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js",
symbol: "jQuery"
},
{
src: "http://the-next-one",
symbol: "nextSymbol"
}
],
doTheNextThing
);
function loadThese(scripts, callback) {
var index = 0;
run("okay");
function run(result) {
var entry;
if (result === "timeout") {
callback(result);
}
else if (index < scripts.length) {
entry = scripts[index++];
loadAndWait(entry.src, entry.symbol, 10000, run);
}
else {
callback("loaded");
}
}
}
There, loadThese sets up a loop using run to load each script in turn.
All of the above is completely off-the-cuff and can probably be tightened and bullet-proofed, but you get the idea.
Off-topic, but my question is: Is there really so much code that it's a problem for the browsers that can't use it to load it? Barring the files getting a lot bigger, you'll actually slow down your site for users with advanced browsers without gaining much of anything on the others. Below a certain size, the overhead of connecting to the server to retrieve the script is as big a factor as transferring it. Is the extra stuff 50k of code? I'd do some benchmarking to test whether it's really necessary... Perhaps it is (perhaps you already have!), but it's worth just mentioning...
Off-topic update: In your updated question, you list five separate scripts you'd be downloading if localStorage is supported. Even assuming you're getting all five from various CDNs, that's a lot of individual script requests (whether done in the usual way or as above), each of which has to be processed one at a time. That's a page load performance issue waiting to happen. Despite (possibly) losing the benefits of CDNs and existing caching, you might look at grabbing all of those scripts, combining them, and hosting your combined version in a single file. See "Minimize HTTP Requests" in the YUI performance "rules" (I prefer the term "guideline", but whatever). It would also simplify your dynamic loading.

You can use the combination of onload and closure function. Something like the following:
function loadScripts(index) {
return function () {
var e = document.createElement('script');
e.src = scripts[index];
document.body.appendChild(e);
if (index + 1 < scripts.length) {
e.onload = loadScripts(index + 1)
}
};
}
And invoke it like this:
loadScripts(0)();

I've come to use this code. Both main functions (addEvent and load_javascript) are found on the web.
I wasn't trying to reduce download size, though: this is the only way I could load resources. So, maybe the idea proposed by Šime Vidas makes sense for you.
addEvent = function(elm, evType, fn, useCapture) {
//Credit: Function written by Scott Andrews
//(slightly modified)
var ret = 0;
if (elm.addEventListener) {
ret = elm.addEventListener(evType, fn, useCapture);
} else if (elm.attachEvent) {
ret = elm.attachEvent('on' + evType, fn);
} else {
elm['on' + evType] = fn;
}
return ret;
};
var left_to_load = 0;
function init() {
--left_to_load;
if (left_to_load > 0) {
return;
}
// all scripts are loaded now
// proceed with your logic
}
// load js file and call function when done
function load_javascript(src, callback) {
var a = document.createElement('script');
a.type = 'text/javascript';
a.src = src;
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(a, s);
++left_to_load;
addEvent(a, 'load', callback, false);
}
load_javascript('url1', init);
load_javascript('url2', init);
...