I am having trouble to understand why this takes so long to execute and whether there is a solution to this. It requires 20 seconds to execute the .setValues() line. The grid is about 2000 (row) by 100 (col) cells.
Here is a link to the test speed sheet for testing material, that is like my data-set.
function test() {
var active = SpreadsheetApp.getActiveSpreadsheet();
var logs_sheet = active.getSheetByName("Logs");
logs_sheet.appendRow([new Date(), "", "", "STARTING EXECUTION"]);
var test_sheet = active.getSheetByName("Test");
var test_data = test_sheet.getRange(1, 1, test_sheet.getLastRow(), test_sheet.getLastColumn()).getValues();
var combine = test_data.slice();
combine[1].splice(3, 1, new Date());
combine.splice(2017, 1);
combine.splice(0, 0, combine[1]);
logs_sheet.appendRow([new Date(), "", "", "finished combine"]);
test_sheet.getRange(1, 1, combine.length, combine[0].length).setValues(combine);
logs_sheet.appendRow([new Date(), "", "", "FINISHED EXECUTION"]);
}
What's more is that setValues() appears to update the sheet quite fast, but then it continues running something invisible before moving on.
Here is a possible solution / workaround I came up with.
Writing the array to a Google Doc for temporary storage only takes 0.06 seconds, rather than 20 seconds storing it in Sheets!
I assume that this can be quite a large array then, but I have not tested the limits of this workaround. I did however try to open the Google Doc and that is not really possible. But there is no need to open it.
var doc_database = DocumentApp.openById(" some doc ID here ").getBody();
var old_array = JSON.parse(doc_database.getText());
doc_database.setText('');
// run old_array calcs and updates here
doc_database.setText(JSON.stringify( ... new array here .... ));
hope this hack helps others.
Best
PAV
i would like to measure the skipped time during video playback if a user skipped some.
Using video.currentTime
First it looks quite trivial
Listen to seeking and get the currentTime A
Listen to seekend and get the currentTime B
B - A = Seeked Time
When i do that in Chrome the result is 0.
Why is that? If you listen to timeupdate TU it gets quite obvious.
Simplified Sequence:
Press Play
TU: 1
TU: 2
TU: 3 // now i use the mouse to seek forward to 19
TU: 19
//chrome & ff fire pause in between, ie not
Seek Start: 19
TU: 19
Seek End: 19
TU: 19
//chrome & ff fire play, ie not
TU: 20
...
I know i can play dirty and save the currentTime somewhere but not in an reliable way ;-)
Using TimeRanges video.played
I can use TimeRanges to calculate the amount of Time which got seeked/skipped. But the Problem is: TimeRanges come in ordered and normalized form as list. So if the User jumps forth an back the Ranges become merged and ordered => not reliable for accurate tracking.
Is there an easier less complicated approch i just dont see?
How I solved it.
I used a RingBuffer (Size 10) from here https://stackoverflow.com/a/28038535/2588818 and push the Math.round(video.currentTime) with every timeupdate to it.
On seekend I go left (prev) in the RingBuffer and take the first datum which differs from the current time.
Works just fine enough to use it for tracking the skipped time.
var createRingBuffer = createRingBuffer || function(length) {
/* https://stackoverflow.com/a/4774081 */
var pointer = 0, buffer = [];
return {
...
push : function(item){
buffer[pointer] = item;
pointer = (pointer + 1) % length;
return item;
},
...
getFirstDifference: function() {
var last_value = buffer[pointer - 1],
prev_value;
for (var i = 1; i <= length; i++) {
prev_value = buffer[(pointer - i) % length]
//check for undefined or initialize the buffer beforehand
if(prev_value === undefined || last_value === prev_value) {
} else {
return prev_value;
}
}
return last_value;
},
print: function() {
console.log(JSON.stringify(buffer));
}
};
};
...
var seekTimes = createRingBuffer(10);
handleUpdateTime = function() {
seekTimes.push(Math.round(video.currentTime));
},
handleSeekEnd = function(e) {
seekTimes.print();
console.log("%s - %s", seekTimes.getFirstDifference(), Math.round(video.currentTime));
},
...
Since localStorage (currently) only supports strings as values, and in order to do that the objects need to be stringified (stored as JSON-string) before they can be stored, is there a defined limitation regarding the length of the values.
Does anyone know if there is a definition which applies to all browsers?
Quoting from the Wikipedia article on Web Storage:
Web storage can be viewed simplistically as an improvement on cookies, providing much greater storage capacity (10 MB per origin in Google Chrome(https://plus.google.com/u/0/+FrancoisBeaufort/posts/S5Q9HqDB8bh), Mozilla Firefox, and Opera; 10 MB per storage area in Internet Explorer) and better programmatic interfaces.
And also quoting from a John Resig article [posted January 2007]:
Storage Space
It is implied that, with DOM Storage,
you have considerably more storage
space than the typical user agent
limitations imposed upon Cookies.
However, the amount that is provided
is not defined in the specification,
nor is it meaningfully broadcast by
the user agent.
If you look at the Mozilla source code
we can see that 5120KB is the default
storage size for an entire domain.
This gives you considerably more space
to work with than a typical 2KB
cookie.
However, the size of this storage area
can be customized by the user (so a
5MB storage area is not guaranteed,
nor is it implied) and the user agent
(Opera, for example, may only provide
3MB - but only time will tell.)
Actually Opera doesn't have 5MB limit. It offers to increase limit as applications requires more. User can even choose "Unlimited storage" for a domain.
You can easily test localStorage limits/quota yourself.
Here's a straightforward script for finding out the limit:
if (localStorage && !localStorage.getItem('size')) {
var i = 0;
try {
// Test up to 10 MB
for (i = 250; i <= 10000; i += 250) {
localStorage.setItem('test', new Array((i * 1024) + 1).join('a'));
}
} catch (e) {
localStorage.removeItem('test');
localStorage.setItem('size', i - 250);
}
}
Here's the gist, JSFiddle and blog post.
The script will test setting increasingly larger strings of text until the browser throws and exception. At that point it’ll clear out the test data and set a size key in localStorage storing the size in kilobytes.
Find the maximum length of a single string that can be stored in localStorage
This snippet will find the maximum length of a String that can be stored in localStorage per domain.
//Clear localStorage
for (var item in localStorage) delete localStorage[item];
window.result = window.result || document.getElementById('result');
result.textContent = 'Test running…';
//Start test
//Defer running so DOM can be updated with "test running" message
setTimeout(function () {
//Variables
var low = 0,
high = 2e9,
half;
//Two billion may be a little low as a starting point, so increase if necessary
while (canStore(high)) high *= 2;
//Keep refining until low and high are equal
while (low !== high) {
half = Math.floor((high - low) / 2 + low);
//Check if we can't scale down any further
if (low === half || high === half) {
console.info(low, high, half);
//Set low to the maximum possible amount that can be stored
low = canStore(high) ? high : low;
high = low;
break;
}
//Check if the maximum storage is no higher than half
if (storageMaxBetween(low, half)) {
high = half;
//The only other possibility is that it's higher than half but not higher than "high"
} else {
low = half + 1;
}
}
//Show the result we found!
result.innerHTML = 'The maximum length of a string that can be stored in localStorage is <strong>' + low + '</strong> characters.';
//Functions
function canStore(strLen) {
try {
delete localStorage.foo;
localStorage.foo = Array(strLen + 1).join('A');
return true;
} catch (ex) {
return false;
}
}
function storageMaxBetween(low, high) {
return canStore(low) && !canStore(high);
}
}, 0);
<h1>LocalStorage single value max length test</h1>
<div id='result'>Please enable JavaScript</div>
Note that the length of a string is limited in JavaScript; if you want to view the maximum amount of data that can be stored in localStorage when not limited to a single string, you can use the code in this answer.
Edit: Stack Snippets don't support localStorage, so here is a link to JSFiddle.
Results
Chrome (45.0.2454.101): 5242878 characters
Firefox (40.0.1): 5242883 characters
Internet Explorer (11.0.9600.18036): 16386 122066 122070 characters
I get different results on each run in Internet Explorer.
Don't assume 5MB is available - localStorage capacity varies by browser, with 2.5MB, 5MB and unlimited being the most common values.
Source: http://dev-test.nemikor.com/web-storage/support-test/
I wrote this simple code that is testing localStorage size in bytes.
https://github.com/gkucmierz/Test-of-localStorage-limits-quota
const check = bytes => {
try {
localStorage.clear();
localStorage.setItem('a', '0'.repeat(bytes));
localStorage.clear();
return true;
} catch(e) {
localStorage.clear();
return false;
}
};
Github pages:
https://gkucmierz.github.io/Test-of-localStorage-limits-quota/
I have the same results on desktop Google chrome, opera, firefox, brave and mobile chrome which is ~10Mbytes
And half smaller result in safari ~4Mbytes
You don't want to stringify large objects into a single localStorage entry. That would be very inefficient - the whole thing would have to be parsed and re-encoded every time some slight detail changes. Also, JSON can't handle multiple cross references within an object structure and wipes out a lot of details, e.g. the constructor, non-numerical properties of arrays, what's in a sparse entry, etc.
Instead, you can use Rhaboo. It stores large objects using lots of localStorage entries so you can make small changes quickly. The restored objects are much more accurate copies of the saved ones and the API is incredibly simple. E.g.:
var store = Rhaboo.persistent('Some name');
store.write('count', store.count ? store.count+1 : 1);
store.write('somethingfancy', {
one: ['man', 'went'],
2: 'mow',
went: [ 2, { mow: ['a', 'meadow' ] }, {} ]
});
store.somethingfancy.went[1].mow.write(1, 'lawn');
BTW, I wrote it.
I've condensed a binary test into this function that I use:
function getStorageTotalSize(upperLimit/*in bytes*/) {
var store = localStorage, testkey = "$_test"; // (NOTE: Test key is part of the storage!!! It should also be an even number of characters)
var test = function (_size) { try { store.removeItem(testkey); store.setItem(testkey, new Array(_size + 1).join('0')); } catch (_ex) { return false; } return true; }
var backup = {};
for (var i = 0, n = store.length; i < n; ++i) backup[store.key(i)] = store.getItem(store.key(i));
store.clear(); // (you could iterate over the items and backup first then restore later)
var low = 0, high = 1, _upperLimit = (upperLimit || 1024 * 1024 * 1024) / 2, upperTest = true;
while ((upperTest = test(high)) && high < _upperLimit) { low = high; high *= 2; }
if (!upperTest) {
var half = ~~((high - low + 1) / 2); // (~~ is a faster Math.floor())
high -= half;
while (half > 0) high += (half = ~~(half / 2)) * (test(high) ? 1 : -1);
high = testkey.length + high;
}
if (high > _upperLimit) high = _upperLimit;
store.removeItem(testkey);
for (var p in backup) store.setItem(p, backup[p]);
return high * 2; // (*2 because of Unicode storage)
}
It also backs up the contents before testing, then restores them.
How it works: It doubles the size until the limit is reached or the test fails. It then stores half the distance between low and high and subtracts/adds a half of the half each time (subtract on failure and add on success); honing into the proper value.
upperLimit is 1GB by default, and just limits how far upwards to scan exponentially before starting the binary search. I doubt this will even need to be changed, but I'm always thinking ahead. ;)
On Chrome:
> getStorageTotalSize();
> 10485762
> 10485762/2
> 5242881
> localStorage.setItem("a", new Array(5242880).join("0")) // works
> localStorage.setItem("a", new Array(5242881).join("0")) // fails ('a' takes one spot [2 bytes])
IE11, Edge, and FireFox also report the same max size (10485762 bytes).
You can use the following code in modern browsers to efficiently check the storage quota (total & used) in real-time:
if ('storage' in navigator && 'estimate' in navigator.storage) {
navigator.storage.estimate()
.then(estimate => {
console.log("Usage (in Bytes): ", estimate.usage,
", Total Quota (in Bytes): ", estimate.quota);
});
}
I'm doing the following:
getLocalStorageSizeLimit = function () {
var maxLength = Math.pow(2,24);
var preLength = 0;
var hugeString = "0";
var testString;
var keyName = "testingLengthKey";
//2^24 = 16777216 should be enough to all browsers
testString = (new Array(Math.pow(2, 24))).join("X");
while (maxLength !== preLength) {
try {
localStorage.setItem(keyName, testString);
preLength = testString.length;
maxLength = Math.ceil(preLength + ((hugeString.length - preLength) / 2));
testString = hugeString.substr(0, maxLength);
} catch (e) {
hugeString = testString;
maxLength = Math.floor(testString.length - (testString.length - preLength) / 2);
testString = hugeString.substr(0, maxLength);
}
}
localStorage.removeItem(keyName);
// Original used this.storageObject in place of localStorage. I can only guess the goal is to check the size of the localStorage with everything but the testString given that maxLength is then added.
maxLength = JSON.stringify(localStorage).length + maxLength + keyName.length - 2;
return maxLength;
};
I really like cdmckay's answer, but it does not really look good to check the size in a real time: it is just too slow (2 seconds for me). This is the improved version, which is way faster and more exact, also with an option to choose how big the error can be (default 250,000, the smaller error is - the longer the calculation is):
function getLocalStorageMaxSize(error) {
if (localStorage) {
var max = 10 * 1024 * 1024,
i = 64,
string1024 = '',
string = '',
// generate a random key
testKey = 'size-test-' + Math.random().toString(),
minimalFound = 0,
error = error || 25e4;
// fill a string with 1024 symbols / bytes
while (i--) string1024 += 1e16;
i = max / 1024;
// fill a string with 'max' amount of symbols / bytes
while (i--) string += string1024;
i = max;
// binary search implementation
while (i > 1) {
try {
localStorage.setItem(testKey, string.substr(0, i));
localStorage.removeItem(testKey);
if (minimalFound < i - error) {
minimalFound = i;
i = i * 1.5;
}
else break;
} catch (e) {
localStorage.removeItem(testKey);
i = minimalFound + (i - minimalFound) / 2;
}
}
return minimalFound;
}
}
To test:
console.log(getLocalStorageMaxSize()); // takes .3s
console.log(getLocalStorageMaxSize(.1)); // takes 2s, but way more exact
This works dramatically faster for the standard error; also it can be much more exact when necessary.
Once I developed Chrome (desktop browser) extension and tested Local Storage real max size for this reason.
My results:
Ubuntu 18.04.1 LTS (64-bit)
Chrome 71.0.3578.98 (Official Build) (64-bit)
Local Storage content size 10240 KB (10 MB)
More than 10240 KB usage returned me the error:
Uncaught DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'notes' exceeded the quota.
Edit on Oct 23, 2020
For a Chrome extensions available chrome.storage API. If you declare the "storage" permission in manifest.js:
{
"name": "My extension",
...
"permissions": ["storage"],
...
}
You can access it like this:
chrome.storage.local.QUOTA_BYTES // 5242880 (in bytes)
I'm working on a JavaScript game that involves throwing a snowball. I need the snowball to render as often as possible during its flight path. Chrome does all the calculations, including setting the style.left and style.top properties, but doesn't actually redraw the snowball until it reaches its destination. Opera doesn't have this problem.
A relevant point is that putting in an alert() after renderSnowball() fixes the problem, except using the alert() is an obvious issue.
Here's my code so far:
function throwSnowball()
{
var theta = parseFloat(angleField.value) * Math.PI/180 ;
var Vir = parseFloat(velocityField.value) ;
if (!isNaN(Vir) && !isNaN(theta) )
{
Vix = Math.cos(theta) * Vir * 50;
Viy = Math.sin(theta) * Vir * 50;
time = new Date() ;
var timeThrown = time.getTime() ;
while (snowballPosY > 0)
{
current = new Date() ;
var currentTime = current.getTime() ;
var timeElapsed = (currentTime - timeThrown)/5000 ;
snowballPosX += Vix * timeElapsed;
snowballPosY += Viy * timeElapsed;
Viy -= GRAVITY * timeElapsed ;
renderSnowball() ; //renderSnowball() sets the style.left
// and style.top properties to snowballPosX pixels
// and snowballPosY pixels respectively
timeThrown = currentTime ;
}
snowballPosX = 0 ;
snowballPosY = 50 ;
renderSnowball() ;
}
}
You're totally blocking the main thread. Have you tried using a setTimeout (even with a zero timeout) to allow other things to happen during your animation?
If you're willing to use experimental technology, requestAnimationFrame would be even better.
Edit: the setTimeout approach would look something like this (replacing the while loop):
var drawAndWait = function() {
if (snowballPosY > 0) {
// movement/drawing code here
setTimeout(drawAndWait, 20 /* milliseconds */);
} else {
// reset code that would normally go after your while loop
}
};
drawAndWait();
So each time the drawing finishes, it arranges for itself to be invoked again, if appropriate. Note that your throwSnowball function will return quickly; the throwing isn't actually done until later on. This takes awhile to get used to doing correctly; don't be too concerned if it's not intuitive at first.
Try getting out of the tight loop. Chrome may not want to redraw until your function exits. Try using setInterval or setTimeout to give Chrome a chance to repaint.
I was thinking of making a game using javascript for the game logic and the HTML5 canvas element to animate the drawing. My goal is to write something that works in browsers and on newer smartphones. So I wrote up a quick program that moves 100 circles around on the screen and shows me the frame rate. I was fairly disappointed with the results:
Chrome: ~90 FPS
Firefox: ~ 25 FPS
iPhone: ~11 FPS
This was a pretty simple test so I don't like my chances when it comes to actually making a complete game. Is this the standard result from the canvas element or are there some tricks to make drawing faster, if you have any good links let me know? Is canvas just a toy at this point or can it be used for real world applications.
Edit Here's the code:
var ctx;
var width;
var height;
var delta;
var lastTime;
var frames;
var totalTime;
var updateTime;
var updateFrames;
var creats = new Array();
function init() {
var canvas =document.getElementById('main');
width = canvas.width;
height = canvas.height;
ctx = canvas.getContext('2d');
for(var i=0; i < 100; ++i) {
addCreature();
}
lastTime = (new Date()).getTime();
frames = 0;
totalTime = 0;
updateTime = 0;
updateFrames =0;
setInterval(update, 10);
}
function addCreature() {
var c = new Creature(Math.random()*100,Math.random()*200);
creats.push(c);
}
function update() {
var now = (new Date()).getTime();
delta = now-lastTime;
lastTime = now;
totalTime+=delta;
frames++;
updateTime+=delta;
updateFrames++;
if(updateTime > 1000) {
document.getElementById('fps').innerHTML = "FPS AVG: " + (1000*frames/totalTime) + " CUR: " + (1000*updateFrames/updateTime);
updateTime = 0;
updateFrames =0;
}
for(var i=0; i < creats.length; ++i) {
creats[i].move();
}
draw();
}
function draw() {
ctx.clearRect(0,0,width,height);
creats.forEach(drawCreat);
}
function drawCreat(c,i,a) {
if (!onScreen(c)) {
return;
}
ctx.fillStyle = "#00A308";
ctx.beginPath();
ctx.arc(c.x, c.y, 10, 0, Math.PI*2, true);
ctx.closePath();
ctx.fill();
}
function onScreen(o) {
return o.x >= 0 && o.y >= 0 && o.x <= width && o.y <=height;
}
function Creature(x1,y) {
this.x = x1;
this.y = y;
this.dx = Math.random()*2;
this.dy = Math.random()*2;
this.move = function() {
this.x+=this.dx;
this.y+=this.dy;
if(this.x < 0 || this.x > width) {
this.dx*=-1;
}
if(this.y < 0 || this.y > height) {
this.dy*=-1;
}
}
}
init();
In order to make animations more efficient, and to synchronize your framerate with the UI updates, Mozilla created a mozRequestAnimationFrame() function that is designed to remove the inefficiencies of setTimeout() and setInterval(). This technique has since been adopted by Webkit for Chrome only.
In Feb 2011 Paul Irish posted a shim that created requestAnimFrame(), and shortly afterwards Joe Lambert extended it by restoring the "timeout" and "interval" delay to slow down animation ticks.
Anyway, I've used both and have seen very good results in Chrome and Firefox. The shim also fallsback to setTimeout() if support requestAnimationFrame() is unavailable. Both Paul and Joe's code is online at github.
Hope this helps!
It's largely dependent on the JavaScript engine. V8 (Chrome) and Carakan (Opera) are probably the two fastest production-quality engines. TraceMonkey (Firefox) and SquirrelFish (Safari) are far behind, with KJS bringing up the rear. This will change as hardware acceleration enters the mainstream.
As for specific optimizations, we'd probably have to see some code. Remember that the canvas supports compositing, so you really only need to redraw areas that changed. Perhaps you should re-run your benchmark without the canvas so you know if the drawing operations really were the limiting factor.
If you want to see what can be done now, check out:
js1k
Bespin
Canvas-stein
Arcs are math-intensive to draw. You can dramatically improve performance by using drawImage or even putImageData instead of drawing the path each frame.
The image can be a file loaded from a URL or it can be an image created by drawing on a separate canvas not visible to the user (not connected to the DOM). Either way, you'll save a ton of processor time.
I have written a simple bouncing ball which gives you points if you click it.
It works fine in Firefox, Safari, Chrome and on the iPad. However, the iPhone 3G/3GS were horribly slow with it. Same goes for my older Android phone.
I am sorry but I do lack specific numbers.
Chrome is the only browser thus far that I've seen high framerate results with.
You might also want to try the latest Preview of IE9. That should give you a decent benchmark of how the next generation of browsers (with hardware acceleration for HTML5) will handle your code.
So far, I've seen that IE9, Chrome 7, and Firefox 4 will all sport some form of hardware acceleration.
There's loads of optimizations to be done with Canvas drawing.
Do you have example code you could share?