Since a call to AudioWorkletProcessor.process is synchronous according to https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletProcessor/process, what happens when an execution takes too long time, more than 1 second, for example? Would some audio samples be skipped in a next call? Or would samples be queued somewhere? I could not find a documentation on this.
A realtime AudioContext is usually tight to a certain physical audio output device. Therefore the currentTime of an AudioContext is driven by the hardware clock of that audio output device.
If the AudioContext somehow fails to deliver the samples in time the result will be silence. Typically that can be heard as a click sound if there is suddenly a little bit of silence where it shouldn't be. The samples that took too long to render will then be used for the next render quantum if they happen to be ready by then.
There is a difference though in how browsers advance the currentTime in case some samples couldn't be delivered in time. Firefox seems to not count any missed samples whereas Chrome progresses time counting all samples that the audio hardware spit out including those that couldn't be rendered in time.
To test this I created an AudioWorkletProcessor that can be paused or blocked from within the main thread.
class BlockableProcessor extends AudioWorkletProcessor {
constructor () {
super();
this.int32Array = null;
this.port.onmessage = ({ data }) => this.int32Array = data;
}
process() {
if (this.int32Array !== null) {
while (Atomics.load(this.int32Array, 0) === 0) {
Atomics.store(this.int32Array, 1, currentTime);
}
Atomics.store(this.int32Array, 1, currentTime);
}
return true;
}
}
registerProcessor('blockable-processor', BlockableProcessor);
This BlockableProcessor expects to receive a SharedArrayBuffer which it uses to check if it should block the process() function or not. And it also uses it to communicate the currentTime back to the main thread.
It can be used like this:
const audioWorkletNode = new AudioWorkletNode(
audioContext,
'blockable-processor'
);
const sharedArrayBuffer = new SharedArrayBuffer(8);
const int32Array = new Int32Array(sharedArrayBuffer);
Atomics.store(int32Array, 0, 1);
Atomics.store(int32Array, 1, 0);
audioWorkletNode.port.postMessage(int32Array);
audioWorkletNode.connect(audioContext.destination);
It can be blocked by setting the first value of the SharedArrayBuffer to 0.
Atomics.store(int32Array, 0, 0);
And likewise it can be unblocked again by setting it to 1 again.
Atomics.store(int32Array, 0, 1);
It's also possible to read the currentTime written from within the AudioWorkletProcessor on the main thread.
Atomics.load(int32Array, 1);
Using this setup I was able to see that Chrome (v95) and Firefox (v93) both stop the time progression on the main thread and in the worklet in case the processor is blocked.
When it gets unblocked Firefox continues where it stopped and Chrome continues where it should be if everything would have been going as expected.
Related
I was testing the accuracy of setTimeout using this test. Now I noticed that (as expected) setTimeout is not very accurate but for most appliances not dramatically inaccurate. Now if I run the test in Chrome and let it run in a background tab (so, switching to another tab and browse on there), returning to the test and inspecting the results (if the test finished) they are dramatically changed. It looks like the timeouts have been running a lot slower. Tested in FF4 or IE9 this didn't occur.
So it looks like Chrome suspends or at least slows down javascript execution in a tab that has no focus. Couldn't find much on the internet on the subject. It would mean that we can't run background tasks, like for example checking periodically on a server using XHR calls and setInterval (I suspect to see the same behavior for setInterval, will write a test if time is with me).
Has anyone encountered this? Would there be a workaround for this suspension/slowing down? Would you call it a bug and should I file it as such?
I recently asked about this and it is behaviour by design. When a tab is inactive, only at a maximum of once per second the function is called. Here is the code change.
Perhaps this will help:
How can I make setInterval also work when a tab is inactive in Chrome?
TL;DR: use Web Workers.
There is a solution to use Web Workers, because they run in separate process and are not slowed down
I've written a tiny script that can be used without changes to your code - it simply overrides functions setTimeout, clearTimeout, setInterval, clearInterval
Just include it before all your code
http://github.com/turuslan/HackTimer
Playing an empty sound forces the browser to retain the performance. I discovered it after reading this comment: How to make JavaScript run at normal speed in Chrome even when tab is not active?
With the source of that comment found here:
The Chromium insider also clarified that aggressive throttling will be automatically disabled for all background tabs “playing audio” as well as for any page where an “active websocket connection is present.”
I need unlimited performance on-demand for a browser game that uses WebSockets, so I know from experience that using WebSockets doesn't ensure unlimited performance, but from tests, playing an audio file seems to ensure it
Here's two empty audio loops I created for this purpose, you can use them freely, commercially:
http://adventure.land/sounds/loops/empty_loop_for_js_performance.ogg
http://adventure.land/sounds/loops/empty_loop_for_js_performance.wav
(They include -58db noise, -60db doesn't work)
I play them, on user-demand, with Howler.js: https://github.com/goldfire/howler.js
function performance_trick()
{
if(sounds.empty) return sounds.empty.play();
sounds.empty = new Howl({
src: ['/sounds/loops/empty_loop_for_js_performance.ogg','/sounds/loops/empty_loop_for_js_performance.wav'],
volume:0.5,
autoplay: true, loop: true,
});
}
It's sad that there is no built-in method to turn full JavaScript performance on/off by default, yet, crypto miners can hijack all your computing threads using Web Workers without any prompt.
I have released worker-interval npm package which setInterval and clearInterval implementation with using Web-Workers to keep up and running on inactive tabs for Chrome, Firefox and IE.
Most of the modern browsers (Chrome, Firefox and IE), intervals (window timers) are clamped to fire no more often than once per second in inactive tabs.
You can find more information on
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setInterval
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Timeouts_and_intervals
For unit tests you can run your chrome/chromium with argument: --disable-background-timer-throttling
I updated my jQuery core to 1.9.1, and it solved the Interval discrepancy in inactive tabs. I would try that first, then look into other code override options.
here is my solution which gets the current millisecond, and compares it to the millisecond that the function was created. for interval, it will update the millisecond when it runs the function. you can also grab the interval/timeout by an id.
<script>
var nowMillisTimeout = [];
var timeout = [];
var nowMillisInterval = [];
var interval = [];
function getCurrentMillis(){
var d = new Date();
var now = d.getHours()+""+d.getMinutes()+""+d.getSeconds()+""+d.getMilliseconds();
return now;
}
function setAccurateTimeout(callbackfunction, millis, id=0){
nowMillisTimeout[id] = getCurrentMillis();
timeout[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisTimeout[id] + +millis)){callbackfunction.call(); clearInterval(timeout[id]);} }, 10);
}
function setAccurateInterval(callbackfunction, millis, id=0){
nowMillisInterval[id] = getCurrentMillis();
interval[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisInterval[id] + +millis)){callbackfunction.call(); nowMillisInterval[id] = getCurrentMillis();} }, 10);
}
//usage
setAccurateTimeout(function(){ console.log('test timeout'); }, 1000, 1);
setAccurateInterval(function(){ console.log('test interval'); }, 1000, 1);
</script>
I'm using the WebAudio API and doing a simple delay on the input using DelayNode. The code is as follows (might not work in the snippet because it requests microphone permissions):
const player = document.querySelector("#player")
const handleSuccess = stream => {
const context = new AudioContext()
const source = context.createMediaStreamSource(stream)
const delay = context.createDelay(179) //seconds
//source.connect(context.destination) connect directly
source.connect(delay)
delay.connect(context.destination)
delay.delayTime.value = 1
}
navigator.mediaDevices.getUserMedia({audio: true, video: false}).then(handleSuccess)
However, when I run a metronome at 60 beats per minute (one click every second), the audio coming from the browser is not delayed by exactly one second, and multiple clicks are heard (the delay is slightly longer than expected). Is there any way to make an exact delay?
I guess the problem is not the DelayNode itself but rather that there are multiple other hidden delays in the audio pipeline. If you want to hear the previous click from the metronome at the very same time as the current click you would need to reduce the time of your delay to account for those other delays (also known as latency).
The signal takes a bit of time to travel from your microphone through your A/D converter into the browser. This time is usually available as part of the settings of the MediaStream
stream.getAudioTracks()[0].getSettings().latency
Piping the MediaStream into the Web Audio API will probably add some more latency. And the AudioContext itself will add some latency due to its internal buffer.
context.baseLatency
It will also take some time to get the signal out of the computer again. It will travel from the browser to the OS which passes it on to the hardware. This value is also exposed on the AudioContext.
context.outputLatency
In theory you would only need to subtract all those values from the delay time and it would just work. However in reality every hardware/OS/browser combination is a bit different and you will probably need to make some adjustments in order to successfully overlay the previous with the current click depending on your personal setup.
You can use the async/await syntax (documentation here), here are two examples (taken from here)
const foo = async () => {
await sleep(1000);
// do something
}
const foo = async evt => {
await sleep(1000);
// do something with evt
}
I was testing the accuracy of setTimeout using this test. Now I noticed that (as expected) setTimeout is not very accurate but for most appliances not dramatically inaccurate. Now if I run the test in Chrome and let it run in a background tab (so, switching to another tab and browse on there), returning to the test and inspecting the results (if the test finished) they are dramatically changed. It looks like the timeouts have been running a lot slower. Tested in FF4 or IE9 this didn't occur.
So it looks like Chrome suspends or at least slows down javascript execution in a tab that has no focus. Couldn't find much on the internet on the subject. It would mean that we can't run background tasks, like for example checking periodically on a server using XHR calls and setInterval (I suspect to see the same behavior for setInterval, will write a test if time is with me).
Has anyone encountered this? Would there be a workaround for this suspension/slowing down? Would you call it a bug and should I file it as such?
I recently asked about this and it is behaviour by design. When a tab is inactive, only at a maximum of once per second the function is called. Here is the code change.
Perhaps this will help:
How can I make setInterval also work when a tab is inactive in Chrome?
TL;DR: use Web Workers.
There is a solution to use Web Workers, because they run in separate process and are not slowed down
I've written a tiny script that can be used without changes to your code - it simply overrides functions setTimeout, clearTimeout, setInterval, clearInterval
Just include it before all your code
http://github.com/turuslan/HackTimer
Playing an empty sound forces the browser to retain the performance. I discovered it after reading this comment: How to make JavaScript run at normal speed in Chrome even when tab is not active?
With the source of that comment found here:
The Chromium insider also clarified that aggressive throttling will be automatically disabled for all background tabs “playing audio” as well as for any page where an “active websocket connection is present.”
I need unlimited performance on-demand for a browser game that uses WebSockets, so I know from experience that using WebSockets doesn't ensure unlimited performance, but from tests, playing an audio file seems to ensure it
Here's two empty audio loops I created for this purpose, you can use them freely, commercially:
http://adventure.land/sounds/loops/empty_loop_for_js_performance.ogg
http://adventure.land/sounds/loops/empty_loop_for_js_performance.wav
(They include -58db noise, -60db doesn't work)
I play them, on user-demand, with Howler.js: https://github.com/goldfire/howler.js
function performance_trick()
{
if(sounds.empty) return sounds.empty.play();
sounds.empty = new Howl({
src: ['/sounds/loops/empty_loop_for_js_performance.ogg','/sounds/loops/empty_loop_for_js_performance.wav'],
volume:0.5,
autoplay: true, loop: true,
});
}
It's sad that there is no built-in method to turn full JavaScript performance on/off by default, yet, crypto miners can hijack all your computing threads using Web Workers without any prompt.
I have released worker-interval npm package which setInterval and clearInterval implementation with using Web-Workers to keep up and running on inactive tabs for Chrome, Firefox and IE.
Most of the modern browsers (Chrome, Firefox and IE), intervals (window timers) are clamped to fire no more often than once per second in inactive tabs.
You can find more information on
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setInterval
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Timeouts_and_intervals
For unit tests you can run your chrome/chromium with argument: --disable-background-timer-throttling
I updated my jQuery core to 1.9.1, and it solved the Interval discrepancy in inactive tabs. I would try that first, then look into other code override options.
here is my solution which gets the current millisecond, and compares it to the millisecond that the function was created. for interval, it will update the millisecond when it runs the function. you can also grab the interval/timeout by an id.
<script>
var nowMillisTimeout = [];
var timeout = [];
var nowMillisInterval = [];
var interval = [];
function getCurrentMillis(){
var d = new Date();
var now = d.getHours()+""+d.getMinutes()+""+d.getSeconds()+""+d.getMilliseconds();
return now;
}
function setAccurateTimeout(callbackfunction, millis, id=0){
nowMillisTimeout[id] = getCurrentMillis();
timeout[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisTimeout[id] + +millis)){callbackfunction.call(); clearInterval(timeout[id]);} }, 10);
}
function setAccurateInterval(callbackfunction, millis, id=0){
nowMillisInterval[id] = getCurrentMillis();
interval[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisInterval[id] + +millis)){callbackfunction.call(); nowMillisInterval[id] = getCurrentMillis();} }, 10);
}
//usage
setAccurateTimeout(function(){ console.log('test timeout'); }, 1000, 1);
setAccurateInterval(function(){ console.log('test interval'); }, 1000, 1);
</script>
I have developed a music sequencer in javascript; something like this: http://stepseq.michd.me/
I have implemented loop using setInterval function in following way:
var Sequencer = {
...
// every step sequencer ...
next: function(callback) {
// restart from begin if arrive to last sequecer step
if(Sequencer.current==Sequencer.steps.length)
Sequencer.current = 0;
// play all sounds in array step contains
if(Sequencer.steps[Sequencer.current].length>0) {
var set = Sequencer.steps[Sequencer.current];
for(var i=0;i<set.length;i++){
set[i].play();
}
}
callback(Sequencer.current);
Sequencer.current++;
},
loop: function(callback) {
Sequencer.interval = $interval(function(){
Sequencer.next(callback);
}, Sequencer.time);
}
}
...
Code below works but i think that there is a better way to implement loop; infact sometimes steps goes out of time. Sequencer.time (time passed to setInterval function) is a time in millisecs (this value is the conversion of a bpm value; for example it can be 125),
Someone can suggest me a better solution?
N.B.: this is a web application angularjs based (for this reason in code above a use $interval insteed of setInterval), but i think that this point is insignificant.
Javascript timer intervals are not guaranteed to run at exactly the time you request, due to the single threaded nature of JS. What you get is a callback that is queued up to run after the interval expires, whenever the engine is free to run it.
John resig has covered this off in some depth:
http://ejohn.org/blog/how-javascript-timers-work/
http://ejohn.org/blog/analyzing-timer-performance/
And from his conclusions, which is going to be important for your app:
If a timer is blocked from immediately executing it will be delayed
until the next possible point of execution (which will be longer than
the desired delay).
I don't really have a better solution for your problem, due to these fundamental issues with timers in JS, but this may at least explain what is happening.
I was testing the accuracy of setTimeout using this test. Now I noticed that (as expected) setTimeout is not very accurate but for most appliances not dramatically inaccurate. Now if I run the test in Chrome and let it run in a background tab (so, switching to another tab and browse on there), returning to the test and inspecting the results (if the test finished) they are dramatically changed. It looks like the timeouts have been running a lot slower. Tested in FF4 or IE9 this didn't occur.
So it looks like Chrome suspends or at least slows down javascript execution in a tab that has no focus. Couldn't find much on the internet on the subject. It would mean that we can't run background tasks, like for example checking periodically on a server using XHR calls and setInterval (I suspect to see the same behavior for setInterval, will write a test if time is with me).
Has anyone encountered this? Would there be a workaround for this suspension/slowing down? Would you call it a bug and should I file it as such?
I recently asked about this and it is behaviour by design. When a tab is inactive, only at a maximum of once per second the function is called. Here is the code change.
Perhaps this will help:
How can I make setInterval also work when a tab is inactive in Chrome?
TL;DR: use Web Workers.
There is a solution to use Web Workers, because they run in separate process and are not slowed down
I've written a tiny script that can be used without changes to your code - it simply overrides functions setTimeout, clearTimeout, setInterval, clearInterval
Just include it before all your code
http://github.com/turuslan/HackTimer
Playing an empty sound forces the browser to retain the performance. I discovered it after reading this comment: How to make JavaScript run at normal speed in Chrome even when tab is not active?
With the source of that comment found here:
The Chromium insider also clarified that aggressive throttling will be automatically disabled for all background tabs “playing audio” as well as for any page where an “active websocket connection is present.”
I need unlimited performance on-demand for a browser game that uses WebSockets, so I know from experience that using WebSockets doesn't ensure unlimited performance, but from tests, playing an audio file seems to ensure it
Here's two empty audio loops I created for this purpose, you can use them freely, commercially:
http://adventure.land/sounds/loops/empty_loop_for_js_performance.ogg
http://adventure.land/sounds/loops/empty_loop_for_js_performance.wav
(They include -58db noise, -60db doesn't work)
I play them, on user-demand, with Howler.js: https://github.com/goldfire/howler.js
function performance_trick()
{
if(sounds.empty) return sounds.empty.play();
sounds.empty = new Howl({
src: ['/sounds/loops/empty_loop_for_js_performance.ogg','/sounds/loops/empty_loop_for_js_performance.wav'],
volume:0.5,
autoplay: true, loop: true,
});
}
It's sad that there is no built-in method to turn full JavaScript performance on/off by default, yet, crypto miners can hijack all your computing threads using Web Workers without any prompt.
I have released worker-interval npm package which setInterval and clearInterval implementation with using Web-Workers to keep up and running on inactive tabs for Chrome, Firefox and IE.
Most of the modern browsers (Chrome, Firefox and IE), intervals (window timers) are clamped to fire no more often than once per second in inactive tabs.
You can find more information on
https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setInterval
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Timeouts_and_intervals
For unit tests you can run your chrome/chromium with argument: --disable-background-timer-throttling
I updated my jQuery core to 1.9.1, and it solved the Interval discrepancy in inactive tabs. I would try that first, then look into other code override options.
here is my solution which gets the current millisecond, and compares it to the millisecond that the function was created. for interval, it will update the millisecond when it runs the function. you can also grab the interval/timeout by an id.
<script>
var nowMillisTimeout = [];
var timeout = [];
var nowMillisInterval = [];
var interval = [];
function getCurrentMillis(){
var d = new Date();
var now = d.getHours()+""+d.getMinutes()+""+d.getSeconds()+""+d.getMilliseconds();
return now;
}
function setAccurateTimeout(callbackfunction, millis, id=0){
nowMillisTimeout[id] = getCurrentMillis();
timeout[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisTimeout[id] + +millis)){callbackfunction.call(); clearInterval(timeout[id]);} }, 10);
}
function setAccurateInterval(callbackfunction, millis, id=0){
nowMillisInterval[id] = getCurrentMillis();
interval[id] = setInterval(function(){ var now = getCurrentMillis(); if(now >= (+nowMillisInterval[id] + +millis)){callbackfunction.call(); nowMillisInterval[id] = getCurrentMillis();} }, 10);
}
//usage
setAccurateTimeout(function(){ console.log('test timeout'); }, 1000, 1);
setAccurateInterval(function(){ console.log('test interval'); }, 1000, 1);
</script>