I'm creating a music related app using Web Audio API, which requires precisely synchronizing sound, visuals and MIDI input processing.
The sound production follows the pattern described in this article: requestAnimationFrame regularly calls a function that schedules events on the AudioContext. This works fine most of the time, except in some occasions where the audio inexplicably lags behind the visuals.
After much poking around, I ran into the AudioContext.currentTime specification which hints:
Elapsed time in this system corresponds to elapsed time in the audio stream generated by the BaseAudioContext, which may not be synchronized with other clocks in the system.
And indeed I was able to verify that these sporadic delays in the audio come down to a problem in the AudioContext clock itself, which seems to pause for a bit sometimes just after starting up. Note that this doesn't happen every time, but frequently enough to be an issue (maybe 10-15% of the time...). Each time it happens the pattern is the same: currentTime starts increasing then gets stuck at 23ms for a bit, then starts going again regularly without any further issue after accumulating a total of ~230ms of lag behind the system/wall clock...
I've created a simple script which reproduces the problem (if you want to try it, just open the console to see the output, and press any key to start the test... as the issue is sporadic you may need to retry or sometimes reload multiple times before it happens...):
<script>
const audioContext = new AudioContext();
// reference points for both clocks (JS and audioContext)
var animStartTime = null;
var audioStartTime = null;
// loop function to be called by requestAnimationFrame
function play(timestamp) {
// set animStartTime on first invocation
if (animStartTime == null) animStartTime = timestamp;
// compute elapsed time for both clocks
var animElapsedTime = timestamp - animStartTime;
var audioElapsedTime = (audioContext.currentTime - audioStartTime) * 1000;
console.log('Animation ts: ' + animElapsedTime +
', Audio ts: ' + audioElapsedTime +
', Diff: ' + (animElapsedTime - audioElapsedTime)
);
// keep this going for 1 second
if (animElapsedTime < 1000)
requestAnimationFrame(play);
}
function start() {
audioStartTime = audioContext.currentTime;
animStartTime = null; // use the timestamp provided by requestAnimationFrame
// create a simple oscillator and schedule it to produce a single beep when it starts
const osc = audioContext.createOscillator();
osc.frequency.value = 800;
osc.connect(audioContext.destination);
osc.start(audioStartTime);
osc.stop(audioStartTime + 0.03);
// launch animation loop
requestAnimationFrame(play);
}
// press any key to start the test
window.addEventListener('keypress', function(e) {start()});
</script>
And here is the console output of a bogus run, where you can see the Audio timestamp freezing up to a delay of ~230ms behind the JS main thread timestamp before starting again:
Could someone explain to me:
What is going on? Why is this freezing sporadically?
Is there anything I can do to prevent this? I can think of some ways to mitigate the issue if I can get convinced that it happens only when starting up, but without fully understanding the root cause I fear seeing these freezes happen again at other times...
This is probably https://crbug.com/693978 Resuming the context may take "some" time.
Unfortunately, they don't really wait for the context has started before resolving the Promise returned by context.resume(), so we have to resort to ugly workarounds.
One such workaround would be to wait for currentTime to start updating after context.resume() resolves before starting your animation.
const audioContext = new AudioContext();
// reference points for both clocks (JS and audioContext)
var animStartTime = null;
var audioStartTime = null;
// loop function to be called by requestAnimationFrame
function play(timestamp) {
// set animStartTime on first invocation
if (animStartTime == null) animStartTime = timestamp;
// compute elapsed time for both clocks
var animElapsedTime = timestamp - animStartTime;
var audioElapsedTime = (audioContext.currentTime - audioStartTime) * 1000;
console.log('Animation ts: ' + animElapsedTime +
', Audio ts: ' + audioElapsedTime +
', Diff: ' + (animElapsedTime - audioElapsedTime)
);
// keep this going for 1 second
if (animElapsedTime < 1000)
requestAnimationFrame(play);
}
async function start() {
// wait for the context to resume
// needs to be there, for we still handle the user-gesture
await audioContext.resume();
// though resume() might be a lie in Chrome,
// so we also wait for currentTime to update
const startTime = audioContext.currentTime;
while (startTime === audioContext.currentTime) {
await new Promise((res) => setTimeout(res));
}
// now our AudioContext is ready.
audioStartTime = audioContext.currentTime;
animStartTime = null; // use the timestamp provided by requestAnimationFrame
// create a simple oscillator and schedule it to produce a single beep when it starts
const osc = audioContext.createOscillator();
osc.frequency.value = 800;
osc.connect(audioContext.destination);
osc.start(0);
osc.stop(0.3);
// launch animation loop
requestAnimationFrame(play);
}
// using a button for a clearer snippet
document.querySelector("button")
.addEventListener('click', start);
<button>start</button>
Related
I'm curious if you can invoke a function after the time changes?
For example invoking a function after the minute changes maybe something like this:
onMinChange(()=>{console.log('the minute has changed!')})
Note:
I do not mean waiting for a minute to invoke a function like this
setInterval(()=>{console.log('the minute has changed!')},60000);
there's a difference between
✓ invoking a function after the minute changes (Current time 2:30:05, invoke function at 2:31:00 )
&
X waiting for a minute to invoke a function (Current time 2:30:05, invoke function at 2:31:05 )
There's nothing built into the web platform that will do this (nor, I think, Node.js's standard library). The closest you can come (without some third-party lib) is to get the time, calculate how long it should be until the next minute, and then use setTimeout to schecule a callback. If you want it repeated, repeat as necessary.
Here's an example:
const MINUTE_IN_MS = 60000;
function callNextMinute(fn) {
const now = Date.now();
let next = now / MINUTE_IN_MS;
const ceil = Math.ceil(next);
if (ceil > next) {
next = ceil;
}
return setTimeout(fn, (next * MINUTE_IN_MS) - now);
}
function tick() {
console.log(`New minute! Time is ${new Date()}`);
callNextMinute(tick);
}
console.log(`Started at ${new Date()}`);
callNextMinute(tick);
There, the margin of error is ~10 seconds.
Beware that:
That kind of recurring execution can have an adverse effect on battery life of mobile devices.
Timers are slowed or even stopped for tabs that aren't the active tab in many browsers, so you may well miss the minute if the tab is inactive.
You can achieve behaviour like this with the npm package cron.
var job = new CronJob('* * * * *', function() {
console.log('You will see this message every full minute');
}, null, true, 'America/Los_Angeles');
I am using WebAudio to play a sequence of notes. I have a playNote function which works well; I send it note frequency and start and stop times for each note. The generation of the sequence parameters occurs before the actual sound starts, which is a little confusing. The function just creates an oscillator for every note. (I tried other methods and this is the cleanest).
But I would like to stop the sequence asynchronously (e.g. when an external event occurs). I tried setting up a master Gain node that could be used to gate the output, but it seems it needs to be "inside" the function, so it can't be controlled later on. If I try and turn off my gain object inside the function then it is too late - because the start & stop times have already been passed to the function.
Here is my function:
function playNote(audioContext,frequency, startTime, endTime, last) {
gainNode = audioContext.createGain(); //to get smooth rise/fall
oscillator = audioContext.createOscillator();
oscillator.frequency.value=frequency;
oscillator.connect(gainNode);
gainNode.connect(analyserScale); //analyser is global
analyserScale.connect(audioContext.destination);
gainNode.gain.exponentialRampToValueAtTime(toneOn, startTime + trf);
gainNode.gain.exponentialRampToValueAtTime(toneOff, endTime+trf);
oscillator.start(startTime);
oscillator.stop(endTime);
}
Any help appreciated!
This does it: Web Audio API: Stop all scheduled sounds from playing. The solution is to keep track of the scheduled oscillators with an array.
The function now becomes:
var oscs = []; //list of oscillators
function playNote(audioContext,frequency, startTime, endTime, last, index) {
gainNode = audioContext.createGain(); //to get smooth rise/fall
oscillator = audioContext.createOscillator();
oscillator.frequency.value=frequency;
oscillator.connect(gainNode);
//keep track of alll the oscs so that they can be switched off if scale is stopped by user
oscs[index] = oscillator;
gainNode.connect(analyserScale); //analyser is global
analyserScale.connect(audioContext.destination);
gainNode.gain.exponentialRampToValueAtTime(toneOn, startTime + trf);
gainNode.gain.exponentialRampToValueAtTime(toneOff, endTime+trf);
oscillator.start(startTime);
oscillator.stop(endTime);
}
Then code to stop the oscillators:
for(let i=0; i<oscs.length; i++) {
if(oscs[i]){
oscs[i].stop(0);
}
}
Trying to create a simple drum machine sequencing app without using any libraries. Basic purpose of the app is to loop audio at specified intervals while some sort of condition is met e.g. pressing the spacebar.
I figured out that looping audio with setInterval isn't really a good idea since it's pretty inconsistent. Found another solution using new Date() but the example requires calling the function with a set duration as an arg. What I'd like to do is use this same code, but have the loop run infinitely until a spacebar keydown or some other condition is met.
Is this possible? Any thoughts on how I can adapt or re-write this to get what I want? I know how to create my trigger for the loop with event handlers, but each keydown or whatever event will trigger a new audio event (causing loops on top of loops) vs. killing the process of the last one which once set to "loop forever" does just that...
function doTimer(totalms, interval, oninstance, kill) {
var time = 0,
start = new Date().getTime();
function instance() {
time += interval;
if (kill) {
} else {
var diff = (new Date().getTime() - start) - time;
window.setTimeout(instance, (interval - diff));
oninstance(time + diff);
}
}
window.setTimeout(instance, interval);
}
function playSample(str){
soundStr = `./audio808/${str}.mp3`;
sound = new Audio(soundStr);
sound.play();
}
doTimer(0, // duration
125, // sets the interval
function() {
playSample("ch");}, // calls the sample to play
true // whether or not it should play
);
You can set the .loop property of AudioNode to true
sound.loop = true;
I have encountered following issue when creating simple task: displaying html clock by using WebKit engine. Additional requirement were to handle system time change and that it should work on Windows.
I have used setInterval to achive this but it seems to freeze browser after I change system time backward.
For me it looks like WebKit issue. It is easy to reproduce on safari by running this simple code:
<p id="date"></p>
setInterval(SetTime, 1000);
function SetTime() {
document.getElementById('date').textContent=new Date();
}
After that I have made another approach with recursive setTimeout call. Same effect.
(function loop() {
document.getElementById('date').textContent=new Date();
setTimeout(loop, 1000);
})();
Any ideas why is that happening and how to go around this?
This is almost definitely an issue with WebKit.
The Problem
When you use setTimeout, you create a 'timer' with two properties:
A timestamp, set to now + the specified delay
A callback to fire once once the system time is greater than the timestamp.
You can imagine a naïve implementation of setTimeout looking something like this:
var timers = [];
function setTimeout(callback, delay) {
var id = timers.length;
timers[id] = {
callback: callback,
timestamp: Date.now() + delay
}
return id;
}
This would simply create a timer and add it to a list. Then, on each tick, the JS runtime would check these timers and execute the callbacks for those that have fired:
var now = Date.now();
for(var id in timers) {
var timer = timers[id];
if(timer && timer.timestamp < now) {
callback();
delete timers[id];
}
}
Given this implementation, imagine now changing the system time (i.e. Date.now() in the examples above) to a value in the past -- the timer's timestamp will still be set relative to the previous system time (i.e. in the future).
The same issue exists with setInterval, which (assuming sane code in WebKit) will be implemented using setTimeout.
Unfortunately, this means that any invocation of setTimeout or setInterval is going to suffer.
The Solution
As an alternative, you can use the lovely window.requestAnimationFrame method to perform a callback on each tick. I haven't tested this at at all, but it should continue to fire on each tick, regardless of the system time.
As a bonus, each time it fires the callback, you get passed the current timestamp as a parameter. By keeping track of the previous timestamp passed to your callback, you could easily detect backwards system time changes:
var lastTimestamp;
var onNextFrame = window.requestAnimationFrame || window.webkitRequestAnimationFrame;
var checkForPast = function(timestamp) {
if(lastTimestamp && timestamp < lastTimestamp) {
console.error('System time moved into the past! I should probably handle this');
}
lastTimestamp = timestamp;
onNextFrame(checkForPast);
};
onNextFrame(checkForPast);
Conclusions
This might not be great news for you, but you should probably rewrite your entire application to use requestAnimationFrame anyway - it seems much more suited to your needs:
var onNextFrame = window.requestAnimationFrame || window.webkitRequestAnimationFrame;
var dateElem = document.getElementById('date');
var updateDate = function(timestamp) {
dateElem.textContent = new Date();
onNextFrame(updateDate);
};
onNextFrame(updateDate);
I am trying to make a function that starts in exact intervals to keep stanble update rate. The problem is that it seems to execute in 2 channels. This is the log:
timeElapsed=141; lastDraw=1314040922291
timeElapsed=860; lastDraw=1314040923151
timeElapsed=141; lastDraw=1314040923292
timeElapsed=860; lastDraw=1314040924152
timeElapsed=141; lastDraw=1314040924293
timeElapsed=860; lastDraw=1314040925153
timeElapsed=141; lastDraw=1314040925294
timeElapsed=860; lastDraw=1314040926154
timeElapsed=141; lastDraw=1314040926295
timeElapsed=859; lastDraw=1314040927154
timeElapsed=143; lastDraw=1314040927297
timeElapsed=858; lastDraw=1314040928155
timeElapsed=143; lastDraw=1314040928298
timeElapsed=858; lastDraw=1314040929156
timeElapsed=142; lastDraw=1314040929298
First, I exectute my function using
drawTimer = setTimeout(function(){ draw() }, 1);
and the function looks like this:
var draw = function(){
if(!running)
return;
var miliseconds = getCurrentMiliseconds();
var timeElapsed = miliseconds - lastDraw;
lastDraw = miliseconds;
console.log("timeElapsed=" + timeElapsed + "; lastDraw=" + lastDraw);
onDrawListener(timeElapsed);
if(timeElapsed < timeLapse)
miliseconds = timeLapse - timeElapsed;
else
miliseconds = 1;
drawTimer = setTimeout(function(){ draw() }, miliseconds);
}
It happens in both, Chrome and Firefox. Do you know what is it caused by? And... How to fix it?
P.S. Since everyone seems to be so confused about the running variable, here it is: it's a private parent object member that indicates whether the mechanism is still running or has stopped. It's set by other functions and is just there to make sure this function doesn't continue working after stop() is called.
-- update --
timeLapse is set to 1000 (1 time per seconds) and never changed again.
onDrawListener is set to this function:
function(timeElapsed){
canvas.clear();
moveSnake();
if(snake.body[0] == food){
food = getRandomFreePosition();
++snake.body.lenght;
}
drawBackground();
drawFood();
drawSnake();
}
to explain it: canvas is kinda the engine that takes care of callbacks, key listening and also has a few functions. Other than that seems kinda self-explaining. they do nothing other than some int algorithms and drawing in the canvas.
-- Figured out --
I understood that I should calculate time spent for current function and not since the last one started. My old method worked not in 2 channels but rather in long-short-long-short-long-... delayes
first of all you dont set the running bool and also when you enter the function immediately do a on clearTimeout on drawTimer.
clearTimeout(drawTimer);
In a loop like that, you should consider to write:
if(timeElapsed >= AMOUNT_OF_TIME)
{
// run code
}