We Keep Coding

AudioContext.currentTime sometimes freezes for ~230ms shortly after start

I'm creating a music related app using Web Audio API, which requires precisely synchronizing sound, visuals and MIDI input processing.
The sound production follows the pattern described in this article: requestAnimationFrame regularly calls a function that schedules events on the AudioContext. This works fine most of the time, except in some occasions where the audio inexplicably lags behind the visuals.
After much poking around, I ran into the AudioContext.currentTime specification which hints:
Elapsed time in this system corresponds to elapsed time in the audio stream generated by the BaseAudioContext, which may not be synchronized with other clocks in the system.
And indeed I was able to verify that these sporadic delays in the audio come down to a problem in the AudioContext clock itself, which seems to pause for a bit sometimes just after starting up. Note that this doesn't happen every time, but frequently enough to be an issue (maybe 10-15% of the time...). Each time it happens the pattern is the same: currentTime starts increasing then gets stuck at 23ms for a bit, then starts going again regularly without any further issue after accumulating a total of ~230ms of lag behind the system/wall clock...
I've created a simple script which reproduces the problem (if you want to try it, just open the console to see the output, and press any key to start the test... as the issue is sporadic you may need to retry or sometimes reload multiple times before it happens...):
<script>
const audioContext = new AudioContext();
// reference points for both clocks (JS and audioContext)
var animStartTime = null;
var audioStartTime = null;
// loop function to be called by requestAnimationFrame
function play(timestamp) {
// set animStartTime on first invocation
if (animStartTime == null) animStartTime = timestamp;
// compute elapsed time for both clocks
var animElapsedTime = timestamp - animStartTime;
var audioElapsedTime = (audioContext.currentTime - audioStartTime) * 1000;
console.log('Animation ts: ' + animElapsedTime +
', Audio ts: ' + audioElapsedTime +
', Diff: ' + (animElapsedTime - audioElapsedTime)
);
// keep this going for 1 second
if (animElapsedTime < 1000)
requestAnimationFrame(play);
}
function start() {
audioStartTime = audioContext.currentTime;
animStartTime = null; // use the timestamp provided by requestAnimationFrame
// create a simple oscillator and schedule it to produce a single beep when it starts
const osc = audioContext.createOscillator();
osc.frequency.value = 800;
osc.connect(audioContext.destination);
osc.start(audioStartTime);
osc.stop(audioStartTime + 0.03);
// launch animation loop
requestAnimationFrame(play);
}
// press any key to start the test
window.addEventListener('keypress', function(e) {start()});
</script>
And here is the console output of a bogus run, where you can see the Audio timestamp freezing up to a delay of ~230ms behind the JS main thread timestamp before starting again:
Could someone explain to me:
What is going on? Why is this freezing sporadically?
Is there anything I can do to prevent this? I can think of some ways to mitigate the issue if I can get convinced that it happens only when starting up, but without fully understanding the root cause I fear seeing these freezes happen again at other times...

This is probably https://crbug.com/693978 Resuming the context may take "some" time.
Unfortunately, they don't really wait for the context has started before resolving the Promise returned by context.resume(), so we have to resort to ugly workarounds.
One such workaround would be to wait for currentTime to start updating after context.resume() resolves before starting your animation.
const audioContext = new AudioContext();
// reference points for both clocks (JS and audioContext)
var animStartTime = null;
var audioStartTime = null;
// loop function to be called by requestAnimationFrame
function play(timestamp) {
// set animStartTime on first invocation
if (animStartTime == null) animStartTime = timestamp;
// compute elapsed time for both clocks
var animElapsedTime = timestamp - animStartTime;
var audioElapsedTime = (audioContext.currentTime - audioStartTime) * 1000;
console.log('Animation ts: ' + animElapsedTime +
', Audio ts: ' + audioElapsedTime +
', Diff: ' + (animElapsedTime - audioElapsedTime)
);
// keep this going for 1 second
if (animElapsedTime < 1000)
requestAnimationFrame(play);
}
async function start() {
// wait for the context to resume
// needs to be there, for we still handle the user-gesture
await audioContext.resume();
// though resume() might be a lie in Chrome,
// so we also wait for currentTime to update
const startTime = audioContext.currentTime;
while (startTime === audioContext.currentTime) {
await new Promise((res) => setTimeout(res));
}
// now our AudioContext is ready.
audioStartTime = audioContext.currentTime;
animStartTime = null; // use the timestamp provided by requestAnimationFrame
// create a simple oscillator and schedule it to produce a single beep when it starts
const osc = audioContext.createOscillator();
osc.frequency.value = 800;
osc.connect(audioContext.destination);
osc.start(0);
osc.stop(0.3);
// launch animation loop
requestAnimationFrame(play);
}
// using a button for a clearer snippet
document.querySelector("button")
.addEventListener('click', start);
<button>start</button>

How do I improve the accuracy of the DelayNode?

I'm using the WebAudio API and doing a simple delay on the input using DelayNode. The code is as follows (might not work in the snippet because it requests microphone permissions):
const player = document.querySelector("#player")
const handleSuccess = stream => {
const context = new AudioContext()
const source = context.createMediaStreamSource(stream)
const delay = context.createDelay(179) //seconds
//source.connect(context.destination) connect directly
source.connect(delay)
delay.connect(context.destination)
delay.delayTime.value = 1
}
navigator.mediaDevices.getUserMedia({audio: true, video: false}).then(handleSuccess)
However, when I run a metronome at 60 beats per minute (one click every second), the audio coming from the browser is not delayed by exactly one second, and multiple clicks are heard (the delay is slightly longer than expected). Is there any way to make an exact delay?

I guess the problem is not the DelayNode itself but rather that there are multiple other hidden delays in the audio pipeline. If you want to hear the previous click from the metronome at the very same time as the current click you would need to reduce the time of your delay to account for those other delays (also known as latency).
The signal takes a bit of time to travel from your microphone through your A/D converter into the browser. This time is usually available as part of the settings of the MediaStream
stream.getAudioTracks()[0].getSettings().latency
Piping the MediaStream into the Web Audio API will probably add some more latency. And the AudioContext itself will add some latency due to its internal buffer.
context.baseLatency
It will also take some time to get the signal out of the computer again. It will travel from the browser to the OS which passes it on to the hardware. This value is also exposed on the AudioContext.
context.outputLatency
In theory you would only need to subtract all those values from the delay time and it would just work. However in reality every hardware/OS/browser combination is a bit different and you will probably need to make some adjustments in order to successfully overlay the previous with the current click depending on your personal setup.

You can use the async/await syntax (documentation here), here are two examples (taken from here)
const foo = async () => {
await sleep(1000);
// do something
}
const foo = async evt => {
await sleep(1000);
// do something with evt
}

JS: Calling async function in for loop does not reference the same iterated variable

I am creating a queue to download YouTube videos in Node.js. When the download is triggered all queued videos should start downloading simultaneously. This is handled simply by getting all the queued videos from the database and iterating over them in a for loop. However when my download finishes I cannot mark it as so in the database because because "video" now references the last video in my for loop. How do I resolve this?
...
var videos = db.videos.find({ status: 'queued' });
for (var video of videos) {
alert(video.id); // This is the correct id
var download = ytdl(`http://www.youtube.com/watch?v=${video.id}`);
download.pipe(fs.createWriteStream(filename));
download.on('end', () => {
video.status = 'done';
db.videos.update(video)
alert(video.id); // This is now the id of the last video in videos!
});
}

var video has functional scope. You want let video or const video, which have block scope.
Alternatively, moving the body of the loop to a separate function will create a new scope where video cannot be overwritten:
for (var video of videos) {
downloadVideo(video)
}
function downloadVideo(video) {
// ... your download logic
}

Java script -All audio plays at once when attempting to play audio one by one in a sequence (through a for loop)

Here is the link to code pen to my project https://codepen.io/RajTheSHow/pen/vZZPqZ
when you press the wrong button it is supposed to play the sequence(with audio and button press in order).however what actually happens is when you press the wrong button it plays all the audio and changes the color at once and doesn't execute what is in the sleep function.you can see the problem in the pen.
the function that runs the sequence when you press the wrong button is below
cal is where the order is stored of the sequence.
//plays the sequence
function callBut(){
onr=0;
for(i=0;i<cal.length;i++){
// or eval ("s"+cal[i])
window["s"+cal[i]].play();
// set the but# to Clicked Color Ccol# then after 1 sec go back to normal color
$("[but="+cal[i]+"]").css("background",window["Ccol"+cal[i]])
sleep(500).then(() => {
// set the button# to Standard color Scol#
$("[but="+cal[i]+"]").css("background",window["Scol"+cal[i]])
});
}

What'd you expect? The loop does not wait for the audio to finish. It's better to use recursion to play the next song.
let arr = cal.map(element => window[`s${element}`]); // asuming 'cal' is something significant
let index = 0;
playAudio(arr[index]);
function playAudio(audio) {
if (!audio || !(audio instanceof Audio)) return;
audio.addEventListener('ended', function() {
index++;
playAudio(arr[index]);
})
audio.play();
}

This is the correct and expected behaviour.
What you want to achieve requires to wait for the end event and only then invoke play on the next clip.
This can be achieved by properly handling the audio events you can look up online and "chaining" them playing one at a time, much like this:
var nextSound;
function playNext()
{
var audio;
if (nextSound >= cal.length)
{
return;
}
audio = $(window["s"+cal[nextSound++]]);
audio.on("ended", playNext);
audio[0].play();
}
nextSound = 0;
playNext();

Wait for image loading to complete in JavaScript

I'm loading images with JavaScript. Something like this:
images[0]=new Image();
images[0].onload=function(){loaded++;console.log(loaded)};
images[0].src="assets/img/image.png";
When I look at the log, I see that all the images are loaded nicely, since the value of the "loaded" variable increases with each loaded image.
However I would like to stop any further action to be executed until this amount reaches it's maximum, so right after setting up the images, I place a while cycle.
while(loaded<11){
document.getElementById("test").innerHTML="Loading "+loaded+"/11";
console.log(loaded);
}
//Some code here which should only run after everything has been loaded
//In other words: when the statement in the while cycle becomes false
However my browser simply crashes, since the while seems to be stuck in an infinite loop. When I check the log, I see that "0" was written 1000 times, and after that, the numbers from 1 to 11 (which implies that the images in fact gets loaded, but the while does not care about it, and crashes faster than it could happen).
I believe that the method I'm trying to use here is not the right approach to solve this problem.
How can I put everything on hold until every asset which is needed for the site is loaded?

Using promises and async functions, there is a nice way to wait until all the images are loaded (no callbacks, no loaded image counting):
async function loadImages(imageUrlArray) {
const promiseArray = []; // create an array for promises
const imageArray = []; // array for the images
for (let imageUrl of imageUrlArray) {
promiseArray.push(new Promise(resolve => {
const img = new Image();
// if you don't need to do anything when the image loads,
// then you can just write img.onload = resolve;
img.onload = function() {
// do stuff with the image if necessary
// resolve the promise, indicating that the image has been loaded
resolve();
};
img.src = imageUrl;
imageArray.push(img);
}));
}
await Promise.all(promiseArray); // wait for all the images to be loaded
console.log("all images loaded");
return imageArray;
}
Or you can wait for a single image to load:
async function loadImage(imageUrl) {
let img;
const imageLoadPromise = new Promise(resolve => {
img = new Image();
img.onload = resolve;
img.src = imageUrl;
});
await imageLoadPromise;
console.log("image loaded");
return img;
}
You can use it like this (using promise chaining):
loadImages(myImages).then(images => {
// the loaded images are in the images array
})
Or inside an async function:
const images = await loadImages(myImages);

Personally I hate using while()... I think the easiest way to do it, is using event listeners.
var img = new Image;
img.addEventListener("load", function () {
//Img loaded
});
img.src= e.target.result;

Javascript is single threaded. This means that if you add an event listener that listener will not run until the current execution has completed. Hence if you start a loop that relies on a event to end it it will never happen as the event will never fire because the current execution is preventing it from running. Also the events are place on the call stack asynchronously, thus if your execution is slower than the rate of event firing (placing a call on the call stack) you also risk the a page crash. This is a common mistake when using setInterval when the interval is set to less time than the code takes to execute. NEVER USE setInterval.
Just remember Javascript can not do two things at once.
The best way to deal with resource monitored loading is to use setTimeout.
var allLoaded = false;
var imgCount = 0; // this counts the loaded images
// list of images to load
const imageURLS =["a.jpg","b.jpg","c.jpg","d.jpg","e.jpg"];
// array of images
var images = [];
const onImageLoad = function(){ imgCount += 1; } // onload event
// loads an image an puts it on the image array
const loadImage = function(url){
images.push(new Image());
images[images.length-1].src = url
images[images.length-1].onload = onImageLoad;
}
const waitForLoaded = function(){
if(imgCount === images.length){
allLoaded = true; // flag that the image have loaded
}else{
// display the progress here
...
setTimeout(waitForLoaded,100); // try again in 100ms
}
}
// create the images and set the URLS
imageURLS.forEach(loadImage);
setTimeout(waitForLoaded,100); // monitor the image loading