After starting to play audio from an AudioBuffer using AudioBufferSourceNode, how do I make it stop after a predetermined time in milliseconds? My setup is simple:
// create audio source
var source = audioCtx.createBufferSource();
// play audio from source
source.start();
Using AudioBufferSourceNode.stop() I can only specify this duration in seconds, but supplying anything non-integer will evaluate to zero for some reason, and make the audio stop immediately:
source.stop(0.25); // should be 250 milliseconds, but audio stops immediately
setTimeout is not precise enough, and occasional jittering/drifting occurs, especially if the UI works a lot, or other calculations are being performed elsewhere.
setTimeout(function () {
source.stop(); // very inaccurate
}, 250);
Is there a trivial solution to stop playing an audio after a certain amount of milliseconds, without resorting to hacks like busy wait in a worker thread, or similar?
I think you've misunderstood what the first argument to stop() does.
It represents the amount of time relative to the audio context time, which means that when calling stop(), you need to add that time to it:
source.stop(context.currentTime + 0.25);
Since the function accepts a double, it shouldn't be rounding it to the nearest second. According to the Web Audio API specification:
The when parameter describes at what time (in seconds) the sound should stop playing. It is in the same time coordinate system as the AudioContext's currentTime attribute. If 0 is passed in for this value or if the value is less than currentTime, then the sound will stop playing immediately.
Related
I want to play a HTML5 video in segments of x seconds. For example, start at t=0 and pause at t=x, then start at t=x and pause at t=2x. The problem is that the updates I receive with ontimeupdate don't match my intervals. There is one update short before the intended stop time and one update shortly after. I could just stop the video whenever currentTime >= x, but the problem here is that this stop point would fall into the new interval. The requirement for this task is to stop at the end of a given interval.
If stopping exactly at a given time is not possible, is there any way to determine the closest possible stop time before that time? That would still be better than stopping too late. I checked the deltas of currentTime (time between each ontimeupdate call), but these are not constant. I read somewhere else that the browser adapts this rate based on some optimization criterions, so that is probably hard to compute.
Background for this question is a tool that I want to develop. The user is shown a video and he is required to answer some questions for each x second interval of this video.
Unfortunately the timeupdate event doesn't provide a lot of granularity. You can read this answer for more information.
One alternative is to set up an interval manually with setInterval, and on each interval check the time passed with Date.now() since the last time that the timeupdate was updated (also using Date.now() and saving that value somewhere).
That would allow knowing the milliseconds from then, although it would need to handle cases like pausing the video and clearing the interval when necessary to avoid memory leaks with clearTimeout.
I want to create an application that when launched on different devices will display the same content (music, photo or video) at the same time.
Here is simple example.
And real-life example. :)
My first idea was based on machine local time.
timestamp = new Date().getTime()
(timestamp(\d{4}$) === "0000") => play music and animation
music = 10s,
animation = 10s
and for every every 10 seconds, start this function.
I know, however, that this solution may not work and the content will still be unsynchronized.
So, does anyone know how to achieve the effect I'm talking about using javascript?
I actually had the same idea as you had and implemented a little proof of concept for it.
You can try it out like this:
Open the deployed site (either in two browsers/windows or even on different devices)
Choose the same unique channel-id for all your opened sites
Click "Start"
One window should have a heading with "Leader". The others should be "Follower"s. By clicking on the Video-id field you can paste the video id of any youtube video (or choose from a small list of recommended ones).
Click "Play" on each window
Wait a bit - it can take up to 1 minute until the videos synchronize
Each follower has a "System time offset". On the same device it should be nearly 0ms. This is the time that the system-time (Date.now()) in the browser differs from the system time on the Leader window.
On the top left of each video you can see a time that changes every few seconds and should be under 20ms (after the videos are synchronized). This is the time the video-feed differs from its optimal time in relation to the system time.
(I would love to know wether it works for you too. My Pusher deployment is EU-based so maybe problems with the increased latency could occur...)
How does it work?
The synchronisation happens in two steps:
Step 1.: Synchronizing the system times
I basically implemented the NTP (Network time protocol) Algorithm in JS via websockets or Pusher JS as my channel of communication between each Follower- and the Leader-clients. Look under "Clock synchronization algorithm" in the Wikipedia article for more information.
Step 2.: Synchronizing the video feed to the "reference time"
At the currentTime (= synchronized system time) we want the currentVideoTimeto be at currentTime % videoLength. Because the currentTime or system time has been synchronized between the clients in Step 1 and the videoLength is obviously the same in all the clients (because they are supposed to play the same video) the currentVideoTime is the same too.
The big problem is that if I would just start the video at the correct time on all clients (via setTimeout()) they probably wouldn't play at the same time, because one system has e.g. network problems and the video still buffers or another program wants in this moment the processing power of the OS. Depending on the device the time from calling the start function of the video player and the actual starting of the video differs too.
I'm solving this by checking every second wether the video is at the right position (= currentTime % videoLength). If the difference to the right position is bigger than 20ms, I'm stopping the video, skipping the video to the position where it should be in 5s + the time it was late before and start it again.
The code is a bit more sophisticated (and complicated) but this is the general idea.
sync-difference-estimator
synchronized-player
I'm working on a web-based music sequencer/tracker, and I've noticed that in my sample playback routine, audio contexts seem to exist only for the duration of of a sample, and that the Web Audio API doesn't seem to adjust playback duration when I pitchshift a sample. For instance, if I shift a note down an octave, the routine only plays the first half of the sound before cutting off. More intense pitch downshifts result in even less of the sound playing, and while I'm not sure I can confirm this, I suspect that speeding up the audio results in relatively long periods of silence before the sound exits the buffer.
Here's my audio playback routine at the moment. So far, a lot more work has gone into making sure other functions send the right data to this than into extending the functionality of this routine.
function playSound(buffer, pitch, dspEffect, dspValue, volume) {
var source = audioEngine.createBufferSource();
source.buffer = buffer;
source.playbackRate.value = pitch;
// Volume adjustment is handled before other DSP effects are added.
var volumeAdjustment = audioEngine.createGain();
source.connect(volumeAdjustment);
// Very basic error trapping in case of bad volume input.
if(volume >= 0 && volume <= 1) {
volumeAdjustment.gain.value = volume;
} else {
volumeAdjustment.gain.value = 0.6;
}
switch(dspEffect){
case 'lowpass':
var createLowPass = audioEngine.createBiquadFilter();
volumeAdjustment.connect(createLowPass);
createLowPass.connect(audioEngine.destination);
createLowPass.type = 'lowpass';
createLowPass.frequency.value = dspValue;
break;
// There are a couple of other optional DSP effects,
// but so far they all operate in about the same fashion.
}
source.start();
}
My intent is for samples to play back fully no matter how much pitch shifting is applied, and to limit the amount of pitch shifting allowed if necessary. I've found that appending several seconds of silence to a sound works around this issue, but it's cumbersome due to the large amount of sounds I would need to process, and I'd prefer a code-based solution.
EDIT: Of the browsers I can test this in, this only appears to be an issue in Google Chrome. Samples play back fully in Firefox, Internet Explorer does not yet support the Web Audio API, and I do not have ready access to Safari or Opera. This definitely changes the nature of the help I'm looking for.
I've found that appending several seconds of silence to a sound works around this issue, but it's cumbersome due to the large amount of sounds I would need to process, and I'd prefer a code-based solution.
You could upload a sound file that is just several seconds of silence and append it to the actual audio file. Here is an SO answer that shows how to do this...
I have a server application which renders a 30 FPS video stream then encodes and muxes it in real-time into a WebM Byte Stream.
On the client side, an HTML5 page opens a WebSocket to the server, which starts generating the stream when connection is accepted. After the header is delivered, each subsequent WebSocket frame consists of a single WebM SimpleBlock. A keyframe occurs every 15 frames and when this happens a new Cluster is started.
The client also creates a MediaSource, and on receiving a frame from the WS, appends the content to its active buffer. The <video> starts playback immediately after the first frame is appended.
Everything works reasonably well. My only issue is that the network jitter causes the playback position to drift from the actual time after a while. My current solution is to hook into the updateend event, check the difference between the video.currentTime and the timecode on the incoming Cluster and manually update the currentTime if it falls outside an acceptable range. Unfortunately, this causes a noticeable pause and jump in the playback which is rather unpleasant.
The solution also feels a bit odd: I know exactly where the latest keyframe is, yet I have to convert it into a whole second (as per the W3C spec) before I can pass it into currentTime, where the browser presumably has to then go around and find the nearest keyframe.
My question is this: is there a way to tell the Media Element to always seek to the latest keyframe available, or keep the playback time synchronised with the system clock time?
network jitter causes the playback position to drift
That's not your problem. If you are experiencing drop-outs in the stream, you aren't buffering enough before playback to begin with, and playback just has an appropriately sized buffer, even if a few seconds behind realtime (which is normal).
My current solution is to hook into the updateend event, check the difference between the video.currentTime and the timecode on the incoming Cluster
That's close to the correct method. I suggest you ignore the timecode of incoming cluster and instead inspect your buffered time ranges. What you've received on the WebM cluster, and what's been decoded are two different things.
Unfortunately, this causes a noticeable pause and jump in the playback which is rather unpleasant.
How else would you do it? You can either jump to realtime, or you can increase playback speed to catch up to realtime. Either way, if you want to catch up to realtime, you have to skip in time to do that.
The solution also feels a bit odd: I know exactly where the latest keyframe is
You may, but the player doesn't until that media is decoded. In any case, keyframe is irrelevant... you can seek to non-keyframe locations. The browser will decode ahead of P/B-frames as required.
I have to convert it into a whole second (as per the W3C spec) before I can pass it into currentTime
That's totally false. The currentTime is specified as a double. https://www.w3.org/TR/2011/WD-html5-20110113/video.html#dom-media-currenttime
My question is this: is there a way to tell the Media Element to always seek to the latest keyframe available, or keep the playback time synchronised with the system clock time?
It's going to play the last buffer automatically. You don't need to do anything. You're doing your job by ensuring media data lands in the buffer and setting playback as close to that as reasonable. You can always advance it forward if a network condition changes that allows you to do this, but frankly it sounds as if you just have broken code and a broken buffering strategy. Otherwise, playback would be simply smooth.
Catching up if fallen behind is not going to happen automatically, and nor should it. If the player pauses due to the buffer being drained, a buffer needs to be built back up again before playback can resume. That's the whole point of the buffer.
Furthermore, your expectation of keeping anything in-time with the system clock is not a good idea and is unreasonable. Different devices have different refresh rates, will handle video at different rates. Just hit play and let it play. If you end up being several seconds off, go ahead and set currentTime, but be very confident of what you've buffered before doing so.
I'm making a simple game in Javascript, in which when an object collides with a wall, it plays a "thud" sound. That sound's loudness depends on the object's velocity (higher velocity => louder sound).
The play function:
playSound = function(id, vol) //ID of the sound in the sounds array, volume/loudness of the sound
{
if (vol) //sometimes, I just want to play the sound without worrying about volume
sounds[id].volume = vol;
else
sounds[id].volume = 1;
sounds[id].play();
}
How I call it:
playSound(2, Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV); //self.TV stands for terminal velocity. This calculates the actual speed using the basic Pythagora's theorem and then divides it by self.TV, which results in a number from 0 to self.TV. 2 is the id of the sound I want to play.
In Chrome, things work quite well. In Firefox, though, each time a collision with a wall happens (=> playSound gets called), there's a pause lasting almost half a second! At first, I thought that the issues were at Math.sqrt, but I was wrong. This is how I tested it:
//playSound(2, 1); //2 Is the id of the sound I want to play, and 1 is max loudness
Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
This completely removed the collision lag, and lead me to believe that Math.sqrt isn't causing any problems at all. Just to be sure, though, I did this:
playSound(2, 1); //2 Is the id of the sound I want to play, and 1 is max loudness
//Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
//Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
//Math.sqrt(p.vx*p.vx + p.vy*p.vy)/self.TV;
And the lag was back! Now I'm sure that playing a sound causes problems. Am I correct? Why is this happening? How do I fix it?
I ran into this same delay issue making a sound when the player fires a weapon. My solution was two-fold:
Play each sound at load time and then pause it immediately. This will allow it to resume playing quickly, rather than playing from scratch. Do this play-pause technique after every play of the sound.
Use a pool of <audio> objects for each sound, rather than a single audio object for each sound type. Instead of just using sounds[id], use a 2D array, accessed with sound[id][count]. Here, sound[id] is a list of audio objects that all have the same sound, and count is the index of current object in use for that sound id. With each call to playSound(id), increment the count associated with that id, so that the next call invokes a different audio object.
I had to use these together, because the play-pause technique does a good job of moving the buffering delay to before the sound needs to be played, but if you need the sound rapidly, you'll still get the delay. This way, the most recently used audio object can "recharge" while another object plays.
Two things that might help you is to either utilize Web workers or to precompute several levels of loudness in advance, which you could also do in the background with worker threads. I'm saying this without going into the peculiarities of the Web audio API or how your computing the sound output, but if you've exhausted all other approaches this might be the next direction you should be focusing on.