Audio track looping in Javascript without a delay - javascript

I'm currently helping a friend develop a web application in which I need ~6 audio tracks (all using the same time signature) to continuously loop and stay in sync. To give context, it is a typeface-music pairing application where as soon as a typeface is chosen, the associated audio loop starts playing and as the user keeps picking typefaces, the tracks layer and begin to resemble a song.
I've tried using SoundJS and the Buzz sound library, but I keep running into the same problem: there is always a slight delay between loops. This would be fine if all my audio tracks were the same length, but they aren't, so very quickly things go out of sync.
This seems to be a known problem, but I can't seem to find any answer to how to fix it. I came across Hivenfour's SeamlessLoop 2.0, but - unless I'm using it completely wrong - it doesn't actually seem to work (setting a volume returns an error).
If anyone has experience with this, I would truly appreciate any input! Thanks :)

SoundJS WebAudioPlugin uses a look ahead approach with web audio that will loop seamlessly, which is described here in a what will probably be a very helpful on audio timing.
Also be aware that some compression formats will insert white noise into sounds. I believe mp3 does this. WAV is supported broadly and does not.
As for HTMLAudioPlugin, we loop as smoothly as the browser will allow but it does not have the same precision as WebAudio.
Hope that helps.

Related

Is there any way to compensate for drift between the css event.elapsedTime with the time passed in the play method of an AudioContext note?

From my tests, and from searching to find out more about the problem, my best guess is that css animations may be using a different physical clock from the one used to stream audio. If so perhaps the answer to this is that it can't be done, but am asking in case I am missing anything.
It is for my online metronome here.
It is able to plays notes reasonably accurately in response to an event listener for the css animationiteration event. The eventlistener is set up using e.g.
x.addEventListener("animationstart",playSoundBall2);
See here.
However if I try to synchronize the bounce with the sample precise timing of the AudioContext method that's when I run into problems.
What I do is to use the first css callback just to record the audio context time for the css elapsed time of 0. Then I play the notes using the likes of:
oscillator.start(desired_start_time);
You can try it out with the option on the page: "Schedule notes in advance for sample-precise timing of the audio" on the page here.
You can check how much it drifts by switching on "Add debug info to extra info" on the same page.
On my development machine it works fine with Firefox. But on Edge and Chrome it drifts away from the bounce. And not in a steady way. Can be fine for several minutes and then the bounce starts to slow down relative to the audio stream.
It is not a problem with browser activity - if I move the browser around and try to interrupt the animation the worst that happens is that it may drop notes and if the browser isn't active it is liable to drop notes. But the ones it plays are exactly in time.
My best guess so far, is that it might be that the browser is using the system time, while the audiocontext play method is scheduling it at a precise point in a continuous audio stream. Those may well be using different hardware clocks, from online searches for the problem.
Firefox may for some reason be using the same hardware clock, maybe just on my development machine.
If this is correct, it rather looks as if there is no way to guarantee to precisely synchronize html audio played using AudioContext with css animations.
If that is so I would also think you probably can't guarantee to synchronize it with any javascript animations as it would depend on which clocks the browser uses for the animations, and how that relates to whatever clock is used for streaming audio.
But can this really be the case? What do animators do who need to synchronize sound with animations for long periods of time? Or do they only ever synchronize them for a few minutes at a time?
I wouldn't have noticed if it weren't that the metronome naturally is used for long periods at a time. It can get so bad that the click is several seconds out from the bounce after just two or three minutes.
At other times - well while writing this I've had the metronome going for ten minutes in my Windows 10 app and it has drifted, but only by 20-30 ms relative to the bounce. So it is very irregular, so you can't hope to solve this by adding in some fixed speed up or slow down to get them in time with each other.
I am writing this just in case there is a way to do this in javascript, anything I'm missing. I'm also interested to know if it makes any difference if I use other methods of animation. I can't see how one could use the audio context clock directly for animation as you can only schedule notes in the audio stream, can't schedule a callback at a particular exact time in the future according to the audio stream.

Web Audio - How to change playbackRate of all sounds instantly?

When using Web Audio, you can connect all sounds you create to one globally created gainNode and use that node to have a "Master Volume" property. This is very handy when you want to be able to change the master volume on the fly and want it to affect all sounds immediately.
Now, I am trying to accomplish the same, but for playbackRate. For reference: this would be for a web game where you can use a power-up to slow down time, which should also slow down all music and sounds.
Each sound I create is a AudioBufferSourceNode linked to a chain of processing nodes. Now I know that the the AudioBufferSourceNode itself has a playbackRate property you can change. This is great, but it'd require me to cache all AudioBufferSourceNodes I create, loop over them and change their playbackRate if I wanted to change a "global playbackRate" on the fly. It'd be perfect if I could accomplish this in the same way as with the global gainNode, but couldn't find a way to do that.
What would be the proper way to implement such a feature? Would you recommend caching all AudioBufferSourceNodes created (can be thousands) and loop over them? That's the way I do this with HTML5 Audio, but it seems hacky for Web Audio, which is much more advanced.
If you want more information, please ask and I will update the question!
You can't directly do that. There are some source nodes that don't have playback rate controls - like live input. In this case, you're best off doing what you suggest - keeping a list of active sounds to loop through.
You could use a granular method to resample and pitch-bend it down - like the "pitch bend" code in my audio input effects demo (https://webaudiodemos.appspot.com/input/). That's a bit costly to keep around just in case you want to make the effect, though.

How to perfectly sync two or more html5 video tags?

Is there any way to have two or more (preferably three) html5 < video > tags playing simultaneously and to be in perfect sync.
If I have let's say three tiles of one video and I want them to appear in browser as one big video. They need to be perfectly synchronized. Without even smallest visual/vertical hint that they are tiled.
Unfortunately I cannot use MediaController because it is not supported well enough.
I've tried some workouts, including canvases, but I still get visual differentiation. Has anyone had any similar problem/solution?
Disclaimer: I'm not a video guy, but here are some thoughts anyway.
If they need to be absolutely perfect...you are fighting several problems at once:
A device might not be powerful enough to acquire, synchronize and render 3 streams at once.
Even if #1 is solved, a device is never totally dedicated to your task. For example, it might pause for garbage collection between processing stream#1 and stream#2--resulting in dropped/unsynchronized frames.
So to give yourself the best chance at perfection, you should first merge your 3 videos into 1 vertical video in the studio (or using studio software).
Then you can use the extended clipping properties of canvas context.drawImage to break each single frame into 2-3 separate frames.
Additionally, buffer a few frames you acquire on the stream (this goes without saying!).
Use requestAnimationFrame (RAF) to control the drawing. RAF does a fairly good job of drawing frames when system resources are available and delaying frames when system resources are lacking.
Your result won't be perfect, but they will be synchronized. You will always have to make the decision whether to drop or delay frames when system resources are unavailable, but at least the frames you do present will be synchronized.
As far as I know it's currently impossible to play HTML5 video frame-by-frame, or seek to a frame accurate time-code. The nearest seek seems to be precise to roughly 1-second.
But you can still get pretty close using the some of the media frameworks:
Popcorn.js library made for synchronizing video with content.
mediagroup.js another library used to add support for mediagroup attributes on HTML5 media elements
The only feature that allowed that is named mediaGroup and it was removed from Chrome(apparently for not being popular enough). It's still present in WebKit. Relevant discussion here and here.
I think you can implement you own "mediagroup"-like tag using wasm though without DOM support it may be tricky.

variable speed control for audio playback in the browser?

Is there a way to change the playback speed of audio in the browser? What is best to accomplish this task, html5 audio, or flash, or something else? Are there any specific libraries that would help with this?
Use the Web Audio API.
In the following code I answered your other question.
best way to loop audio in the browser?
Modify the code in my answer above as follows for a playback speed example.
Right below
source.loop = loopOnOff;
add
source.playbackRate.value = 1; // change number to between 0.10 to 10 (or larger/smaller) to test.
You can also run the html audio tag through the web audio api and add effects processing.
Interesting question there,
HTMl5 will have player speed control will have speed control..
A couple of noteworthy upcoming features are playbackRate and defaultPlaybackRate. As you can probably imagine, these fellas let us alter the speed and direction of playback. This functionality could be used for fast-forward and rewind functions or perhaps to allow users to tweak the playback speed so they can fit more podcasts into their day.
audio.playbackRate returns 1 at normal speed and acts as a multiple that is applied to the rate of playback. For example, setting playbackRate to 2 would double the speed, while setting it to -1 would play the media backwards.
audio.defaultPlaybackRate is the rate at which the audio will play after you pause and restart the media (or issue any event for that matter).
Flash Player may help( but it will be customized one you may create, with stream buffer, you need to define the player speed once buffer has the content to play.
Sound easy but will take a lot effort,
Refer VLC opesource for better Idea, its documented with ffmpeg which works with Audio,
and works with client software, in browser ti will be heavy, refer Just to have idea.
I hope this may help :)

Javascript Game: What should i know?

I am thinking as a challenge i should write a javascript based game. I want sound, images and input. A background to simulate a screen (like 640x480 with all my images in it) would be useful to separate the rest of the page from the 'game'. What should i look at?
Some things i would need
Framecontrol. A way to get the current time (or delta).
Image, displaying it and moving it. How do i display full image. Knowing pixel access may be cool.
Input A way to lock it in a box (like flash does) is cool.
Sound play simple sounds on demand (like when i get a hit). Several sounds at once would be great
Bottlenecks. What are things that will kill the CPU?
Restrictions. What cant i do? I hear i cant 'sleep' to wait. I must set a callback
Good or best pratice. What are good things i can do to either keep speed up or to lower glitch or compatibility problems.
I'm going to answer this looking at things from a mootools javascript perspective:
Framecontrol. A way to get the current time (or delta).
periodical()
Image, displaying it and moving it. How do i display full image.
setStyles()
Input A way to lock it in a box (like
flash does) is cool.
Plain old CSS
Sound play simple sounds on demand
(like when i get a hit).
Swiff, remote();
Bottlenecks. What are things that will
kill the CPU?
Internet Explorer.
Restrictions.
3D ... ?
What are good things i can do ... to
lower glitch or compatibility
problems.
Use a framework.
As a starting point, you may want to write it for Opera, as Opera provides a game canvas that will help you out.
For some examples of games in javascript:
http://dev.opera.com/articles/view/3d-games-with-canvas-and-raycasting-part/
http://my.opera.com/WebApplications/blog/show.dml/200788
This one is interesting, it is demos of games using the canvas element.
http://www.canvasdemos.com/tag/games/
The best way to see where the problems are is to start writing the game, and then you will see what may be a problem. By looking at demos you can get an idea what performance issues they encountered. For example, a full 3D Doom game will have problems, but, as the first article above explains, there are some ways to optimize for javascript.
Once you get it working with Opera, then you can look at Firefox 3.5+ and Safari, as well as Chrome, and see if you can make some changes to have it work on those. How many platforms it works on depends on how much work you want to do for it. For a proof-of-concept pick the easiest browser for your task.
The best place to start would be to get very familiar with the <canvas> tag (it allows you to draw anything on screen)
This may help a lot:
http://benfirshman.com/projects/jsnes/
its an online NES emulator that renders everything on screen - the source is also available
Hope that helps =)

Categories