I'm building a video for my website with HTML5. Ideally, I'd have only one silent video file, and five different audio tracks in different languages that sync up with the video.
Then I'd have a button that allows users to switch between audio tracks, even as the video is playing; and the correct audio track would come to life (without the video pausing or starting over or anything; much like a DVD audio track selection).
I can do this quite simply in Flash, but I don't want to. There has to be a way to do this in pure HTML5 or HTML5+jQuery. I'm thinking you'd play all the audio files at 0 volume, and only increase the volume of the active track... but I don't know how to even do that, let alone handle it when the user pauses or rewinds the video...
Thanks in advance!
Synchronization between audio and video is far more complex than simply starting the audio and video at the same time. Sound cards will playback at slightly different rates. (What is 44.1 kHz to me, might actually be 44.095 kHz to you.)
Often, the video is synchronized to the audio stream, but the player is what handles this. By loading up multiple objects for playback, you are effectively pulling them out of sync.
The only way this is going to work is if you can find a way to control the different audio streams from the video player. I don't know if this is possible.
Instead, I propose that you encode the video multiple times, with the different streams. You can use FFMPEG for this, and even automate the process, depending on your workflow. Switching among streams becomes a problem, but most video players are robust enough to guess the byte offset in the file, when given the bitrate.
If you only needed two languages, you could simply adjust the balance between a left and right stereo audio channel.
If you're willing to let all five tracks download, why not just mux them into the video? Videos are not limited to a single audio track (even AVI could do multiple audio tracks). Then syncing should be handled for you. You'd just enable/disable the audio tracks as needed.
It is doable with Web Audio API.
Part of my program listens to video element events and stops or restarts audio tracks created using web audio API. This gives me an ability to turn on and off any of the tracks in perfect sync.
There are some drawbacks.
There is no Web Audio API support in Internet Explorers except for Edge.
The technique works with buffered audio only and that's limiting. There are some problems with large files: https://bugs.chromium.org/p/chromium/issues/detail?id=71704
Related
I have a web app that records sound and plays music from an embed youtube player (youtube player api). For the audio, I continuously record by starting the recorder after it ends in an infinite loop (speech recognition class). But I want to also play the youtube video at the same time. On a desktop, they both don't interfere with each other (or at least with a headphone and mic). But on my android phone, everytime it starts recording, it pauses the video and then resumes it when the recording ends. Is there a way to make the video not pause?
You would have to break the stream into separate threads, and hope that you have enough power to handle them all.
I say "all" because while the your music stream may only be carried on one thread, the video from YouTube is still there.
In effect, you have to pretend to be the origininator and produce all of the a/v signals separately, then allow the mobile device to consume them (assuming that the signals are properly encoded to begin with).
I need a way to play a live audio stream using HTML5 (primarily in Google Chrome), so I tried using the following:
<audio>
<source src="my-live-stream.ogg" type="audio/ogg">
</audio>
While this does work for a live stream, there seems to be a very large, uncontrollable delay/buffer of around 30 seconds or so when this is played.
I need the delay to be a couple of seconds or less so this method doesn't work.
As an alternative I have tried sending the audio over a WebSocket connection as 1 second individual audio files, which are then appended to a SourceBuffer and played in an audio element using Media Source Extensions.
After experimenting with a number of formats (MediaSource.isTypeSupported seems to be rather limited in audio support), I got this working using a Vorbis audio stream in a WebM container, which sounds perfect with no audible gaps. Other formats worked less well as they need to be gapless - e.g. MP3 and AAC end up with tiny audible gaps between each 1 second segment.
While this seems to work at first, when looking at chrome://media-internals, the following errors repeatedly appear:
00:00:09 544 info Estimating WebM block duration to be 3ms for the last (Simple)Block in the Cluster for this Track. Use BlockGroups with BlockDurations at the end of each Track in a Cluster to avoid estimation.
00:00:09 585 error Large timestamp gap detected; may cause AV sync to drift. time:8994999us expected:9231000us delta:-236001us
00:01:05 239 debug Skipping splice frame generation: not enough samples for splicing new buffer at 65077997us. Have 1us, but need 1000us.
Eventually the playback stops as though the pause button has been pressed on the audio element. It still shows the pause rather than play button, but the current time stops advancing:
Pressing the pause button and then the play button that replaces it doesn't make it start playing again, but manually dragging the position slider further ahead makes it continue playing.
I have tried setting sourceBuffer.mode = 'sequence'; but this doesn't seem to help.
Is there anything that needs to be changed in how the audio files are being encoded, or how they are played back in JavaScript to fix this?
Additional details:
The audio stream is encoded into 1 second WebM/Vorbis files using FFmpeg on Windows.
A background worker is used in the browser to receive the audio segments and pass them to the main page thread, which appends them to the audio stream. Otherwise the playback freezes.
Source code:
Web player: https://github.com/SamuelFisher/WebSocketAudio
WebSocket server and encoder: https://github.com/SamuelFisher/WebSocketAudioServer
I'm building a video player where each scene is filmed from multiple angles. All videos are hosted on YouTube. I'd like to allow the user to be able to switch between angles seamlessly during playback.
To facilitate this, I need a way to load videos from YouTube without playing them. That way I can load alternate angles in the background while one angle is playing. When the user switches angle, the new angle should be at least partially loaded and ready to play immediately.
Unfortunately, I can't find a way to load a video without playing it.
The loadVideoById method autoplays the video as soon as the request to load the video has returned so that won't work.
Is this possible?
There's no way to cue up a video and force it to pre-buffer.
You can load (as opposed to cue) a video and then immediately pause it, and it may or may not pre-buffer, but that's dependent on a number of factors and is outside your control as someone using the API.
Is there a way to change the playback speed of audio in the browser? What is best to accomplish this task, html5 audio, or flash, or something else? Are there any specific libraries that would help with this?
Use the Web Audio API.
In the following code I answered your other question.
best way to loop audio in the browser?
Modify the code in my answer above as follows for a playback speed example.
Right below
source.loop = loopOnOff;
add
source.playbackRate.value = 1; // change number to between 0.10 to 10 (or larger/smaller) to test.
You can also run the html audio tag through the web audio api and add effects processing.
Interesting question there,
HTMl5 will have player speed control will have speed control..
A couple of noteworthy upcoming features are playbackRate and defaultPlaybackRate. As you can probably imagine, these fellas let us alter the speed and direction of playback. This functionality could be used for fast-forward and rewind functions or perhaps to allow users to tweak the playback speed so they can fit more podcasts into their day.
audio.playbackRate returns 1 at normal speed and acts as a multiple that is applied to the rate of playback. For example, setting playbackRate to 2 would double the speed, while setting it to -1 would play the media backwards.
audio.defaultPlaybackRate is the rate at which the audio will play after you pause and restart the media (or issue any event for that matter).
Flash Player may help( but it will be customized one you may create, with stream buffer, you need to define the player speed once buffer has the content to play.
Sound easy but will take a lot effort,
Refer VLC opesource for better Idea, its documented with ffmpeg which works with Audio,
and works with client software, in browser ti will be heavy, refer Just to have idea.
I hope this may help :)
There is a program at enounce.com that it will increase the play speed of a video in a browser. I think that 95% of videos on the internet run on flash therefore this tool can be useful. I am wondering how that program was created. Maybe they modify the html source in the browser? perhaps it looks for the swf video playing on your browser and it injects some JavaScript on that html element to increase the speed. I been researching on Google and I think it is possible to alter the playback speed of a video with JavaScript. If it is not modifying the html page then it will be nice to at least know how this can be achieved. Also if a video plays on your browser it has to be saved somewhere in your computer I believe. That's why you can seek back and forth once the video finished downloading. why is it that it is almost impossible to find it and the only way of getting that video will be by capturing the packages with a package sniffer? anyways that is not my question I am just really curious on how that program achieves doing what it does. it speeds up everything even Pandora songs.
MySpeed seems to intercept the media stream coming from the server into the Flash player that sits in your browser. It changes the speed on the fly, and sends the result to the Flash player.
PS: If you need to control the playing speed of your own videos I recommend looking into the VLC Browser Plug-in, or the QuickTime player, which also has very good speed control features (from Javascript). Or you could use the HTML5 <video> tag.
Afaik, Flash-based players like Longtail/JWPlayer and Nonverbla don't have very good support for this.