I'm creating a 1s audio snippet by programmatically filling a AudioBuffer. The AudioBufferSourceNode has looping enabled. It plays back just fine in Chrome and Firefox.
Now I want to dynamically update the AudioBuffer and have the new audio picked up immediately (or at the next loop). In Chrome this works perfectly by simply getting the channel data (getChannelData(0)) and writing to it. Chrome updates the playing audio on-the-fly. Firefox keeps playing the original buffer over and over again. In fact in Firefox the AudioBuffer needs to be written before assigning it to the AudioBufferSourceNode (source.buffer = buffer).
This should not be done this way. You're trying to update an object across a thread boundary.
Chrome has a bug where we don't currently implement the memory protection (i.e. you can update the contents of the AudioBuffer and it will change what the looping buffer sounds like). FF currently has a different bug, where it allows you to set the .buffer more than once. These should both get fixed.
To address this scenario, you need to loop each buffer until you get then next one, then cross-fade between them. It's unlikely just looping a 1s buffer is really what you want anyway? (unless it's noise.)
Simply assigning the same buffer again will update Firefox's internal state. So just do source.buffer = buffer again after updating the buffer. Even though it should be a NOOP, since it's the exact same reference.
Even source.buffer = source.buffer does the trick.
Related
Ive been playing around with JS normalizer to bring widely different volume recordings to play at a fairly constant volume.
This code is doing it's job quite nicely.
My final goal though is to allow the user to record audio with the mic, then save the normalized audio to a file (I don't need to play it back, And I don't need the original recording once i have the normalized version of it).
Does anyone know how can this be accomplished?
BTW, since i first used this sample chrome now refuses to let the AudioContext object be set without user interaction, so a quick workaround is to move the declaration:
var audioCtx = new AudioContext();
to the beginning of normalizedAudioElement function
Cheers!
I need to implement a spectrogram of WAV files for a tool we're building. It needs to display the spectrogram for the entire file in one go (think Audacity).
Here's a jsfiddle of my starting point. This function only logs in 'playback time'. Is it possible to get the frequency data any other way?
var freqData = new Uint8Array(analyser.frequencyBinCount);
scp.onaudioprocess = function()
{
analyser.getByteFrequencyData(freqData);
console.log(freqData);
};
Use an OfflineAudioContext as the destination instead of an AudioContext. You probably also want to check that the buffer size for the ScriptProcessorNode makes sense for the FFT size of the AnalyserNode.
I'm not sure every browser allows the ScriptProcessorNode to work nicely with an OfflineAudioContext.
This is sort of expanding on my previous question Web Audio API- onended event scope, but I felt it was a separate enough issue to warrant a new thread.
I'm basically trying to do double buffering using the web audio API in order to get audio to play with low latency. The basic idea is we have 2 buffers. Each is written to while the other one plays, and they keep playing back and forth to form continuous audio.
The solution in the previous thread works well enough as long as the buffer size is large enough, but latency takes a bit of a hit, as the smallest buffer I ended up being able to use was about 4000 samples long, which at my chosen sample rate of 44.1k would be about 90ms of latency.
I understand that from the previous answer that the issue is in the use of the onended event, and it has been suggested that a ScriptProcessorNode might be of better use. However, it's my understanding that a ScriptProcessorNode has its own buffer of a certain size that is built-in which you access whenever the node receives audio and which you determine in the constructor:
var scriptNode = context.createScriptProcessor(4096, 2, 2); // buffer size, channels in, channels out
I had been using two alternating source buffers initially. Is there a way to access those from a ScriptProcessorNode, or do I need to change my approach?
No, there's no way to use other buffers in a scriptprocessor. Today, your best approach would be to use a scriptprocessor and write the samples directly into there.
Note that the way AudioBuffers work, you're not guaranteed in your previous approach to not be copying and creating new buffers anyway - you can't simultaneously be accessing a buffer from the audio thread and the main thread.
In the future, using an audio worker will be a bit better - it will avoid some of the thread-hopping - but if you're (e.g.) streaming buffers down from a network source, you won't be able to avoid copying. (It's not that expensively, actually.) If you're generating the audio buffer, you should generate it in the onaudioprocess.
I am writing a simple mpeg-dash streaming player using HTML5 video element.
I am creating MediaSource and attaching a SourceBuffer to it. Then I am appending dash fragments into this sourcebuffer and everything is working fine.
Now, what I want to do is, I want to pre-fetch those segments dynamically depending upon current time of the media element.
While doing this there are lot of doubts and which are not answered by MediaSource document.
Is it possible to know how much data sourceBuffer can support at a time? If I have a very large video and append all the fragments into sourcebuffer, will it accommodate all fragments or cause errors or will slow down my browser?
How to compute number of fragments in sourcebuffer?
How to compute the presentation time or end time of the last segment in SourceBuffer?
How do we remove only specific set of fragments from SourceBuffer and replace them with segments with other resolutions? (I want to do it to support adaptive resolution switching run time.)
Thanks.
The maximum amount of buffered data is an implementation detail and is not exposed to the developer in any way AFAIK. According to the spec, when appending new data the browser will execute the coded frame eviction algorithm which removes any buffered data deemed unnecessary by the browser. Browsers tend to remove any part of the stream that has already been played and don't remove parts of the stream that are in the future relative to current time. This means that if the stream is very large and the dash player downloads it very quickly, faster than the MSE can play it, then there will be a lot of the stream that cannot be remove by the coded frame eviction algorithm and this may cause the append buffer method to throw a QuotaExceededError. Of course a good dash player should monitor the buffered amount and not download excessive amounts of data.
In plain text: You have nothing to worry about, unless your player downloads all of the stream as quickly as possible without taking under consideration the current buffered amount.
The MSE API works with a stream of data (audio or video). It has no knowledge of segments. Theoretically you could get the buffered timerange and map to to a pair of segments using the timing data provided in the MPD. But this is fragile IMHO. Better is to keep track of the downloaded and fed segments.
Look at the buffered property. The easiest way to get the end time in seconds of the last appended segments is simply: videoElement.buffered.end(0)
If by presentation time you mean the Presentation TimeStamp of the last buffered frame then there is no way of doing this apart from parsing the stream itself.
To remove buffered data you can use the remove method.
Quality switching is actually quite easy although the spec doesn't say much about it. To switch the qualities the only thing you have to do is append the init header for the new quality to the SourceBuffer. After that you can append the segments for the new quality as usual.
I personally find the youtube dash mse test player a good place to learn.
The amount of data a sourceBuffer can support depends on the MSE implementation and therefore the browser vendor. Once you reached the maximum value, this will of course result in an error.
You cannot directly get the number of segments in SourceBuffer, but you can get the actual buffered time. In combination with the duration of the segments you are able to compute it.
I recommend to have a look in open source DASH player projects like dashjs or ExoPlayer, which implement all your desired functionalities. Or maybe even use a commercial solution like bitdash.
I'm testing a streaming web application that uses MediaSourceAPI. Everything works fine, however when i stream big files (i.e 240MB or more), the buffer of the video has a strange behavior. To be more clear i attached three images you can check. My script creates a mediaSource object, then it calls addSourceBuffer and then it calls appendBuffer many time as there are chunks to append. I think that i do not configure well the buffer and so the mediaSource API use a default value for the buffer length.
Could you help me please?
Visit https://productforums.google.com/forum/#!category-topic/chrome/report-a-problem-and-get-troubleshooting-help/windows8/Stable/0igRzDJQ7ds
There is a max limit on the size of the SourceBuffers, maybe you're exceeding those? When they exceed the limits, the browsers will start evicting buffer segments according to some defined algorithm.
If you are appending as much data to the source buffers as you can, you might want to introduce a limit. E.g. for us, when playing HD video at 4.5mps, we could have a buffer size of about 3-4 minutes before we saw some odd behaviour (e.g. segments being evicted in front of the videos currentTime)