I'm doing a project where I have a server offering up audio via WebSocket to the browser. The browser is then using WebSocket to capture the audio blocks and play them out via the Web Audio API.
I create an AudioContext which then has a predefined sample rate of X to play back this audio on the local hardware. Fair enough. However, the sample rate of the audio I am getting from the server is Y (which is not necessarily the same value as X, but I do know what it is via the WebSocket protocol I have defined).
After doing a lot of digging I can't seem to see a way in which the input audio would be resampled via Web Audio itself. I can't imagine this is an uncommon problem but since I'm not quite an expert JavaScript coder I have yet to discern the solution.
Just assigning the samples to getChannelData only really works when X==Y. Otherwise, the audio skips and is at the wrong pitch. Using decodeAudioData allows me access to do the resampling, it seems, but I don't want to have to write my own resampler.
Thoughts?
Confused, are you needing to use decodeAudioData?
If you just have the raw PCM data transferred, you just create an AudioBuffer in your local AudioContext at the served buffer's sampleRate (i.e. you set the third parameter of http://webaudio.github.io/web-audio-api/#widl-AudioContext-createBuffer-AudioBuffer-unsigned-long-numberOfChannels-unsigned-long-length-float-sampleRate based on the data you're getting), and call start() with an appropriate time (and a playbackRate of 1). You WILL get minor glitches at block boundaries, but only minor ones.
There's an active discussion on this, watch issue: https://github.com/WebAudio/web-audio-api/issues/118.
Related
I'm trying to build an app that will take mic input, modulate it, and then broadcast it as an audio source that my web browser can then use.
Use cases for this would be like on a video call. If I need to change my voice to artificially make it louder or perform other modulations, I would use this app.
Right now I have the first 2 steps of getting the mic input and modulating it done, but I'm stuck on the third. How do I broadcast it back to the system as a viable audio source (or in the terms of the app, how to do I get the video call to pick up the sound I'm outputting)?
Some of the ideas I've seen so far have been trying to play the modulated sound back into the microphone object but I don't see a reliable way to do that.
In summary: Is there any way with Javascript (in the browser) or NodeJS to take a sound and register or broadcast it as an audio source that the browser can use?
I don't even know if this is possible and if the answer is "no", then please answer that. There might be some security issues with this that I am totally overlooking. In the end, this question is if nothing, conceptional.
I am trying to make a live spectral plot of a SHOUTcast audio stream. I have found this page http://www.aerodynes.fr/2014/04/14/a-pure-javascript-audio-waterfall/ of someone doing almost exactly what I would like but with the audio from the sound card. How do I open a SHOUTcast stream for processing in the same way as he did? I can't seem to find info on it in the Web Audio API
// Open the microphone
function init() {
var audioConstraints = {
audio: true
};
getUserMedia(audioConstraints, gotStream);
}
...
Thanks for any advice/info.
You can't directly. You'll have to MacGyver some solution.
The first problem you'll run into is that you cannot connect directly to a SHOUTcast stream from most browsers. While SHOUTcast is essentially HTTP, there is one small difference that breaks compatibility, especially with more modern clients. A normal HTTP server returns a status line in its response like this:
HTTP/1.1 200 OK
SHOUTcast servers return this:
ICY 200 OK
The only way around this (assuming you need to still use SHOUTcast) is to proxy the data server-side while rewriting the response status line.
The next problem is that SHOUTcast/Icecast streams use a codec, usually MP3 or AAC in ADTS, to compress the audio into bandwidth suitable for internet streaming. The Web Audio API deals with floating point PCM samples. You will have to decode the audio stream. While this can often be done in-browser, it depends on the codec you're using. Otherwise, you'll have to do it server-side at which point you might as well do the spectrum analysis server side and stream frequency band values.
I think that the best way to handle this is to get your stream playing in an audio element or object and use that as a Web Audio API node which is then connected to your analyser node to get the spectrum. You'll want to use Icecast for your server, and you will have to transcode your streams to a couple codecs to get broader browser support.
I'm developing a collaborative audio recording platform for musicians (something like a cloud DAW married with GitHub).
In a nutshell, a session (song) is made of a series of audio tracks, encoded in AAC and played through HTML5 <audio> elements. Each track is connected to the Web Audio API through a MediaElementAudioSourceNode and routed through a series of nodes (gain and pan, at the moment) until the destination. So far so good.
I am able to play them in sync, pause, stop and seek with no problems at all, and successfully implemented the usual mute, solo functionalities of the common DAW, as well as waveform visualization and navigation. This is the playback part.
As for the recording part, I connected the output from getUserMedia() to a MediaStreamAudioSourceNode, which is then routed to a ScriptProcessorNode that writes the recorded buffer to an array, using a web worker.
When the recording process ends, the recorded buffer is written into a PCM wave file and uploaded to the server, but, at the same time, hooked up to a <audio> element for immediate playback (otherwise I would have to wait for the wav file to be uploaded to the server to be available).
Here is the problem: I can play the recorded track in perfect sync with the previous ones if I play them from the beginning, but I can't seek properly. If I change the currentTime property of the newly recorded track, it becomes messy and terribly out of sync — I repeat that this happens only when the "local" track is added, as the other tracks behave just fine when I change their position.
Does anyone have any idea of what may be causing this? Is there any other useful information I can provide?
Thank you in advance.
Fundamentally, there's no guarantee that elements will sync properly. If you really want audio to be in sync, you'll have to load the audio files into AudioBuffers and play them with BufferSourceNodes.
You'll find in some relatively straightforward circumstances you can get them to sync - but it won't necessarily work across devices and OSes, and once you start trying to seek, as you found, it will fall apart. The way wraps downloading, decoding and playing into one step doesn't lend itself to syncing.
I want to extract audio from a video file using javascript, is it possible to make a mp3 file or any format so that i can play it in an audio tag in html5 ? if yes how? OR what API should I use ?
JavaScript is a script running at the client-end for handling user interactions usually. Long computations cause the browser to halt the script anyway and can turn the page unresponsive. Besides, it is not suitable for video encoding/decoding.
You should use a Java applet for video processing/audio extraction. That way, processing is still offloaded to the client. Processing on the server is expensive in terms of memory and processor cycles. Not Recommended.
Sooo, this is NOT an optimal solution. It is sort of terrible actually since you will forcing users to download a video without seeing it. What you could do is play the file as HTML5 video and just hide the video display then show controls as if it were just audio. This is sort of like playing a video, but just not watching the video while listening to the audio.
Checkout jPlayer: http://jplayer.org/
It will let you play an MP4 basically on any browser by telling it to use flash then falling back to HTML5 when you need it to. Then you can just style things how you want using CSS/HTML/Images.
You might want to try a small test first to see if the display:none; css style will also prevent the video's audio from playing. If that is the case you could also use height:0;;
Once again, this is sub-optimal since you are taking up the user's bandwidth. There are other solutions out there that could potentially strip out just the audio for you in a streaming environment.
I am using audio element to stream an ogg source from an icecast server. Does the audio element extract icy metadata and is there a way to access it through javascript?
Basically, I want to show "currently playing" information of a radio station I am streaming. What are my options?
Thanks!
No, your HTML5 elements are only aware of the stream data, and aren't even making the appropriate request to fetch the metadata.
See these posts:
Developing the client for the icecast server
Pulling Track Info From an Audio Stream Using PHP
http://www.smackfu.com/stuff/programming/shoutcast.html
Now, this isn't impossible by any means. You just have to do it server-side. (See that second link in particular.)
I should also point out that on a full-blown SHOUTcast Server (haven't tested with Icecast, but its worth a try) generates "7.html" which contains data on the number of listeners, max listeners, peak listeners, stereo/mono, bitrate, and current track name as comma-separated values. Like this:
2,1,33,625,2,128,J Mascis - Not Enough
If you can fetch http://yourstreamingserver:port/7.html, then you can get this data very easily.
In case of an Ogg container stream (So both Vorbis and Opus), at least Firefox supports javascript access to the metadata. This is currently only "vendor specific" Javascript API: audio.mozGetMetadata();
On the other hand if you go with Icecast 2.4.1 you can access metadata independently through a JSON API. Lack of synchronization as pointed out in other comments applies. It's usually "close enough" though.