I'm trying write a web application that takes information from the user, generates audio on the server from that information, and then plays it back in the user's browser. I've been googling a whole bunch, and I'm kind of unsure exactly what it is that I need to do to get this to happen. What is it that programs like Icecast are doing "behind the scenes" to create these streams? I feel a little bit like I don't even know how to ask the right question or search as almost all the information I'm finding is either about serving files or assumes I know more than I do about how the server side of things works.
This question may help with how to generate music programmatically; it suggests several tools that are designed for this purpose: What's a good API for creating music via programming?
Icecast is a bit of a red herring - that is the tool to broadcast an audio stream, so really you'd be looking at feeding the output of whatever tool you use to generate your music into Icecast or something similar in order to broadcast it to the world at large. However, this is more for situations where you want a single stream to be broadcast to multiple users (e.g. internet radio). If you simply want to generate audio from user input and serve it back to that user, then this isn't necessary.
I'm aware this isn't a full answer, as the question is not fully formed, but I couldn't fit it all into a comment. Hopefully it should help others who stumble upon this question... I suspect the original question writer has moved on by now.
Just have look at Media source API( under implementation). this would be what you are required.
window.MediaSource = window.MediaSource || window.WebKitMediaSource;
var ms = new MediaSource();
var audio = document.querySelector('audio');
audio.src = window.URL.createObjectURL(ms);
ms.addEventListener('webkitsourceopen', function(e) {
...
var sourceBuffer = ms.addSourceBuffer('type; codecs="codecs"');
sourceBuffer.append(oneAudioChunk); //append chunks of data
....
}, false);
Related
Ive been playing around with JS normalizer to bring widely different volume recordings to play at a fairly constant volume.
This code is doing it's job quite nicely.
My final goal though is to allow the user to record audio with the mic, then save the normalized audio to a file (I don't need to play it back, And I don't need the original recording once i have the normalized version of it).
Does anyone know how can this be accomplished?
BTW, since i first used this sample chrome now refuses to let the AudioContext object be set without user interaction, so a quick workaround is to move the declaration:
var audioCtx = new AudioContext();
to the beginning of normalizedAudioElement function
Cheers!
In WebRTC we have MediaStream and MediaStreamTrack interfaces.
MediaStreamTrack represents a audio or video stream of a media source. So a consumer like video or audio tag can simply take an MediaStreamTrack object and and get the stream from it. So what is the need for MediaStream interface?
According to official documentation, MediaStream synchronises one or more tracks. Does that mean it combines multiple streams from tracks and produces a single stream so that we have video data with audio?
For example: Does a video tag read the stream from MediaStream object or reads streams from the individual tracks?
This concept is not explained clearly anywhere.
Thanks in advance.
MediaStream has devolved into a simple container for tracks, representing video and audio together (a quite common occurrence).
It doesn't "combine" anything, it's merely a convenience keeping pieces together that need to be played back in time synchronization with each other. No-one likes lips getting out of sync with spoken words.
It's not even really technically needed, but it's a useful semantic in APIs for:
Getting output from hardware with camera and microphone (usually video and audio), and
Connecting it (the output) to a sinc, like the html video tag (which accepts video and audio).
Reconstituting audio and video at the far end of an RTCPeerConnection that belong together, in the sense that they should generally be played in sync with each other (browsers have more information about expectations on the far end this way, e.g. if packets from one track are lost but not the other).
Whether this is a useful abstraction may depend on the level of detail you're interested in. For instance the RTCPeerConnection API, which is still in Working Draft stage, has over the last year moved away from streams as inputs and outputs to dealing directly with tracks, because the working group believes that details matter more when it comes to transmission (things like tracking bandwidth use etc.)
In any case, going from one to the other will be trivial:
var tracks = stream.getTracks();
console.log(tracks.map(track => track.kind)); // audio,video
video.srcObject = new MediaStream(tracks);
once browsers implement the MediaStream constructor (slated for Firefox 44).
This is sort of expanding on my previous question Web Audio API- onended event scope, but I felt it was a separate enough issue to warrant a new thread.
I'm basically trying to do double buffering using the web audio API in order to get audio to play with low latency. The basic idea is we have 2 buffers. Each is written to while the other one plays, and they keep playing back and forth to form continuous audio.
The solution in the previous thread works well enough as long as the buffer size is large enough, but latency takes a bit of a hit, as the smallest buffer I ended up being able to use was about 4000 samples long, which at my chosen sample rate of 44.1k would be about 90ms of latency.
I understand that from the previous answer that the issue is in the use of the onended event, and it has been suggested that a ScriptProcessorNode might be of better use. However, it's my understanding that a ScriptProcessorNode has its own buffer of a certain size that is built-in which you access whenever the node receives audio and which you determine in the constructor:
var scriptNode = context.createScriptProcessor(4096, 2, 2); // buffer size, channels in, channels out
I had been using two alternating source buffers initially. Is there a way to access those from a ScriptProcessorNode, or do I need to change my approach?
No, there's no way to use other buffers in a scriptprocessor. Today, your best approach would be to use a scriptprocessor and write the samples directly into there.
Note that the way AudioBuffers work, you're not guaranteed in your previous approach to not be copying and creating new buffers anyway - you can't simultaneously be accessing a buffer from the audio thread and the main thread.
In the future, using an audio worker will be a bit better - it will avoid some of the thread-hopping - but if you're (e.g.) streaming buffers down from a network source, you won't be able to avoid copying. (It's not that expensively, actually.) If you're generating the audio buffer, you should generate it in the onaudioprocess.
I am using html5 web audio api in my application. Application is simple, I have
BufferSourceNode -> GainNode -> lowpass filter -> context.destination
Now I want to save the output after applying the filters. So I decided to add recorder before
context.destination. But this doesn't work, it gives some noise sound while playing the audio, though my recorder records filter effects successfully.
Am I doing it in right way or is there any better way to do this?
Two things:
1) if you are going to use the buffer anyway - even if you're not() - you might want to consider using an OfflineAudioContext (https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#OfflineAudioContext-section). OACs can run faster than real-time, so you don't need to "record" it in real-time; you set up your nodes, call startRendering(), and the oncomplete event hands you an audiobuffer. () If you still want a .WAV file, you can pull the WAV encoding function out of Recordjs and use it to encode an arbitrary buffer.
2) That sounds like an error in your code - it should work either way, without causing extra noise. Do you have a code sample you can send me?
I am trying to save the output from webAudio API for future use , so far i think getting PCM data and saving it as a file will do my expectation , I am wondering if the webAudio or mozAudio already supports saving the output stream if not how can i get the pcm data from the output stream
There isn't a good sense of the requirements here outside of attempting to capture web audio in some programmatic way. The presumption here is you want to do this from code executing in JavaScript on the page that's currently being browsed, but that also isn't entirely clear.
As Incognito points out, you can do this in Chrome by using a callback hanging off decodeAudioData(). But, this may be overly complicated for your uses if you're simply trying to capture, for example, the output of a single web stream and decode it into PCM for use in your sound tools of choice.
Another strategy you might consider, for cases when the media URL is obscured or otherwise difficult to decode using your current tools, is capture from your underlying sound card. This gives you the decoding for free, at the potential expense of a lower sampling rate if (and only if) your sound card isn't able to sample the stream effectively.
As we know, you're already encoding analog signals digitally anyway via your desire for PCM encoding. Obviously, only do this if you have the legal right to use the files being sampled.
Regardless of the route you choose, best of luck to you. Be it programmatic stream dissection or spot sampling, you should now have more than enough information to proceed.
Edit: Based on additional information from the OP, this seems like the needed solution (merged from here and here, using NodeJS' implementation of fs):
var fs = require('fs');
function saveAudio(data, saveLocation) {
var context = new (window.AudioContext || window.webkitAudioContext)();
var source = context.createBufferSource();
if(context.decodeAudioData) {
context.decodeAudioData(data, function(buffer) {
fs.writeFile(saveLocation, buffer, function (err) {
if (err) throw err;
console.log('It\'s saved!');
});
}, function(e) {
console.log(e);
});
} else {
var buffer = context.createBuffer(data, false /*mixToMono*/);
fs.writeFile(saveLocation, buffer, function (err) {
if (err) throw err;
console.log('It\'s saved!');
});
}
}
(Warning: untested code. If this doesn't work, edits are welcome.)
This effectively spools out decodeAudioData from the Web Audio API, decodes PCM from the supplied data, then attempts to save it to the target saveLocation. Simple enough, really.
The latest WebAudio API draft introduced the OfflineAudioContext exactly for this purpose.
You use it exactly the same way as a regular AudioContext, but with an additional startRendering() method to trigger offline rendering, as well as an oncomplete callback so that you can act upon finishing rendering.
Chrome should support it (or at the least, mostly support this new feature).
decodeAudioData()
When decodeAudioData() is finished, it calls a callback function which provides the decoded PCM audio data as an AudioBuffer
It's nearly identical to the XHR2 way of doing things, so you'll likely want to make an abstraction layer for it.
Note: I haven't tested that it works, but I only see one bug in chromium regarding this, indicating it works but fails for some files.
I think that what you are looking for can be achieved with the startRendering-function in Web Audio. I dunno if the answers above did the trick, but if they didn't - here's a little something to get you going:
https://bugs.webkit.org/show_bug.cgi?id=57676 (scroll down to comment three)
This part is still undocumented, so it's nowhere to be seen in the spec, but you can console.log the audio context to confirm that it's actually there. I've only done some preliminary test with it, but I think it should be the answer to your question.