I am creating a MediaStream object and adding a video track to it from a canvas using the captureStream() function. This works fine.
However I am trying to add audio as a separate track from a video element. I cant seem to find a way to get an AudioTrack object from a html video element.
Currently HTMLMediaElement.audioTracks is not supported in Chrome. According to the mozilla developer site I should be able to use HTMLMediaElement.captureStream() to return a MediaStream object from which I should be able to retrieve the separate tracks but I just get 'captureStream is not a function' error.
Perhaps i'm missing something very obvious but I would greatly appreciate any help on this.
Below is my current code:
var stream = new MediaStream();
//Works fine for adding video source
var videotracks = myCanvas.captureStream().getTracks();
var videostream = videotracks[0];
stream.addTrack(videostream);
//Currently not supported in Chrome
var audiotracks = myVid.audioTracks;
var audiostream = audiotracks[0];
stream.addTrack(audiostream);
To get an audio stream from a video element in a cross-browser way :
AudioContext API createMediaStreamDestination + createMediaElementSource.
// if all you need is the audio, then you should even probably load your video in an Audio element
var vid = document.createElement('video');
vid.onloadedmetadata = generateAudioStream;
vid.crossOrigin = 'anonymous';
vid.src = 'https://dl.dropboxusercontent.com/s/bch2j17v6ny4ako/movie720p.mp4';
function generateAudioStream() {
var audioCtx = new AudioContext();
// create a stream from our AudioContext
var dest = audioCtx.createMediaStreamDestination();
// connect our video element's output to the stream
var sourceNode = audioCtx.createMediaElementSource(this);
sourceNode.connect(dest)
// start the video
this.play();
// your audio stream
doSomethingWith(dest.stream)
}
function doSomethingWith(audioStream) {
// the audio element that will be shown in the doc
var output = new Audio();
output.srcObject = audioStream;
output.controls = true;
output.play();
document.body.appendChild(output);
}
To add audio to a canvas stream :
MediaStream Capture Canvas and Audio Simultaneously
Related
I have 3 AudioContexts in my document and a canvas that is rendering random things. I'm trying to capture the audio that is playing in the AudioContext and adding it to the canvasStream so the audio + video is being send over webrtc.
Now the code looks like this:
here I create the media streams
this.mediaStreams = [];
window.activeAudioContexts.forEach(context=>{
const gainNode = context.createGain();
gainNode.gain.value = 1;
const destination = context.createMediaStreamDestination();
gainNode.connect(destination);
this.mediaStreams.push(destination.stream);
});
Here i create the canvasStream and add the audio tracks:
const stream = this.targetCanvas.captureStream(30)
this.mediaStreams.forEach((audioStream) ={
stream.addTrack( audioStream.getAudioTracks()[0] );
});
broadCaster.attachStream(stream);
On the receiving client I can see the video and I can see the MediaStream has 3 Audio Tracks, however no audio is being played in the video.
Any ideas where this is going wrong?
Thanks for pointing me in the right direction!
Cheers,
Erik
I want to use the web audio OfflineAudioContext to save some short sound files into a new longer sound file. It seems that the OfflineAudioContext uses an audiobuffer to save the audio. I'm using the audio tag to load my sound files. How can I convert the sound files into an audio buffer that I can pass to the OfflineAudioContext buffer. I don't want to have to use the XMLHttpRequest to create the audio buffer. I wanted to be able to load each sound file then add it to a buffer, and then pass it to the offline context. I've tried several different ways, but the most I've got done is to pass an empty audio buffer to the context which just gives me a file with no sound.
I've tried several features of the web audio context to create a new buffer, but I either get errors, or no sound
<html>
<body>
<audio controls controlsList = "nodownload" id="audio"><source id="audioSource" src="" />
Your browser does not support the audio format.</audio>
<script>
// define online and offline audio context
var audioCtx1 = new AudioContext();
var offlineCtx = new OfflineAudioContext(2,44100*40,44100);
var channels = 2;
// Create an empty two second stereo buffer at the
// sample rate of the AudioContext
var frameCount = audioCtx1.sampleRate * 2.0;
var myArrayBuffer = audioCtx1.createBuffer(2, frameCount, audioCtx1.sampleRate);
source = offlineCtx.createBufferSource();
var array = ['sound1.mp3','sound2.mp3','sound3.mp3']
var audio = document.querySelector('audio');
audio.src = array[0];
source.buffer = myArrayBuffer;
source.connect(offlineCtx.destination);
source.start();
//source.loop = true;
offlineCtx.startRendering();
offlineCtx.oncomplete = function(e) {
var audioCtx = new (window.AudioContext || window.webkitAudioContext)();
var song = audioCtx.createBufferSource();
song.buffer = e.renderedBuffer;
song.connect(audioCtx.destination);
song.start();
console.log("completed!");
}
</script>
</body>
</html>
I've changed the code several times during testing, I think with this code I posted I just get no sound playing.
I'm pretty new to the Web Audio API and Javascript in general so this may seem really stupid to some of you but I'm in the process of trying to create a basic audio visualiser in a javascript canvas.
I'm having issues with the audio context and more so connecting an analyser to the audio source which is a locally stored mp3 file. 'source.connect()' is apparently not a function, but I've copied the syntax exactly from the Web Audio API guide at: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API.
function SetUpAudio()
{
let audio = document.createElement('audio');
audio.src = 'never let u go master 3.mp3';
let source = audio.src
audio.controls = 'true';
document.body.appendChild(audio);
audio.style.width = window.innerWidth + 'px';
let audioCtx = new (window.AudioContext)();
let analyser = audioCtx.createAnalyser();
source.connect(analyser);
audio.play();
}
This is the error that appears:
Uncaught TypeError: source.connect is not a function
You mixed up the variable names. The docs have this example:
source = audioCtx.createMediaStreamSource(stream);
source.connect(analyser);
But in your case, source is set to the value of audio.src which is just a string: 'never let u go master 3.mp3', not a MediaStreamSource.
Change it as follows:
audioCtx = new (window.AudioContext)();
let analyser = audioCtx.createAnalyser();
audioCtx.connect(analyser);
Then take it from there. And if you have further issues, please post a new question.
Is there a way to play the audio I am recording while I'm still recording it?
This is my code. I'm recording the audio and everytime an audio chunk is ready a call to onaudioprocess is made. It receives an instance of AudioProcessingEvent. I'm not familiar with the Web Audio API, so I'm not sure what to do to listen to it. I want my speakers to output the sound.
function startRecording(stream){
var context = new AudioContext();
var audio_input = context.createMediaStreamSource(stream);
var buffer_size = 2048;
var recorder = context.createScriptProcessor(buffer_size, 1, 1);
recorder.onaudioprocess = function(e){
// var data = e.inputBuffer.getChannelData(0);
// AudioStream.write(data);
var source = context.createBufferSource();
source.buffer = e.inputBuffer;
source.connect(context.destination);
source.start(0);
console.log(e.inputBuffer)
};
audio_input.connect(recorder);
recorder.connect(context.destination);
}
The code inside onaudioprocess does not work. There is no error in the console, but nothing happens.
The output of e.inputBuffer looks fine:
AudioBuffer {length: 4096, duration: 0.09287981859410431, sampleRate: 44100, numberOfChannels: 1}
Read up on the audio processing event at
https://webaudio.github.io/web-audio-api/#dom-audioprocessingevent to
see what the onprocess event does. You basically need to copy the
events inputBuffer to the outputBuffer and you'll be able to hear the
audio.
I am using a SoundCloud URL as audio.src . It is only playing the unprocessed version when i run it through the delay chain i have.
Here is the fiddle:
http://jsfiddle.net/ehsanziya/nwaH3/
var context = new webkitAudioContext();
var audio = new Audio(); //creates a HTML5 Audio Element
url = 'http://api.soundcloud.com/tracks/33925813/stream' + '?client_id=c625af85886c1a833e8fe3d740af753c';
//wraps the soundcloud stream to an audio element.
audio.src = url;
var source = context.createMediaElementSource(audio);
var input = context.createGainNode();
var output = context.createGainNode();
var fb = context.createGainNode();
fb.gain.value = 0.4;
var delay = context.createDelayNode();
delay.delayTime.value = 0.5;
//dry
source.connect(input);
input.connect(output);
//wet
input.connect(delay);
delay.connect(fb);
fb.connect(delay);
delay.connect(output);
source.mediaElement.play();
The chain works with Oscillator node.
What is the reason for it?
And is there any other way of processing a streaming sound from SoundCloud with Web Audio API?
You need to wait for the canplaythrough event on your audio element to fire before you can use it with createMediaElementSource.
So just add the event listener, and wait until the callback fires before you assign source = context.createMediaElementSource(audio); and make all of your connections.
Here's an updated jsFiddle that'll do what you want: http://jsfiddle.net/nwaH3/3/