I have 3 AudioContexts in my document and a canvas that is rendering random things. I'm trying to capture the audio that is playing in the AudioContext and adding it to the canvasStream so the audio + video is being send over webrtc.
Now the code looks like this:
here I create the media streams
this.mediaStreams = [];
window.activeAudioContexts.forEach(context=>{
const gainNode = context.createGain();
gainNode.gain.value = 1;
const destination = context.createMediaStreamDestination();
gainNode.connect(destination);
this.mediaStreams.push(destination.stream);
});
Here i create the canvasStream and add the audio tracks:
const stream = this.targetCanvas.captureStream(30)
this.mediaStreams.forEach((audioStream) ={
stream.addTrack( audioStream.getAudioTracks()[0] );
});
broadCaster.attachStream(stream);
On the receiving client I can see the video and I can see the MediaStream has 3 Audio Tracks, however no audio is being played in the video.
Any ideas where this is going wrong?
Thanks for pointing me in the right direction!
Cheers,
Erik
Related
I need to convert the mp3 files I play in a project from mono to stereo with the web audio api. But do this a1 = new Audio(/1.mp3); I can't with. My entire system is based on this build. Converting all the sounds playing on the page to stereo or new Audio(/1.mp3); Is there a way to convert a sound created with .
var a1 = new Audio(`/1.mp3`);
a1.volume = .5;
a1.play()
I am using a simple code structure as above.
https://stackoverflow.com/a/54990085/15929287
I couldn't adapt the above answer for myself. In no way can I convert the sound I created with new audio() to stereo. In the example in the link, the oscillator is also added. I'm just trying to do something where I can adjust the mono/stereo setting. I need your help.
The linked answer assumes the mono audio source is an oscillator created with the Web Audio API but it is also possible to use an audio element as the source by using a MediaElementAudioSourceNode.
The audio graph would then need to be wired up like this:
const audio = new Audio('/1.mp3');
const audioCtx = new AudioContext();
const source = audioCtx.createMediaElementSource(audio);
const gainNodeL = audioCtx.createGain();
const gainNodeR = audioCtx.createGain();
const merger = audioCtx.createChannelMerger(2);
source.connect(gainNodeL);
source.connect(gainNodeR);
gainNodeL.connect(merger, 0, 0);
gainNodeR.connect(merger, 0, 1);
merger.connect(audioCtx.destination);
Please note that it's probably still necessary to call resume() on the AudioContext in response to a user gesture to make sure the AudioContext is running.
I am currently working on a simple audio project which uses HTML / JS audio elements to play audio using buttons.
Is it somehow possible to record that audio being played instead of recording the microphone (by using the MediaRecorder API)?
After following the steps provided by lawrence-witt, I figured it out. Here an example for everyone trying to achieve the same:
const audioContext = new AudioContext();
const destination = audioContext.createMediaStreamDestination();
const mediaRecorder = new MediaRecorder(destination.stream);
// array of 'new Audio()' elements
audioArray.forEach(audio => {
let stream = audioContext.createMediaStreamSource(audio.captureStream());
stream.connect(destination);
})
mediaRecorder.start();
I would like to create a Web Audio panner to position the sound from a WebRTC stream.
I have the stream connecting OK and can hear the audio and see the video, but the panner does not have any effect on the audio (changing panner.setPosition(10000, 0, 0) to + or - 10000 makes no difference to the sound).
This is the onaddstream function where the audio and video get piped into a video element and where I presume i need to add the panner.
There are no errors, it just isn't panning at all.
What am I doing wrong?
Thanks!
peer_connection.onaddstream = function(event) {
var AudioContext = window.AudioContext || window.webkitAudioContext;
var audioCtx = new AudioContext();
audioCtx.listener.setOrientation(0,0,-1,0,1,0)
var panner = audioCtx.createPanner();
panner.panningModel = 'HRTF';
panner.distanceModel = 'inverse';
panner.refDistance = 1;
panner.maxDistance = 10000;
panner.rolloffFactor = 1;
panner.coneInnerAngle = 360;
panner.coneOuterAngle = 0;
panner.coneOuterGain = 0;
panner.setPosition(10000, 0, 0); //this doesn't do anything
peerInput.connect(panner);
panner.connect(audioCtx.destination);
// attach the stream to the document element
var remote_media = USE_VIDEO ? $("<video>") : $("<audio>");
remote_media.attr("autoplay", "autoplay");
if (MUTE_AUDIO_BY_DEFAULT) {
remote_media.attr("muted", "false");
}
remote_media.attr("controls", "");
peer_media_elements[peer_id] = remote_media;
$('body').append(remote_media);
attachMediaStream(remote_media[0], event.stream);
}
Try to get the event stream before setting the panner
var source = audioCtx.createMediaStreamSource(event.stream);
Reference: Mozilla Developer Network - AudioContext
CreatePaneer Refernce: Mozilla Developer Network - createPanner
3rd Party Library: wavesurfer.js
Remove all the options you've set for the panner node and see if that helps. (The cone angles seem a little funny to me, but I always forget how they work.)
If that doesn't work, create a smaller test with the panner but use a simple oscillator as the input. Play around with the parameters and positions to make sure it does what you want.
Put this back into your app. Things should work then.
Figured this out for myself.
The problems was not the code, it was because I was connected with Bluetooth audio.
Bluetooth apparently can only do stereo audio with the microphone turned off. As soon as you activate the mic, that steals one of the channels and audio output downgrades to mono.
If you have mono audio, you definitely cannot do 3D positioned sound, hence me thinking the code was not working.
I am creating a MediaStream object and adding a video track to it from a canvas using the captureStream() function. This works fine.
However I am trying to add audio as a separate track from a video element. I cant seem to find a way to get an AudioTrack object from a html video element.
Currently HTMLMediaElement.audioTracks is not supported in Chrome. According to the mozilla developer site I should be able to use HTMLMediaElement.captureStream() to return a MediaStream object from which I should be able to retrieve the separate tracks but I just get 'captureStream is not a function' error.
Perhaps i'm missing something very obvious but I would greatly appreciate any help on this.
Below is my current code:
var stream = new MediaStream();
//Works fine for adding video source
var videotracks = myCanvas.captureStream().getTracks();
var videostream = videotracks[0];
stream.addTrack(videostream);
//Currently not supported in Chrome
var audiotracks = myVid.audioTracks;
var audiostream = audiotracks[0];
stream.addTrack(audiostream);
To get an audio stream from a video element in a cross-browser way :
AudioContext API createMediaStreamDestination + createMediaElementSource.
// if all you need is the audio, then you should even probably load your video in an Audio element
var vid = document.createElement('video');
vid.onloadedmetadata = generateAudioStream;
vid.crossOrigin = 'anonymous';
vid.src = 'https://dl.dropboxusercontent.com/s/bch2j17v6ny4ako/movie720p.mp4';
function generateAudioStream() {
var audioCtx = new AudioContext();
// create a stream from our AudioContext
var dest = audioCtx.createMediaStreamDestination();
// connect our video element's output to the stream
var sourceNode = audioCtx.createMediaElementSource(this);
sourceNode.connect(dest)
// start the video
this.play();
// your audio stream
doSomethingWith(dest.stream)
}
function doSomethingWith(audioStream) {
// the audio element that will be shown in the doc
var output = new Audio();
output.srcObject = audioStream;
output.controls = true;
output.play();
document.body.appendChild(output);
}
To add audio to a canvas stream :
MediaStream Capture Canvas and Audio Simultaneously
I have a strange Problem. I'm using Web Audio to play a stream from the server. I do that the following way:
var d2 = new DataView(evt.data);
var data = new Float32Array(d2.byteLength / Float32Array.BYTES_PER_ELEMENT);
for (var jj = 0; jj < data.length; ++jj) {
data[jj] = d2.getFloat32(jj * Float32Array.BYTES_PER_ELEMENT, true);
}
var buffer = context.createBuffer(1, data.length, 44100);
buffer.getChannelData(0).set(data);
source = context.createBufferSource();
source.buffer = buffer;
source.start(startTime);
source.connect(context.destination);
startTime += buffer.duration;
This works fine.
If i play the stream on my Computer i don't have any problems.
If i play the same stream on my Windows 8 tablet (same Chrome version) i have a lot of clicking sounds in the audio. There are multiple of them within one second.
It kinda seams that on the end of each buffer i hear a click.
I don't understand the difference... The only difference i could find was that the samplingrate of the soundcard on my computer is 44100 and on the tablet it's 48000.
The transmitted stream is in 44100 and i don't have any samplerate problems. just the clicking sounds.
Does anybody have an idea why this is happening?
Thank you,
metabolic
AudioBufferSourceNode resample their buffers to the AudioContext samplerate. As you can imagine, the API does not allow you to keep the resampler state between one AudioBufferSourceNode and the other, so there is a discontinuity between the two buffers.
I think the easiest way is to provide a stream at the sample-rate of the device, by resampling server-side. When the AudioWorkerNode will be ready and implemented, you'll be able to fix this yourself as well client side, but it's not.
Alternatively also you can just stream using an element, and pipe that to Web Audio API using AudioContext.createMediaElementSource().
I had the same issue, thanks to Padenot's answer I checked the sample rates. AudioContext.sampleRate defaulted to 44100, but the PCM data and AudioBuffer was 48000. Initialising the AudioContext with a matching sampleRate solved the problem:
var AudioContext = window.AudioContext || window.webkitAudioContext;
var audioCtx = new AudioContext({
latencyHint: 'interactive',
sampleRate: 48000,
});
With this, I can schedule the playback of 20ms 48khz PCM16 AudioBuffers back-to-back without any clicks or distortion.