We Keep Coding

VideoTexture is not playing in VR

At first the texture works fine and the video plays as expected, but when VR is entered via VRDisplay.requestPresent it stops. Why is this and how to fix it?

The VR display has its own render loop. Usually needsUpdate is automatically set to true on every animation frame by three.js, but this is only true for the default display.
To fix this, get the VR display from the vrdisplayconnect event and create your own update loop. E.g.
let display = e.display;
let displayUpdateLoop = () =>
{
// May get a warning if getFrameData is not called.
let frameData = new VRFrameData();
display.getFrameData(frameData);
videoTexture.needsUpdate = true;
// Stop loop if no longer presenting.
if (display.isPresenting)
display.requestAnimationFrame(displayUpdateLoop);
}
display.requestAnimationFrame(displayUpdateLoop);

oddity about createMediaElementSource()

I was building an audio program and hit a stumbling block on the .createMediaElementSource method. I was able to solve the problem, but I do not quite know why the solution works.
In my HTML, I created an audio player: <audio id="myAudio><source src="music.mp3"></audio>
Now in my JS:
context = new AudioContext();
audio = document.getElementById('myAudio');
source = context.createMediaElementSource(audio);
audio.play();
doesn't work. The audio element loads, but doesn't play the song, nor is there audio.
However! This JS code works:
context = ...; //same as above
audio...;
source = context.createMediaElementSource(audio[0]);
audio.play();
All I changed was adding a [0] to the audio and the program suddenly works again. Since .getElementById doesn't return an array, I don't know why referring to audio as an array works, but just referring to audio does not.

A few months late, but in case others stumble upon this and want an answer:
This behaviour is described in the Web Audio API spec:
The createMediaElementSource method
Creates a MediaElementAudioSourceNode given an HTMLMediaElement. As a consequence of calling this method, audio playback from the HTMLMediaElement will be re-routed into the processing graph of the AudioContext.
Emphasis mine. Since the output from the audio element is now routed into the newly created MediaElementAudioSourceNode instance (instead of the original destination, usually your speakers), you need to route the output of the instance back to the original destination:
var audio = document.getElementById('myAudio');
var ctx = new AudioContext();
var src = ctx.createMediaElementSource(audio);
src.connect(ctx.destination); // connect the output of the source to your speakers
audio.play();
The reason it worked when you added [0] is that document.getElementById doesn't return an array, or an element with a defined key of "0". As such, you might as well have written ctx.createMediaElementSource(undefined), which doesn't re-route the audio from the #myAudio element.

web audio, createMediaElementSource variable sources

lets say i wan't to have an app that has variable audio sources as audio tags like so:
<audio preload="auto" src="1.mp3" controls="" class="muzz"></audio>
<audio preload="auto" src="track.mp3" controls="" class="muzz"></audio>
Depending on which of them is played it should be passed to createMediaElementSource and then the sound would be sent to analyser and various things would be done with it, but it doesn't work:
var trackName;
//get the source of clicked track
$(".muzz").on("play", function(){
trackName = $(this).attr("src");
console.log("got a source: ", trackName);
audio = new Audio();
audio.src=trackName;
context = new AudioContext();
analyser = context.createAnalyser();
source = context.createMediaElementSource(audio);
source.connect(analyser);
analyser.connect(context.destination);
letsDraw();
});
the console log displays the correct source name, the letsDraw() method is supposed to draw a spectrogram of the audio playing:
function letsDraw(){
console.log("draw called");
window.requestAnimationFrame(letsDraw);
fbc_array = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteFrequencyData(fbc_array); //get frequency from the analyser node
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.fillStyle="white";
ctx.font = "bold 12px Arial";
ctx.fillText("currently playing:" + trackName, 10, 20);//this works
bars = 150;
for(var i = 0; i < analyser.frequencyBinCount; i++){ //but this doesn't
/*fill the canvas*/
x = i *2;
barWidth = 1;
barHeight = -(fbc_array[i]/1.8);
//colours react to the frequency loudness
hue = parseInt(500 * (1 - (barHeight / 200)), 10);
ctx.fillStyle = 'hsl(' + hue + ',75%,50%)';
ctx.fillRect(x, canvas.height, barWidth, barHeight);
}
}
it was working fine with one set audio source, but fails with variable sources, any ideas would be very appreciated.
of course, no errors are even thrown in the console at all.

What I don't get is why you take the src and put that in a new Audio object, as you already have them.
It is also way better to create a source from both <audio> tags. You just create a function that runs on page load (so when everything on the page is ready so you won't get any errors about elements not yet existing etc.).
Before I start writing a piece of code, what do you expect to happen? Should it be possible to have both tags playing at the same time, or should one be stopped if you click play on the other? If it shouldn't play at the same time, you'd better make one <audio> tag and create two buttons which each set the src of the tag.
Another problem with your code is that you already have the <audio> elements, and when you want them to play you just create a new audio element and append the src to it.. What is the logic behind that?
EDIT:
Here is an example of using multiple sources with only one <audio> element.
The HTML code should look like this:
<audio id="player" src="" autoplay="" controls=""></audio>
<div id="buttons">
<!--The javascript code will generate buttons with which you can play the audio-->
</div>
Then you use this JS code:
onload = function () { //this will be executed when the page is ready
window.audioFiles = ['track1.mp3', 'track2.mp3']; //this is gonna be the array with all file names
window.player = document.getElementById('player');
window.AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();
source = context.createMediaElementSource(player);
analyser = context.createAnalyser();
source.connect(analyser);
analyser.connect(context.destination);
//now we take all the files and create a button for every file
for (var x in audioFiles) {
var btn = document.createElement('button');
btn.innerHTML = audioFiles[x];
btn.onclick = function () {
player.src = audioFiles[x];
//so when the user clicks the button, the new source gets appended to the audio element
}
document.getElementById('buttons').appendChild(btn);
}
}
Hope the comments explain it good enough
EDIT2:
You want to know how to do this for multiple elements. What you want to do is to create all the audio elements when the page loads, and create all sources for it. This will decrease mess when starting to play audio. The code is here.
All you need to do is have the for loop that runs for every media file you have, which it will create an audio element for with the appropriate source, and then create a sourcenode for that (createMediaElementSource), and connect that sourcenode to the analyser.
I also want to say something about your visualiser code though. If you do not override the font, color or anything, you don't need to execute that every animationframe. Once on init is enough.

At the top of your first code block, try this:
var trackName,
context = new AudioContext();
And remove context = new AudioContext(); from the click handler.
You should only have one AudioContext for the page.

Buffered Audio For Generating Sounds

I have an oscillator to generate the frequencies of a keyboard. It all works when I output to speakers, but as well as outputting to speakers I would like to buffer it so that I can turn it into base 64 and use again later. The only examples of this I have seen use xhr which I do not need as obviously I want to be able to just add a node into the modular routing to take input, store it in an array, then output to the hardware.
Something like this:
var osc = ctx.createOscillator();
osc.type = 3;
osc.frequency.value = freq;
osc.connect(buffer);
buffer.connect(ctx.destination);
Is this possible?

Have you considered utilizing a ScriptProcessorNode?
See: http://www.w3.org/TR/webaudio/#ScriptProcessorNode
You would attach an eventListener to this node, allowing you to capture arrays of audio samples as they pass through. You could then save these buffers and manipulate them as you wish.

Have you checked out RecorderJs? https://github.com/mattdiamond/Recorderjs. I think it does what you need.

I have solved my problem by using Matt's Recorder.js https://github.com/mattdiamond/Recorderjs and connecting it to a GainNode which acts as an intermediary from a number of oscillators to the ctx.destination. I will be using localStorage but here is an example using an array (this does not include the oscillator setup).
var recorder;
recorder = new Recorder(gainNode, { workerPath: "../recorderWorker.js"});
recorder.record();
var recordedSound = [];
function recordSound() {
recorder.exportWAV(function(blob) {
recordedSound.push(blob);
});
}
function play(i) {
var audio = new Audio(window.URL.createObjectURL(recordedSound[i]));
audio.play();
}

HTML5 object selection and audio playback

In my software engineering course we're designing a language-learning website in HTML5 but none of us have experience with it.
The main functionality of the website is essentially playing an audio clip, having the user select an image based on what they think the audio clip corresponds to, and then playing (and internally recording) a noise based on whether they correctly identified the object.
I'm just looking for some general ideas as to how this can be done. I'm assuming the use of javascript is a given but not entirely sure about how to 'link' an image to the audio clip to determine if the user selection was correct.
Any help is appreciated and thanks a lot for taking a look!

This is a sketch: http://jsfiddle.net/pimvdb/eZWq5/.
Define a pair, and each pair can be "displayed"; it will make the audio and images appear. Clicking on an image will check whether it was the image of the same pair that the audio was from.
It does not have the noise implemented among other things you'd probably like to alter, but it might be a start.
function Pair(image, sound) {
this.image = image;
this.sound = sound;
}
Pair.prototype.display = function() {
var div1 = document.getElementById("div1");
var div2 = document.getElementById("div2");
div1.innerHTML = "";
div2.innerHTML = "";
var audio = document.createElement("audio"); // make audio
audio.src = this.sound;
audio.autoplay = true;
audio.controls = true;
div1.appendChild(audio);
var thisPair = this;
for(var i = 0; i < pairs.length; i++) { // make images
var img = document.createElement("img");
img.src = pairs[i].image;
img.onclick = function() {
if(this.src === thisPair.image) {
alert('yey');
} else {
alert('nop');
}
showRandom();
};
div2.appendChild(img);
}
};
var pairs = [
new Pair("http://lorempixel.com/100/100/?1",
"http://upload.wikimedia.org/wikipedia/"
+ "commons/a/a9/Tromboon-sample.ogg"),
new Pair("http://lorempixel.com/100/100/?2",
"http://upload.wikimedia.org/wikipedia/"
+ "commons/c/c8/Example.ogg")
];
function showRandom() {
pairs[Math.random() * pairs.length | 0].display();
}
showRandom();

To learn more about the APIs for javascript and the HTML5 audio check out Mozilla Docs.
To answer your question on how to 'link' things here is a rough sketch about how it would work. You have a list of audio clips and a list of images that goes with each and which image is correct (like a multiple choice question). Then you display the images and play the sound clip. Wait for the user to click on one of the images and then check whether it is right or not. This all can be accomplished with pretty straightforward javscript. I haven't worked with the audio API as much but you can come back with a specific question if you need more help.