I need to loop an audio file (WAV, OGG or raw PCM) in the browser that contains segments which are unheard (ultrasonic) by the human ear (yet contain data which is valuable to me).
Using Chrome on Mac, I've noticed that if the segments of unheard sound are relatively short, I get all the data back (heard + unheard). In contrast, if the segments of unheard sound are longer than a certain threshold, it will fade out the whole channel quickly and effectively cancel the rest of the file completely, until the next loop cycle begins.
The way I'm loading and playing the sound is like so:
var b = msg.data; // binary msg received from websocket
b.type = "audio/wav";
var URLObject = window.webkitURL || window.URL;
var url = URLObject.createObjectURL(b);
var snd = document.createElement("audio");
snd.setAttribute("src", url);
snd.addEventListener("loadeddata", function() {
snd.loop = true;
snd.muted = false;
snd.play();
});
I'm looking for a way to cancel this automatic filtering of unheard sounds. Eventually, I would like a way to do this cross-browser. If not possible using JavaScript, a Flash solution will also be accepted.
Sample ultrasonic WAV files (~1MB each):
https://drive.google.com/file/d/0B5sMkxczD6sNbm04MmxMTmIwdlk/edit?usp=sharing
https://drive.google.com/file/d/0B5sMkxczD6sNal91WUhRNWo2d3c/edit?usp=sharing
There isn't a single approach that will work on all browsers, unfortunately.
For most browsers on the desktop and iOS too you can use the Web Audio API as shown here:
http://www.html5rocks.com/en/tutorials/webaudio/intro/
For IE/Android you need to use Flash to play a WAV/PCM, or play OGG with HTML5 Audio tag, but the latter may lose the ultrasonic frequencies.
So in general, you need to write code that will check what the current browser supports and use that, starting with Web Audio API, then trying HTML5 Audio, then Flash.
Related
I'm developing an Elecron app (JavaScript) to audio visualization. There is a Playlist() instance which receives audio file paths the user wants to play. When the first audio finishes, it plays the next one. So far so good. The app does an intense computational work extracting audio features from each channel, re-rendering canvases and animating plots. It does it beautifully.
The problem is: each time the app plays a next file, the more slow it gets, as if all the audio data before is still somewhere. I've found in documentation the method close() from AudioContext():
"The close() method of the AudioContext Interface closes the audio context, releasing any system audio resources that it uses."
"An AudioContext can now be explicitly closed, thereby releasing any hardware resources associated with the AudioContext. Without this, developers had to depend on garbage collection of the AudioContext to release hardware resources."
I also have found this example of closing and restarting audio contexts:
https://github.com/mdn/webaudio-examples/blob/master/audiocontext-states/index.html
https://mdn.github.io/webaudio-examples/audiocontext-states/
The problem is that I use a audioContext.createMediaElementSource(HTMLelementID) and it doesn't allow me to restart everything recreating all the nodes like in the example. A simplified code that represents what I did before is:
class Audio() {
constructor(audioElementID, playlistObj) {
this.audioContext = new AudioContext();
this.audioElement = document.getElementById(audioElementID);
this.track = this.audioContext.createMediaElementSource(this.audioElement);
this.gainNode = this.audioContext.createGain();
this.track.connect(this.gainNode);
this.gainNode.connect(this.audioContext.destination);
this.audioElement.addEventListener('ended', () => {
playlistObj.playnextTrack() // changes the src from the html element (audioElementID) and sets this.audioElement.currentTime to 0
}
}
// everything is a property here for debugging reasons
}
const audio = new Audio('audioID', playlist);
// playlist defined somewhere else
To implement the close() method I had to change (just exactly the example, a function that recreates everything again):
class Audio() {
constructor(audioElementID, playlistObj) {
this.createAudioContext = () => {
this.audioContext = new AudioContext();
this.audioElement = document.getElementById(audioElementID);
this.track = this.audioContext.createMediaElementSource(this.audioElement);
this.gainNode = this.audioContext.createGain();
this.track.connect(this.gainNode);
this.gainNode.connect(this.audioContext.destination);
this.audioElement.addEventListener('ended', () => {
playlistObj.playNextTrack() // changes the src from the html element (audioElementID) and sets this.audioElement.currentTime to 0
}
}
this.createAudioContext();
}
}
and in playlist.playNextTrack() I pause the audioElement, call audio.audioContext.close(), wait for it (it's a promise), call audio.createAudioContext() to recreate everything and plays. The logic returns an error at this.track = this.audioContext.createMediaElementSource(this.audioElement) with:
"Failed to execute 'createMediaElementSource' on 'BaseAudioContext': HTMLMediaElement already connected previously to a different MediaElementSourceNode, at Audio.createAudioContext"
In the example, the audio source is just a random oscillator and not a mp3 audio file.
I'm really stuck here. Don't know what to do. I'm not even sure if AudioContext() really holds data from all the audio files before causing this performance problem. And if so, how could I reconnect the HTMLMediaElement to a new node audio.createAudioContext() creates? I've already tried audio.track.disconnect()but it doesn't work (as it shouldn't because here I'm disconnecting track from gainNode). And also audioElement doesn't have a disconnect()method as It's just a html element.
Any idea?
UPDATE:
I passed over the problem of recreating the audio context deleting and creating again the html element. But the problem persist: the more new audio files are played, the app gets slower. More precisely now: the more new AudioContext() is created, the slower it gets (even if I close the previous one).
I'm really stuck here. Don't know what to do. I'm not even sure if AudioContext() really holds data from all the audio files before causing this performance problem.
No, it's really unlikely this is the case. The AudioContext sets up things like the sample rate, output destination, and the graph. That's all.
The close() method of the AudioContext Interface closes the audio context, releasing any system audio resources that it uses.
You're misunderstanding what this means. Those "system audio resources" are the sound devices. While the AudioContext is running, there is an audio device requested. This is particularly meaningful in low power environments, like mobile. Another example would be Bluetooth. If the AudioContext is kept running, your Bluetooth headset may just stay on. If the AudioContext is allowed to close, then the Bluetooth headset may go to sleep.
And if so, how could I recconect the HTMLMediaElement to a new node audio.createAudioContext() creates?
You don't. While it would be nice if the API supported this, it seems it doesn't. Simply create a new HTMLMediaElement.
What you should do is properly profile your application to figure out where the slowdown is occurring. Use your developer tools. Might be faster though just to start commenting out sections of things that are running. We certainly can't tell you where the problem is, specifically, from the code you've shown.
I'm trying to get sound working on my iPhone game using the Web Audio API. The problem is that this app is entirely client side. I want to store my mp3s in a local folder (and without being user input driven) so I can't use XMLHttpRequest to read the data. I was looking into using FileSystem but Safari doesn't support it.
Is there any alternative?
Edit: Thanks for the below responses. Unfortunately the Audio API is horribly slow for games. I had this working and the latency just makes the user experience unacceptable. To clarify, what I need is sounething like -
var request = new XMLHttpRequest();
request.open('GET', 'file:///./../sounds/beep-1.mp3', true);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(request.response, function(buffer) {
dogBarkingBuffer = buffer;
}, onError);
}
request.send();
But this gives me the errors -
XMLHttpRequest cannot load file:///sounds/beep-1.mp3. Cross origin requests are only supported for HTTP.
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101
I understand the security risks with reading local files but surely within your own domain should be ok?
I had the same problem and I found this very simple solution.
audio_file.onchange = function(){
var files = this.files;
var file = URL.createObjectURL(files[0]);
audio_player.src = file;
audio_player.play();
};
<input id="audio_file" type="file" accept="audio/*" />
<audio id="audio_player" />
You can test here:
http://jsfiddle.net/Tv8Cm/
Ok, it's taken me two days of prototyping different solutions and I've finally figured out how I can do this without storing my resources on a server. There's a few blogs that detail this but I couldn't find the full solution in one place so I'm adding it here. This may be considered a bit hacky by seasoned programmers but it's the only way I can see this working, so if anyone has a more elegent solution I'd love to hear it.
The solution was to store my sound files as a Base64 encoded string. The sound files are relatively small (less than 30kb) so I'm hoping performance won't be too much of an issue. Note that I put 'xxx' in front of some of the hyperlinks as my n00b status means I can't post more than two links.
Step 1: create Base 64 sound font
First I need to convert my mp3 to a Base64 encoded string and store it as JSON. I found a website that does this conversion for me here - xxxhttp://www.mobilefish.com/services/base64/base64.php
You may need to remove return characters using a text editor but for anyone that needs an example I found some piano tones here - xxxhttps://raw.github.com/mudcube/MIDI.js/master/soundfont/acoustic_grand_piano-mp3.js
Note that in order to work with my example you're need to remove the header part data:audio/mpeg;base64,
Step 2: decode sound font to ArrayBuffer
You could implement this yourself but I found an API that does this perfectly (why re-invent the wheel, right?) - https://github.com/danguer/blog-examples/blob/master/js/base64-binary.js
Resource taken from - here
Step 3: Adding the rest of the code
Fairly straightforward
var cNote = acoustic_grand_piano.C2;
var byteArray = Base64Binary.decodeArrayBuffer(cNote);
var context = new webkitAudioContext();
context.decodeAudioData(byteArray, function(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer;
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.noteOn(0);
}, function(err) { console.log("err(decodeAudioData): "+err); });
And that's it! I have this working through my desktop version of Chrome and also running on mobile Safari (iOS 6 only of course as Web Audio is not supported in older versions). It takes a couple of seconds to load on mobile Safari (Vs less than 1 second on desktop Chrome) but this might be due to the fact that it spends time downloading the sound fonts. It might also be the fact that iOS prevents any sound playing until a user interaction event has occured. I need to do more work looking at how it performs.
Hope this saves someone else the grief I went through.
Because ios apps are sandboxed, the web view (basically safari wrapped in phonegap) allows you to store your mp3 file locally. I.e, there is no "cross domain" security issue.
This is as of ios6 as previous ios versions didn't support web audio api
Use HTML5 Audio tag for playing audio file in browser.
Ajax request works with http protocol so when you try to get audio file using file://, browser mark this request as cross domain request. Set following code in request header -
header('Access-Control-Allow-Origin: *');
I am in the process of developing an HTML5 canvas interactive piece that uses Createjs and the Web Audio API. I've managed to get audio working in Chrome/Firefox/Safari despite the deprecation of webkitAudioContext by Chrome and FF but not Safari. However, filters for some reason are not working in Safari, but sound still plays. Filters DO work in Chrome/FF.
I have my filters set up like this:
var sound = new Audio();
sound.src = './sounds/sound.mp3';
sound.autoplay = false;
sound.loop = true;
soundSource = context.createMediaElementSource(sound);
var soundFilter = context.createBiquadFilter();
soundFilter.type = "lowpass";
soundFilter.frequency.value = 500;
soundSource.connect(soundFilter);
soundFilter.connect(context.destination);
Am I unknowingly using a deprecated term or something? Live project can be found here. Cheers.
UPDATE: This has been recognised as a genuine bug by the WebKit team, and will be patched. Full details here
Apparently Safari doesn't implement createMediaElementSource correctly. So instead of redirecting the sound through your Web Audio nodes, it still just plays the sound directly to the audio device.
Is there any particular reason why you can't use a BufferSourceNode? It makes you jump through extra hoops to get the sound file and decode it, but it should work.
I'm developing a webapp that (in part) records some audio using recorder.js, and sends it to a server. I'm trying to target Firefox, so I have to use this hack to keep the audio source from cutting off:
// Hack for a Firefox bug that stops input after a few seconds
window.source = audio_context.createMediaStreamSource(stream);
source.connect(audio_context.destination);
I think that this is causing audio to be played back through the computer speakers, but I'm not sure. I'm kind of a newbie when it comes to web audio. My goal is to eliminate the audio that is being played out of the speakers.
EDIT:
Here's a link to my JS file on Github: https://github.com/miller9904/Jonathan/blob/master/js/main.js#L87
you have to connect the source to the node( through which you retrieve data which you are going to record), replace this.node with what variable name you have assigned to yuor node used for processing.
window.source.connect(this.node);
//this.node.connect(this.context.destination);
edit: just checked, connecting to destination is not compulsory, also make sure your node variable does not get garbage collected( which i am assuming is happening in your case, since recording stops after few seconds.)
In the document.ready function, I have this:
audioElement = document.createElement('audio');
audioElement.setAttribute('src', 'http://www.mfiles.co.uk/mp3-downloads/Toccata-and-Fugue-Dm.mp3');
$('#ToggleStart').click(function () {
audioElement.play();
});
$('#ToggleStop').click(function () {
audioElement.pause();
});
The problem is that the MP3 is downloaded when the page loads, which causes significant load time since the MP3 is over 2MB. What I want is the MP3 to be streamed. Is this possible and if so, what do I need to change?
jsFiddle here
You're very close to getting it right. I've had a look at your JSFiddle and noticed that the audio does stream already (I can play the file before it's finished downloading). You can easily see this by checking the Network traffic in your browser:
Chrome displays 'partial content' but is playing the mp3 at the same time. Your specific problem seems to be that it is downloading and playing too early. So if we take a look a the spec we can see some options.
preload = "none" or "metadata" or "auto" or "" (empty string) or empty
Represents a hint to the UA about whether optimistic downloading of the audio stream itself or its metadata is considered worthwhile.
- "none": Hints to the UA that the user is not expected to need the audio stream, or that minimizing unnecessary traffic is desirable.
- "metadata": Hints to the UA that the user is not expected to need the audio stream, but that fetching its metadata (duration and so on) is desirable.
- "auto": Hints to the UA that optimistically downloading the entire audio stream is considered desirable.
As you're not displaying any information about the audio file we can ignore the metdata option, this means you want to set the preload="none" attribute. Therefore if you change your JSFiddle slightly to dynamically set this:
audioElement.setAttribute('preload', "none");
audioElement.setAttribute('src', 'http://www.mfiles.co.uk/mp3-downloads/Toccata-and-Fugue-Dm.mp3');
Here is a JSFiddle showing the result, if you bring up the network tab in Chrome you'see see that the download doesn't start till you begin playing the mp3.