We Keep Coding

How to seek audio with MediaSource and continue playing?

I am currently trying to implement an audio player for my Angular web application following a tutorial on Google Developers and also some help I've found on " Can't seek video when playing from MediaSource ".
The big difference in my case though is that I want to stream the audio chunk-wize such that the user does not have to download the entire file in order to listen to it right away.
Listening to a track from the start to the beginning is already working as I am just downloading byte-chunks from the server and simply append each chunk to the SourceBuffer as they arrive.
However, I am not able to implement the "seek" functionality.
I do not quite understand how to handle this on the client. At the moment I only work with mp3 files. I cannot find any example where seeking is explained.
I know that setting currentTime of an audio element will trigger a seeking event according to the Media Events doc.
We have:
this.audioObj = document.createElement('audio');
and a setter:
public seekTo(seconds) {
this.logger.debug(`seekTo ${seconds.toFixed(2)} seconds`);
// Setting currentTime will cause a "seeking" event to be emitted.
this.audioObj.currentTime = seconds;
}
I think that I have to load new data before I set currentTime and append it to the sourceBuffer. However, this simply cannot work for obvious reasons.
How can I make this work?

If you've set the duration, you should be able to set the currentTime but there will be nothing to play. You can use the currentTime to inform your chunk fetcher which portion of the file you need and append it so the media element can continue to play.
Note that when using mp3, the SourceBuffer will be operating in sequence mode since there are no timestamps, which means that if you just blindly append bytes they will not be at the correct point in time - I believe you need to set timestampOffset to the relevant time for the frame in question (I haven't tried this myself).

Html Audio Tag and cache

Im currently building a Web App which plays sound files now and again via JS:
var sound = new Audio();
function playSound(audioUrl) {
sound.src = audioUrl;
sound.play();
}
playSound("/my/audio/url.wav");
As timing is essential i want to preload all the possible soundfiles before the user can interact with the application. im doing this simply via this snippet of code:
var preloader = new Audio();
preloader.addEventListener("loadeddata", checkPreload(), true);
preloader.src = audioUrl;
This works fine and i can see in Chromes DevTools network-section that all the Soundfiles are loaded. Also when im playing a sound it does not make another request.
If i wait a minute however and play the sound again (without reloading the page or anything like that), then the wav file is reloaded from the webserver and taking again 200 ms of loading time.
I wonder how can i prevent this. Can i manually extend the life of the audiofile cache through some setting in the response headers on the server? Can i manipulate the Audio() Tag somehow to allow me to play the same audio without reloading it from the server?
I looked into the ServiceWorkers and AppCache, but ServiceWorkers doesent support all Browsers and AppCache seems to be also depricated on Chromium browsers, so i dont exactly know how to proceed here.
Any help would be appriciated,
cheers

Using createObjectURL, it will only request it once.
var url;
fetch("/path/to/file")
.then(response => response.blob())
.then(blob => {
url = URL.createObjectURL(blob);
// now you can play all the audio, setting the src value to the url variable
// e.g.
playSound(url);
});

Trouble with playing sounds from JavaScript [duplicate]

I'm trying to get sound working on my iPhone game using the Web Audio API. The problem is that this app is entirely client side. I want to store my mp3s in a local folder (and without being user input driven) so I can't use XMLHttpRequest to read the data. I was looking into using FileSystem but Safari doesn't support it.
Is there any alternative?
Edit: Thanks for the below responses. Unfortunately the Audio API is horribly slow for games. I had this working and the latency just makes the user experience unacceptable. To clarify, what I need is sounething like -
var request = new XMLHttpRequest();
request.open('GET', 'file:///./../sounds/beep-1.mp3', true);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(request.response, function(buffer) {
dogBarkingBuffer = buffer;
}, onError);
}
request.send();
But this gives me the errors -
XMLHttpRequest cannot load file:///sounds/beep-1.mp3. Cross origin requests are only supported for HTTP.
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101
I understand the security risks with reading local files but surely within your own domain should be ok?

I had the same problem and I found this very simple solution.
audio_file.onchange = function(){
var files = this.files;
var file = URL.createObjectURL(files[0]);
audio_player.src = file;
audio_player.play();
};
<input id="audio_file" type="file" accept="audio/*" />
<audio id="audio_player" />
You can test here:
http://jsfiddle.net/Tv8Cm/

Ok, it's taken me two days of prototyping different solutions and I've finally figured out how I can do this without storing my resources on a server. There's a few blogs that detail this but I couldn't find the full solution in one place so I'm adding it here. This may be considered a bit hacky by seasoned programmers but it's the only way I can see this working, so if anyone has a more elegent solution I'd love to hear it.
The solution was to store my sound files as a Base64 encoded string. The sound files are relatively small (less than 30kb) so I'm hoping performance won't be too much of an issue. Note that I put 'xxx' in front of some of the hyperlinks as my n00b status means I can't post more than two links.
Step 1: create Base 64 sound font
First I need to convert my mp3 to a Base64 encoded string and store it as JSON. I found a website that does this conversion for me here - xxxhttp://www.mobilefish.com/services/base64/base64.php
You may need to remove return characters using a text editor but for anyone that needs an example I found some piano tones here - xxxhttps://raw.github.com/mudcube/MIDI.js/master/soundfont/acoustic_grand_piano-mp3.js
Note that in order to work with my example you're need to remove the header part data:audio/mpeg;base64,
Step 2: decode sound font to ArrayBuffer
You could implement this yourself but I found an API that does this perfectly (why re-invent the wheel, right?) - https://github.com/danguer/blog-examples/blob/master/js/base64-binary.js
Resource taken from - here
Step 3: Adding the rest of the code
Fairly straightforward
var cNote = acoustic_grand_piano.C2;
var byteArray = Base64Binary.decodeArrayBuffer(cNote);
var context = new webkitAudioContext();
context.decodeAudioData(byteArray, function(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer;
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.noteOn(0);
}, function(err) { console.log("err(decodeAudioData): "+err); });
And that's it! I have this working through my desktop version of Chrome and also running on mobile Safari (iOS 6 only of course as Web Audio is not supported in older versions). It takes a couple of seconds to load on mobile Safari (Vs less than 1 second on desktop Chrome) but this might be due to the fact that it spends time downloading the sound fonts. It might also be the fact that iOS prevents any sound playing until a user interaction event has occured. I need to do more work looking at how it performs.
Hope this saves someone else the grief I went through.

Because ios apps are sandboxed, the web view (basically safari wrapped in phonegap) allows you to store your mp3 file locally. I.e, there is no "cross domain" security issue.
This is as of ios6 as previous ios versions didn't support web audio api

Use HTML5 Audio tag for playing audio file in browser.
Ajax request works with http protocol so when you try to get audio file using file://, browser mark this request as cross domain request. Set following code in request header -
header('Access-Control-Allow-Origin: *');

How to play ultrasonic WAV/PCM file in javascript without losing data

I need to loop an audio file (WAV, OGG or raw PCM) in the browser that contains segments which are unheard (ultrasonic) by the human ear (yet contain data which is valuable to me).
Using Chrome on Mac, I've noticed that if the segments of unheard sound are relatively short, I get all the data back (heard + unheard). In contrast, if the segments of unheard sound are longer than a certain threshold, it will fade out the whole channel quickly and effectively cancel the rest of the file completely, until the next loop cycle begins.
The way I'm loading and playing the sound is like so:
var b = msg.data; // binary msg received from websocket
b.type = "audio/wav";
var URLObject = window.webkitURL || window.URL;
var url = URLObject.createObjectURL(b);
var snd = document.createElement("audio");
snd.setAttribute("src", url);
snd.addEventListener("loadeddata", function() {
snd.loop = true;
snd.muted = false;
snd.play();
});
I'm looking for a way to cancel this automatic filtering of unheard sounds. Eventually, I would like a way to do this cross-browser. If not possible using JavaScript, a Flash solution will also be accepted.
Sample ultrasonic WAV files (~1MB each):
https://drive.google.com/file/d/0B5sMkxczD6sNbm04MmxMTmIwdlk/edit?usp=sharing
https://drive.google.com/file/d/0B5sMkxczD6sNal91WUhRNWo2d3c/edit?usp=sharing

There isn't a single approach that will work on all browsers, unfortunately.
For most browsers on the desktop and iOS too you can use the Web Audio API as shown here:
http://www.html5rocks.com/en/tutorials/webaudio/intro/
For IE/Android you need to use Flash to play a WAV/PCM, or play OGG with HTML5 Audio tag, but the latter may lose the ultrasonic frequencies.
So in general, you need to write code that will check what the current browser supports and use that, starting with Web Audio API, then trying HTML5 Audio, then Flash.

Streaming Video to Web Browser

I would like to display a live video stream in a web browser. (Compatibility with IE, Firefox, and Chrome would be awesome, if possible.) Someone else will be taking care of streaming the video, but I have to be able to receive and display it. I will be receiving the video over UDP, but for now I am just using VLC to stream it to myself for testing purposes. Is there an open source library that might help me accomplish this using HTML and/or JavaScript? Or a good website that would help me figure out how to do this on my own?
I've read a bit about RTSP, which seems like the traditional option for something like this. That might be what I have to fall back on if I can't accomplish this using UDP, but if that is the case I still am unsure of how to go about this using RTSP/RTMP/RTP, or what the differences between all those acronyms are, if any.
I thought HTTP adaptive streaming might be the best option for a while, but it seemed like all the solutions using that were proprietary (Microsoft IIS Smooth Streaming, Apple HTTP Live Streaming, or Adobe HTTP Dynamic Streaming), and I wasn't having much luck figuring out how to accomplish it on my own. MPEG-DASH sounded like an awesome solution as well, but it doesn't seem to be in use yet since it is still so new. But now I am told that I should expect to receive the video over UDP anyways, so those solutions probably don't matter for me anymore.
I've been Googling this stuff for several days without much luck on finding anything to help me implement it. All I can find are articles explaining what the technologies are (e.g. RTSP, HTTP Adaptive Streaming, etc.) or tools that you can buy to stream your own videos over the web. Your guidance would be greatly appreciated!

It is incorrect that most video sites use FLV, MP4 is the most widely supported format and it is played via Flash players as well.
The easiest way to accomplish what you want is to open a S3Amzon/cloudFront account and work with JW player. Then you have access to RTMP software to stream video and audio. This service is very cheap. if you want to know more about this, check out these tutorials:
http://www.miracletutorials.com/category/s3-amazon-cloudfront/ Start at the bottom and work your way up to the tutorials higher up.
I hope this will help you get yourself on your way.

If you don't need sound, you can send JPEGs with header like this:
Content-Type: multipart/x-mixed-replace
This is a simple demo with nodejs, it uses library opencv4nodejs to generate images. You can use any other HTTP server which allows to append data to the socket while keeping connection opened. Tested on Chrome and FF on Ubuntu Linux.
To run the sample you will need to install this library with npm install opencv4nodejs, it might take while, then start the server like this: node app.js, here is app.js itself
var http = require('http')
const cv = require('opencv4nodejs');
var m=new cv.Mat(300, 300, cv.CV_8UC3);
var cnt = 0;
const blue = new cv.Vec3(255, 220, 120);
const yellow = new cv.Vec3(255, 220, 0);
var lastTs = Date.now();
http.createServer((req, res) => {
if (req.url=='/'){
res.end("<!DOCTYPE html><style>iframe {transform: scale(.67)}</style><html>This is a streaming video:<br>" +
"<img src='/frame'></img></html>")
} else if (req.url=='/frame') {
res.writeHead(200, { 'Content-Type': 'multipart/x-mixed-replace;boundary=myboundary' });
var x =0;
var fps=0,fcnt=0;
var next = function () {
var ts = Date.now();
var m1=m.copy();
fcnt++;
if (ts-lastTs > 1000){
lastTs = ts;
fps = fcnt;
fcnt=0;
}
m1.putText(`frame ${cnt} FPS=${fps}`, new cv.Point2(20,30),1,1,blue);
m1.drawCircle(new cv.Point2(x,50),10,yellow,-1);
x+=1;
if (x>m.cols) x=0;
cnt++;
var buf = cv.imencode(".jpg",m1);
res.write("--myboundary\r\nContent-type:image/jpeg\r\nDaemonId:0x00258009\r\n\r\n");
res.write(buf,function () {
next();
});
};
next();
}
}).listen(80);
A bit later I've found this example with some more details in python: https://blog.miguelgrinberg.com/post/video-streaming-with-flask
UPDATE: it also works, if you stream this into html img tag.

True cross-browser streaming is only possible through "rich media" clients like Flash, which is why almost all video websites default to serving video using Adobe's proprietary .flv format.
For non-live video the advent of video embeds in HTML5 shows promise, and using Canvas and JavaSCript streaming should be technically possible, but handling streams and preloading binary video objects would have to be done in JavaScript and would not be very straightforward.