I would like to display a live video stream in a web browser. (Compatibility with IE, Firefox, and Chrome would be awesome, if possible.) Someone else will be taking care of streaming the video, but I have to be able to receive and display it. I will be receiving the video over UDP, but for now I am just using VLC to stream it to myself for testing purposes. Is there an open source library that might help me accomplish this using HTML and/or JavaScript? Or a good website that would help me figure out how to do this on my own?
I've read a bit about RTSP, which seems like the traditional option for something like this. That might be what I have to fall back on if I can't accomplish this using UDP, but if that is the case I still am unsure of how to go about this using RTSP/RTMP/RTP, or what the differences between all those acronyms are, if any.
I thought HTTP adaptive streaming might be the best option for a while, but it seemed like all the solutions using that were proprietary (Microsoft IIS Smooth Streaming, Apple HTTP Live Streaming, or Adobe HTTP Dynamic Streaming), and I wasn't having much luck figuring out how to accomplish it on my own. MPEG-DASH sounded like an awesome solution as well, but it doesn't seem to be in use yet since it is still so new. But now I am told that I should expect to receive the video over UDP anyways, so those solutions probably don't matter for me anymore.
I've been Googling this stuff for several days without much luck on finding anything to help me implement it. All I can find are articles explaining what the technologies are (e.g. RTSP, HTTP Adaptive Streaming, etc.) or tools that you can buy to stream your own videos over the web. Your guidance would be greatly appreciated!
It is incorrect that most video sites use FLV, MP4 is the most widely supported format and it is played via Flash players as well.
The easiest way to accomplish what you want is to open a S3Amzon/cloudFront account and work with JW player. Then you have access to RTMP software to stream video and audio. This service is very cheap. if you want to know more about this, check out these tutorials:
http://www.miracletutorials.com/category/s3-amazon-cloudfront/ Start at the bottom and work your way up to the tutorials higher up.
I hope this will help you get yourself on your way.
If you don't need sound, you can send JPEGs with header like this:
Content-Type: multipart/x-mixed-replace
This is a simple demo with nodejs, it uses library opencv4nodejs to generate images. You can use any other HTTP server which allows to append data to the socket while keeping connection opened. Tested on Chrome and FF on Ubuntu Linux.
To run the sample you will need to install this library with npm install opencv4nodejs, it might take while, then start the server like this: node app.js, here is app.js itself
var http = require('http')
const cv = require('opencv4nodejs');
var m=new cv.Mat(300, 300, cv.CV_8UC3);
var cnt = 0;
const blue = new cv.Vec3(255, 220, 120);
const yellow = new cv.Vec3(255, 220, 0);
var lastTs = Date.now();
http.createServer((req, res) => {
if (req.url=='/'){
res.end("<!DOCTYPE html><style>iframe {transform: scale(.67)}</style><html>This is a streaming video:<br>" +
"<img src='/frame'></img></html>")
} else if (req.url=='/frame') {
res.writeHead(200, { 'Content-Type': 'multipart/x-mixed-replace;boundary=myboundary' });
var x =0;
var fps=0,fcnt=0;
var next = function () {
var ts = Date.now();
var m1=m.copy();
fcnt++;
if (ts-lastTs > 1000){
lastTs = ts;
fps = fcnt;
fcnt=0;
}
m1.putText(`frame ${cnt} FPS=${fps}`, new cv.Point2(20,30),1,1,blue);
m1.drawCircle(new cv.Point2(x,50),10,yellow,-1);
x+=1;
if (x>m.cols) x=0;
cnt++;
var buf = cv.imencode(".jpg",m1);
res.write("--myboundary\r\nContent-type:image/jpeg\r\nDaemonId:0x00258009\r\n\r\n");
res.write(buf,function () {
next();
});
};
next();
}
}).listen(80);
A bit later I've found this example with some more details in python: https://blog.miguelgrinberg.com/post/video-streaming-with-flask
UPDATE: it also works, if you stream this into html img tag.
True cross-browser streaming is only possible through "rich media" clients like Flash, which is why almost all video websites default to serving video using Adobe's proprietary .flv format.
For non-live video the advent of video embeds in HTML5 shows promise, and using Canvas and JavaSCript streaming should be technically possible, but handling streams and preloading binary video objects would have to be done in JavaScript and would not be very straightforward.
Related
I want to send audio signal coming from my audio interface (focusrite saffire) to my nodejs server. How should I go about doing this? The easiest way would be to access the audio interface from the browser (html5) like capturing microphone output with getUserMedia (https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia), but couldn't find a way to access my audio interface through that library. Otherwise, I'm planning on creating a desktop application, but don't know if there is a function/library to allow access to my usb-connected audio interface.
This probably has little to do with the Javascript or the MediaDevices API. On Linux, using Firefox, PulseAudio is required to interface with your audio hardware. Since your soundcard is an Input/Output interface, you should be able to test it pretty easily by simply playing any sound file in the browser.
Most of PulseAudio configuration can be achieved using pavucontrol GUI. You should check the "configuration" tab and both "input device" & "output device" tabs to make sure your Focusrite is correctly set up and used as sound I/O.
Once this is done, you should be able to access an audio stream using the following (only available if on a "secure context", ie localhost or served through HTTPS, as stated in the MDN page you mentionned):
navigator.mediaDevices.getUserMedia({ audio: true })
.then(function(stream) {
// do whatever you want with this audio stream
})
(code snippet taken from the MDN page about MediaDevices)
Sending audio to the server is a whole other story. Have you tried anything? Are you looking for a real-time communication with the server? If so, I'd start by having a look at the WebSockets API. The WebRTC docs might be worth reading too, but it is more oriented to client-to-client communications.
How to use exact device id
Use media device constrain to pass exact device id.
An sample would be
const preferedDeviceName = '<your_device_name>'
const deviceList = await navigator.mediaDevices.enumerateDevices()
const audio = devices.find((device) => device.kind === 'audioinput' && device.label === preferedDeviceName)
const {deviceId} = audio;
navigator.mediaDevices.getUserMedia({audio: { deviceId }}
How to process and send to the backend
Possible duplicate of this
Still not clear follow this
I'm trying to get sound working on my iPhone game using the Web Audio API. The problem is that this app is entirely client side. I want to store my mp3s in a local folder (and without being user input driven) so I can't use XMLHttpRequest to read the data. I was looking into using FileSystem but Safari doesn't support it.
Is there any alternative?
Edit: Thanks for the below responses. Unfortunately the Audio API is horribly slow for games. I had this working and the latency just makes the user experience unacceptable. To clarify, what I need is sounething like -
var request = new XMLHttpRequest();
request.open('GET', 'file:///./../sounds/beep-1.mp3', true);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(request.response, function(buffer) {
dogBarkingBuffer = buffer;
}, onError);
}
request.send();
But this gives me the errors -
XMLHttpRequest cannot load file:///sounds/beep-1.mp3. Cross origin requests are only supported for HTTP.
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101
I understand the security risks with reading local files but surely within your own domain should be ok?
I had the same problem and I found this very simple solution.
audio_file.onchange = function(){
var files = this.files;
var file = URL.createObjectURL(files[0]);
audio_player.src = file;
audio_player.play();
};
<input id="audio_file" type="file" accept="audio/*" />
<audio id="audio_player" />
You can test here:
http://jsfiddle.net/Tv8Cm/
Ok, it's taken me two days of prototyping different solutions and I've finally figured out how I can do this without storing my resources on a server. There's a few blogs that detail this but I couldn't find the full solution in one place so I'm adding it here. This may be considered a bit hacky by seasoned programmers but it's the only way I can see this working, so if anyone has a more elegent solution I'd love to hear it.
The solution was to store my sound files as a Base64 encoded string. The sound files are relatively small (less than 30kb) so I'm hoping performance won't be too much of an issue. Note that I put 'xxx' in front of some of the hyperlinks as my n00b status means I can't post more than two links.
Step 1: create Base 64 sound font
First I need to convert my mp3 to a Base64 encoded string and store it as JSON. I found a website that does this conversion for me here - xxxhttp://www.mobilefish.com/services/base64/base64.php
You may need to remove return characters using a text editor but for anyone that needs an example I found some piano tones here - xxxhttps://raw.github.com/mudcube/MIDI.js/master/soundfont/acoustic_grand_piano-mp3.js
Note that in order to work with my example you're need to remove the header part data:audio/mpeg;base64,
Step 2: decode sound font to ArrayBuffer
You could implement this yourself but I found an API that does this perfectly (why re-invent the wheel, right?) - https://github.com/danguer/blog-examples/blob/master/js/base64-binary.js
Resource taken from - here
Step 3: Adding the rest of the code
Fairly straightforward
var cNote = acoustic_grand_piano.C2;
var byteArray = Base64Binary.decodeArrayBuffer(cNote);
var context = new webkitAudioContext();
context.decodeAudioData(byteArray, function(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer;
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.noteOn(0);
}, function(err) { console.log("err(decodeAudioData): "+err); });
And that's it! I have this working through my desktop version of Chrome and also running on mobile Safari (iOS 6 only of course as Web Audio is not supported in older versions). It takes a couple of seconds to load on mobile Safari (Vs less than 1 second on desktop Chrome) but this might be due to the fact that it spends time downloading the sound fonts. It might also be the fact that iOS prevents any sound playing until a user interaction event has occured. I need to do more work looking at how it performs.
Hope this saves someone else the grief I went through.
Because ios apps are sandboxed, the web view (basically safari wrapped in phonegap) allows you to store your mp3 file locally. I.e, there is no "cross domain" security issue.
This is as of ios6 as previous ios versions didn't support web audio api
Use HTML5 Audio tag for playing audio file in browser.
Ajax request works with http protocol so when you try to get audio file using file://, browser mark this request as cross domain request. Set following code in request header -
header('Access-Control-Allow-Origin: *');
I need to loop an audio file (WAV, OGG or raw PCM) in the browser that contains segments which are unheard (ultrasonic) by the human ear (yet contain data which is valuable to me).
Using Chrome on Mac, I've noticed that if the segments of unheard sound are relatively short, I get all the data back (heard + unheard). In contrast, if the segments of unheard sound are longer than a certain threshold, it will fade out the whole channel quickly and effectively cancel the rest of the file completely, until the next loop cycle begins.
The way I'm loading and playing the sound is like so:
var b = msg.data; // binary msg received from websocket
b.type = "audio/wav";
var URLObject = window.webkitURL || window.URL;
var url = URLObject.createObjectURL(b);
var snd = document.createElement("audio");
snd.setAttribute("src", url);
snd.addEventListener("loadeddata", function() {
snd.loop = true;
snd.muted = false;
snd.play();
});
I'm looking for a way to cancel this automatic filtering of unheard sounds. Eventually, I would like a way to do this cross-browser. If not possible using JavaScript, a Flash solution will also be accepted.
Sample ultrasonic WAV files (~1MB each):
https://drive.google.com/file/d/0B5sMkxczD6sNbm04MmxMTmIwdlk/edit?usp=sharing
https://drive.google.com/file/d/0B5sMkxczD6sNal91WUhRNWo2d3c/edit?usp=sharing
There isn't a single approach that will work on all browsers, unfortunately.
For most browsers on the desktop and iOS too you can use the Web Audio API as shown here:
http://www.html5rocks.com/en/tutorials/webaudio/intro/
For IE/Android you need to use Flash to play a WAV/PCM, or play OGG with HTML5 Audio tag, but the latter may lose the ultrasonic frequencies.
So in general, you need to write code that will check what the current browser supports and use that, starting with Web Audio API, then trying HTML5 Audio, then Flash.
I am currently using nodejs and express to stream video to tags simply by waiting for app.get on the video tag src address and then using res.writeHead and res.write to provide the data.
I would like to be able to do the something similar but with lower latency using WebRTC. However, I am bit confused as to how to achieve this and haven't really found any good information resource.
Can anyone recommend any good examples, nodejs packages etc... that might be helpful?
I was hoping to do something like:
// Nodejs Server
rtcServer.on('connection', function(connection) {
var videoSource = getVideoDataSource();
videoSource.on('data', function(data) {
connection.write(data);
});
});
rtcServer.listen(8000);
-
// HTML Client
<video src="???:8000"/>
I have seen a number of questions that don't answer this, is it possible to check someones bandwidth using java script and load specific content based on it?
The BBC seem to give me low quality images when using my mobile and in the middle of nowhere.
by the looks of this this cool service does this and its a CDN so it could be server side.
http://www.resrc.it/docs/
Does anyone know how they do it? or how I could do it using asp.net or javascript, or an community opensource plug in.
I think it may be possible with https://github.com/yahoo/boomerang/ but not sure this is its true purpose.
Basically you do this like this:
Start a timer
Load an fixed size file e.g a image through an ajax call
Stop the timer
Take some samples and compute the average badwidth
Somethign like this could work:
//http://upload.wikimedia.org/wikipedia/commons/5/51/Google.png
//Size = 238 KB
function measureBW(cnt, cb) {
var start = new Date().getTime();
var bandwidth;
var i = 0;
(function rec() {
var xmlHttp = new XMLHttpRequest();
xmlHttp.open('GET', 'http://upload.wikimedia.org/wikipedia/commons/5/51/Google.png', true);
xmlHttp.onreadystatechange = function () {
if (xmlHttp.readyState == 4) {
var x = new Date().getTime() - start;
bw = Number(((238 / (x / 1000))));
bandwidth = ((bandwidth || bw) + bw) / 2;
i++;
if (i < cnt) {
start = new Date().getTime();rec();
}
else cb(bandwidth.toFixed(0));
}
};
xmlHttp.send(null);
})();
}
measureBW(10, function (e) {
console.log(e);
});
Not that var xmlHttp = new XMLHttpRequest(); won't work on all browsers, you should check for the UserAgent and use the right one
And of course its just an estimated value.
Heres a JSBin example
Start a timer.
Send a AJAX request to your server, requesting a file of known size.
When the AJAX request's done loading, stop the timer, and calculate the bandwidth from the passed time and file size.
The problem with JavaScript is that users can disable it. (Which is more common on phones, that happen to be better off with smaller images)
I've knocked this up based on timing image downloads (ref: http://www.ehow.com/how_5804819_detect-connection-speed-javascript.html)
Word of warning though:
It says my speed is 1.81Mbps,
But according to SpeedTest.Net my speeds are this:
The logic of timing the download seems right but not sure if it's accurate?
Well, like I said in my comments, you can choose 2 approaches:
1) You are in the context of a mobile app, then you can query the technology used by the device directly so you can notify the server directly what type (and size) of content you area able to render. I think phone gap can help you with accessing some of the native mobile API's using JavaScript.
2) The server-timer thing. You can "serve" some files yourself, lets say you have a magic file in your landing page, that, as soon as the client request the file, you grab this HTTP request with a custom handler. You "manually" serve the file by writing to the output stream, and you measure the bytes send and the time it took to reach the EOF, then you can somehow measure the bandwith. Combine this with the session cookie and you have this information per connected browser.
While this isn't an answer, it may be important to note that measuring bandwidth isn't always reliable.
http://www.smashingmagazine.com/2013/01/09/bandwidth-media-queries-we-dont-need-em/
To paraphrase the above:
...the number of bits downloaded divided by the time it took to download them...is true when you download a large file over a single warmed-up TCP connection. That is rarely the case.
Typical page load scenario:
Initial HTML page is downloaded using slow-start mechanism, so measurement will significantly underestimate the available bandwidth
CSS and JavaScript external resources are loaded -- a collection of new TCP connections, all in their slow-start phase, and they are not all necessarily to the same destination server
Images are loaded -- multiple connections, each one downloading a resource. The problem is that these connections are not always in the same phase of their life cycle. Some might be in the slow-start phase; some may have suffered a packet loss and, thus, reduced their window and the bandwidth they are trying to fill; and some might be warmed-up TCP connections, ready to fill the bandwidth. These TCP connections are not necessarily all to the same destination server, and the bandwidth towards the various destination servers might be different between one another.
So, estimating bandwidth is possible, but it is far from simple, and it is possible only for certain phases of the page-loading process. And because having several TCP connections to various destination servers is common (for example, a CDN could host the image resources of a Web page), we cannot really tell what is the bandwidth we want to measure.
Since this is an older question, the alternative suggestion at the end of the article is to consider the more recent srcset attribute for responsive imagery, which lets the browser decide which asset to load based on whatever it knows (which should be more than us). It sounds like it's weighted more towards just determining resolution, but maybe it'll get smarter as support goes up.
I have released BwCh which is an open-source JavaScript API to detect bandwidth for web-based environments
It is built with ES2015. It uses some of the latest JavaScript innovation (window.navigator.connection currently supported in Chrome 48+ for Android as of April 2016) in order to provide a flexible method to detect bandwidth for both mobile and desktop devices. It fallbacks/complements to image pre-loading to detect bandwidth where those newest API are not available.