Suppose I am receiving data from a video chat and I need to add the data to the video element in the HTML page.
So here is the code:
var payload = []; // This array keeps updating, since it is getting the data from the network using media stream.
var remoteVideo = document.getElementById('remoteVideo');
var buffer = new Blob([], { type: "video/x-matroska;codecs=avc1,opus" });
var url = URL.createObjectURL(buffer);
remoteVideo.src = url;
Now, I am getting data in the buffer. How do I update the url i have created instead of creating a new one again to view the video?
I think you might not need to update the url at all using MediaSource,the process goes like this:
Create a MediaSource Object.
Create an object URL from the MediaSource using createObjectURl
Set the video's src to the object URL
listen tosourceopen event and when it occurs, create a SourceBuffer instance.
Use SourceBuffer.appendBuffer() to add all of your chunks to the video.
But pay close attention to MediaSource limitations and considerations.
EDIT:
I found this Answer which explains the process described above more precisely and also points out the considerations you should take, and also an example.
Related
I made changes to an audio buffer like gain and panning, connected them to an audio context.
Now I want to save to a file with all the implemented changes.
Saving the buffer as is would give me the original audio without the changes.
Any idea of a method or a procedure existed to do that?
On way is to use a MediaRecorder to save the modified audio.
So, in addition to connecting to the destination, connect to a MediaStreamDestinationNode. This node has a stream object that you can use to initialize a MediaRecorder. Set up the recorder to save the data when data is available. When you're down recording, you have a blob that you can then download.
Many details are missing here, but you can find out how to use a MediaRecorder using the MDN example.
I found a solution, with OfflineAudioContext.
Here is an example with adding a gain change to my audio and saving it.
On the last line of the code I get the array buffer with the changes I made.
From there, I can go on saving the file.
let offlineCtx = new OfflineAudioContext(this.bufferNode.buffer.numberOfChannels, this.bufferNode.buffer.length, this.bufferNode.buffer.sampleRate);
let obs = offlineCtx.createBufferSource();
obs.buffer = this.buffer;
let gain = offlineCtx.createGain();
gain.gain.value = this.gain.gain.value;
obs.connect(gain).connect(offlineCtx.destination);
obs.start();
let obsRES = this.ctx.createBufferSource();
await offlineCtx.startRendering().then(r => {
obsRES.buffer = r;
});
Whenever I run this code
var blob = new Blob(["ninja.mp3"], {type:"audio/mp3"});
var audio = new Audio(URL.createObjectURL(blob));
audio.play().catch(err => console.log(err));
I am given the following error
DOMException index.html:3
I expect it to play the audio file ninja.mp3 but instead I'm faced with this error. Any help would be greatly appreciated.
When you do
var blob = new Blob(["ninja.mp3"], {type:"audio/mp3"});
What you just created is a Binary file in your browser's memory which holds the USVString ninja.mp3, and for which the browser will send a Content-Type: audio/mp3 header in some network actions.
Id est, you just created an UTF-8 text file. And yes, the MediaElement is not able to read that.
var blob = new Blob(["ninja.mp3"], {type:"audio/mp3"});
// read as text
new Response(blob).text().then(console.log);
For a comparison, here is what a real mp3 file looks like when read as text:
fetch("https://dl.dropboxusercontent.com/s/agepbh2agnduknz/camera.mp3")
.then(resp => resp.text())
.then(console.log)
Blob constructor doesn't expect an URL, but a list of Blob parts (which are either USVStrings, Blobs or ArrayBuffers), but in no way will it ever fetch anything.
So what you want seems to be as simple as
var audio = new Audio("ninja.mp3");
audio.play().catch(console.log);
But if one day you need to build a Blob (which you don't now), then be sure that what you pass in the Blob() constructor is actually the binary content of your file.
The DOMException interface represents an abnormal event (called an exception) which occurs as a result of calling a method or accessing a property of a web API. This is basically how error conditions are described in web APIs.
I think you call the method wrongly. Pls Check it.
I have an array of Blobs (binary data, really -- I can express it however is most efficient. I'm using Blobs for now but maybe a Uint8Array or something would be better). Each Blob contains 1 second of audio/video data. Every second a new Blob is generated and appended to my array. So the code roughly looks like so:
var arrayOfBlobs = [];
setInterval(function() {
arrayOfBlobs.append(nextChunk());
}, 1000);
My goal is to stream this audio/video data to an HTML5 element. I know that a Blob URL can be generated and played like so:
var src = URL.createObjectURL(arrayOfBlobs[0]);
var video = document.getElementsByTagName("video")[0];
video.src = src;
Of course this only plays the first 1 second of video. I also assume I can trivially concatenate all of the Blobs currently in my array somehow to play more than one second:
// Something like this (untested)
var concatenatedBlob = new Blob(arrayOfBlobs);
var src = ...
However this will still eventually run out of data. As Blobs are immutable, I don't know how to keep appending data as it's received.
I'm certain this should be possible because YouTube and many other video streaming services utilize Blob URLs for video playback. How do they do it?
Solution
After some significant Googling I managed to find the missing piece to the puzzle: MediaSource
Effectively the process goes like this:
Create a MediaSource
Create an object URL from the MediaSource
Set the video's src to the object URL
On the sourceopen event, create a SourceBuffer
Use SourceBuffer.appendBuffer() to add all of your chunks to the video
This way you can keep adding new bits of video without changing the object URL.
Caveats
The SourceBuffer object is very picky about codecs. These have to be declared, and must be exact, or it won't work
You can only append one blob of video data to the SourceBuffer at a time, and you can't append a second blob until the first one has finished (asynchronously) processing
If you append too much data to the SourceBuffer without calling .remove() then you'll eventually run out of RAM and the video will stop playing. I hit this limit around 1 hour on my laptop
Example Code
Depending on your setup, some of this may be unnecessary (particularly the part where we build a queue of video data before we have a SourceBuffer then slowly append our queue using updateend). If you are able to wait until the SourceBuffer has been created to start grabbing video data, your code will look much nicer.
<html>
<head>
</head>
<body>
<video id="video"></video>
<script>
// As before, I'm regularly grabbing blobs of video data
// The implementation of "nextChunk" could be various things:
// - reading from a MediaRecorder
// - reading from an XMLHttpRequest
// - reading from a local webcam
// - generating the files on the fly in JavaScript
// - etc
var arrayOfBlobs = [];
setInterval(function() {
arrayOfBlobs.append(nextChunk());
// NEW: Try to flush our queue of video data to the video element
appendToSourceBuffer();
}, 1000);
// 1. Create a `MediaSource`
var mediaSource = new MediaSource();
// 2. Create an object URL from the `MediaSource`
var url = URL.createObjectURL(mediaSource);
// 3. Set the video's `src` to the object URL
var video = document.getElementById("video");
video.src = url;
// 4. On the `sourceopen` event, create a `SourceBuffer`
var sourceBuffer = null;
mediaSource.addEventListener("sourceopen", function()
{
// NOTE: Browsers are VERY picky about the codec being EXACTLY
// right here. Make sure you know which codecs you're using!
sourceBuffer = mediaSource.addSourceBuffer("video/webm; codecs=\"opus,vp8\"");
// If we requested any video data prior to setting up the SourceBuffer,
// we want to make sure we only append one blob at a time
sourceBuffer.addEventListener("updateend", appendToSourceBuffer);
});
// 5. Use `SourceBuffer.appendBuffer()` to add all of your chunks to the video
function appendToSourceBuffer()
{
if (
mediaSource.readyState === "open" &&
sourceBuffer &&
sourceBuffer.updating === false
)
{
sourceBuffer.appendBuffer(arrayOfBlobs.shift());
}
// Limit the total buffer size to 20 minutes
// This way we don't run out of RAM
if (
video.buffered.length &&
video.buffered.end(0) - video.buffered.start(0) > 1200
)
{
sourceBuffer.remove(0, video.buffered.end(0) - 1200)
}
}
</script>
</body>
</html>
As an added bonus this automatically gives you DVR functionality for live streams, because you're retaining 20 minutes of video data in your buffer (you can seek by simply using video.currentTime = ...)
Adding to the previous answer...
make sure to add sourceBuffer.mode = 'sequence' in the MediaSource.onopen event handler to ensure the data is appended based on the order it is received. The default value is segments, which buffers until the next 'expected' timeframe is loaded.
Additionally, make sure that you are not sending any packets with a data.size === 0, and make sure that there is 'stack' by clearing the stack on the broadcasting side, unless you are wanting to record it as an entire video, in which case just make sure the size of the broadcast video is small enough, and that your internet speed is fast. The smaller and lower the resolution the more likely you can keep a realtime connection with a client, ie a video call.
For iOS the broadcast needs to made from a iOS/macOS application, and be in mp4 format. The video chunk gets saved to the app's cache and then removed once it is sent to the server. A client can connect to the stream using either a web browser or app across nearly any device.
I am trying to upload a video to server, and on client end. I am reading it using FileReader's readAsBinaryString().
Now, my problem is, I don't know how to read duration of this video file.
If i try reading the file, and assigning the reader's data to a video tag's source, then none of the events associated to the video tag are fired. I need to find the duration of file uploaded on client end.
Can somebody please suggest me something?
You can do something like this for that to work:
read the file as ArrayBuffer (this can be posted directly to server as a binary stream later)
wrap it in a Blob object
create an object URL for the blob
and finally set the url as the video source.
When the video object triggers the loadedmetadata event you should be able to read the duration.
You could use data-uri too, but notice that browsers may apply size limits (as well as other disadvantages) for them which is essential when it comes to video files, and there is a significant encoding/decoding overhead due to the Base-64 process.
Example
Select a video file you know the browser can handle (in production you should of course filter accepted file types based on video.canPlayType()).
The duration will show after the above steps has performed (no error handling included in the example, adjust as needed).
var fileEl = document.querySelector("input");
fileEl.onchange = function(e) {
var file = e.target.files[0], // selected file
mime = file.type, // store mime for later
rd = new FileReader(); // create a FileReader
rd.onload = function(e) { // when file has read:
var blob = new Blob([e.target.result], {type: mime}), // create a blob of buffer
url = (URL || webkitURL).createObjectURL(blob), // create o-URL of blob
video = document.createElement("video"); // create video element
video.preload = "metadata"; // preload setting
video.addEventListener("loadedmetadata", function() { // when enough data loads
document.querySelector("div")
.innerHTML = "Duration: " + video.duration + "s"; // show duration
(URL || webkitURL).revokeObjectURL(url); // clean up
// ... continue from here ...
});
video.src = url; // start video load
};
rd.readAsArrayBuffer(file); // read file object
};
<input type="file"><br><div></div>
you can do something like below, the trick is to use readAsDataURL:
var reader = new FileReader();
reader.onload = function() {
var media = new Audio(reader.result);
media.onloadedmetadata = function(){
media.duration; // this would give duration of the video/audio file
};
};
reader.readAsDataURL(file);
Fiddle Demo
I want to use the HTML5 FileApi to read a SWF to an OBJECT (or EMBED, if it's better to do?).
My current code crashes on Chrome/Iron (the only stable browser which also supports the xmlhttprequest v2 FormData). I got it to read image data into a on-the-fly created IMG. But the object one crashes the current tab in the browser.
else if (file.type == "application/x-shockwave-flash") {
var show = document.createElement("object");
show.type = "application/x-shockwave-flash"
show.style.width = "100%";
show.style.height = "100%";
show.id = "thumb";
document.getElementById("thumbnails").appendChild(show);
var reader = new FileReader();
reader.onload = (function (aImg) {
return function (e) { aImg.data = e.target.result; };
})(show);
reader.readAsDataURL(file);
Do I really read to the object.data part? How is it done right? Anybody know? Or is this incomplete and I have to wait for better implementation?
A few things I'd recommend trying (in order of increasing complexity):
base64 encode the data with btoa and set it using a data: URI,
instead of creating the object using createElement, construct the <object> tag with all attributes as an HTML string (including the base64 advice above), then inject it into a DOM element with innerHTML,
create a reflector web service where you POST the swf content, it gives you a URL, then pass the URL off to the object,
similar to the previous, create a reflector web service where you POST the swf content, targeting a full-screen IFRAM as the target, have the service spits back an HTML doc including an <object> pointing back to the server.
The later of these options is more intense, and requires round-trips from the server that you'd probably want to avoid - just some more options you might want to consider.
ActionScript 3 has a Loader which may be useful as well. I don't know if it supports data: URI's, but if it does, you could write a boot loader SWF which runs the contents of the local swf file directly.