Use local files with MediaSource - javascript

I am trying to build a local video player in nwjs (node-webkit). I was able to play the local files by adding their path as the video element's src attribute, but now I want to make use of MediaSource and ,probably necessary, URL.createObjectUrl().
The problem is I haven't found any documentation that allowed me to achieve this, during my tests I am unable to append a new source to MediaSource as the local file. I tried direct paths and XHR requests, the closest I have been was with the XHR request of the file but I cannot convert the xhr.response into a usable item for my purpose, such as a objecturl.
For some reason just changing the src attribute directly each time a new video is selected causes the memory usage to grow constantly, which is why I would like to try doing this via the MediaSource api.
Since there's a lack of such information I would appreciate if anyone could help.

I was able to discover how this could be done, first there needs to be a mediasource object with which you use URL.createObjectUrl to link to the video.src, and then you create a buffer on it. It is to that buffer that is appended the media which is loaded via XMLHttpRequest with the Content-type set as an arraybuffer.
Careful with big files, if you don't segment them and load everything at once it will eat your ram and even crash your application.

Related

Share local blob data between two client-only JS applications deployed on different domains

I have two client-only (backend is static file server) applications written in JS. One is a recorder that records videos (e.g. the webcam) via the MediaRecorder API (this results in a Blob object). The other application is a video editor (don't mind how that one is implemented).
The two applications are deployed on different domains (e.g. foo-recorder.com and foo-editor.com). My goal is to make editing your recordings as easy as possible. I would like the recorder to have a button "edit recording in editor" that opens the editor with the recording already loaded.
A naive solution would be to URL.createObjectUrl(blob) in the recorder, URL-encode that URL and pass it as query parameter to the editor. E.g. https://foo-editor.com?video=blob://.... However, as far as I can tell, browsers do not allow sites to access blob URLs from another domain. (That's probably for good privacy reasons.)
I also thought about:
Storing the blob as a file on the user's device and pass the path to the other site. But we can't just willy nilly write files with JS.
Storing the blob in local storage: local storage usually is limited to a few MB and can't be access cross-domain either.
Passing the whole video base64 encoded as query parameter: that's ridiculous. No.
So I can't come up with a working solution. And I wouldn't be surprised if it doesn't work at all because it's probably very tricky privacy and security wise.
Does anyone have an idea how I could pass this blob from one app to the other? Obviously, the blob should not be uploaded.
Thanks to Musa's hint, I found a solution: postMessage allows me to pass the Blob to the editor, even if it's deployed on another domain. The recorder has to open a new browsing contect with the editor loaded. This can be:
windows.open to open a popup window
An <iframe> with the editor as src. Acquire the context via iframeDomNode.contentWindow.
It's now possible to use postMessage to send a message to that context:
const iframe = document.getElementById('editor-iframe');
iframe.contentWindow.postMessage(videoBlob, "*");
// ^ TODO: change "*" to the specific target origin in production code!!!
On the editor side, I can receive that message via:
window.addEventListener("message", event => {
// TODO: in real code, you should check `event.origin` here!
const v = document.getElementById("a-video-element");
v.src = URL.createObjectURL(event.data);
}, false);
Note: if you generate the blob URL on the recorder side, pass that to the editor, the editor still cannot access that. So you need to pass the blob directly.
Also note that sending a Blob via postMessage is not expensive and does not copy the Blob data.

Does browser cache blob src?

I'm using videoJS to play my videos which are hosted on Brightcove. I just give it a videoId and it automatically fetches the src which I guess gets translated into a blob src like src="blob:https://crap.crap".
However, it seems like when I remove the <video> component then put it back, the video gets refetched from the source and I'm not sure if the browser cached it or not. Does anyone know if it's possible to manually cache a blob src?
edit: It is streamed via this protocol

Is it possible to preload and cache video files without adding them to the DOM?

I'm working on a game that involves trigging one of 30 small video files depending on what result you get. As the videos need to play immediately after the user interacts, ideally I'd like to have the videos preloaded and ready to go.
I've added PreloadJS, queued up all of the assets I need.
Looking at the Network tab in inspector, I can see all 20mb of videos transferring on the loading screen.
However, when it comes time to play the clips, it seems to be re-downloading them rather than playing them from memory...
I thought that once the files were downloaded, they'd just stay in the browser cache, and once I tried to load a file with the same src, it would pull it from the pool of downloaded assets, but this doesn't seem to be the case...
Any idea how I can keep the downloaded files in memory without adding 30 video players to the page?
Thanks!
Ger
You could try to load the entire file into memory using Blob and Object-URL. This way the non-attached video element can play directly via the object-URL.
If it's a good strategy in regard to system resources is of course something you need to decide yourself.
Load through XHR as blob
Create Object URL: var url = (URL || webkitURL).createObjectURL(blob);
The video is now in memory, so when you need to play it, set it as source for the video element and you should be ready to go:
var video = document.createElement("video");
video.oncanplay = ...; // attach to DOM, invoke play() etc.
video.src = url; // set the object URL
An object-URL is kept in memory during the life-cycle of a page. You can manually revoke it this way if needed:
(URL || webkitURL).revokeObjectURL(url);

testing blob urls

I'm using the Window.URL.createObjectURL function to generate a blob url for a local video file, which I then use to set the source of a <video> element. This loads the video when the URL is first constructed, and everything works well. But when the web page is reloaded, the generated URL is no longer valid -- the browser automatically revokes the generated URL.
My question: Is there a way to determine if this Blob URL has actually been revoked? In other words, how do I determine if I can still use this Blob url using javascript, jquery, or whatever options are out there?
I came up with a simple solution, which works... albeit, probably isn't ideal.
Basically, I take the url, set the <video> src to that url, and then attach a jquery error event handler to it. If the error event is called (which it is, if the blob url has been revoked) I then prompt the user to reselect their video file.

How to save the window.URL.createObjectURL() result for future use?

I'm making an application in HTML5 where you choose a video file, and then the application plays it with the HTML5 video tag and the window.URL.createObjectURL(). The problem is that I want to save the data about this video in localStorage and play it again when the user uses my application, but as Mozilla MDN states about the results of this method:
Browsers will release these automatically when the document is unloaded
So is it possible to do what I'm trying to do? Or do the same thing without the window.URL.createObjectURL() but with something else?
I haven't used createObjectURL(), but if I understand correctly, it's essentially a temporary reference to a file or an in-memory object. If you want to save the actual video, it won't be useful, because the video itself will no longer be referenced by this pointer the next time the user visits the application.
I think you might be able to do this with a data: URL instead, as that URL actually includes the full data from the file. This example demonstrates using a FileReader to generate a data URL. I think you should be able to do this:
var reader = new FileReader();
reader.onload = function(e) {
var myDataUrl = e.target.result;
// do something with the URL in the DOM,
// then save it to local storage
};
reader.readAsDataURL(file);
Update: If you want to go up to 1GB, as you note in your comment, you'd probably be better served by the FileSystem API. This would require you to get the local file, save a copy of the file to persistent filesystem storage, and then use createObjectURL() to get a URL for the file copy. You still have a problem with disk space - you just added 1GB of duplicative file content to the user's filesystem - but I don't think it's possible to keep a persistent reference to a file outside of the browser sandbox otherwise.

Categories