I want to record user webcam stream using web's MediaRecorder and generate a file which has:
correctly set duration (which can be seen and used by browsers e.g. html5 video player shows correct duration when video is in preload state)
correctly set codecs for both audio and video tracks
Here you can see a fully working example (Chrome: 54.0.2840.71, Firefox: 49.0.2 -> PLEASE USE RELATIVELY NEW BROWSERS SINCE APIS USED IN THE DEMO ARE QUITE RECENT): https://jsfiddle.net/2vr7vL2p/12/ where all works perfectly besides:
in chrome the duration returns Infinity (in firefox it works fine)
in both firefox and chrome resulting file has broken codecs (for instance VLC media player returns: VLC does not support the audio or video format "undf". Unfortunately there is no way for you to fix this.)
How can one solve those issues?
Related
I am trying to stream hls on safari iOS with Aframe that has three.js under the hood. But the video shows a black screen with just the audio playing. The video src is of type .m3u8. I tried to read through a lot of related posts but none seem to have a proper solution. Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS? If not, can some one please help me with a solution.
A couple of discussions on the issues that are available on github:
HLS m3u8 video streaming
HLS on Safari
To your question:
Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS?
Yes, wishful thinking :-) The problem/issue/bug is with Apple, not any library. No matter what the JS library, A-Frame, Three, etc, this will always be an issue on any browser in iOS (all browsers on iOS are basically wrappers for Safari), and OSX Safari.
The issue is this (from my understanding):
At some point in the history of WebGL, there were restrictions on cross-origin content (videos, images, etc). I can't find a source for this, but I remember reading it somewhere, so this might not be 100% accurate.
Recently (a couple years ago? 2015?) all major browsers came to the conclusion that cross-origin media for use in WebGL was safe. Except Apple/Safari.
For most browsers, the crossorigin attribute on a <video> element could signal that this content came from another source. In Safari, for whatever reason, this attribute is ignored or not implemented. In fact, it looks like WebKit, which Safari is based on, fixed this as far back as 2015, but Apple still does not implement it. Even Apple refuses to comment on any progress.
Possible workarounds:
WebGL on Safari works with progressive (not a stream like HLS/Dash) .mp4 videos. Check out any 360 video on Facebook (website, not app) in iOS/Safari, and you'll note the source is an .mp4.
Use HLS (or Dash), but play the video flat, without WebGL. Check out any 360 video on YouTube (website, not app), and I think they are using HLS or Dash, but the point is they stream the video, whereas Facebook doesn't.
Here's a good starting point to the real issue: link.
Here's another detailed thread: link.
https://github.com/video-dev/hls.js#compatibility
Please note: iOS Safari "Mobile" does not support the MediaSource API.
Safari browsers have however built-in HLS support through the plain
video "tag" source URL. See the example above (Getting Started) to run
appropriate feature detection and choose between using Hls.js or
natively built-in HLS support.
When a platform has neither MediaSource nor native HLS support, you
will not be able to play HLS.
<script src="https://cdn.jsdelivr.net/npm/hls.js#latest"></script>
<!-- Or if you want a more recent canary version -->
<!-- <script src="https://cdn.jsdelivr.net/npm/hls.js#canary"></script> -->
<video id="video"></video>
<script>
var video = document.getElementById('video');
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource('https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function() {
video.play();
});
}
// hls.js is not supported on platforms that do not have Media Source Extensions (MSE) enabled.
// When the browser has built-in HLS support (check using `canPlayType`), we can provide an HLS manifest (i.e. .m3u8 URL) directly to the video element through the `src` property.
// This is using the built-in support of the plain video element, without using hls.js.
// Note: it would be more normal to wait on the 'canplay' event below however on Safari (where you are most likely to find built-in HLS support) the video.src URL must be on the user-driven
// white-list before a 'canplay' event will be emitted; the last video event that can be reliably listened-for when the URL is not on the white-list is 'loadedmetadata'.
else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = 'https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8';
video.addEventListener('loadedmetadata', function() {
video.play();
});
}
</script>
I am trying to programatically find out whether some media will play or not based on it's URL.
Based on browser testing, HLS will play fine on my Chrome browser using the videoJs player video library, however CanIUse is saying it won't which is my first confusion.
I am then running it through this JavaScript...
using this npm package to get the mime-types
https://www.npmjs.com/package/mime-types
var mime = require('mime-types');
// get the extension of the file
var mimeType = mime.contentType(file.split('.').pop());
// returns 'application/vnd.apple.mpegurl'
var video = document.createElement('video');
console.log(Boolean(video.canPlayType(mimeType)));
Which returns false also, however it does play which is making me confused at how this is happening, does anyone have any ideas on why this is happening, or how I could correctly detect if the video will play or not?
The "Can I Use HLS" page lists the compatibility matrix for the HTML5 video tag, meaning that it checks if the browser has native support for that format. For example most desktop browsers except Safari on MacOS cannot play HLS natively.
That's why for formats like Apple HLS and MPEG-DASH you need a JavaScript player which will transmux (repackage the streams without re-encoding) that specific format into one that the video element can process natively.
Regarding codecs you might either go for the lowest common denominator like H.264 baseline with AAC, or provide different formats tailored to the targeted device with a fallback mechanism.
I'm currently attempting to integrate a custom audio filter on a video player handling both HLS and raw MP4 files. I'm having little to no issue integrating it on Chrome and Firefox - Safari on the other hand is not behaving accordingly. I've stumbled upon an inconsistency in the way safari routes the audio from a createMediaElementSource call.
I've created a small demo to illustrate the problem (Safari ONLY for native HLS support), I've expressedly left out any audio filter code as it has little relevance concerning the main issue :
https://codepen.io/edvincandonus/pen/ZEGVbPG
In the demo, once the audioContext is instantiated and "unlocked" via a user gesture, I create the MediaElementAudioSourceNode via audioCtx.createMediaElementSource(video) and leave it dangling (as in unconnected). As a consequence, the HTMLMediaElement will be re-routed into the processing graph of the AudioContext, and as no connections were made to the audioCtx.destination, audio playback should be missing when video playback starts.
This is the behaviour on Firefox. Chrome goes even further and blocks video playback if the MediaElementAudioSourceNode has no final destination node (try playing the MP4 from the demo on chrome)
As for Safari : this expectation is only met when the video source is a raw MP4 file - But when switched to an m3u8/hls stream, the HTMLMediaElement's audio is not correctly routed to the AudioContext as you can clearly hear the video's sound.
Is this a known safari limitation ? If not, would anyone have any workarounds to correctly access and route the audio from an HLS stream using Safari's native support for this protocol?
I've stumbled upon old stackoverflow answers indicating createMediaElementSource had always been buggy on Safari, but I figure they would have fixed it by now.
I'm using a .mp3 file, the .mp3 file plays okay when viewed directly and also when embeded using the HTML5 audio tag, however when creating the HTML5 audio tag in JS it does not play! (very strange)
I do not have this issue in any other browser/device, for example Desktop - Chrome works perfectly.
sound = document.createElement('audio');
sound.setAttribute('src', 'sound.mp3');
sound.play();
I've tested sound.canPlayType('audio/mpeg') and this produces true (so it is supported).
Perhaps there's a bug in Android - Chrome? (it is the latest version)
Looks like this is intended feature that spans more then just the Chrome browser. User interaction is required to get media elements to play.
Blink and WebKit have a setting for requiring a “user gesture” to play or pause an audio or video element, which is enabled in Opera for Android, Chrome for Android, the default Android browser, Safari for iOS and probably other browsers. This makes some sense, since mobile devices are used in public and in bed, where unsolicited sound from random Web sites could be a nuisance. Also, autoplaying video ads would waste bandwidth. Block Quote from 'blog.foolip.org'
Duplicate Threads from Other Users
Autoplay audio on mobile safari
How can I autoplay media in ios 4.2.1
Autoplay audio with ios 5 workaround?
Current Status
Developers have requested the deletion of 'mediaPlaybackRequiresUserGesture' which was reviewed and denied (for now). "We're going to gather some data about how users react to autoplaying videos in order to decide whether to keep this restriction."
Upon further inspection i found this...
"I misunderstood the outcome of the discussion (removing mediaPlaybackRequiresUserGesture) surrounding this topic. We need to keep this code in order to not break google.com while gathering data about this feature."
Google.com relies on the feature being disabled, otherwise it breaks (they didn't say what it breaks).
Original Bug Report
Try appending it to the document body.
document.body.appendChild(sound);
Though it is possible that mobile devices will not automatically play the audio or videos. If you are targeting mobile devices, autoplaying is considered bad practice since it can consume bandwidth. So it may be worth considering adding controls.
sound.setAttribute('controls', 'true');
OK, well, now that we know it won't work with audio, the only path left to you is to switch to the Web Audio API. You'll need to load the mp3 into an ArrayBuffer (e.g. using an XHR), then pass that to the decodeAudioData method, which gets you an Audio buffer that you can play back at will from an AudioBufferSourceNode.
Not every browser on every platform can play the mp3 audio format. Generally, as I would recommend, you should provide two <source> elements within your audio element, one providing the mp3 format, and another one providing the ogg vorbis format.
You can read more here: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
I have flowplayer running on a page in a slight variation of the playlist demo on their site, and for the most part it seems okay.
However, when I pause the video and switch to another video in the playlist on Chrome, once in a while, it will say HTML5 video not found (it will list the current video, not the one selected).
This will proceed to break the player and the page, forcing a refresh. The message also seems to show up on page refresh sometimes, but that doesn't matter much since it will be reloaded.
I am not sure what is wrong, but I have two theories.
1) MP4 file not streamable
-possible but unlikely considering I can jump around the video easily
2) Timeout from S3
-maybe, but I don't see any errors.
Has anyone seen this/know how to debug it?
The first thing to do would be to test this in another browser. If it works ok then it will be a video compatibility issue with the browser.
Mozilla has a very good article on Media formats supported. In it it states:
The MP4 container format with the H.264 video codec and either the AAC audio codec or the MP3 audio codec is natively supported by Internet Explorer, Safari and Chrome, but Chromium and Opera do not support the format. Firefox will soon support the format, but only when a third-party decoder is available.
For the best cross browser support you really need two video formats MP4 and WebM