I am trying to stream hls on safari iOS with Aframe that has three.js under the hood. But the video shows a black screen with just the audio playing. The video src is of type .m3u8. I tried to read through a lot of related posts but none seem to have a proper solution. Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS? If not, can some one please help me with a solution.
A couple of discussions on the issues that are available on github:
HLS m3u8 video streaming
HLS on Safari
To your question:
Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS?
Yes, wishful thinking :-) The problem/issue/bug is with Apple, not any library. No matter what the JS library, A-Frame, Three, etc, this will always be an issue on any browser in iOS (all browsers on iOS are basically wrappers for Safari), and OSX Safari.
The issue is this (from my understanding):
At some point in the history of WebGL, there were restrictions on cross-origin content (videos, images, etc). I can't find a source for this, but I remember reading it somewhere, so this might not be 100% accurate.
Recently (a couple years ago? 2015?) all major browsers came to the conclusion that cross-origin media for use in WebGL was safe. Except Apple/Safari.
For most browsers, the crossorigin attribute on a <video> element could signal that this content came from another source. In Safari, for whatever reason, this attribute is ignored or not implemented. In fact, it looks like WebKit, which Safari is based on, fixed this as far back as 2015, but Apple still does not implement it. Even Apple refuses to comment on any progress.
Possible workarounds:
WebGL on Safari works with progressive (not a stream like HLS/Dash) .mp4 videos. Check out any 360 video on Facebook (website, not app) in iOS/Safari, and you'll note the source is an .mp4.
Use HLS (or Dash), but play the video flat, without WebGL. Check out any 360 video on YouTube (website, not app), and I think they are using HLS or Dash, but the point is they stream the video, whereas Facebook doesn't.
Here's a good starting point to the real issue: link.
Here's another detailed thread: link.
https://github.com/video-dev/hls.js#compatibility
Please note: iOS Safari "Mobile" does not support the MediaSource API.
Safari browsers have however built-in HLS support through the plain
video "tag" source URL. See the example above (Getting Started) to run
appropriate feature detection and choose between using Hls.js or
natively built-in HLS support.
When a platform has neither MediaSource nor native HLS support, you
will not be able to play HLS.
<script src="https://cdn.jsdelivr.net/npm/hls.js#latest"></script>
<!-- Or if you want a more recent canary version -->
<!-- <script src="https://cdn.jsdelivr.net/npm/hls.js#canary"></script> -->
<video id="video"></video>
<script>
var video = document.getElementById('video');
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource('https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function() {
video.play();
});
}
// hls.js is not supported on platforms that do not have Media Source Extensions (MSE) enabled.
// When the browser has built-in HLS support (check using `canPlayType`), we can provide an HLS manifest (i.e. .m3u8 URL) directly to the video element through the `src` property.
// This is using the built-in support of the plain video element, without using hls.js.
// Note: it would be more normal to wait on the 'canplay' event below however on Safari (where you are most likely to find built-in HLS support) the video.src URL must be on the user-driven
// white-list before a 'canplay' event will be emitted; the last video event that can be reliably listened-for when the URL is not on the white-list is 'loadedmetadata'.
else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = 'https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8';
video.addEventListener('loadedmetadata', function() {
video.play();
});
}
</script>
Related
I am trying to programatically find out whether some media will play or not based on it's URL.
Based on browser testing, HLS will play fine on my Chrome browser using the videoJs player video library, however CanIUse is saying it won't which is my first confusion.
I am then running it through this JavaScript...
using this npm package to get the mime-types
https://www.npmjs.com/package/mime-types
var mime = require('mime-types');
// get the extension of the file
var mimeType = mime.contentType(file.split('.').pop());
// returns 'application/vnd.apple.mpegurl'
var video = document.createElement('video');
console.log(Boolean(video.canPlayType(mimeType)));
Which returns false also, however it does play which is making me confused at how this is happening, does anyone have any ideas on why this is happening, or how I could correctly detect if the video will play or not?
The "Can I Use HLS" page lists the compatibility matrix for the HTML5 video tag, meaning that it checks if the browser has native support for that format. For example most desktop browsers except Safari on MacOS cannot play HLS natively.
That's why for formats like Apple HLS and MPEG-DASH you need a JavaScript player which will transmux (repackage the streams without re-encoding) that specific format into one that the video element can process natively.
Regarding codecs you might either go for the lowest common denominator like H.264 baseline with AAC, or provide different formats tailored to the targeted device with a fallback mechanism.
I'm currently attempting to integrate a custom audio filter on a video player handling both HLS and raw MP4 files. I'm having little to no issue integrating it on Chrome and Firefox - Safari on the other hand is not behaving accordingly. I've stumbled upon an inconsistency in the way safari routes the audio from a createMediaElementSource call.
I've created a small demo to illustrate the problem (Safari ONLY for native HLS support), I've expressedly left out any audio filter code as it has little relevance concerning the main issue :
https://codepen.io/edvincandonus/pen/ZEGVbPG
In the demo, once the audioContext is instantiated and "unlocked" via a user gesture, I create the MediaElementAudioSourceNode via audioCtx.createMediaElementSource(video) and leave it dangling (as in unconnected). As a consequence, the HTMLMediaElement will be re-routed into the processing graph of the AudioContext, and as no connections were made to the audioCtx.destination, audio playback should be missing when video playback starts.
This is the behaviour on Firefox. Chrome goes even further and blocks video playback if the MediaElementAudioSourceNode has no final destination node (try playing the MP4 from the demo on chrome)
As for Safari : this expectation is only met when the video source is a raw MP4 file - But when switched to an m3u8/hls stream, the HTMLMediaElement's audio is not correctly routed to the AudioContext as you can clearly hear the video's sound.
Is this a known safari limitation ? If not, would anyone have any workarounds to correctly access and route the audio from an HLS stream using Safari's native support for this protocol?
I've stumbled upon old stackoverflow answers indicating createMediaElementSource had always been buggy on Safari, but I figure they would have fixed it by now.
I'm using a .mp3 file, the .mp3 file plays okay when viewed directly and also when embeded using the HTML5 audio tag, however when creating the HTML5 audio tag in JS it does not play! (very strange)
I do not have this issue in any other browser/device, for example Desktop - Chrome works perfectly.
sound = document.createElement('audio');
sound.setAttribute('src', 'sound.mp3');
sound.play();
I've tested sound.canPlayType('audio/mpeg') and this produces true (so it is supported).
Perhaps there's a bug in Android - Chrome? (it is the latest version)
Looks like this is intended feature that spans more then just the Chrome browser. User interaction is required to get media elements to play.
Blink and WebKit have a setting for requiring a “user gesture” to play or pause an audio or video element, which is enabled in Opera for Android, Chrome for Android, the default Android browser, Safari for iOS and probably other browsers. This makes some sense, since mobile devices are used in public and in bed, where unsolicited sound from random Web sites could be a nuisance. Also, autoplaying video ads would waste bandwidth. Block Quote from 'blog.foolip.org'
Duplicate Threads from Other Users
Autoplay audio on mobile safari
How can I autoplay media in ios 4.2.1
Autoplay audio with ios 5 workaround?
Current Status
Developers have requested the deletion of 'mediaPlaybackRequiresUserGesture' which was reviewed and denied (for now). "We're going to gather some data about how users react to autoplaying videos in order to decide whether to keep this restriction."
Upon further inspection i found this...
"I misunderstood the outcome of the discussion (removing mediaPlaybackRequiresUserGesture) surrounding this topic. We need to keep this code in order to not break google.com while gathering data about this feature."
Google.com relies on the feature being disabled, otherwise it breaks (they didn't say what it breaks).
Original Bug Report
Try appending it to the document body.
document.body.appendChild(sound);
Though it is possible that mobile devices will not automatically play the audio or videos. If you are targeting mobile devices, autoplaying is considered bad practice since it can consume bandwidth. So it may be worth considering adding controls.
sound.setAttribute('controls', 'true');
OK, well, now that we know it won't work with audio, the only path left to you is to switch to the Web Audio API. You'll need to load the mp3 into an ArrayBuffer (e.g. using an XHR), then pass that to the decodeAudioData method, which gets you an Audio buffer that you can play back at will from an AudioBufferSourceNode.
Not every browser on every platform can play the mp3 audio format. Generally, as I would recommend, you should provide two <source> elements within your audio element, one providing the mp3 format, and another one providing the ogg vorbis format.
You can read more here: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
I have video which is delivered over HLS. Now I'd like to test in JavaScript if the device actually can play HLS video in HTML5.
Usually in Javascript I did something like
document.createElement('video').canPlayType('video/mp4')
However I can't figure out which 'type' is the right one for HLS.
Apple's Safari HTML5 Audio and Video Guide seems to suggest "vnd.apple.mpegURL" ("Listing 1-7 Falling back to a plug-in for IE")
<video controls>
<source src="HttpLiveStream.m3u8" type="vnd.apple.mpegURL">
<source src="ProgressiveDowload.mp4" type="video/mp4">
....
but canPlayType("vnd.apple.mpegURL") return an empty string even on iOS devices which can play actual HLS streams perfectly fine.
Is there any way to check for playback capabilities without 'external knowledge' (e.g. "check for iOS user agent and assume it can play hls")?
I know I can specify multiple sources in a element and the browser will use the first playable source. However in my case I need feed a single URL to JW Player which I can't modify. So somehow I need to find the "best playable URL" from a set of video encodings. (An open source JS library which handles source selection would be a nice workaround though.)
I haven't tested this across the board, but it looks like you should be testing for the full HLS mimetype application/vnd.apple.mpegURL instead of just just vnd.apple.mpegURL.
application/x-mpegURL and audio/mpegurl are also suitable mimetypes for the HLS m3u8 file. audio/x-mpegurl is also listed as an acceptable mimetype according to Apple, but it doesn't appear to be mentioned in the actual HLS draft spec.
In Safari on iOS and OS X,
document.createElement('video').canPlayType('application/vnd.apple.mpegURL')
returns maybe. I'm not sure if there are any other browsers that support HLS -- Android doesn't seem to like this syntax (despite some assertions I've seen to the contrary), and I believe that it may be due to the fact that the actual video playback is delegated to an external application, rather than the browser itself.
References:
http://developer.apple.com/library/ios/#technotes/tn2235/_index.html
http://www.longtailvideo.com/html5/hls
https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-03
http://developer.apple.com/library/safari/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Using_HTML5_Audio_Video.pdf
I'm looking for a web video player with which I can keep the full compatibility with iOS devices: iPad, iPhone, etc (so I would exclude all Flash video players).
Until now I've used Flowplayer but I have some problems:
the main problem is that using flv files I can start very fastly to play the video but I don't have any compatibility with iOS devices. Instead, using mp4 files, I have the full compatibility with iOS devices but before playing the video I have to wait that all content of the file has been loaded (few minutes).
So my question is: does it exist a video format that allows me to play videos very fastly and also compatible also with iOS devices?
Not talking about HTML5 just yet, lets assume you are first interested in supporting most users and legacy devices. Unless you are using an embedded player, (such as Flash), there is nothing inherent in all browsers and/or JavaScript to allow you to play a video that is standardized across these devices. If you simply reference a link to the video file, you are asking the device to natively download and decode the video file. This is why it does not typically begin playing until the entire file is downloaded. This is dependent on each device having a compatible MIME encoding configured for the file type which points to a player that the browser can invoke to handle the file. When you use something like Flowplayer, usually these Flash applications can begin playing video before it is fully downloaded because it knows how to download the video from your server over HTTP and once it has received enough of the video stream (buffered the video), it can begin playing it. Currently your best option is to use something like you have been using for most devices, and have a separate link to the mp4 for iOS devices. If you just re-encode any videos you already have in FLV or whatever older formats you have been using to mp4, you should be able to play that in a current version of any Flash based player, as Flash will work with those files as well as it's legacy formats.
I've found this resource and it seems to be very good: http://code.google.com/p/php-mobile-detect/