Chrome is wrongly detecting can it play HLS video media - javascript

I am trying to programatically find out whether some media will play or not based on it's URL.
Based on browser testing, HLS will play fine on my Chrome browser using the videoJs player video library, however CanIUse is saying it won't which is my first confusion.
I am then running it through this JavaScript...
using this npm package to get the mime-types
https://www.npmjs.com/package/mime-types
var mime = require('mime-types');
// get the extension of the file
var mimeType = mime.contentType(file.split('.').pop());
// returns 'application/vnd.apple.mpegurl'
var video = document.createElement('video');
console.log(Boolean(video.canPlayType(mimeType)));
Which returns false also, however it does play which is making me confused at how this is happening, does anyone have any ideas on why this is happening, or how I could correctly detect if the video will play or not?

The "Can I Use HLS" page lists the compatibility matrix for the HTML5 video tag, meaning that it checks if the browser has native support for that format. For example most desktop browsers except Safari on MacOS cannot play HLS natively.
That's why for formats like Apple HLS and MPEG-DASH you need a JavaScript player which will transmux (repackage the streams without re-encoding) that specific format into one that the video element can process natively.
Regarding codecs you might either go for the lowest common denominator like H.264 baseline with AAC, or provide different formats tailored to the targeted device with a fallback mechanism.

Related

Expo video not showing on iOS [duplicate]

I am trying to stream hls on safari iOS with Aframe that has three.js under the hood. But the video shows a black screen with just the audio playing. The video src is of type .m3u8. I tried to read through a lot of related posts but none seem to have a proper solution. Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS? If not, can some one please help me with a solution.
A couple of discussions on the issues that are available on github:
HLS m3u8 video streaming
HLS on Safari
To your question:
Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS?
Yes, wishful thinking :-) The problem/issue/bug is with Apple, not any library. No matter what the JS library, A-Frame, Three, etc, this will always be an issue on any browser in iOS (all browsers on iOS are basically wrappers for Safari), and OSX Safari.
The issue is this (from my understanding):
At some point in the history of WebGL, there were restrictions on cross-origin content (videos, images, etc). I can't find a source for this, but I remember reading it somewhere, so this might not be 100% accurate.
Recently (a couple years ago? 2015?) all major browsers came to the conclusion that cross-origin media for use in WebGL was safe. Except Apple/Safari.
For most browsers, the crossorigin attribute on a <video> element could signal that this content came from another source. In Safari, for whatever reason, this attribute is ignored or not implemented. In fact, it looks like WebKit, which Safari is based on, fixed this as far back as 2015, but Apple still does not implement it. Even Apple refuses to comment on any progress.
Possible workarounds:
WebGL on Safari works with progressive (not a stream like HLS/Dash) .mp4 videos. Check out any 360 video on Facebook (website, not app) in iOS/Safari, and you'll note the source is an .mp4.
Use HLS (or Dash), but play the video flat, without WebGL. Check out any 360 video on YouTube (website, not app), and I think they are using HLS or Dash, but the point is they stream the video, whereas Facebook doesn't.
Here's a good starting point to the real issue: link.
Here's another detailed thread: link.
https://github.com/video-dev/hls.js#compatibility
Please note: iOS Safari "Mobile" does not support the MediaSource API.
Safari browsers have however built-in HLS support through the plain
video "tag" source URL. See the example above (Getting Started) to run
appropriate feature detection and choose between using Hls.js or
natively built-in HLS support.
When a platform has neither MediaSource nor native HLS support, you
will not be able to play HLS.
<script src="https://cdn.jsdelivr.net/npm/hls.js#latest"></script>
<!-- Or if you want a more recent canary version -->
<!-- <script src="https://cdn.jsdelivr.net/npm/hls.js#canary"></script> -->
<video id="video"></video>
<script>
var video = document.getElementById('video');
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource('https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function() {
video.play();
});
}
// hls.js is not supported on platforms that do not have Media Source Extensions (MSE) enabled.
// When the browser has built-in HLS support (check using `canPlayType`), we can provide an HLS manifest (i.e. .m3u8 URL) directly to the video element through the `src` property.
// This is using the built-in support of the plain video element, without using hls.js.
// Note: it would be more normal to wait on the 'canplay' event below however on Safari (where you are most likely to find built-in HLS support) the video.src URL must be on the user-driven
// white-list before a 'canplay' event will be emitted; the last video event that can be reliably listened-for when the URL is not on the white-list is 'loadedmetadata'.
else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = 'https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8';
video.addEventListener('loadedmetadata', function() {
video.play();
});
}
</script>

HTML5 audio tag does not work in Android - Chrome when created in JS?

I'm using a .mp3 file, the .mp3 file plays okay when viewed directly and also when embeded using the HTML5 audio tag, however when creating the HTML5 audio tag in JS it does not play! (very strange)
I do not have this issue in any other browser/device, for example Desktop - Chrome works perfectly.
sound = document.createElement('audio');
sound.setAttribute('src', 'sound.mp3');
sound.play();
I've tested sound.canPlayType('audio/mpeg') and this produces true (so it is supported).
Perhaps there's a bug in Android - Chrome? (it is the latest version)
Looks like this is intended feature that spans more then just the Chrome browser. User interaction is required to get media elements to play.
Blink and WebKit have a setting for requiring a “user gesture” to play or pause an audio or video element, which is enabled in Opera for Android, Chrome for Android, the default Android browser, Safari for iOS and probably other browsers. This makes some sense, since mobile devices are used in public and in bed, where unsolicited sound from random Web sites could be a nuisance. Also, autoplaying video ads would waste bandwidth. Block Quote from 'blog.foolip.org'
Duplicate Threads from Other Users
Autoplay audio on mobile safari
How can I autoplay media in ios 4.2.1
Autoplay audio with ios 5 workaround?
Current Status
Developers have requested the deletion of 'mediaPlaybackRequiresUserGesture' which was reviewed and denied (for now). "We're going to gather some data about how users react to autoplaying videos in order to decide whether to keep this restriction."
Upon further inspection i found this...
"I misunderstood the outcome of the discussion (removing mediaPlaybackRequiresUserGesture) surrounding this topic. We need to keep this code in order to not break google.com while gathering data about this feature."
Google.com relies on the feature being disabled, otherwise it breaks (they didn't say what it breaks).
Original Bug Report
Try appending it to the document body.
document.body.appendChild(sound);
Though it is possible that mobile devices will not automatically play the audio or videos. If you are targeting mobile devices, autoplaying is considered bad practice since it can consume bandwidth. So it may be worth considering adding controls.
sound.setAttribute('controls', 'true');
OK, well, now that we know it won't work with audio, the only path left to you is to switch to the Web Audio API. You'll need to load the mp3 into an ArrayBuffer (e.g. using an XHR), then pass that to the decodeAudioData method, which gets you an Audio buffer that you can play back at will from an AudioBufferSourceNode.
Not every browser on every platform can play the mp3 audio format. Generally, as I would recommend, you should provide two <source> elements within your audio element, one providing the mp3 format, and another one providing the ogg vorbis format.
You can read more here: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats

HTML5 base64 encoded audio on mobile devices

I'm writing a platform with an audio playback component. Audio is uploaded to the server as an wav/mp3/ogg file, and then (like the rest of our media), converted to base64 and stored within our redis database.
To play the audio back at the client side we make an AJAX request to the server for the base64 encoded audio. We have a desktop version that compliments the mobile application, at the moment audio playback works like this:
recording.sound = new Audio("data:audio/ogg;base64," + recording.audio);
recording.sound.play(); // this works
Today we started our tests on mobile devices, and have so far been unable to get it working, even on mobile browsers that apparently support HTML5 audio.
Any ideas? Or if this is not possible, is there a different approach we can take? Ideally there should be a mobile compatible version of the web app, and there has to be a phonegap version.
The reason might not be a technical one here, from Apple developer site:
In Safari on iOS (for all devices, including iPad), where the user may be on a cellular network and be charged per data unit, preload and autoplay are disabled. No data is loaded until the user initiates it. This means the JavaScript play() and load() methods are also inactive until the user initiates playback, unless the play() or load() method is triggered by user action. In other words, a user-initiated Play button works, but an onLoad="play()" event does not.
same applies to Android devices.
read more here: Safari HTML5 Audio and Video Guide
But „audio/wav“ doesn't exist. See spec here: http://www.iana.org/assignments/media-types/audio
You should use „audio/vnd.dts“ for .wav file, „audio/mpeg“ for .mp3 file and „audio/ogg“ for .ogg file...
OK, try StackOverflow search, see:
https://stackoverflow.com/search?q=audio+codec+support+mobile+devices+html5
or https://stackoverflow.com/search?q=audio+codec+support+mobile+devices+html
or try Google
Some search results, that might be useful:
In search for a library that knows to detect support for specific Audio Video format files
or html5 vs flash - full comparison chart anywhere?

HTML5 video: How to test for HLS playing capability? (video.canPlayType)

I have video which is delivered over HLS. Now I'd like to test in JavaScript if the device actually can play HLS video in HTML5.
Usually in Javascript I did something like
document.createElement('video').canPlayType('video/mp4')
However I can't figure out which 'type' is the right one for HLS.
Apple's Safari HTML5 Audio and Video Guide seems to suggest "vnd.apple.mpegURL" ("Listing 1-7 Falling back to a plug-in for IE")
<video controls>
<source src="HttpLiveStream.m3u8" type="vnd.apple.mpegURL">
<source src="ProgressiveDowload.mp4" type="video/mp4">
....
but canPlayType("vnd.apple.mpegURL") return an empty string even on iOS devices which can play actual HLS streams perfectly fine.
Is there any way to check for playback capabilities without 'external knowledge' (e.g. "check for iOS user agent and assume it can play hls")?
I know I can specify multiple sources in a element and the browser will use the first playable source. However in my case I need feed a single URL to JW Player which I can't modify. So somehow I need to find the "best playable URL" from a set of video encodings. (An open source JS library which handles source selection would be a nice workaround though.)
I haven't tested this across the board, but it looks like you should be testing for the full HLS mimetype application/vnd.apple.mpegURL instead of just just vnd.apple.mpegURL.
application/x-mpegURL and audio/mpegurl are also suitable mimetypes for the HLS m3u8 file. audio/x-mpegurl is also listed as an acceptable mimetype according to Apple, but it doesn't appear to be mentioned in the actual HLS draft spec.
In Safari on iOS and OS X,
document.createElement('video').canPlayType('application/vnd.apple.mpegURL')
returns maybe. I'm not sure if there are any other browsers that support HLS -- Android doesn't seem to like this syntax (despite some assertions I've seen to the contrary), and I believe that it may be due to the fact that the actual video playback is delegated to an external application, rather than the browser itself.
References:
http://developer.apple.com/library/ios/#technotes/tn2235/_index.html
http://www.longtailvideo.com/html5/hls
https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-03
http://developer.apple.com/library/safari/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Using_HTML5_Audio_Video.pdf

Javascript player for web video streaming compatible with iPad, iPhone, etc

I'm looking for a web video player with which I can keep the full compatibility with iOS devices: iPad, iPhone, etc (so I would exclude all Flash video players).
Until now I've used Flowplayer but I have some problems:
the main problem is that using flv files I can start very fastly to play the video but I don't have any compatibility with iOS devices. Instead, using mp4 files, I have the full compatibility with iOS devices but before playing the video I have to wait that all content of the file has been loaded (few minutes).
So my question is: does it exist a video format that allows me to play videos very fastly and also compatible also with iOS devices?
Not talking about HTML5 just yet, lets assume you are first interested in supporting most users and legacy devices. Unless you are using an embedded player, (such as Flash), there is nothing inherent in all browsers and/or JavaScript to allow you to play a video that is standardized across these devices. If you simply reference a link to the video file, you are asking the device to natively download and decode the video file. This is why it does not typically begin playing until the entire file is downloaded. This is dependent on each device having a compatible MIME encoding configured for the file type which points to a player that the browser can invoke to handle the file. When you use something like Flowplayer, usually these Flash applications can begin playing video before it is fully downloaded because it knows how to download the video from your server over HTTP and once it has received enough of the video stream (buffered the video), it can begin playing it. Currently your best option is to use something like you have been using for most devices, and have a separate link to the mp4 for iOS devices. If you just re-encode any videos you already have in FLV or whatever older formats you have been using to mp4, you should be able to play that in a current version of any Flash based player, as Flash will work with those files as well as it's legacy formats.
I've found this resource and it seems to be very good: http://code.google.com/p/php-mobile-detect/

Categories