In firefox, I'm able to request a video stream of a window with
navigator.mediaDevices.getUserMedia({
video: {
mediaSource: 'window',
},
})
This produces a dialog like this:
I only care about the current window. Is there a way to specify in my call to getUserMedia that I would like the current tab (or window) only?
I don't think so no...
What FF implements here is not really specified yet, but w3c is working on a new API which will take care of Screen Capture: MediaDevices.getDisplayMedia.
While it's not what Firefox did implement, there is a clear paragraph in this paper about why deviceId can't and will not work on such requests:
Each potential source of capture is treated by this API as a discrete media source. However, display capture sources MUST NOT be enumerated by enumerateDevices, since this would reveal too much information about the host system.
Display capture sources therefore cannot be selected with the deviceId constraint, since this would allow applications to influence selection; setting deviceId constraint can only cause the resulting MediaStreamTrack to become overconstrained.
So, once again even if FF does not implement this API yet, we can assume they do follow this same rule for their current implementation, for the same reasons.
However, what will apparently be possible to do when this API will come to life, is to use the "browser" constraint, instead of "window". While the specs are not really clear as to what it is exactly ("a browser display surface, or single browser window"), I guess it will be closer to what you want than "window", and someone even asked 2 days ago for a "tab" constraint, could we even hope for a "current-tab" constraint? That might need someone to open an issue on w3c github's page.
Related
I have several audio tracks, which i got from getUserMedia (microphone). Those are being transmitted via WebRTC.
I want to highlight the stream that is active at the moment, but checking the documentation for MediaTrack i cannot find any method to access something that allows me to determine if that object is the most active at the moment.
So, if there is a method to get the current output, with some filter i should be able to determine which one is the "most active" one and highlight it.
Does such method exist in the API? Is there another approach that i can take for that?
MediaStream Object has APIs refer to detect active stream but not its MediaStreamTrack.
Even if you want to detect active speaker via volume level you need to pass MediaStream to Web audio API - AudioContext to analyse it. example
If you have proper RTCPeerConnection then you can use getStats API. example
MediaStreamTrack doesn't have such a property. You can use the webaudio api as done by hark to get a volume indication and then determine who is speaking.
Your mileage may vary though, active speaker detection is a hard problem.
I have read the section on The AnalyserNode Interface on the W3C docs, which states that the AnalyserNode will pass the input audio to the output untouched. It also describes the process of computing its "current frequency data".
I am wondering whether this processing of the input audio is done continuously or on-demand, f.i. when getFloatFrequencyData() is called.
Does anyone know? Is it browser specific?
It might depend on the browser, but as a performance optimization, it could be done only on demand. Of course, the node needs to keep enough information around to do the computation on demand, but that should be much cheaper than continuously computing the frequency data.
I asked Paul Adenot who was kind enough to link me to the source of Chrome and Firefox respectively:
Chrome: Updated when requested. https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/webaudio/RealtimeAnalyser.cpp?cl=GROK&gsn=discreteTimeConstantForSampleRate&rcl=1469603944&l=209
Firefox: Updated when requested. http://searchfox.org/mozilla-central/source/dom/media/webaudio/AnalyserNode.cpp#197
I have a site where I collect both the Flash screen resolution data using a .swf file and the same data from JavaScript and send it to my servers.
For some users, this data looks wrong - the Flash resolution is very occasionally larger or smaller than the reported screen resolution in the browser.
For example, one user has a 1600x900 resolution screen as reported in the JavaScript, but a screen size of 1366x768 as reported by Flash.
The ActionScript properties I am using are:
flash.system.Capabilities.screenResolutionX
flash.system.Capabilities.screenResolutionY
The JavaScript properties I am using are:
window.screen.width
window.screen.height
Are there any conditions when one would expect these values to be consistently incorrect in either ActionScript or JavaScript?
I imagine the issue lies in the fact that flash player's flash.system.Capabilities.screenResolutionX/Y properties only report the primary monitor's resolution. (as mentioned in the comments on the question by #akmozo)
From the documentation: (emphasis mine)
This property does not update with a user's screen resolution and instead only indicates the resolution at the time Flash Player or an Adobe AIR application started. Also, the value only specifies the primary screen.
In JS (and this may be dependent on browser), window.screen.width/height will report based on the actual monitor the browser window is on.
So your discrepancies are likely in the scenario that the browser window is NOT on the primary display. (So AS3 reports the primary display, and JS reports the actual display being used)
If you need to use Flash and get an accurate value, I'd recommend using one of the following techniques:
Use the stage.fullScreenWidth & stage.fullScreenHeight values which should give you the current monitors resolution.
Use ExternalInterface and get the data from JavaScript:
if(ExternalInterface.available){
var screenW:int = int(ExternalInterface.call("window.screen.width"));
var screenH:int = int(ExternalInterface.call("window.screen.height"));
}
I am creating a Video Sniffing Framework where I have to sniff different browsers' HTML5 Video playing capability. For that I used the canPlayType() method which is giving me three possible results:
the empty String (when unable to run the video)
"maybe"
"probably"
I need to know the exact difference between "maybe" and "probably". Please let me to know if anyone can resolve my confusion. Thanks in advance.
probably means that the browser can play the media type described. maybe means that the type might be playable. Usually, this is because the media type described is not specific enough to make a decision.
For example, the type audio/ogg may or may not be playable, because Ogg is a container type that could contain several different codecs. Vorbis and Opus are two Ogg-containable codecs. A browser's ability to play Ogg files in general says nothing about the browser's ability to play Vorbis or Opus codecs, so it can't say whether it can play your Ogg file.
If you ask about a specific codec with audio/ogg; codecs=vorbis, then the browser can say for sure whether it can play that type.
To make an analogy: suppose you ask me if I am able to drive your boat. I am good at driving tiny speedboats, but I cannot drive a massive cruise boat. I must answer the question "Can you drive my boat?" with "Maybe," because you haven't told me exactly what type of boat it is.
Stating the W3 specification: http://www.w3.org/TR/2011/WD-html5-20110113/video.html#mime-types
media.canPlayType(type) returns the empty string (a negative response), "maybe", or "probably" based on how confident the user agent is that it can play media resources of the given type.
More details are given on MDN: https://developer.mozilla.org/en/docs/Web/API/HTMLMediaElement#Methods
"probably": if the specified type appears to be playable.
"maybe": if it's impossible to tell whether the type is playable without playing it.
The empty string: if the specified type definitely cannot be played.
Also, in some cases (although that seems to happen only for <audio> elements), the returned value is "no" instead of the empty string:
http://24ways.org/2010/the-state-of-html5-audio
http://diveintohtml5.info/everything.html
Source : http://www.w3schools.com/tags/av_met_canplaytype.asp
The canPlayType() method can return one of the following values:
"probably" - the browser most likely supports this audio/video type
"maybe" - the browser might support this audio/video type
"" - (empty string) the browser does not support this audio/video
type
I have a server that generates pngs very rapidly and I need to make this into a poor-man's video feed. Actually creating a video feed is not an option.
What I have working right now is a recursive loop that looks a little like this (in pseudo-code):
function update() {
image.src = imagepath + '?' + timestamp; // ensures the image will update
image.onload = function () {update()};
}
This works, however after a while, it crashes the browser (Google Chrome, after more than 10 minutes or so). These images are being updated very frequently (several times a second). It seems the images are caching, which causes the browser to run out of memory.
Which of these solutions would solve the problem while maintaining fast refresh:
HTML5 canvas with drawImage
HTML5 canvas with CanvasPixelArray (raw pixel manipulation)
I have access to the raw binary as a Uint8Array, and the image isn't too large (less than 50 kb or so, 720 x 480 pixels).
Alternatively, is there anyway to clear old images from the cache or to avoid caching altogether?
EDIT:
Note, this is not a tool for regular users. It's a tool for diagnosing analog hardware problems for engineers. The reason for the browser is platform independence (should work on Linux, Windows, Mac, iPad, etc without any software changes).
The crashing is due to http://code.google.com/p/chromium/issues/detail?id=36142. Try creating object URLs (use XHR2 responseType = "arraybuffer" along with BlobBuilder) and revoking (using URL.revokeObjectURL) the previous frame after the next frame is loaded.
Edit: You really should be processing these into a live low-fps video stream on the server side, which will end up giving you greatly decreased latency and faster load times.
#Eli Grey seems to have identified the source of your crashing. It looks like they have a fix in the works, so if you don't want to modify your approach hopefully that will be resolved soon.
With regard to your other question, you should definitely stick with drawImage() if you can. If I understand your intention of using the CanvasPixelArray, you are considering iterating over each pixel in the canvas and updating it with your new pixel information? If so, this will be nowhere near as efficient as drawImage(). Furthermore, this approach is completely unnecessary for you because you (presumably) do not need to reference the data in the previous frame.
Whether fortunately or not, you cannot directly swap out the internal CanvasPixelArray object stored within an HTML5 canvas. If you have a properly-formatted array of pixel data, the only way you can update a canvas element is by calling either drawImage() or putImageData(). Right now, putImageData() is much slower than drawImage(), as you can see here: http://jsperf.com/canvas-drawimage-vs-putimagedata. If you have any sort of transparency in the frames of your video, you will likely want to clear the canvas and then use drawImage() (otherwise you could see through to previous frames).
Having said all that, I don't know that you really even need to use a canvas for this. Was your motivation for looking into using a canvas so that you could avoid caching (which now doesn't seem to be the underlying issue for you)?
If the "movie" is data-driven (ie. based on numbers and calculations), you may be able to push MUCH more data to the browser as text and then have javascript render it client-side into a movie. The "player" in the client can then request the data in batches as it needs it.
If not, one thing you could do is simply limit the frames-per-second (fps) of the script, possibly a hard-coded value, or a slider / setable value. Assuming this doesn't limit the utility of the tool, at the very least it would let the browser run longer w/o crashing.
Lastly, there are lots of things that can be done with headers (eg. in the .htaccess file) to indicate to browsers to cache or not cache content.
iPad, you say ?.. Nevertheless, i would advice using Flash/video or HTML5/video.
Because WebKit is very easily crashed with even moderate influx of images, either just big images or just a huge number of small ones..
From the other side, XHR with base64 image data or pixel array MIGHT work. I have had short polling app, which was able to run for 10-12 hours with XHR polling server every 10 seconds.
Also, consider delta compression, - like, if its histogram with abscissa being time scale - you can only send a little slice from the rigth, - of course, for things like heat-maps, you cannot do that.
These images are being updated very frequently (several times a
second).
.. if its critical to update at such a high rate - you MUST use long polling.