Video textures in Three.js with Google Cardboard - javascript

I want to implement a video texture in Three.js following the method used in this example: http://stemkoski.github.io/Three.js/Video.html . However, I'd also like to use my site with the Google Cardboard Chrome API, as detailed here: https://vr.chromeexperiments.com/ , yet when I test the program on Chrome for Android, I get a 'S3TC textures not supported' error. Is there a way around this error?

It's not an error, it just that S3TC is not supported. It's a file format error - like asking a program that only supports JPG to read an PNG. You need to either convert the video to an accepted format, or perhaps see if there's a way of getting the Chrome API to turn on OpenGL extensions, the one you want is EXT_texture_compression_s3tc - if you can access this then you can read in S3TC files. S3TC is usually targeted on mobile devices - where file size is critical. If it's just for a web browser you might want to investigate useing a more standardized video format - like H.264
You can also looks for a library that supports the formats you want to use and if they can uncode a frame, you can directly pass it to the graphics API.

Related

Is there a way for WebGL/THREE.js to read a video DOM element into YUV textures?

In most platform specific APIs, there is an option to read the video frame as it's native pixel format, YUV.
However, the DOM Video Element wrapped with a THREE.Texture, always returns RGB.
Is there a way to get textures in the native pixel format of YUV?
I am using JS and THREE.js, but if it's possible in WebGL, that's also good.
There is the incoming Web codecs API which should expose a Plane interface from where we should be able to read the YUV (already available for testing in Chrome 86, under a flag).
This is all still very experimental, it's still not well defined how this can then be used with webgl, though it is apparently already doable (I personally don't know how), but there will definitely be some work in the near future around that, you can follow this repo for more info.
No, it's not possible. Or at least the browser itself provides no API to get video data in YUV format.

How to generate a video file on a browser

I'm working on a project with kinetic typography where the user can write something and download it on a video format but I'm having trouble generating the (video) file.
I thought of screen capturing but most browsers are blocking those features due to security concerns (and I wanted to add audio later on).
I've looked into https://www.w3.org/TR/screen-capture/ but its documentation doesn't say much nor gives examples.
I've seen https://mgechev.github.io/jscapture/ but it only works with versions bellow Chrome 37.
I've seen https://github.com/spite/ccapture.js but it only records canvases and I'm working with text.
How can I generate a video client-side (preferably) on a browser?

Capture images in web browser

I am working on Image Processing projects.
I want to capture the images(Face) from the web-camera using web browser and i am not allowed to install any additional plug-ins.
Currently, i am using Flex/Flash to capture the images.As per the requirement, i am also not allowed to use Flex/Flash.
Is there any package to capture images from webcamera?
Note: Actually, i want to apply some image processing algorithms like face Detection , filters at browser end. I will be good if the same package used for image capturing has some image processing algorithms.
Updated by author of the post:
Can anybody tell me the possibility with webgl?
If you wanted to let users take a snapshot of themselves with the webcam, that's possible with capture=camera in one line using HTML5'sgetUserMedia/Stream API. here is how:
<input type="file" accept="image/*;capture=camera">
Supported in latest chrome, firefox & opera. for others, flash or other plugins are the only option.
Link: can I use webcam
As Gaurish mentions, getUserMedia is what you can use for this, although its cross-browser support is not guaranteed.
To see how you can use this, and canvas to get the image, take a look at Filtering a webcam using getUserMedia and HTML5 canvas which uses it.

How to save an image with JavaScript?

I'm working on a Chrome Extension: when you drag an image, it will be saved to your computer.
I learned that in HTML5 there is FileWriter API, but really can't find any example of it, and does Chrome support it?
No, there isn't a FileWriter API in HTML5. What you probably mean is the File API that allows you to read files. And in Chrome even extensions aren't allowed to write files, for reasons of security. So unless you want to bundle an NPAPI plugin with your extension (which would trigger a huge warning upon installation) all you can do is trigger a download message that the user might choose to accept - or not. See Cross-browser Save As .txt for a possible approach (Flash objects like Downloadify being the other).
Edit: I was wrong, there is a FileWriter API proposal. It is very far from being done however.
I found this. http://www.html5rocks.com/en/tutorials/file/filesystem/
You can find some examples.
Edit This is screenshot from the article. I'm using chrome 12.0
FileWriter API browser support message http://s3.amazonaws.com/twitpic/photos/full/329503613.png?AWSAccessKeyId=AKIAJF3XCCKACR3QDMOA&Expires=1308932374&Signature=DXBdFSjbNqaeJPr%2F0fSAqPWyh2E%3D
I don't think the FileWriter API will be ready and usable for some time yet.
You could get the image data in hex, then use a DataURI to 'export' it from the browser. Although this leads to a file saved with a filename such as "download(1)". Each browser seems to have different size limitations for DataURIs, and they're not big, although ahould be fine for a reasonably sized image.
http://en.wikipedia.org/wiki/Data_URI_scheme
Alternatively you could use a Downloadify to save it with a proper filename (Requires flash and may be tricky to embed into the chrome extension).
http://davidwalsh.name/downloadify

HTML5 Local Storage of audio element source - is it possible?

I've been experimenting with the audio and local storage features of html5 of late and have run into something that has me stumped.
I'd like to be able to cache or store the source of the audio element locally to enable speedier and offline playback. The problem is I can't see how this is possible with the current implementation.
I have tried the following using WebKit:
Creating a manifest file to set up local caching but the audio file appears not to be a cacheable item maybe due to the way it is stream or something
I have also attempted to use javascript to put an audio object into local storage but the size of the mp3 makes this impossible due to memory issues (i think).
I have tried to use the data uri and base64 to use the html as a audio transport that can be cached but again the filesize makes this prohibitive. Also the audio element does not seem to like this in WebKit (works fine in mozilla)
I have tried several methods of putting the data into the local database store. Again suffering the same issues as the other cases.
I'd love to hear any other ideas anyone may have as to how I could achieve my goal of offline playback using caching/local storage in WebKit.
I've been trying to do this myself, on the iOS (for iPhone / iPad) but it refuses to cache audio files in offline, even if in Cache Manifest.
It does not error, but instead simply pretends to have played the audio element if invoked via JavaScript without a control. If it's embedded with a control, it displays an alternate control that says "Cannot play audio file.". It works fine if the application is able to go online.
It seems not to cache the audio, playing another sound resource seems to cause it to clear the previous resource from memory - this is pretty worthless functionality even when online.
I have experimented with base64 encoding the audio as data URI's. This works in Safari on the desktop (at least for fairly short samples of around 20-30k that I've been using) but seems not to be supported at all on iOS - it silently does nothing, which is highly annoying.
I don't know about other vendors - Google Chrome used to not support data URI's for audio but perhaps they fixed it... - it seems like it's not possible for now though.
Update: Minor discrepancy with iPhone OS 3.x (tested with 3.1.2): If an audio element is specified in an offline web app but it doesn't have a control, it displays a non-interactive control with a non-animated spinner on it (which it definitely shouldn't do). I assume this is fixed in iOS 4.x (which should be out next week).
So it's been a while since I asked this question and I thought i'd give some info about how we solved it. Basically we encoded the data into PNG's using a similar technique to this:
http://audioscene.org/scene-files/yury/pngencoding/sample.html
Then cached the image on the mobile device using html5 local storage and accessed it as needed. The PNG's were pretty big but this worked for us.
I spent a while trying to do this for a game I'm making, and since as far as I could tell browsers (Firefox and Chrome) still don't support caching of audio elements I thought I'd post the solution I found.
There is a workaround described here: http://dougx.net/plunder/index.php#code
I can confirm it works pretty well, but is probably better suited to smaller files. As he describes here (http://dougx.net/plunder/GameSounds.txt), you encode the audio as base64 strings, and give them a data:audio/ogg;base64 (or any compatible audio format) header, which HTML5 audio can then read in. Because this is just a string, the browser will cache it.
I guess it would be preferable to get the manifest approach working, since this feels like the most relevant mechanism for locally caching the file.
What happens if you alter the audio file's HTTP headers, e.g. Content-Type and Expires? Does the browser do something different if the file extension is changed?
I see you've had no luck so far.
You might want to take a look at JAI (JavaScript Audio Interface) ("the world's first javascript interface for web <audio>"). Or get in touch with Alastair MacDonald, who wrote it.
Failing that, the HTML5 Doctor may be able to assist.
Adding video and audio files to local storage works with iOS 4.3.
I just added a video and an audio file to manifest and they both got downloaded to offline storage on iPad.

Categories