How can I decode a JPEG in a web worker to get access to the pixel data, without including code (written by myself or 3rd party) to decode the JPEG, but using a built-in browser API.
I understand that I can render a JPEG to a canvas in the main thread, read off the pixel data, and pass it to the worker if that's where I want it, but I am specifically investigating the possibilities of decoding the JPEG in the worker.
I also understand that I can use a third party library, but I am wondering if there is anything built into browsers that can do this, since it seems strange to have to send code that duplicates browser ability.
tl;dr: No.
After your edit, the question whether you can use built-in API (which is only canvas at this moment) the answer is no. You cannot create HTMLElements, including canvas in web workers because they are GUI components and most GUI frameworks refuse to allow GUI component instantiation outside GUI thread.
This is of course quite unfortunate, it means there's no way to use native API (CanvasRenderingContext2D) to do any image operations at all.
Your question as I answer it now was already asked and answered: Web Workers and Canvas
I would suggest favoriting that question for updates - I hope as well that in future image processing will be possible in web workers.
Related
I'm trying to render HTML as a H.264 stream, and then streaming it to another PC on my network.
I've got the last part, streaming to to another PC on my network down.
Now my only problem is rendering the webpage.
I can't render it once, because it isn't a static webpage.
I need to load the webpage, fetch images, run javascript and open websockets.
The only way I can imagine this working, is if I run a browser (or maybe something like CEF?), and "capture" the output, and render it as H.264
I'm basically trying to do the same as OBS' BrowserSource, but the only reason I'm NOT using OBS, is because I can't find a good way to run it headless.
NOTE: I need to be able to do it through the commandline, completely headless.
I've done this with the Chromium Tab Capture API, and the Off-Screen Tab Capture API.
Chromium will conveniently handle all the rendering, including bringing in WebGL rendered stuff, and composite all together in a nice neat MediaStream for you. From there, you can use it in a WebRTC call or pass off to a MediaRecorder instance.
The off-screen tab capture even has a nice separate isolated process that cannot access local cameras and such.
https://developer.chrome.com/extensions/tabCapture
We are using Caspar CG to render a rotating news display for live TV broadcast. Extremely powerful open source tool built by the Swedish public service company SVT. A true Swiss army knife type of software, highly recommend:
https://github.com/CasparCG/server
I'm trying to figure out what is the best and most efficient way to display a QPixmap or a QImage in a QWebEngine frame.
My goal is to display a video that is being loaded from a custom network camera, which we have a C++ API, in the QWebEngine frame, with maximum efficiency. So saving the frames to the disk and then loading them on the QWebEngine frame, is not an option.
Previously, I was using Qt WebKit and it was pretty straight forward to load a QPixmap created in C++, because it was accessible through a Javascript object. But now, I'm interested in QWebEngine due to multiple purposes.
However, in QWebEngine, it seems that there is no way to directly attach QObjects to the frame window object.
So my question is what is the most efficient way to transfer a image, or a byte array, to be shown in QWebEngine?
One of the fastest ways to draw QPixmap is QGraphicsScene::addPixmap. And it is very comfortable way. Or just draw it on QLabel.
Drawing pixmap with QWebEngine and JS is really bad idea. It is slow and WebEngine is big Chromium browser under the hood, so it is + about ~60 mb libraries.
My goal is to display a video that is being loaded from a custom
network camera, which we have a C++ API
If you need only this as you said, you don't need QWebEngine, only your network method to obtain image and QGraphicsScene to draw. It is really fast and good for video.
But if you want it so much you can, for example, paste image with JS through clipboard :)
How would you emulate a memory mapped file in FirefoxOS, Tizen or any other mobile pure-JS solution?
The use case is for a mobile browser and you need lots of data which does not fit in the RAM or you don't want to waste RAM for it yet and prefer to lazy load it.
The only thing I found is IndexedDB or what can I do about it? Any better tricks or APIs?
Hmmh it looks like Web SQL Database could be also a solution on Android, Tizen or iOS. But Firefox does not support it (?)
Update: I'm asking because of some experiments
First thing first, Web SQL won't be ever standardised as explained in the specification, so it should be considered only for WebKit/Blink-based browsers.
There is an awesome overview of offline storage options in this queston, even though the map tiles are considered in that question, I think it is still relevant to your use case.
I believe you are on the right track with IndexedDB for the graph data. On a high level it is a key-value asynchronous object store (see the Basic Concepts document). For your use case, you could index graph nodes in an object store. There is, for example, LevelGraph library which stores graph data in IndexedDB, though it is built for Semantic Web triples. HeliosJS is also worth mentioning, though it is an in-memory graph database.
Edit: Current API for IndexedDB is asynchronous. There is synchronous API drafted in the spec, which could be used only in web workers. Unfortunately no engine currently implements this feature. There is a pending patch for Gecko, but I did not find any plans for Blink or WebKit, so it is not a meaningful option now.
It is possible to access raw files through Web APIs. You could use XHR2 to load a (local) file as a binary Blob. Unfortunately XHR2 is mostly designed for streaming files and not for a random access, though you could split-up the data into multiple files and request them on demand, but that may be slow.
The direct access to files is currently quite limited, FileList and createObjectURL are primarily used for direct file user input (through Drag and Drop or file input field), FileSystem API was recently killed, and the DeviceStorage is non-standard and privileged (Firefox OS-specific). You can also store files in IndexedDB, which is described for FileHandle API. However, once you manage to get access to the raw File object, you can use the Blob.slice method to load chunks of the file – there is a great example of reading file chunks via upload form.
You may also want to look at jDataView library & friends, which eases handling of binary data through the more efficient ArrayBuffer.
Edit: As for the synchronous API, localStorage (aka DOM Storage) could be considered too. It is also a key-value storage, but much much simpler and more limited than IndexedDB:
Size of the storage is limited, usually to 5 MB
Only one localStorage per domain/application (you can have multiple named object stores in IndexedDB).
Only strings can be stored.
In general, localStorage is useful cookies replacement, but it is not really useful for storing large offline data.
So to sum it up:
IndexedDB is the easiest and widely available option, though it may be slow, inefficient or hit memory limits with very large data; also, only asynchronous API is currenlty possible.
Raw file access is hard to obtain without user interaction and the APIs are unstable and non-standard.
In the end, you can combine both approaches, two options come in mind:
Use XHR2 to parse the large file in chunks and store the parsed nodes into IndexedDB
Store the large file into IndexedDB (via XHR), use FileHandle.getFile to load the File object and Blob.slice to read its content.
In all cases, you can (should) use Web Workers to handle data manipulation and calculations in the background.
Anyway, GraphHopper looks great, we really lack such non-trivial offline applications for Firefox OS, so good luck!
Let's say I have a canvas element, and I need to turn the image on the canvas into a PNG or JPEG. Of course, I can simply use canvas.toDataURL, but the problem is that I need to do this a twenty times a second, and canvas.toDataURL is extremely slow -- so slow that the capturing process misses frames because the browser is busy converting to PNG.
My idea is to call context.getImageData(...), which evidently is much faster, and send the returned CanvasPixelArray to a Web Worker, which will then process the raw image data into a PNG or JPEG. The problem is that I can't access the native canvas.toDataURL from within the Web Worker, so instead, I'll need to resort to pure JavaScript. I tried searching for libraries intended for Node.js, but those are written in C++. Are there any libraries in pure JavaScript that will render raw image data into a PNG or JPEG?
There have been several JavaScript ports of libpng, including pnglets (very old), data:demo, and PNGlib.
The PNG spec itself is pretty straightforward – the most difficult part you may have is with implementing a simple PNG encoder yourself is getting ZLIB right (for which there are also many independent JavaScript implementations out there).
There's actually a C++ to JavaScript compiler called Emscripten.
Someone made a port of libpng, which you might want to check.
I was able to write my own PNG encoder, which supports both RGB and palette depending on how many colors there are. It's intended to be run as a Web Worker. I've open-sourced it as usain-png.
The project is a servlet to which people can upload files via, at present, HTTP POST. This is accompanied by Web page(s) providing a front-end to trigger the upload. We have more or less complete control over the servlet, and the Web pages, but don't want to impose any restrictions on the client beyond being a reasonably modern browser with Javascript. No Java applets etc.
Files may potentially be large, and a possible use case is mobile devices on less reliable networks. Some people on the project are demanding the ability to resume an upload if the network connection goes down. I don't think this is possible with plain HTTP and Javascript in a browser, but I'd love to be proved wrong.
Any suggestions?
Not with Plain Ol' JS. It doesn't have access to the file system, not even a file added to an input type=file control and so it cannot manipulate the data and upload via XHR instead.
You would have to look into a Flash or Java based alternative.
With your current restrictions, no.
(There may be a tiny chance that using the HTML5 file api could be capable of doing this. Maybe someone more knowledgeable can comment because I usually cannot make heads or tails of technical specifications from the w3c : http://www.w3.org/TR/file-upload/ )
Firefox 3.6 implements a FileReader interface, however it doesn't seem to support any form of skipping. Therefor, you would need to read the file and split it where you need it to resume.
This would not be especially useful for large file since you would probably crash the browser anyway because of the memory-allocation it would need.
https://developer.mozilla.org/en/DOM/FileReader