I need to display a quite complex 2D shape in a canvas using PixiJS and to do so I'd like to create and define all graphic elements in a separate thread (web worker) to not block the rest of the UI.
The problem is that when I import PixiJS in the web worker file like this
importScripts('https://cdnjs.cloudflare.com/ajax/libs/pixi.js/4.5.2/pixi.js');
it gives me an error because it accesses DOM elements (like window and document) and this is not allowed in web workers. How can I solve this?
This is the error:
Uncaught ReferenceError: window is not defined
at Object.179../Math.sign (Object.assign.js:3)
at s (_prelude.js:1)
at _prelude.js:1
at Object.<anonymous> (WebGLPrepare.js:101)
at Object.187../accessibility (index.js:44)
at s (_prelude.js:1)
at e (_prelude.js:1)
at _prelude.js:1
at _prelude.js:1
at _prelude.js:1
Well I guess you cannot. Webworkers have their own DedicatedWorkerGlobalScope with no access to the DOM, window, etc. If you have heavy computations, for instance to calculate animations, all you can do is let the webworker do the number crunching and the main thread do the rendering.
The Worker and the main thread cannot share objects. Even if the following explanation is not 100% technically correct, you can imagine that if you:
var obj = { a: { b: 100 } };
worker.postMessage(obj);
It will be more like:
//really dirty object clone here
worker.postMessage(JSON.parse(JSON.stringify(obj)));
With that I want to point out, that you cannot share objects with the worker and vice verca.
Find a tecnically correct explaintion here:
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Transferring_data_to_and_from_workers_further_details
Philipp is right on his answer, I just want to elaborate this a bit further, from the games and web workers point-of-view.
In my experience, it is ofter hard to take advantage of web workers in games. Especially if you use a library to handle canvas / webGL, since that does most of the number crunching anyway. Web workers are designed for number crunching as philipp mentioned. And passing data between web worker and main thread is quite expensive, so passing data and doing tiny operations on them, will not be beneficial.
Without providing a direct link, I read already long while ago that some game engines (I think construct) used web workers for their pathfinding module. So nothing related to direct graphic handling, but numerical operations.
Also there is a proposal for possibility to use canvases in web worker contexts, so it is clearly an issue with others too. If my memory serves right, I believe it was this: https://developer.mozilla.org/fi/docs/Web/API/OffscreenCanvas
You can include PixiJS in your Web Worker, although you can not perform graphics operations.
Although the other two answers are technically correct, there are legitimate use cases for including Pixi's code in your Worker's scope without actually rendering graphics. An example is using Pixi's Point and Rectangle classes. In my case, I wrote several modules that rely on definitions exported by Pixi and perform heavy calculations based on them.
The errors you had were because Pixi relies on the globals window and document to define some graphics-related constants. In order to allow Pixi to finish loading you can provide mock values for window and document with empty methods with the same names as the ones used by Pixi.
As an example, while using PixiJS 4.8.6 the following code worked for me:
window = self
document = {
createElement () {
return {
getContext () {
return {
fillRect () {},
drawImage () {},
getImageData () {},
}
},
}
},
}
importScripts('pixi-4-8-6.js');
/* Web Worker code follows */
// ...
Depending on Pixi's version you may need to adjust this boilerplate. An alternative would be to try one of those packages that mock document and Canvas for headless browsers. I haven't tried that since the code above worked well enough.
Pixi 5 came out recently and it may have changed this completely. I skimmed the code and it looks like they removed the constant definition that was causing problems so it's also possible that this new version just works out of the box. I know they have been looking into letting people use Workers more easily and they have been exploring OffscreenCanvas like Hachi said.
pixi team is working on adding headless environments support like web workers; see https://github.com/pixijs/pixijs/issues/7123
Related
As is known, we can't access the dom element from the workers. When I create AudioContext:
var audioCtx = new (canvas.AudioContext || canvas.webkitAudioContext)();
I get:
Uncaught ReferenceError: window is not defined
Is there any way to walk around this problem?
Not yet no. Here is a specs issue discussing this very matter, and it is something a lot of actors would like to see, so there is hope it comes, one day.
Note that there is an AudioWorklet API available, which will create its own Worklet (which also works in a parallel thread), but you still need to instantiate it from the UI thread, and you don't have access to everything that can be done in an AudioContext. Still, it may suit your needs.
Also, note that it might be possible to do the computing you have to do in a Worker already, by transferring ArrayBuffers from your UI thread to the Worker's one, or by using SharedArrayBuffers.
After sweating blood and tears I've finally managed to set up a Node C++ addon and shove a web-platform standard MediaStream object into one of its C++ methods for good. For compatibility across different V8 and Node.js versions, I'm using Native Abstractions for Node.js (nan):
addon.cc
NAN_METHOD(SetStream)
{
Nan::HandleScope scope;
v8::Local<v8::Object> mediaStream = info[0]->ToObject();
}
addon.js
setStream(new MediaStream());
For what it's worth, this works correctly (i.e. it does not obliterate the renderer process on sight), and I can verify the presence of the MediaStream object, e.g. by returning its constructor name from the C++ method:
addon.cc
info.GetReturnValue().Set(mediaStream->GetConstructorName());
When called from JavaScript through setStream, this would return the string MediaStream, so the object is definitely there. I can also return the mediaStream object itself and everything will work correctly, so it's indeed the object I need.
So, how would I read audio data (i.e. audio samples) from this MediaStream object in C++? As a sidenote, the actual data read (and processing) would be done in a separate std::thread.
Bounty Update
I understand this would be sort of easier/possible if I were compiling Electron and/or Chromium myself, but I'd rather not get involved in that maintenance hell.
I was wondering if it would be possible without doing that, and as far as my research goes, I'm convinced I need 2 things to get this done:
The relevant header files, for which I believe blink public should be adequate
A chromium/blink library file (?), to resolve external symbols, similarly to the node.dylib file
Also, as I said, I believe I could compile chromium/blink myself and then I would have this lib file, but that would be a maintenance hell with Electron. With this in mind, I believe this question ultimately comes down to a C++ linking question. Is there any other approach to do what I'm looking for?
Edit
ScriptProcessorNode is not an option in my case, as its performance makes it nearly unusable in production. This would require to process audio samples on the ui/main thread, which is absolutely insane.
Edit 2
AudioWorklets have been available in Electron for some time now, which, unlike the ScriptProcessorNode (or worse, the AnalyzerNode), is low-latency and very reliable for true C++ backed audio processing even in real time.
If someone wants to go ahead and write an AudioWorklet-based answer, I'll gladly accept, but beware: it's a very advanced field and a very deep rabbit hole, with countless obstacles to get through even before a very simple, general-purpose pass-through prototype (especially so because currently in Electron, Atomics-synced, buffered cross-thread audio processing is required to pull this off because https://github.com/electron/electron/issues/22503 -- although getting a native C++ addon into one audio renderer thread, let alone multiple threads at the same time, is probably equally as challenging).
The MediaStream header is part of Blink's renderer modules, and it's not obvious to me how you could retrieve this from nan plugin.
So, instead let's look at what you do have, namely a v8::Object. I believe that v8::Object exposes all the functionality you need, it has:
GetPropertyNames()
Get(context, index)
Set(context, key, value)
Has(context, key)
Unless you really need a strictly defined interface, why not avoid the issue altogether and just use the dynamic type that you already have?
For getting audio data out specifically, you would need to call getAudioTracks() on the v8::Object, which probably looks something like this?
Note: I don't think you need a context, v8 seems to be happy with it being empty: v8/src/api/api.cc
Should look something like this, plus some massaging of types in and out of v8.
v8::MaybeLocal<v8::Value> get_audio_tracks = mediaStream->Get("getAudioTracks");
// Maybe needs to be v8::Object or array?
if (!get_audio_tracks.IsEmpty()) {
v8::Local<v8::Value> audio_tracks = get_audio_tracks.ToLocalChecked()();
}
I'm learning Node.js (-awesome-), and I'm toying with the idea of using it to create a next-generation MUD (online text-based game). In such games, there are various commands, skills, spells etc. that can be used to kill bad guys as you run around and explore hundreds of rooms/locations. Generally speaking, these features are pretty static - you can't usually create new spells, or build new rooms. I however would like to create a MUD where the code that defines spells and rooms etc. can be edited by users.
That has some obvious security concerns; a malicious user could for example upload some JS that forks the child process 'rm -r /'. I'm not as concerned with protecting the internals of the game (I'm securing as much as possible, but there's only so much you can do in a language where everything is public); I could always track code changes wiki-style, and punish users who e.g. crash the server, or boost their power over 9000, etc. But I'd like to solidly protect the server's OS.
I've looked into other SO answers to similar questions, and most people suggest running a sandboxed version of Node. This won't work in my situation (at least not well), because I need the user-defined JS to interact with the MUD's engine, which itself needs to interact with the filesystem, system commands, sensitive core modules, etc. etc. Hypothetically all of those transactions could perhaps be JSON-encoded in the engine, sent to the sandboxed process, processed, and returned to the engine via JSON, but that is an expensive endeavour if every single call to get a player's hit points needs to be passed to another process. Not to mention it's synchronous, which I would rather avoid.
So I'm wondering if there's a way to "sandbox" a single Node module. My thought is that such a sandbox would need to simply disable the 'require' function, and all would be bliss. So, since I couldn't find anything on Google/SO, I figured I'd pose the question myself.
Okay, so I thought about it some more today, and I think I have a basic strategy:
var require = function(module) {
throw "Uh-oh, untrusted code tried to load module '" + module + "'";
}
var module = null;
// use similar strategy for anything else susceptible
var loadUntrusted = function() {
eval(code);
}
Essentially, we just use variables in a local scope to hide the Node API from eval'ed code, and run the code. Another point of vulnerability would be objects from the Node API that are passed into untrusted code. If e.g. a buffer was passed to an untrusted object/function, that object/function could work its way up the prototype chain, and replace a key buffer function with its own malicious version. That would make all buffers used for e.g. File IO, or piping system commands, etc., vulnerable to injection.
So, if I'm going to succeed in this, I'll need to partition untrusted objects into their own world - the outside world can call methods on it, but it cannot call methods on the outside world. Anyone can of course feel free to please tell me of any further security vulnerabilities they can think of regarding this strategy.
I'm currently using a mediator that sits in-between all my modules and allows them to communicate between one another. All modules must go through the mediator to send out messages to anything that's listening. I've been doing some reading on RequireJS but I've not found any documentation how best you facilitate communication between modules.
I've looked at signals but if I understand correctly signals aren't really that useful if you're running things through a mediator. I'm just left wondering what else I could try. I'm quite keen on using a callback pattern of some kind but haven't got past anything more sophisticated than a simple lookup table in the mediator.
Here's the signal implementation I found: https://github.com/millermedeiros/js-signals
Here's something else I found: http://ryanflorence.com/publisher.js/
Is there a standardized approach to this problem or must everything be dependency-driven?
Using a centralized event manager is a fairly common and pretty scalable approach. It's hard to tell from your question what problem, if any, you're having with an events model. The typical thing is as follows (using publisher):
File 1:
require(['publisher','module1'],function(Publisher,Module1) {
var module = new Module1();
Publisher.subscribe('globaleventname', module.handleGlobalEvent, module);
});
File 2:
require(['publisher','module2'],function(Publisher,Module2) {
var module = new Module2();
module.someMethod = function() {
// method code
// when method needs module1 to run its handler
Publisher.publish('globaleventname', 'arguments', 'to', 'eventhandlers');
};
});
The main advantage here is loose coupling; rather than objects knowing methods of other objects, objects can fire events and other objects know how to handle that particular application state. If an object doesn't exist that handles the event, no error is thrown.
What problems are you having with this approach?
Here's something you might want to try out:
https://github.com/naugtur/overlord.js
It can do a bit more than an ordinary publisher or mediator. It allows creating a common API for accessing any methods of any modules.
This is kind of a shameless plug, because it's my own tool, but it seems quite relevant to the question.
Support for require.js has been added.
I'm experimenting with web workers, and was wondering how well they would deal with embarassingly parallell problems. I therefore implemented Connaway's Game of Life. (To have a bit more fun than doing a blur, or something. The problems would be the same in that case however.)
At the moment I have one web worker performing iterations and posting back new ImageData for the UI thread to place in my canvas. Works nicely.
My experiment doesn't end there however, cause I have several CPU's available and would like to parallellize my application.
So, to start off simply I split my data in two, down the middle, and make two workers each dealing with a half each. The problem is of course the split. Worker A needs one column of pixels from worker B and vice versa. Now, I can clearly fix this by letting my UI-thread give that column down to the workers, but it would be much better if my threads could pass them to eachother directly.
When splitting further, each worker would only have to keep track of it's neighbouring workers, and the UI thread would only be responsible for updating the UI (as it should be).
My problem is, I don't see how I can achieve this worker-to-worker communication. I tried handing the neighbours to eachother by way of an initialization postMessage, but that would copy my worker rather than hand down a reference, which luckily chrome warned me about being impossible.
Uncaught Error: DATA_CLONE_ERR: DOM Exception 25
Finally I see that there's something called a SharedWorker. Is this what I should look into, or is there a way to use the Worker that would solve my problem?
You should be able to use channel messaging:
var channel = new MessageChannel();
worker1.postMessage({code:"port"}, [channel.port1]);
worker2.postMessage({code:"port"}, [channel.port2]);
Then in your worker threads:
var xWorkerPort;
onmessage = function(event) {
if (event.data.code == "port") {
xWorkerPort = event.ports[0];
xWorkerPort.onmessage = function(event) { /* do stuff */ };
}
}
There's not much documentation around, but you could try this MS summary to get started.