We Keep Coding

Can't find 'path' in worker using Electron & Webpack

My electron app uses a worker to compile WASM in the background (through worker-loader). However, I get the following error when running it:
How would I solve this error? It looks like the module is trying to use path, so I've added the path fallback in Webpack config:
However, this didn't fix my issue
worker.js
import { Essentia, EssentiaWASM } from 'essentia.js';
let essentia = new Essentia(EssentiaWASM);
self.addEventListener('message', (event) => {
console.log(essentia.version);
})

Issue with webpack in production mode ( minification issue )

I am working on this little project for Augmented Reality, ARnft it is based on a lighter version of Jsartoolkit5, JsartoolkitNFT for only NFT markers. The code follows the ES6 standard (partially) and use webpack as bundler. All is fine in development mode but when i go in production mode, the example stuck with this error:
05ff8846-4121-4380-86c3-9612f404732a:1 Uncaught SyntaxError: Function statements require a function name
It stop at the embedded Worker. The app don't enter inside because i otherwise i will receive some messages in the dev console.
I Inject the Worker in a Blob object:
// create a Worker to handle loading of NFT marker and tracking of it
const workerBlob = new Blob(
[workerRunner.toString().replace(/^function .+\{?|\}$/g, '')],
{ type: 'text/js-worker' }
)
const workerBlobUrl = URL.createObjectURL(workerBlob)
worker = new Worker(workerBlobUrl)
https://github.com/kalwalt/ARnft/blob/8322585aa0f863917c6d1cee541356ff3b7c36a0/src/utils/Utils.js#L207-L213
workerRunner defined at this line:
https://github.com/kalwalt/ARnft/blob/8322585aa0f863917c6d1cee541356ff3b7c36a0/src/utils/Utils.js#L272
I think that is a minification issue i tried to add --optimize-minimize in the script:
"build-es6": "webpack --mode production --optimize-minimize",
, but not helped. How can i solve this?
Thank you

This issue can be solved with the worker-loader plugin.
Instead of inlining the worker in a Blob as explained in the question:
create an external Worker.js and import in the file (in this case Utils.js):
import Worker from './Worker.js'
use the worker as usual:
let worker
// other code
worker = new Worker()
// other code with postMessage and onmesssage...
in wepback.config.js
{
test: /\worker\.js$/,
use: {
loader: 'worker-loader',
options: { inline: true, fallback: false }
}
}
You can also see this commit and the issue on webpack.

Using RequireJS with node to optimize creating single output file does not include all the required files

I use the FayeJS and the latest version has been modified to use RequireJS, so there is no longer a single file to link into the browser. Instead the structure is as follows:
/adapters
/engines
/mixins
/protocol
/transport
/util
faye_browser.js
I am using the following nodejs build script to try and end up with all the above minified into a single file:
var fs = require('fs-extra'),
requirejs = require('requirejs');
var config = {
baseUrl: 'htdocs/js/dev/faye/'
,name: 'faye_browser'
, out: 'htdocs/js/dev/faye/dist/faye.min.js'
, paths: {
dist: "empty:"
}
,findNestedDependencies: true
};
requirejs.optimize(config, function (buildResponse) {
//buildResponse is just a text output of the modules
//included. Load the built file for the contents.
//Use config.out to get the optimized file contents.
var contents = fs.readFileSync(config.out, 'utf8');
}, function (err) {
//optimization err callback
console.log(err);
});
The content of faye_browser.js is:
'use strict';
var constants = require('./util/constants'),
Logging = require('./mixins/logging');
var Faye = {
VERSION: constants.VERSION,
Client: require('./protocol/client'),
Scheduler: require('./protocol/scheduler')
};
Logging.wrapper = Faye;
module.exports = Faye;
As I under stand it the optimizer should pull in the required files, and then if those files have required files, it should pull in those etc..., and and output a single minified faye.min.js that contains the whole lot, refactored so no additional serverside calls are necessary.
What happens is faye.min.js gets created, but it only contains the content of faye_browser.js, none of the other required files are included.
I have searched all over the web, and looked at a heap of different examples and none of them work for me.
What am I doing wrong here?

For anyone else trying to do this, I mist that on the download page it says:
The Node.js version is available through npm. This package contains a
copy of the browser client, which is served up by the Faye server when
running.
So to get it you have to pull down the code via NPM and then go into the NPM install dir and it is in the "client" dir...

Node process object made available to browser client code

I'm trying to understand how webpack uses DefinePlugin. I have:
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('development'),
}),
and a function:
export const foo = () => {
console.log(process)
console.log(process.env.NODE_ENV)
}
window.foo = foo
when I print foo, I see the following in my browser console:
ƒ foo() {
console.log(process);
console.log("development");
}
It seems like the variable "development" was injected while webpack was compiling the input file. At the same time webpack also injected the process object into the JavaScript code, and the browser did print out the process object when foo was called:
{title: "browser", browser: true, env: {…}, argv: Array(0), nextTick: ƒ, …}
My question is, how can the process object, which is a Node concept, be made available to the browser?
In fact, if I do:
window.process = process
I can use process.nextTick right inside the browser console! I thought the nextTick function was a Node-specific implementation! Could anybody explain this?
Thank you!

As mentioned here https://webpack.js.org/configuration/node/#node-process, the webpack can make polyfills for different node functions, but it appears as the node.process is a "mock";
"mock": Provide a mock that implements the expected interface but has little or no functionality.
Have you tested it to see if it actually works? It might just be an empty shell.
If it works I assume that the plugin actually uses something like node-process as shown in this blog-post: http://timnew.me/blog/2014/06/23/process-nexttick-implementation-in-browser/
Copied from that blogpost:
process.nextTick = (function () {
var canSetImmediate = typeof window !== 'undefined'
&& window.setImmediate;
var canPost = typeof window !== 'undefined'
&& window.postMessage && window.addEventListener;
if (canSetImmediate) {
return function (f) { return window.setImmediate(f) };
}
if (canPost) {
var queue = [];
window.addEventListener('message', function (ev) {
var source = ev.source;
if ((source === window || source === null) && ev.data === 'process-tick') {
ev.stopPropagation();
if (queue.length > 0) {
var fn = queue.shift();
fn();
}
}
}, true);
return function nextTick(fn) {
queue.push(fn);
window.postMessage('process-tick', '*');
};
}
return function nextTick(fn) {
setTimeout(fn, 0);
};
})();
It is a bit hard to know for sure from the info you provided. If it truly works I think it is very likely you have Browserify enabled in your node app. Perhaps you find some of what you need here: https://webpack.js.org/loaders/transform-loader/#src/components/Sidebar/Sidebar.jsx
Hopefully you find this answer somewhat helpful.
The bottom line is that I believe it is a polyfill from somewhere.

How webpack deals with Node globals and webpack.DefinePlugin are actually two different concerns.
Default node globals are globally injected, while constants defined in webpack.DefinePlugin are physically replaced one by one trough all the codebase.
eg:
// config
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('development'),
'process.env.MY_VAR': {foo: JSON.stringify('bar')},
}),
// source
console.log('process.env.NODE_ENV', process.env.NODE_ENV);
console.log('process.env.MY_VAR', process.env.MY_VAR);
console.log('process.env', process.env);
console.log('process', process);
// compiled
console.log('process.env.NODE_ENV', "development");
console.log('process.env.MY_VAR', __webpack_require__.i({"foo":"bar"}));
console.log('process.env', process.env);
console.log('process', process);
Note that process.env.NODE_ENV and process.env.MY_VAR physically are replaced, while process.env and process keep their reference to the injected process mock.
But webpack.DefinePlugin is also able to override the mocked process object (or just part of it): a lot of power which implies the risk of getting unexpected behaviours.
From Webpack docs:
When defining values for process prefer 'process.env.NODE_ENV': JSON.stringify('production') over process: { env: { NODE_ENV: JSON.stringify('production') } }. Using the latter will overwrite the process object which can break compatibility with some modules that expect other values on the process object to be defined.
eg:
// config
new webpack.DefinePlugin({
'process': JSON.stringify('override'),
'process.env.NODE_ENV': JSON.stringify('development'),
}),
// source
console.log('process', process);
console.log('process.env', process.env);
console.log('process.env.NODE_ENV', process.env.NODE_ENV);
// compiled
console.log('process', "override");
console.log('process.env', "override".env); // [boum!]
console.log('process.env.NODE_ENV', "development"); // [replaced by DefinePlugin!]

This doesn't directly answer this question, but it was the solution I needed. I was trying to access process in code that webpack compiled, intending the compiled code to be run in a NodeJS environment rather than in the browser. The process variable doesn't exist on window, but on global.
The solution was to set the target in the webpack config.
webpack.config.js
const config = {
// ...
target: 'node',
// ...
};
module.exports = config;
This removes the window mock.

see
https://webpack.js.org/configuration/node/#node
These options configure whether to polyfill or mock certain Node.js globals and modules. This allows code originally written for the Node.js environment to run in other environments like the browser.
This feature is provided by webpack's internal NodeStuffPlugin plugin. If the target is "web" (default) or "webworker", the NodeSourcePlugin plugin is also activated.

I am going to assume you are experimenting with this on your local webpack dev server and not in your production build. Webpack dev server utilizes websockets for real time communication between the underlying node processes and the front-end bundle. If you wanted to utilize this in a production environment you would need to set up a socket between your front end JS and your node instance to send commands. From a security perspective it sounds like a nightmare, but may have some valid use cases.

Is it feasible to use web worker (multi-threading) in Angular and Typescript in NativeScript?

I'm currently develop an App that is based on NativeScript and Angular2.
My screen freeze for while when my App fetching data through HTTP, and I'd like to put the fetching action into another thread.
I did a lot of search on the web, and all I got is the code in javascript like the official doc - https://docs.nativescript.org/angular/core-concepts/multithreading-model.html
Is there any way to implement the muli-threading with WebWorker in "Typescript"(which contain the support of Angular injected HTTP service) instead of the "Javascript" code(the code from the official doc)
It's appreciated if someone could give me some guide or hint, and it'll be great if I could got some relative example code.
Thanks.

There shouldn't be any big draw back for using WebWorkers in {N} + Angular but be aware that currently the WebWorker is not "exactly" compatible with Angular AoT compilation.
For me when creating an WebwWrker (var myWorker = new Worker('~/web.worker.js');) throws and error after bundling the application with AoT. I have seen soem talk about this in the community and possible the way to fix this is by editing the webpack.common.js and adding an "loaded" like so:
{
test: /\.worker.js$/,
loaders: [
"worker-loader"
]
}
Disclaimer: I have not tried this approach for fixing the error.

If someone have some problems adding workers in Nativescript with Angular and Webpack, you must follow the steps listed here.
Keep special caution in the next steps:
When you import the worker, the route to the worker file comes after nativescript-worker-loader!.
In the webpack.config.js keep caution adding this piece of code:
{
test: /.ts$/, exclude: /.worker.ts$/, use: [
"nativescript-dev-webpack/moduleid-compat-loader",
"#ngtools/webpack",
]
},
because is probable that you already have configured the AoT compilation, like this:
{
test: /(?:\.ngfactory\.js|\.ngstyle\.js|\.ts)$/,
use: [
"nativescript-dev-webpack/moduleid-compat-loader",
"#ngtools/webpack",
]
},
and you only need to add the exclude: /.worker.ts$/,
Finally, there is an example of a worker, in this case it use an Android native library:
example.worker.ts:
import "globals";
const context: Worker = self as any;
declare const HPRTAndroidSDK;
context.onmessage = msg => {
let request = msg.data;
let port = request.port;
let result = HPRTAndroidSDK.HPRTPrinterHelper.PortOpen("Bluetooth," + port.portName);
context.postMessage(result);
};
example.component.ts (../../workers/example.worker is the relative route to my worker):
import * as PrinterBTWorker from "nativescript-worker-loader!../../workers/example.worker";
import ...
connect(printer: HPRTPrinter): Observable<boolean> {
if (this.isConnected()){
this.disconnect(); //Disconnect first if it's already connected
}
return Observable.create((observer) => {
const worker = new PrinterBTWorker();
worker.postMessage({ port: printer });
worker.onmessage = (msg: any) => {
worker.terminate();
if (msg.data == 0) { // 0: Connected, -1: Disconnected
observer.next(true);
}
else {
observer.next(false);
}
};
worker.onerror = (err) => {
worker.terminate();
observer.next(false);
}
}).pipe(timeout(5000), catchError(err => of(false)));
}
Note: I use an Observable to make my call to the worker async and to add a timeout to the call to the native code, because in the case that it is not possible to connect to the printer (ex. it's turned off), it takes almost 10 seconds to notify, and this caused in my case the frezing of the app for all that time.
Important: It seem that it's necessary to run again the code manually every time that a change is made, because the worker isn't compiled using AoT.