We Keep Coding

Problem with Emscripten with MINIMAL_RUNTIME

I am new to emscripten, so this may be an easy one to answer for others. I can't get access to my C functions. Here is the setup:
Simple C file square.c:
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
double square(double x){
return x*x;
}
Content of ready.js:
ready = function () {
startup();
}
startup is a function that I added to square.html to know when everything is ready.
Emcc command Line:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 --pre-js ./ready.js -s EXPORT_NAME=wasmMod -o square.html
output is as expected: square.html, square.js, square.wasm
Everything runs in Chrome as expected, startup is called. Now I want to access the square function:
function startup(){
console.log('startup called');
let y = _square(2);
console.log(`square: ${y}`);
}
this gives me an error: "VM291:1 Uncaught ReferenceError: _square is not defined"
If I use
let y = wasmMod._square(2);
instead, I get
"square.js:783 TypeError: wasmMod._square is not a function"
I have tried a lot of things and searched the web, but I can't seem to find the error.
If I remove -s MODULARIZE=1, I can call _square(2) without problems, however, in that case wasmMod doesn't hold anything, all variables and functions are in Global context, which is something I want to avoid.
Once it works I want to embed this part into a larger JS project using ES6 modules, so my goal is to keep everything related to emscripten in one module.
Any help is very much appreciated. Thanks in advance!
2020/10/27: added script from html file:
// Depending on the build flags that one uses, different files need to be downloaded
// to load the compiled page. The right set of files will be expanded to be downloaded
// via the directive below.
function binary(url) { // Downloads a binary file and outputs it in the specified callback
return new Promise((ok, err) => {
var x = new XMLHttpRequest();
x.open('GET', url, true);
x.responseType = 'arraybuffer';
x.onload = () => { ok(x.response); }
x.send(null);
});
}
function script(url) { // Downloads a script file and adds it to DOM
return new Promise((ok, err) => {
var s = document.createElement('script');
s.src = url;
s.onload = () => {
var c = wasmMod;
delete wasmMod;
ok(c);
};
document.body.appendChild(s);
});
}
Promise.all([script('square.js'), binary('square.wasm')]).then((r) => {
// Detour the JS code to a separate variable to avoid instantiating with 'r' array as "this" directly to avoid strict ECMAScript/Firefox GC problems that cause a leak, see https://bugzilla.mozilla.org/show_bug.cgi?id=1540101
var js = r[0];
js({ wasm: r[1] });
});
2020/10/27: added some content from square.js:
var wasmMod=
function(wasmMod) {
wasmMod = wasmMod || {};
var Module = wasmMod;
...lots of javascript follows ...
var imports = {
"env": asmLibraryArg,
"wasi_snapshot_preview1": asmLibraryArg
};
var _square, _fflush, stackSave, stackRestore, stackAlloc, _emscripten_get_sbrk_ptr, _sbrk;
if (!Module["wasm"]) throw "Must load WebAssembly Module in to variable Module.wasm before adding compiled output .js script to the DOM";
WebAssembly.instantiate(Module["wasm"], imports).then(function(output) {
var asm = output.instance.exports;
_square = asm["square"];
_fflush = asm["fflush"];
stackSave = asm["stackSave"];
stackRestore = asm["stackRestore"];
stackAlloc = asm["stackAlloc"];
_emscripten_get_sbrk_ptr = asm["emscripten_get_sbrk_ptr"];
_sbrk = asm["sbrk"];
wasmTable = asm["__indirect_function_table"];
initRuntime(asm);
ready();
}).catch(function(error) {
console.error(error);
});
return {}
}

I did some more testing, and I have a working solution now based on the recommendation that #Thomas provided. Thanks again for that.
However, I got it to work only if I drop the MINIMAL_RUNTIME flag.
The difference:
without MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html
Call from square.html:
let res = binary('square.wasm')
.then(wasm => { wasmMod({wasm: wasm})
.then(MyModule => {
console.log('WebAssembly loaded!');
console.log(MyModule._square(3));
});
});
MyModule contains everything I need. I can use my wasm functions without problems.
Now with MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 --pre-js ./ready.js -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html -s EXTRA_EXPORTED_RUNTIME_METHODS="['getValue','setValue']"
Using the same calling sequence in the HTML file gives an error:
square.html:44 Uncaught (in promise) TypeError: wasmMod(...).then is not a function at square.html:44
I can't get this to work like the version without MINIMAL_RUNTIME. Inside function ready() I have access to _square() inside the wasmMod closure, however, outside that closure no module exists that I can access. wasmMod points to the js function only, not to a module.
I expect the error to be on my side, however, I can't spend any more time on this, so I will for now cruise with the full version.

The `Module` variable's mechanism in Emscripten while compiling Pthread to workers

I am confused by the Module variable in Emsripten while compiling Pthread to Web Worker + Wasm
There is a simple Pthread code which just adds 1 in the shared variable sum in each thread. (simple.c is attached in the end.)
We can use the command to compile the Pthread code:
$ emcc simple.c -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=4 -o simple.html
Emscripten will generate simple.html, simple.js, simple.worker.js, and simple.wasm.
In simple.worker.js, there is a snippet:
// simple.worker.js
var Module = {};
// ...
Module['instantiateWasm'] = function(info, receiveInstance) {
// Instantiate from the module posted from the main thread.
// We can just use sync instantiation in the worker.
var instance = new WebAssembly.Instance(Module['wasmModule'], info);
// We don't need the module anymore; new threads will be spawned from the main thread.
Module['wasmModule'] = null;
receiveInstance(instance); // The second 'module' parameter is intentionally null here, we don't need to keep a ref to the Module object from here.
return instance.exports;
};
Note that it declares var Module = {} in the worker, and defines Module['instantiateWasm'].
However, Module['instantiateWasm'] is only called by simple.js, which the code snippet looks like:
//simple.js
var Module = {}
// ...
if (Module['instantiateWasm']) {
try {
var exports = Module['instantiateWasm'](info, receiveInstance);
return exports;
} catch(e) {
err('Module.instantiateWasm callback failed with error: ' + e);
return false;
}
}
// ...
As we can see, simple.js also declares var Module = {} too.
AFAIK, VAR global variables cannot be accessed across the main thread and its worker. I don't understand why simple.js can call Module['instantiateWasm'] as the Module of simple.js and the Module of simple.worker.js should be not the same thing.
Pthread Code:
// simple.c
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define NUMTHRDS 4
#define MAGNIFICATION 1e9
typedef struct
{
int thread_id;
double *sum;
} Arg;
pthread_t callThd[NUMTHRDS];
pthread_mutex_t mutexsum;
void *count_pi(void *arg)
{
Arg *data = (Arg *)arg;
int thread_id = data->thread_id;
double *sum = data->sum;
pthread_mutex_lock(&mutexsum);
*sum += 1;
pthread_mutex_unlock(&mutexsum);
printf("Thread %d: sum=%f\n", thread_id, *sum);
pthread_exit((void *)0);
}
int main(int argc, char *argv[])
{
pthread_mutex_init(&mutexsum, NULL);
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
double *sum = malloc(sizeof(*sum));
*sum = 0;
Arg arg[NUMTHRDS];
for (int i = 0; i < NUMTHRDS; i++)
{
arg[i].thread_id = i;
arg[i].sum = sum;
pthread_create(&callThd[i], &attr, count_pi, (void *)&arg[i]);
}
pthread_attr_destroy(&attr);
void *status;
for (int i = 0; i < NUMTHRDS; i++)
{
pthread_join(callThd[i], &status);
}
printf("Final Sum = %f \n", *sum);
free(sum);
pthread_mutex_destroy(&mutexsum);
pthread_exit(NULL);
}

A main program sends self to worker.
// simple.js
// Ask the new worker to load up the Emscripten-compiled page. This is a heavy operation.
worker.postMessage({
'cmd': 'load',
// If the application main .js file was loaded from a Blob, then it is not possible
// to access the URL of the current script that could be passed to a Web Worker so that
// it could load up the same file. In that case, developer must either deliver the Blob
// object in Module['mainScriptUrlOrBlob'], or a URL to it, so that pthread Workers can
// independently load up the same main application file.
'urlOrBlob': Module['mainScriptUrlOrBlob'] || _scriptDir,
'wasmMemory': wasmMemory,
'wasmModule': wasmModule,
'DYNAMIC_BASE': DYNAMIC_BASE,
'DYNAMICTOP_PTR': DYNAMICTOP_PTR
});
and worker import that.
// simple.worker.js
if (typeof e.data.urlOrBlob === 'string') {
importScripts(e.data.urlOrBlob);
} else {
var objectUrl = URL.createObjectURL(e.data.urlOrBlob);
importScripts(objectUrl);
URL.revokeObjectURL(objectUrl);
}
Therefore Module isn't shared, but it is initialized independently.

How can I load .wasm files stored in a subdirectory?

I am trying out a simple example to call a C function compiled to .wasm with JavaScript.
This is the counter.c file:
#include <emscripten.h>
int counter = 100;
EMSCRIPTEN_KEEPALIVE
int count() {
counter += 1;
return counter;
}
I compiled it using emcc counter.c -s WASM=1 -o counter.js.
My main.js JavaScript file:
Module['onRuntimeInitialized'] = onRuntimeInitialized;
const count = Module.cwrap('count ', 'number');
function onRuntimeInitialized() {
console.log(count());
}
My index.html file only loads both .js files in the body, nothing else:
<script type="text/javascript" src="counter.js"></script>
<script type="text/javascript" src="main.js"></script>
It works fine / prints 101 to the console, but when I move the counter.c file to a wasm subdirectory, recompile it with emscripten and update the script tag to src="wasm/counter.js", the counter.js script tries to load counter.wasm from the root directory instead of the wasm subdirectory and I get the error:
counter.js:190 failed to asynchronously prepare wasm: failed to load wasm binary file at 'counter.wasm'
I did some research, but I didn't find any way to tell emscripten to let the generated .js file load the .wasm from the same subdirectory.

As explained by ColinE in the other answer, you should look at the integrateWasmJS() function generated by the emcc compiler (counter.js). The body of that function has changed recently and now it looks like this:
function integrateWasmJS() {
...
var wasmBinaryFile = 'counter.wasm';
if (typeof Module['locateFile'] === 'function') {
...
if (!isDataURI(wasmBinaryFile)) {
wasmBinaryFile = Module['locateFile'](wasmBinaryFile);
}
...
}
}
If this is the case, then you should add a "locateFile" function to the global Module variable. So in your HTML, you could add the following snippet before importing the counter.js file:
<script>
var Module = {
locateFile: function(s) {
return 'wasm/' + s;
}
};
</script>

If you look at the generated 'loader' file that emscripten creates, it has an integrateWasmJS function as follows:
function integrateWasmJS(Module) {
var method = Module['wasmJSMethod'] || 'native-wasm';
Module['wasmJSMethod'] = method;
var wasmTextFile = Module['wasmTextFile'] || 'hello.wast';
var wasmBinaryFile = Module['wasmBinaryFile'] || 'hello.wasm';
var asmjsCodeFile = Module['asmjsCodeFile'] || 'hello.temp.asm.js';
...
}
You can see that the wasmBinaryFile indicates the location of the binary. If it is not set it provides a default.
It looks like you should be able to override this in your main.js file as follows:
Module['wasmBinaryFile'] = 'wasm/counter.wasm';

Unable to concat typescript modules (.ts) files into single output(.js) using --out Comment: Getting Error

I am new to Typescript. I've been trying to concatenate typescript modules by using the tsc --out command. But I'm getting some error. There is no information about the error. Below is the process, I've tired.
.ts files I have:
Validation.ts:
module Validation {
export interface StringValidator {
isAcceptable(s: string): boolean;
}
}
ZipCodeValidator.ts:
/// <reference path="Validation.ts" />
module Validation {
var numberRegexp = /^[0‐9]+$/;
export class ZipCodeValidator implements StringValidator {
isAcceptable(s: string) {
return s.length === 5 && numberRegexp.test(s);
}
}
}
LettersOnlyValidator.ts:
/// <reference path="Validation.ts" />
module Validation {
var lettersRegexp = /^[A‐Za‐z]+$/;
export class LettersOnlyValidator implements StringValidator {
isAcceptable(s: string) {
return lettersRegexp.test(s);
}
}
}
Test.ts:
/// <reference path="Validation.ts" />
/// <reference path="LettersOnlyValidator.ts" />
/// <reference path="ZipCodeValidator.ts" />
// Some samples to try
var strings = ['Hello', '98052', '101'];
// Validators to use
var validators: { [s: string]: Validation.StringValidator; } = {};
validators['ZIP code'] = new Validation.ZipCodeValidator();
validators['Letters only'] = new Validation.LettersOnlyValidator();
// Show whether each string passed each validator
strings.forEach(s => {
for (var name in validators) {
console.log('"' + s + '" ' + (validators[name].isAcceptable(s) ? ' matches ' : ' does not match ') + name);
}
});
tsc command:
tsc --out sample.js Test.ts
Error:
error TS6053: File 'ΓÇÉout.ts' not found.
error TS6054: File 'sample.js' must have extension '.ts' or '.d.ts'
Please let me know the way to resolve. And one more, is there any way to concat the typescript module in gulp?

I won't answer you exactly because i don't use the tsc command.
But if I were you, I would make it automatic (with Gulp for example) :
var gulp = require('gulp');
var typescript = require('gulp-typescript');
var sourcemaps = require('gulp-sourcemaps');
var addsrc = require('gulp-add-src');
var concat = require('gulp-concat');
gulp.task('tsc', function () {
return gulp.src(['*.ts']) // all your ts files here (check the path)
.pipe(sourcemaps.init())
.pipe(typescript({
sortOutput: true
}))
.js // compile with tsc, ordered with _reference.ts
.pipe(addsrc('external.js')) // add an external javascript file (if needed)
.pipe(concat('app.js')) // concat all in one file
.pipe(sourcemaps.write()) // generate the .map
.pipe(gulp.dest('dist')); // write all in the dist folder
});
In short :
gulp-typescript : tsc for gulp
gulp-sourcemaps : allows you to generate the .map (to link between the .ts files and the .js files)
gulp-add-src : allows you to add sources files in the pipe. It's usefull to push some javascript files in the process.
gulp-concat : concat all the files

It looks like you're using wrong dash character in your tsc --out.
TypeScript compiler looks at your command and instead of "invoke tsc with file Test.ts and output to sample.js" sees something along the lines of "invoke tsc with 3 files: --out, sample.js, Test.ts". It then looks for these files and tries to compile them.
To fix the problem, just copy&paste the command with the correct dash character:
tsc --out sample.js Test.ts

Writing ASYNC CPP nodejs (0.5.3+) modules

I´m searching for a way to build c++ modules for NodeJs with the current release (0.5.9).
By using the following tutorial and abstracting from nodes node_file.cc I was able to build a module by my self for node 0.5.3.
https://github.com/jedp/node-rot13/blob/master/src/rot13.cpp
But with node 0.5.4 some of the API´s must have changed because because I´m no longer able to warp functions with eio_* anymore.
Looking at node_file.cc I discovered that the eio_* wrapping is replaced by new ReqWrap classes.
For Example in this macro: https://gist.github.com/1303926
No I wonder what's the best way of writing async extensions?

Basically looking at the crypto node module solved my problem this module was not poluted with macros like the file module. I came up with a simple async module that calculates the sum of two integers:
#include <v8.h>
#include <node.h>
#include <stdlib.h>
#include <errno.h>
using namespace node;
using namespace v8;
struct Test_req
{
ssize_t result;
ssize_t int1;
ssize_t int2;
Persistent<Function> callback;
};
void TestWorker(uv_work_t* req)
{
Test_req* request = (Test_req*)req->data;
request->result = request->int1 + request->int2;
}
void TestAfter(uv_work_t* req)
{
HandleScope scope;
Test_req* request = (Test_req*)req->data;
delete req;
Handle<Value> argv[2];
// XXX: Error handling
argv[0] = Undefined();
argv[1] = Integer::New(request->result);
TryCatch try_catch;
request->callback->Call(Context::GetCurrent()->Global(), 2, argv);
if (try_catch.HasCaught())
{
FatalException(try_catch);
}
request->callback.Dispose();
delete request;
}
static Handle<Value> Test(const Arguments& args)
{
HandleScope scope;
if ( args.Length() < 3 || !args[0]->IsNumber() || !args[1]->IsNumber() )
{
return ThrowException(Exception::TypeError(String::New("Bad argument")));
}
ssize_t int1 ( args[0]->Int32Value() );
ssize_t int2 ( args[1]->Int32Value() );
if ( args[2]->IsFunction() )
{
Local<Function> callback = Local<Function>::Cast(args[2]);
Test_req* request = new Test_req;
request->callback = Persistent<Function>::New(callback);
request->int1 = int1;
request->int2 = int2;
uv_work_t* req = new uv_work_t();
req->data = request;
uv_queue_work(uv_default_loop(), req, TestWorker, TestAfter);
}
else
{
return ThrowException(Exception::TypeError(String::New("Callback missing")));
}
return Undefined();
}
extern "C"
{
static void init(Handle<Object> target)
{
HandleScope scope;
}
}
NODE_MODULE(node_AsyncTest, init);
On the node side you call the module like this:
var foo = process.binding('AsyncTest');
foo.Test(1,2,function(err,data){
console.log(err,data);
});
result:
undefined 3
Hope this is helpful ;)
Ps: Since the lack of node compiling extensions under windows. I´m building this directly into the core of the node windows port with the Visual Studio build solution.