We Keep Coding

Problem with Emscripten with MINIMAL_RUNTIME

I am new to emscripten, so this may be an easy one to answer for others. I can't get access to my C functions. Here is the setup:
Simple C file square.c:
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
double square(double x){
return x*x;
}
Content of ready.js:
ready = function () {
startup();
}
startup is a function that I added to square.html to know when everything is ready.
Emcc command Line:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 --pre-js ./ready.js -s EXPORT_NAME=wasmMod -o square.html
output is as expected: square.html, square.js, square.wasm
Everything runs in Chrome as expected, startup is called. Now I want to access the square function:
function startup(){
console.log('startup called');
let y = _square(2);
console.log(`square: ${y}`);
}
this gives me an error: "VM291:1 Uncaught ReferenceError: _square is not defined"
If I use
let y = wasmMod._square(2);
instead, I get
"square.js:783 TypeError: wasmMod._square is not a function"
I have tried a lot of things and searched the web, but I can't seem to find the error.
If I remove -s MODULARIZE=1, I can call _square(2) without problems, however, in that case wasmMod doesn't hold anything, all variables and functions are in Global context, which is something I want to avoid.
Once it works I want to embed this part into a larger JS project using ES6 modules, so my goal is to keep everything related to emscripten in one module.
Any help is very much appreciated. Thanks in advance!
2020/10/27: added script from html file:
// Depending on the build flags that one uses, different files need to be downloaded
// to load the compiled page. The right set of files will be expanded to be downloaded
// via the directive below.
function binary(url) { // Downloads a binary file and outputs it in the specified callback
return new Promise((ok, err) => {
var x = new XMLHttpRequest();
x.open('GET', url, true);
x.responseType = 'arraybuffer';
x.onload = () => { ok(x.response); }
x.send(null);
});
}
function script(url) { // Downloads a script file and adds it to DOM
return new Promise((ok, err) => {
var s = document.createElement('script');
s.src = url;
s.onload = () => {
var c = wasmMod;
delete wasmMod;
ok(c);
};
document.body.appendChild(s);
});
}
Promise.all([script('square.js'), binary('square.wasm')]).then((r) => {
// Detour the JS code to a separate variable to avoid instantiating with 'r' array as "this" directly to avoid strict ECMAScript/Firefox GC problems that cause a leak, see https://bugzilla.mozilla.org/show_bug.cgi?id=1540101
var js = r[0];
js({ wasm: r[1] });
});
2020/10/27: added some content from square.js:
var wasmMod=
function(wasmMod) {
wasmMod = wasmMod || {};
var Module = wasmMod;
...lots of javascript follows ...
var imports = {
"env": asmLibraryArg,
"wasi_snapshot_preview1": asmLibraryArg
};
var _square, _fflush, stackSave, stackRestore, stackAlloc, _emscripten_get_sbrk_ptr, _sbrk;
if (!Module["wasm"]) throw "Must load WebAssembly Module in to variable Module.wasm before adding compiled output .js script to the DOM";
WebAssembly.instantiate(Module["wasm"], imports).then(function(output) {
var asm = output.instance.exports;
_square = asm["square"];
_fflush = asm["fflush"];
stackSave = asm["stackSave"];
stackRestore = asm["stackRestore"];
stackAlloc = asm["stackAlloc"];
_emscripten_get_sbrk_ptr = asm["emscripten_get_sbrk_ptr"];
_sbrk = asm["sbrk"];
wasmTable = asm["__indirect_function_table"];
initRuntime(asm);
ready();
}).catch(function(error) {
console.error(error);
});
return {}
}

I did some more testing, and I have a working solution now based on the recommendation that #Thomas provided. Thanks again for that.
However, I got it to work only if I drop the MINIMAL_RUNTIME flag.
The difference:
without MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html
Call from square.html:
let res = binary('square.wasm')
.then(wasm => { wasmMod({wasm: wasm})
.then(MyModule => {
console.log('WebAssembly loaded!');
console.log(MyModule._square(3));
});
});
MyModule contains everything I need. I can use my wasm functions without problems.
Now with MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 --pre-js ./ready.js -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html -s EXTRA_EXPORTED_RUNTIME_METHODS="['getValue','setValue']"
Using the same calling sequence in the HTML file gives an error:
square.html:44 Uncaught (in promise) TypeError: wasmMod(...).then is not a function at square.html:44
I can't get this to work like the version without MINIMAL_RUNTIME. Inside function ready() I have access to _square() inside the wasmMod closure, however, outside that closure no module exists that I can access. wasmMod points to the js function only, not to a module.
I expect the error to be on my side, however, I can't spend any more time on this, so I will for now cruise with the full version.

How to call a webassembly method in vue.js?

I'm trying to transpose to vue.js this simple html page add.html:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
</head>
<body>
<input type="button" value="Add" onclick="callAdd()" />
<script>
function callAdd() {
const result = Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
console.log(`Result: ${result}`);
}
</script>
<script src="js_plumbing.js"></script>
</body>
</html>
which calls the Add function defined in add.c :
#include <stdlib.h>
#include <emscripten.h>
// If this is an Emscripten (WebAssembly) build then...
#ifdef __EMSCRIPTEN__
#include <emscripten.h>
#endif
#ifdef __cplusplus
extern "C" { // So that the C++ compiler does not rename our function names
#endif
EMSCRIPTEN_KEEPALIVE
int Add(int value1, int value2)
{
return (value1 + value2);
}
#ifdef __cplusplus
}
#endif
and converted to js_plumbing and js_plumbling.wasm files through the command:
emcc add.c -o js_plumbing.js -s EXTRA_EXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -s
ENVIRONMENT='web','worker'
In console of google chrome I get these errors:
GET http://localhost:8080/dist/js_plumbing.wasm 404 (Not Found) # js_plumbing.js?2b2c:1653
Where in js_plumbing_js :
// Prefer streaming instantiation if available.
function instantiateAsync() {
if (!wasmBinary &&
typeof WebAssembly.instantiateStreaming === 'function' &&
!isDataURI(wasmBinaryFile) &&
typeof fetch === 'function') {
fetch(wasmBinaryFile, { credentials: 'same-origin' }).then(function (response) { // <---------------!!!
var result = WebAssembly.instantiateStreaming(response, info);
return result.then(receiveInstantiatedSource, function(reason) {
// We expect the most common failure cause to be a bad MIME type for the binary,
// in which case falling back to ArrayBuffer instantiation should work.
err('wasm streaming compile failed: ' + reason);
err('falling back to ArrayBuffer instantiation');
instantiateArrayBuffer(receiveInstantiatedSource);
});
});
} else {
return instantiateArrayBuffer(receiveInstantiatedSource);
}
}
In Google Chrome: createWasm # js_plumbing.js?2b2c:1680
line 1680 of js_plumbing.js:
instantiateAsync();
in Google Chrome: eval # js_plumbing.js?2b2c:1930
line 1930 of js_plumbing.js:
<pre><font color="#4E9A06">var</font> asm = createWasm();</pre>
And many other errors related to wasm :
So... how should I modify the callAdd() method in Result.vue in order to correctly execute the Add function in js_plumbing.js and in js_plumbing.wasm files?
methods: {
callAdd() {
const result = Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
console.log('Result: ${result}');
}
}
Updates:
1 update)
I compiled the add.c with this command:
emcc add.c -o js_plumbing.mjs -s EXTRA_EXPORTED_RUNTIME_METHODS=
['ccall','cwrap'] -s ENVIRONMENT='web' .
Then created a js_plumbing.js file :
. import wasm from './js_plumbing.mjs';
const instance = wasm({
onRuntimeInitialized() {
console.log(instance._addTwoNumbers(3,2));
}
}) .
Doing npm run dev:
Failed to compile.
./src/components/js_plumbing.mjs 3:25
Module parse failed: Unexpected token (3:25)
You may need an appropriate loader to handle this file type, currently
no loaders are configured to process this file.
See https://webpack.js.org/concepts#loaders
|
| var Module = (function() {
> var _scriptDir = import.meta.url;
|
| return (
Update 2)
I solved the 404 error by putting the wasm file into a /div subfolder within the same folder of the index.html file.
Now I’m facing this problem: “Cannot read property ‘ccall’ of undefined”
But I compiled the add.c file, creating js_plumbing.js and js_plumbing.wasm files, with this command, which exports the methods ‘ccall’ and ‘cwrap’ :
emcc add.c -o js_plumbing.js -s EXTRA_EXPORTED_RUNTIME_METHODS=[‘ccall’,‘cwrap’] -s ENVIRONMENT=‘web’,‘worker’
3° Update)
I "solved" through a sort of an hack, which I do not like at all.
This is the Result.vue file:
<template>
<div>
<p button #click="callAdd">Add!</p>
<p>Result: {{ result }}</p>
</div>
</template>
<script>
import * as js_plumbing from './js_plumbing'
import Module from './js_plumbing'
export default {
data () {
return {
result: null
}
},
methods: {
callAdd () {
const result = js_plumbing.Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
this.result = result;
}
}
}
</script>
which is exactly the same as the one used before
The only thing I've done to make it working, is to add export to the definition of Module in js_plumbing.js :
js_plumbing.js
// Copyright 2010 The Emscripten Authors. All rights reserved.
// Emscripten is available under two separate licenses, the MIT
license and the
// University of Illinois/NCSA Open Source License. Both these
licenses can be
// found in the LICENSE file.
// The Module object: Our interface to the outside world. We import
// and export values on it. There are various ways Module can be used:
// 1. Not defined. We create it here
// 2. A function parameter, function(Module) { ..generated code.. }
// 3. pre-run appended it, var Module = {}; ..generated code..
// 4. External script tag defines var Module.
// We need to check if Module already exists (e.g. case 3 above).
// Substitution will be replaced with actual code on later stage of
the build,
// this way Closure Compiler will not mangle it (e.g. case 4. above).
// Note that if you want to run closure, and also to use Module
// after the generated code, you will need to define var Module =
{};
// before the code. Then that object will be used in the code, and you
// can continue to use Module afterwards as well.
export var Module = typeof Module !== 'undefined' ? Module : {};
But, as I said, I do not like this hack.
Any suggestions on how to make the Module exportable, thus importable, without manually adding 'export' in js_plumbing.js file?

First, the 404 error should be addressed. Does file /dist/js_plumbing.wasm exist? I've needed to copy .wasm files manually in the past because some automatic build systems (like Parcel) currently don't.
You can build with the MODULARIZE option to import into your build system.
addTwoNumbers.c
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
int addTwoNumbers(int value1, int value2)
{
return (value1 + value2);
}
build command
$ emcc -o dist/addTwoNumbers.js -s MODULARIZE=1 src/addTwoNumbers.c
Vue Implementation
import myMathModule from './js_plumbing';
let instance = {
ready: new Promise(resolve => {
myMathModule({
onRuntimeInitialized() {
instance = Object.assign(this, {
ready: Promise.resolve()
});
resolve();
}
});
})
};
export default {
data () {
return {
result: null
};
},
methods: {
callAdd(a, b) {
instance.ready
.then(_ => this.result = instance._add(a, b));
}
}
};
Use the onRuntimeInitialized method to detect when the WASM module is ready. Your exported functions will have an underscore in front of them.
require() could possibly be used in place of import:
const wasmModule = require('./addTwoNumbers.js');
...

How to access Javascript module with Duktape in Android

I am successfully parsing and evaluating a javascript file with Duktape in my Android application using Kotlin.
val file_name = "lib.js"
val js_string = application.assets.open(file_name).bufferedReader().use {
it.readText()
}
val duktape = Duktape.create()
try {
Log.d("Greeting", duktape.evaluate("'hello world'.toUpperCase();").toString())
duktape.evaluate(js_string)
} finally {
duktape.close()
}
The javascript file was created with Browserify, so it is one single file with everything and it is working fine. But I need to request a module and a method from the module, example:
var test = require('testjs-lib');
test.EVPair.makeRandom().toWTF();
I have no idea of how to do it and have not found any example, besides this link: http://wiki.duktape.org/HowtoModules.html
It tells me to use a modsearch, but I don't have a clue how to do it or where it should be placed, not even if it is applicable for the Duktape Android (https://github.com/square/duktape-android).
Has anybody done it successfully that could shed some light on this matter?

in the testjs-lib.js, add the JS code that makes use of the module testjs-lib.js itself exports. For example:
function myModule() {
this.hello = function() {
return 'hello!';
}
this.goodbye = function() {
return 'goodbye!';
}
}
module.exports = myModule;
//You code goes here
console.log(myModule.hello());
console.log(myModule.goodbye());
Then ask Duktape to evaluate the entire file.

Say you want to include Underscore in duktape.
Put your module/library code in a separate js file. In an android project, you can put this js file in Assets folder. In our example, it'd be sth like: underscore.js
Create a java interface that'd be used by duktape to get inputstream to this js file.
Sth like:
```
public interface DuktapeHelper {
#JavascriptInterface
String getUnderScore();
}
````
Bind this java interface to a js interface in your duktape instance.
```
duktape.bind("helper", DuktapeHelper.class, <instance of your DuktapeHelperImplementation>);
```
Implment modSearch function in duktape using helper interface that you injected before.
```
duktape.evaluate("Duktape.modSearch = function (id) {\n" +
" if (id == \"underscore\") {" +
" return helper.getUnderScore();" +
" } " +
" throw new Error('cannot find module: ' + id);" +
" };" +
"var _ = require('underscore')._; ");
```

How to get Gulp4 to wait for file write tasks to compete?

I'm trying to write a workflow with Gulp 4 (see below for specific version info) that will
watch a local folder for an .html file
strip multiple tables out into individual .html files per table
convert said tables into .csv for further processing
clean the temporary directory all these files are dumped too.
The problem I'm running into is no matter what I try I cannot get my cleaning task to wait for the rest of the tasks to write files to the disk. I've tried nesting the data collection functions, including all the alteration methods into one long stream, and a handful of other clumsy solutions offered up here and other places - none of them work though. Any pointers would be a great help.
var gulp = require('gulp');
var exec = require('child_process').exec;
var rename = require('gulp-rename');
var inject = require('gulp-inject-string');
var htmlSplit = require('gulp-htmlsplit');
var del = require('del');
// Clean all non-csv files from ./data/temp
function clean() {
return del(['data/temp/*', '!data/temp/*.csv']);
}
// Convert HTML tables to CSV files
function convertCSV(filename) {
return exec('node node_modules/html-table-to-csv data/temp/' + filename + '.html data/temp/' + filename + '.csv');
}
// Move a renamed copy of original report to .data/temp/
function getData() {
return gulp.src('data/report/*.html')
.pipe(rename('injected.html'))
.pipe(gulp.dest('data/temp'));
}
// Inject split start comments before each <table> tag
function injectBefore() {
return gulp.src('data/temp/*.html')
.pipe(inject.beforeEach('<table', '<!-- split table.html -->\n'))
.pipe(gulp.dest('data/temp'));
}
// Inject split stop comments after each </table> tag
function injectAfter() {
return gulp.src('data/temp/*.html')
.pipe(inject.afterEach('</table>', '\n<!-- split stop -->'))
.pipe(gulp.dest('data/temp'));
}
// Split each table into its own HTML file for CSV conversion
function htmlCSV(done) {
var i = 0;
return gulp.src('data/temp/injected.html')
.pipe(htmlSplit())
.pipe(rename(function(file) {
// Append unique number to end of each HTML file
file.basename += i >= 9 ? ++i : '0' + ++i;
// Send unique numbered HTML file to convertCSV()
convertCSV(file.basename);
}))
.pipe(gulp.dest('data/temp'));
done();
}
gulp.task('default', gulp.series(getData, injectBefore, injectAfter, htmlCSV, clean));
// FILE STRUCTURE
// analytics
// |_bower_components
// |_data
// |_report <-- Original report in HTML dumped here
// |_temp <-- Injected and converted files dumped here
// |_node_modules
// |_gulpfile.js and other files
//
// Gulp - CLI version 1.2.2
// Gulp - Local version 4.0.0-alpha.2
// Node - v6.9.5
// NPM - 3.10.10
// OS - Windows 7 6.1.7601 Service pack 1 Build 7601

I removed the regular gulp plugins and the actual csv transformation as that is just a child_process execution.
The main issue with your code is that Node core child_process.exec is Asnyc, and will not return the end unless you add a callback. Replacing it with sync-exec will allow a sync process call since the gulp-rename callback does not have a callback.
var gulp = require('gulp');
var exec = require('sync-exec');
var rename = require('gulp-rename');
var del = require('del');
// Clean all non-csv files from ./data/temp
function clean() {
return del(['temp']);
}
// Convert HTML tables to CSV files
function convertCSV(filename) {
// return exec('node node_modules/html-table-to-csv data/temp/' + filename + '.html data/temp/' + filename + '.csv');
return exec('sleep 5;');
}
// Move a renamed copy of original report to .data/temp/
function getData() {
return gulp.src('t.html')
.pipe(gulp.dest('temp/'));
}
// Split each table into its own HTML file for CSV conversion
function htmlCSV() {
var i = 0;
return gulp.src('t.html')
.pipe(rename(function(file) {
// Append unique number to end of each HTML file
file.basename += i >= 9 ? ++i : '0' + ++i;
// Send unique numbered HTML file to convertCSV()
convertCSV(file.basename);
}))
.pipe(gulp.dest('dist'));
}
gulp.task('default', gulp.series(getData, htmlCSV, clean));

Use es7 async/await syntax as well as util.promisify to wait for it to finish:
const util = require('util');
const exec = util.promisify(require('child_process').exec);
// Convert HTML tables to CSV files
async function convertCSV(filename) {
return await exec('node node_modules/html-table-to-csv',
['data/temp/' + filename + '.html',
'data/temp/' + filename + '.csv']);
}
No need for third party libraries

Import / read variable from separate gulp file

I'm looking to split my gulpfile.js assets or src variables into separate files so that I can manage them better. For example:
....
var scripts = ['awful.js', 'lot.js', 'of.js', 'js.js', 'files.js']
....(somewhere down the line)
gulp.task('vendorjs', function() {
return gulp.src(scripts)
.pipe(concat('vendor.js'))
.pipe(rename({suffix: '.min'}))
.pipe(uglify())
.pipe(gulp.dest(paths.root + 'dist'))
.pipe(notify({ message: 'vendorjs task completed' }));
});
So what I'm basically interested if theres a way to actually move to a separate file the scripts variable and be able to access it from gulpfile.js.
I've been looking into something like:
require("fs").readFile('gulp/test.js', function(e, data) {
//(test.js would be the file that holds the scripts var)
});
Howerver while it does read the contents of the file, I still can't access it from the gulpfile.js. Any tips or ideas are much appreciated.

Node.js allows you to import other files using require(). It supports three types of files:
JSON files. See DavidDomain's answer for that.
Binary Node.js addons. Not useful for your use case.
JavaScript files. That's what you want.
For JavaScript files the value returned from require() is the one that is assigned to module.exports in the imported file.
So for your use case:
gulp/test.js
var arrayOfFiles = ["awful.js", "lots.js"];
arrayOfFiles.push("of.js");
arrayOfFiles.push("js.js");
arrayOfFiles.push("files.js");
for (var i = 0; i < 10; i++) {
arrayOfFiles.push("some_other_file" + i + ".js");
}
module.exports = {
scripts: arrayOfFiles
};
gulpfile.js
var test = require('gulp/test.js');
gulp.task('vendorjs', function() {
return gulp.src(test.scripts)
.pipe(concat('vendor.js'))
.pipe(rename({suffix: '.min'}))
.pipe(uglify())
.pipe(gulp.dest(paths.root + 'dist'))
.pipe(notify({ message: 'vendorjs task completed' }));
});

You could use a json file to store your assets or source file location in and load that into your gulp file.
For example:
// config.json
{
"scripts": ["awful.js", "lot.js", "of.js", "js.js", "files.js"]
}
And in your gulp file you would do
// gulpfile.js
var config = require('./config');
var scripts = config.scripts;
console.log(scripts);