I am new to emscripten, so this may be an easy one to answer for others. I can't get access to my C functions. Here is the setup:
Simple C file square.c:
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
double square(double x){
return x*x;
}
Content of ready.js:
ready = function () {
startup();
}
startup is a function that I added to square.html to know when everything is ready.
Emcc command Line:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 --pre-js ./ready.js -s EXPORT_NAME=wasmMod -o square.html
output is as expected: square.html, square.js, square.wasm
Everything runs in Chrome as expected, startup is called. Now I want to access the square function:
function startup(){
console.log('startup called');
let y = _square(2);
console.log(`square: ${y}`);
}
this gives me an error: "VM291:1 Uncaught ReferenceError: _square is not defined"
If I use
let y = wasmMod._square(2);
instead, I get
"square.js:783 TypeError: wasmMod._square is not a function"
I have tried a lot of things and searched the web, but I can't seem to find the error.
If I remove -s MODULARIZE=1, I can call _square(2) without problems, however, in that case wasmMod doesn't hold anything, all variables and functions are in Global context, which is something I want to avoid.
Once it works I want to embed this part into a larger JS project using ES6 modules, so my goal is to keep everything related to emscripten in one module.
Any help is very much appreciated. Thanks in advance!
2020/10/27: added script from html file:
// Depending on the build flags that one uses, different files need to be downloaded
// to load the compiled page. The right set of files will be expanded to be downloaded
// via the directive below.
function binary(url) { // Downloads a binary file and outputs it in the specified callback
return new Promise((ok, err) => {
var x = new XMLHttpRequest();
x.open('GET', url, true);
x.responseType = 'arraybuffer';
x.onload = () => { ok(x.response); }
x.send(null);
});
}
function script(url) { // Downloads a script file and adds it to DOM
return new Promise((ok, err) => {
var s = document.createElement('script');
s.src = url;
s.onload = () => {
var c = wasmMod;
delete wasmMod;
ok(c);
};
document.body.appendChild(s);
});
}
Promise.all([script('square.js'), binary('square.wasm')]).then((r) => {
// Detour the JS code to a separate variable to avoid instantiating with 'r' array as "this" directly to avoid strict ECMAScript/Firefox GC problems that cause a leak, see https://bugzilla.mozilla.org/show_bug.cgi?id=1540101
var js = r[0];
js({ wasm: r[1] });
});
2020/10/27: added some content from square.js:
var wasmMod=
function(wasmMod) {
wasmMod = wasmMod || {};
var Module = wasmMod;
...lots of javascript follows ...
var imports = {
"env": asmLibraryArg,
"wasi_snapshot_preview1": asmLibraryArg
};
var _square, _fflush, stackSave, stackRestore, stackAlloc, _emscripten_get_sbrk_ptr, _sbrk;
if (!Module["wasm"]) throw "Must load WebAssembly Module in to variable Module.wasm before adding compiled output .js script to the DOM";
WebAssembly.instantiate(Module["wasm"], imports).then(function(output) {
var asm = output.instance.exports;
_square = asm["square"];
_fflush = asm["fflush"];
stackSave = asm["stackSave"];
stackRestore = asm["stackRestore"];
stackAlloc = asm["stackAlloc"];
_emscripten_get_sbrk_ptr = asm["emscripten_get_sbrk_ptr"];
_sbrk = asm["sbrk"];
wasmTable = asm["__indirect_function_table"];
initRuntime(asm);
ready();
}).catch(function(error) {
console.error(error);
});
return {}
}
I did some more testing, and I have a working solution now based on the recommendation that #Thomas provided. Thanks again for that.
However, I got it to work only if I drop the MINIMAL_RUNTIME flag.
The difference:
without MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html
Call from square.html:
let res = binary('square.wasm')
.then(wasm => { wasmMod({wasm: wasm})
.then(MyModule => {
console.log('WebAssembly loaded!');
console.log(MyModule._square(3));
});
});
MyModule contains everything I need. I can use my wasm functions without problems.
Now with MINIMAL_RUNTIME:
emcc square.c -DNDEBUG -s MINIMAL_RUNTIME -s ALLOW_MEMORY_GROWTH=1 -s SAFE_HEAP=1 -s INVOKE_RUN=0 -s ENVIRONMENT=web,worker -s MODULARIZE=1 --pre-js ./ready.js -s SUPPORT_ERRNO=0 -s "EXPORT_NAME='wasmMod'" -o square.html -s EXTRA_EXPORTED_RUNTIME_METHODS="['getValue','setValue']"
Using the same calling sequence in the HTML file gives an error:
square.html:44 Uncaught (in promise) TypeError: wasmMod(...).then is not a function at square.html:44
I can't get this to work like the version without MINIMAL_RUNTIME. Inside function ready() I have access to _square() inside the wasmMod closure, however, outside that closure no module exists that I can access. wasmMod points to the js function only, not to a module.
I expect the error to be on my side, however, I can't spend any more time on this, so I will for now cruise with the full version.
Related
I'm trying to transpose to vue.js this simple html page add.html:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
</head>
<body>
<input type="button" value="Add" onclick="callAdd()" />
<script>
function callAdd() {
const result = Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
console.log(`Result: ${result}`);
}
</script>
<script src="js_plumbing.js"></script>
</body>
</html>
which calls the Add function defined in add.c :
#include <stdlib.h>
#include <emscripten.h>
// If this is an Emscripten (WebAssembly) build then...
#ifdef __EMSCRIPTEN__
#include <emscripten.h>
#endif
#ifdef __cplusplus
extern "C" { // So that the C++ compiler does not rename our function names
#endif
EMSCRIPTEN_KEEPALIVE
int Add(int value1, int value2)
{
return (value1 + value2);
}
#ifdef __cplusplus
}
#endif
and converted to js_plumbing and js_plumbling.wasm files through the command:
emcc add.c -o js_plumbing.js -s EXTRA_EXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -s
ENVIRONMENT='web','worker'
In console of google chrome I get these errors:
GET http://localhost:8080/dist/js_plumbing.wasm 404 (Not Found) # js_plumbing.js?2b2c:1653
Where in js_plumbing_js :
// Prefer streaming instantiation if available.
function instantiateAsync() {
if (!wasmBinary &&
typeof WebAssembly.instantiateStreaming === 'function' &&
!isDataURI(wasmBinaryFile) &&
typeof fetch === 'function') {
fetch(wasmBinaryFile, { credentials: 'same-origin' }).then(function (response) { // <---------------!!!
var result = WebAssembly.instantiateStreaming(response, info);
return result.then(receiveInstantiatedSource, function(reason) {
// We expect the most common failure cause to be a bad MIME type for the binary,
// in which case falling back to ArrayBuffer instantiation should work.
err('wasm streaming compile failed: ' + reason);
err('falling back to ArrayBuffer instantiation');
instantiateArrayBuffer(receiveInstantiatedSource);
});
});
} else {
return instantiateArrayBuffer(receiveInstantiatedSource);
}
}
In Google Chrome: createWasm # js_plumbing.js?2b2c:1680
line 1680 of js_plumbing.js:
instantiateAsync();
in Google Chrome: eval # js_plumbing.js?2b2c:1930
line 1930 of js_plumbing.js:
<pre><font color="#4E9A06">var</font> asm = createWasm();</pre>
And many other errors related to wasm :
So... how should I modify the callAdd() method in Result.vue in order to correctly execute the Add function in js_plumbing.js and in js_plumbing.wasm files?
methods: {
callAdd() {
const result = Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
console.log('Result: ${result}');
}
}
Updates:
1 update)
I compiled the add.c with this command:
emcc add.c -o js_plumbing.mjs -s EXTRA_EXPORTED_RUNTIME_METHODS=
['ccall','cwrap'] -s ENVIRONMENT='web' .
Then created a js_plumbing.js file :
. import wasm from './js_plumbing.mjs';
const instance = wasm({
onRuntimeInitialized() {
console.log(instance._addTwoNumbers(3,2));
}
}) .
Doing npm run dev:
Failed to compile.
./src/components/js_plumbing.mjs 3:25
Module parse failed: Unexpected token (3:25)
You may need an appropriate loader to handle this file type, currently
no loaders are configured to process this file.
See https://webpack.js.org/concepts#loaders
|
| var Module = (function() {
> var _scriptDir = import.meta.url;
|
| return (
Update 2)
I solved the 404 error by putting the wasm file into a /div subfolder within the same folder of the index.html file.
Now I’m facing this problem: “Cannot read property ‘ccall’ of undefined”
But I compiled the add.c file, creating js_plumbing.js and js_plumbing.wasm files, with this command, which exports the methods ‘ccall’ and ‘cwrap’ :
emcc add.c -o js_plumbing.js -s EXTRA_EXPORTED_RUNTIME_METHODS=[‘ccall’,‘cwrap’] -s ENVIRONMENT=‘web’,‘worker’
3° Update)
I "solved" through a sort of an hack, which I do not like at all.
This is the Result.vue file:
<template>
<div>
<p button #click="callAdd">Add!</p>
<p>Result: {{ result }}</p>
</div>
</template>
<script>
import * as js_plumbing from './js_plumbing'
import Module from './js_plumbing'
export default {
data () {
return {
result: null
}
},
methods: {
callAdd () {
const result = js_plumbing.Module.ccall('Add',
'number',
['number', 'number'],
[1, 2]);
this.result = result;
}
}
}
</script>
which is exactly the same as the one used before
The only thing I've done to make it working, is to add export to the definition of Module in js_plumbing.js :
js_plumbing.js
// Copyright 2010 The Emscripten Authors. All rights reserved.
// Emscripten is available under two separate licenses, the MIT
license and the
// University of Illinois/NCSA Open Source License. Both these
licenses can be
// found in the LICENSE file.
// The Module object: Our interface to the outside world. We import
// and export values on it. There are various ways Module can be used:
// 1. Not defined. We create it here
// 2. A function parameter, function(Module) { ..generated code.. }
// 3. pre-run appended it, var Module = {}; ..generated code..
// 4. External script tag defines var Module.
// We need to check if Module already exists (e.g. case 3 above).
// Substitution will be replaced with actual code on later stage of
the build,
// this way Closure Compiler will not mangle it (e.g. case 4. above).
// Note that if you want to run closure, and also to use Module
// after the generated code, you will need to define var Module =
{};
// before the code. Then that object will be used in the code, and you
// can continue to use Module afterwards as well.
export var Module = typeof Module !== 'undefined' ? Module : {};
But, as I said, I do not like this hack.
Any suggestions on how to make the Module exportable, thus importable, without manually adding 'export' in js_plumbing.js file?
First, the 404 error should be addressed. Does file /dist/js_plumbing.wasm exist? I've needed to copy .wasm files manually in the past because some automatic build systems (like Parcel) currently don't.
You can build with the MODULARIZE option to import into your build system.
addTwoNumbers.c
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
int addTwoNumbers(int value1, int value2)
{
return (value1 + value2);
}
build command
$ emcc -o dist/addTwoNumbers.js -s MODULARIZE=1 src/addTwoNumbers.c
Vue Implementation
import myMathModule from './js_plumbing';
let instance = {
ready: new Promise(resolve => {
myMathModule({
onRuntimeInitialized() {
instance = Object.assign(this, {
ready: Promise.resolve()
});
resolve();
}
});
})
};
export default {
data () {
return {
result: null
};
},
methods: {
callAdd(a, b) {
instance.ready
.then(_ => this.result = instance._add(a, b));
}
}
};
Use the onRuntimeInitialized method to detect when the WASM module is ready. Your exported functions will have an underscore in front of them.
require() could possibly be used in place of import:
const wasmModule = require('./addTwoNumbers.js');
...
I am trying out a simple example to call a C function compiled to .wasm with JavaScript.
This is the counter.c file:
#include <emscripten.h>
int counter = 100;
EMSCRIPTEN_KEEPALIVE
int count() {
counter += 1;
return counter;
}
I compiled it using emcc counter.c -s WASM=1 -o counter.js.
My main.js JavaScript file:
Module['onRuntimeInitialized'] = onRuntimeInitialized;
const count = Module.cwrap('count ', 'number');
function onRuntimeInitialized() {
console.log(count());
}
My index.html file only loads both .js files in the body, nothing else:
<script type="text/javascript" src="counter.js"></script>
<script type="text/javascript" src="main.js"></script>
It works fine / prints 101 to the console, but when I move the counter.c file to a wasm subdirectory, recompile it with emscripten and update the script tag to src="wasm/counter.js", the counter.js script tries to load counter.wasm from the root directory instead of the wasm subdirectory and I get the error:
counter.js:190 failed to asynchronously prepare wasm: failed to load wasm binary file at 'counter.wasm'
I did some research, but I didn't find any way to tell emscripten to let the generated .js file load the .wasm from the same subdirectory.
As explained by ColinE in the other answer, you should look at the integrateWasmJS() function generated by the emcc compiler (counter.js). The body of that function has changed recently and now it looks like this:
function integrateWasmJS() {
...
var wasmBinaryFile = 'counter.wasm';
if (typeof Module['locateFile'] === 'function') {
...
if (!isDataURI(wasmBinaryFile)) {
wasmBinaryFile = Module['locateFile'](wasmBinaryFile);
}
...
}
}
If this is the case, then you should add a "locateFile" function to the global Module variable. So in your HTML, you could add the following snippet before importing the counter.js file:
<script>
var Module = {
locateFile: function(s) {
return 'wasm/' + s;
}
};
</script>
If you look at the generated 'loader' file that emscripten creates, it has an integrateWasmJS function as follows:
function integrateWasmJS(Module) {
var method = Module['wasmJSMethod'] || 'native-wasm';
Module['wasmJSMethod'] = method;
var wasmTextFile = Module['wasmTextFile'] || 'hello.wast';
var wasmBinaryFile = Module['wasmBinaryFile'] || 'hello.wasm';
var asmjsCodeFile = Module['asmjsCodeFile'] || 'hello.temp.asm.js';
...
}
You can see that the wasmBinaryFile indicates the location of the binary. If it is not set it provides a default.
It looks like you should be able to override this in your main.js file as follows:
Module['wasmBinaryFile'] = 'wasm/counter.wasm';
I tested the IBMIoTF in a node.js server and it worked well.
IBMIoTF you can find here: https://www.npmjs.com/package/ibmiotf
Now I want to use the IBMIoTF in a WebApplication and I notice this little note in the documentation: https://www.npmjs.com/package/ibmiotf#load-the-library-in-browser
Load the library in browser
load iotf-client-bundle.js or iotf-client-bundle-min.js from the dist directory
I also took a look into the http://browserify.org/, but I am not able to get it working.
It is able to load the library in the index.html
<script src="libs/iotf/iotf-client-bundle.min.js"></script>
, but how can I create a object instance in the angular module?
Option 1
I am not able to use require in a WebApplication.
var config = {
"org": "THEORG",
"id": "IOT_WEB_APPLICATION",
"auth-key": "THEKEY",
"auth-token": "THETOKEN",
"type" : "shared"
};
var IotClient = require('ibmiotf');
var iotClient = new IotClient.IotfApplication(config);
In this situation I get
angular.js:14110 ReferenceError: require is not defined
Option 2
I also tried to use a object, I found in iotf-client.js file.
module.exports = {
IotfDevice: _IotfDevice['default'],
IotfManagedDevice: _IotfManagedDevice['default'],
IotfGateway: _IotfGateway['default'],
IotfManagedGateway: _IotfManagedGateway['default'],
IotfApplication: _IotfApplication['default']
};
and did a implementation like this in my controller:
var config = {
"org": "THEORG",
"id": "IOT_WEB_APPLICATION",
"auth-key": "THEKEY",
"auth-token": "THETOKEN",
"type" : "shared"
};
var iotClient = new IotfApplication(config);
Here I get:
angular.js:14110 ReferenceError: IotfApplication is not defined
These options didn't work, but how to create a instance for the IBMIoTF?
Can anyone help me?
You need to browserify the ibmiotf as part of your buildprocess:
1. in your package.json add dependency to ibmiotf npm
2. do npm install
3. add a script command to your package.json for browserify/uglify like this
"scripts": {
"build": "browserify your.js | uglifyjs -m -c warnings=false > bundle.js"
}
do npm build, this will produce a bundle.js with all your javascript files and the dependencies specified to bundle.js
Include the bundle.js in your web html file. ...<script src="bundle.js"></script>
in "your.js" do something like this
var config = require(YOURCONFIG);
var deviceType = "YOURDEVICETYPE";
var appClient = new client.IotfApplication(config);
appClient.connect();
appClient.on("connect", function () {
console.log("Connected");
appClient.subscribeToDeviceEvents(deviceType);
});
appClient.on("deviceEvent", function (deviceType, deviceId, eventType, format, payload) {
console.log("Device Event from :: "+deviceType+" : "+deviceId+" of event "+eventType+" with payload : "+payload);
});
We are trying to use only nodeJS with minimal dependencies to other packages, the challenge we now encounter is HandelbarsJS. We found a package, Assemble who can generate html for us. Only, it is very very slow, about 3 seconds each time, of these 3 seconds, there are 2,5 / 2,7 seconds of the next line:
var assemble = require('assemble');
Our package.json script section:
"scripts": {
"build:handlebars": "node scripts/handlebars.js",
"watch:handlebars": "nodemon --watch assets --exec \"npm run build:handlebars\"" }
the script/handlebars.js file
#! /usr/bin/env node
var assemble = require('assemble');
var extname = require('gulp-extname');
console.log(Date.now() - start);
assemble.data('assets/templates/data/*.json');
assemble.layouts('assets/templates/layouts/*.hbs');
assemble.partials('assets/templates/partials/*.hbs');
assemble.src('assets/templates/*.hbs', { layout: 'default' })
.pipe(extname())
.pipe(assemble.dest('build/'));
Each time, when we save a .hbs file, Nodemon restart and the external javascript file will be called.
How can we ensure that 'require' get called only once, or whether they remain in memory?
Thank you!
Since you want to accomplish using this with assemble, but without gulp, I recommend chokidar.
npm install chokidar --save
Now you can require chokidar like this:
var chokidar = require('chokidar');
Then define a little helper that runs handler whenever something in a pattern changes:
function watch(patterns, handler) {
chokidar.watch(patterns, {
ignoreInitial: false
}).on('add', handler).on('change', handler).on('unlink', handler);
}
Now we can alter the script like this:
#! /usr/bin/env node
var assemble = require('assemble');
var extname = require('gulp-extname');
var chokidar = require('chokidar');
console.log(Date.now() - start);
assemble.data('assets/templates/data/*.json');
assemble.layouts('assets/templates/layouts/*.hbs');
assemble.partials('assets/templates/partials/*.hbs');
// Enable --watch command line for Chokidar, otherwise, just run!
if (process.argv.pop() === '--watch') {
watch('assets', runOnce);
} else {
runOnce();
}
function watch(patterns, handler) {
chokidar.watch(patterns, {
ignoreInitial: false
}).on('add', handler).on('change', handler).on('unlink', handler);
}
function runOnce() {
assemble.src('assets/templates/*.hbs', { layout: 'default' })
.pipe(extname())
.pipe(assemble.dest('build/'));
}
And instead of nodemon, this will keep your script alive and running. So, in npm, you want this:
"scripts": {
"build:handlebars": "node scripts/handlebars.js",
"watch:handlebars": "node scripts/handlebars.js --watch"
}
Whenever a file changes, the script will now run, without re-invoking from scratch.
The beta version of assemble is based on gulp and has a cli that you can use just like you would use gulp, but if you don't want to use the cli and use npm scripts instead, you can do something based on #roel-van-uden's answer without chokidar and also be able to reload the actual assets (e.g. data, layouts, partials)
#! /usr/bin/env node
var start = Date.now();
var assemble = require('assemble');
var extname = require('gulp-extname');
assemble.task('assets', function () {
console.log(Date.now() - start);
assemble.data('assets/templates/data/*.json');
assemble.layouts('assets/templates/layouts/*.hbs');
assemble.partials('assets/templates/partials/*.hbs');
return assemble.src('assets/templates/*.hbs', { layout: 'default' })
.pipe(extname())
.pipe(assemble.dest('build/'));
});
assemble.task('watch', ['assets'], function () {
assemble.watch('./assets/**/*.*', ['assets]');
});
// Enable --watch command line
if (process.argv.pop() === '--watch') {
assemble.run(['watch']);
} else {
assemble.run(['assets']);
}
When working in Python I always have this simple utility function which returns the file name and line number from where the function is called:
from inspect import getframeinfo, stack
def d():
""" d stands for Debug. It returns the file name and line number from where this function is called."""
caller = getframeinfo(stack()[1][0])
return "%s:%d -" % (caller.filename, caller.lineno)
So in my code I simply put a couple debug lines like this to see how far we get before some error occurs:
print d()
# Some buggy code here
print d()
# More buggy code here
print d(), 'here is the result of some var: ', someVar
This works really well for me because it really helps debugging quickly.
I'm now looking for the equivalent in a node backend script. I was searching around but I can't find anything useful (maybe I'm looking for the wrong words?).
Does anybody know how I can create a Javascript/nodejs function which outputs the file name and line number from where the function is called? All tips are welcome!
You can create an Error to get where the Error is, and its stack trace. Then you can put that into a function, to get the line where it is.
function thisLine() {
const e = new Error();
const regex = /\((.*):(\d+):(\d+)\)$/
const match = regex.exec(e.stack.split("\n")[2]);
return {
filepath: match[1],
line: match[2],
column: match[3]
};
}
console.log(thisLine());
This works for me in Google Chrome.
And also in node.
Note to #j08691's comment:
Both this and this seem to be using lineNumber, which is not present (as far as I could test) in NodeJS.
Printing line number with custom string
const moment = require('moment');
const log = console.log;
const path = require('path');
function getTime(time) { return moment().format('YYYY-MM-DD HH:mm:ss') };
function line(num = 2) {
const e = new Error();
const regex = /\((.*):(\d+):(\d+)\)$/
const match = regex.exec(e.stack.split("\n")[num]);
const filepath = match[1];
const fileName = path.basename(filepath);
const line = match[2];
const column = match[3];
return {
filepath,
fileName,
line,
column,
str: `${getTime()} - ${fileName}:${line}:${column}`
};
}
log(line().str, "mylog1");
log(line().str, "mylog2");
log(line().str, "mylog3");
OUTPUT
2021-11-22 13:07:15 - test.js:44:5 mylog1
2021-11-22 13:07:15 - test.js:45:5 mylog2
2021-11-22 13:07:15 - test.js:46:5 mylog3
You can use this gulp plugin gulp-log-line . It Logs file and line number without the extra cost of reading the stack.
you just have to install gulp and gulp-log-line using the
npm install gulp --save and npm install gulp-log-line command. after that you need to create and write the below code in gulpfile.js and run
gulp log-line to create a duplicate file in the build folder :-
var gulp = require('gulp');
var logLine = require('gulp-log-line');
gulp.task('line-log', function() {
return gulp.src("file.js", {buffer : true})
//Write here the loggers you use.
.pipe(logLine(['console.log', 'winston.info']))
.pipe(gulp.dest('./build'))
})
gulp.task('default', ['line-log'])
Example
file.js :-
console.log('First log')
var someVariable
console.log('Second log')
Becomes
console.log('file.js:1', 'First log')
var someVariable
console.log('file.js:3', 'Second log')
The only way I've found to get anything relating to line numbers is to trap the window.onerror function, and when there's an error that will get passed the error message, the file URL and the line number:
window.onerror = function(msg, url, line) {
alert(msg + "\n" + url + ":" + line);
};
This works for me on Chrome - I don't know about other browsers.
EDIT when this answer was given in Feb' 15 there was no mention of NodeJS in the question. That was only added in November '17.