I'm trying to port the haskell library minisat to JavaScript using ghcjs, for integration in a larger haskell-ghcjs project.
minisat contains a couple of ffi imports from a c library. I've manged to compile the c library to javascript using emscripten, and to export the functions that minisat requires. So far, so good.
However, there are a couple of imports that look like this:
foreign import ccall safe minisat_solve :: Solver -> Int -> Ptr (Lit) -> IO (Bool)
which imports a function that looks like this:
int minisat_solve(minisat_solver *s, int len, minisat_Lit *ps)
My understanding, from the documentation, is that when emscripten exports a function that takes or returns a pointer, the pointer becomes a JavaScript number type.
The ghcjs documentation suggests that it should be possible to leave the existing foreign imports in place, by appropriately wrapping a JavaScript function. However, ghcjs represents pointer types as roughly a pair consisting of a JavaScript object and number.
I think the wrapper code should be roughly
function h$minisat_solve(...){
...
minisat_solve(...)
...
}
function minisat_solve = Module.cwrap('minisat_solve',...,...)
But I'm stumped by the type mismatch.
So, here's the challenge: Explain how to properly wrap an emscripten export for ccall import by ghcjs, using the above wrapper code as an example (or a counterexample, if I've got it completely wrong)
Pointer types can be converted to and from integers: https://hackage.haskell.org/package/base-4.10.0.0/docs/Foreign-Ptr.html#t:IntPtr . Thus, you ought to be able to convert to / from any format that emscripten requires using those functions.
Related
When I add a function to the Math object, it does not show inside IntelliSense of VSCode.
Math.dot = function (a,b) {};
If you don't have one already, create a type declaration module file (Ex. index.d.ts) in the root of your project folder, and add to it the following:
declare interface Math {
doc: (a:number, b:number) => number;
}
You can read more about type declaration module files in the official docs at https://www.typescriptlang.org/docs/handbook/declaration-files/templates/module-d-ts.html.
In simpler cases, you can just use JS Doc comments, since a basic subset of JS Doc comments are supported by VS Code's intellisense facilities. As far as I know, this is not one of those supported cases (declaring the type of a function on an existing global object).
Kotlin enables the extension of existing types. For example we can do this:
fun String.replaceSpaces(): String {
return this.replace(' ', '_')
}
val formatted = str.replaceSpaces()
However in JavaScript this is an antipattern.
Does Kotlin sidestep the issues that this causes in Javascript?
No this is not an antipattern. In js its an antipattern because js is dynamic and therefore changing a prototype changes how code works at runtime making it an antipattern. This is also extremely dangerous based on how the in operator works and based on the fact that you can rewrite everything, so changing a prototype can affect code somewhere on your page:
Number.prototype.toString = function(){
return "bullshit";
};
alert(""+12);
In kotlin this is not the case as kotlin is static, and all references are built at compile time. Additionally, you cant overwrite existing methods so its not dangerous at all.
You cannot compare a prototyped language like JS with Kotlin. All extensions are resolved statically and do not modify the extended type ("receiver"). This is really important and invalidates your worry. Please have a look at the documentation to learn more about the things happening with extensions in the background (compiler).
In my opinion, you need to be cautious with extensions though. Don't allow each and every developer in a Kotlin project to add new extensions to random types. I think the project must define certain rules handling the process of defining new functions/properties on existing types, because otherwise it can get hard to read foreign code. Also there should be firm arranged locations where to put those extensions.
No it is not good. It is very bad that Kotlin allows extensions of existing types.
With extensions there is no reason at all to create a class hierarchy or even create a new class definition for that matter.
Kotlin creators might as well have just used erlang instead of going to the trouble of creating extensions.
Extensions mean that you can no longer rely on a class definition to be constant. Imagine all the time people are going to spend just finding a developer's "creative" extensions let alone debugging them.
You should have compiled this example and seen the generated code. Everything would become clear:
function replaceSpaces($receiver) {
return replace($receiver, 32, 95);
}
function foo(str) {
var formatted = replaceSpaces(str);
}
There's no monkey patching at all! Extension functions are just a syntactic sugar in Kotlin. It's just another way of passing first argument to a static function.
Several languages had this already. The problem with JavaScript is the way it works, as stated in the answer of the link.
JavaScript has a couple way of overriding property. A library could have very well defined a override, but the other one override it again. The function get called from both library, all hell breaks loose.
It’s more a type system and visibility issue in my opinion.
The main arguments against extending prototypes in JavaScript are twofold:
The methods you add might have the same name as methods added by some library used in the same application, but have different behaviour.
Future version of JavaScript/ECMAScript itself might include a method with the same name as the one you're adding, but with different behaviour.
In JavaScript, both of these scenarios will lead to the wrong version of a method being called at runtime, unexpectedly, resulting in a runtime crash or unexpected runtime behaviour.
In Kotlin, most scenarios akin to this will result in a compile-time error or at least a warning, and so the perils encountered when extending types in JavaScript are mostly avoided in Kotlin. However, there is still some slight scope to encounter similar runtime bugs in rare cases.
To illustrate this, let's write some examples of code where conflicting implementations of a method exist, and see what happens.
Scenario 1: Two libraries provide an extension method with the same signature
Suppose we have this code, in Main.kt and Library1.kt respectively:
import library1.*
import library2.*
fun main() {
listOf(99, 101, 103, 104).printOddVals()
}
package library1
fun List<Int>.printOddVals() {
for (x in this) {
if (x % 2 != 0) {
println(x)
}
}
}
So the library1 package defines a printOddVals method that prints the odd values in a list, and main() uses it. Now suppose that in library2, we introduce a conflicting printOddVals method, that prints the values with odd indices:
package library2
fun List<Int>.printOddVals() {
for ((i, x) in this.withIndex()) {
if (i % 2 != 0) {
println(x)
}
}
}
In the equivalent scenario in JavaScript, this would probably cause a runtime bug. In Kotlin, it merely leads to a compile-time error:
Main.kt:5:31: error: overload resolution ambiguity:
public fun List<Int>.printOddVals(): Unit defined in library1 in file Library1.kt
public fun List<Int>.printOddVals(): Unit defined in library2 in file Library2.kt
listOf(99, 101, 103, 104).printOddVals()
^
IntelliJ IDEA will also tell you how to fix the issue - by introducing an import alias:
Do that, and we get this code, with the ambiguity about which printOddVals we want to call resolved:
import library1.*
import library2.*
import library2.printOddVals as printOddVals1
fun main() {
listOf(99, 101, 103, 104).printOddVals1()
}
Scenario 2: A library introduces a member function that shadows an extension method you already wrote
Suppose we have the following files:
package library1
class Cow {
fun chewCud() {}
}
import library1.*
fun Cow.moo() {
println("MOOOOOOO!")
}
fun main() {
val cow = Cow()
cow.moo()
}
So Cow.moo is initially an extension method we wrote. But then we update library1, and then new version has a moo member function:
package library1
class Cow {
fun chewCud() {}
fun moo() { println("moo") }
}
Now, because member functions are preferred to extension functions when resolving method calls, our extension function is not used when we call cow.moo() in main(), and then library's new member function is used instead. This is a change in runtime behaviour, and potentially a bug if the library's new moo() implementation isn't an adequate substitute for the extension function we'd written before. However, the saving grace is that this at least produces a compiler warning:
Main.kt:3:9: warning: extension is shadowed by a member: public final fun moo(): Unit
fun Cow.moo() {
^
Scenario 3: A library introduces a member function that shadows an extension method you already wrote, and doesn't produce a compiler warning
Once we add inheritance (or interface implementation) into the mix, we can contrive a situation similar to the one above where an extension function we were previously using gets partially shadowed by a member function after a library update, causing changes in runtime behaviour, and no compiler warning occurs.
This time, suppose we have these files:
import library1.*
fun Animal.talk() {
println("Hello there! I am a " + this::class.simpleName)
}
fun main() {
val cow = Cow()
val sheep = Sheep()
cow.talk()
sheep.talk()
}
package library1
interface Animal
class Cow : Animal
class Sheep : Animal
At first, when we run our main() function, our extension function gets used for both cow.talk() and sheep.talk():
Why hello there. I am a Cow.
Why hello there. I am a Sheep.
. But suppose we add a talk member to Cow:
package library1
interface Animal
class Cow : Animal {
fun talk() {
println("moo")
}
}
class Sheep : Animal
Now if we run our program, the member function on Cow gets preferred to the extension method, and our program's behaviour has been changed - all with no compiler errors or warnings:
moo
Why hello there. I am a Sheep.
So although extension functions are mostly safe in Kotlin, there's still a tiny bit of potential to run into the same pitfalls as in JavaScript.
Is this theoretical danger sufficient to mean that extensions should be used sparingly, or perhaps avoided entirely? Not according to the official Coding Conventions, which take the view that extensions are great and you should use them lots:
Extension functions
Use extension functions liberally. Every time you have a function that works primarily on an object, consider making it an extension function accepting that object as a receiver. To minimize API pollution, restrict the visibility of extension functions as much as it makes sense. As necessary, use local extension functions, member extension functions, or top-level extension functions with private visibility.
You are, of course, free to form your own view!
In JavaScript, specifically in node.js setting, one can spell module.exports = 13; in module.js, then x = import ("module.js"); elsewhere and have 13 assigned to x directly.
This saves some code when a module exports a single function, and I notice a lot of widely used packages (such as through2) make use of it.
Is there a way to do the same in Python? With some black magic, maybe?
I do have heard of a thing called loader that's, I guess, supposed to do some manipulations with a module before making it available. In particular, I think SaltStack makes use of something like that in salt.loader, but the code is too hard for me to follow. I imagine we could write a function similar to this:
def loader(module):
m = __import__(module)
return m["__exports__"]
— Then define __exports__ somewhere in a module we want to import and enjoy functionality very similar to JavaScript's module.exports mechanics. But unfortunately TypeError: 'module' object has no attribute '__getitem__' prevents us from doing that.
Python has importing built in to the language at a more basic level than Javascript does, almost all use cases are covered by a simple import statement.
For your example, all it really boils down to is:
from module import exports as x
So, there's not need to look for code savings by changing module.
The other part of the question is how, as a module author, would you restrict people to seeing only a single symbol.
Generally this is not required except to help users know what are public functions vs implementation details. Python has a few common idioms for this:
Any names that start with a leading underscore, such as _helper, are considered private. They can be accessed as normal, but the implication is you should not.
If a module level variable __all__ = [...] exists, only the strings it contains are considered public. The names must seperatedly be declared in the module.
As well as being documentation, both of these do affect one aspect of the module import:
from module import *
Using a star import is generally discouraged, but only public names will be brought in to the local namespace.
After some thinking I understood that, while we can't say m["__exports__"] due to module object's class not having __getitem__ method, we can still access some of the module's elements with "dot" notation: m.__exports__ works.
Another way: screen all module level names off with an underscore and assign the object to be exported to a variable named after the module, then from module import *.
loader.py:
def loader(module):
m = __import__(module)
return m.__exports__
exports.py:
def _f():
return 13
_a = 31
exports = {"p6": _f, "p8": _a}
__exports__ = exports
Python 2.7:
>>> import loader
>>> e = loader.load ("exports")
>>> e
{'p8': 31, 'p6': <function _f at 0x7fb79d494cf8>}
>>> from exports import *
>>> exports
{'p8': 31, 'p6': <function _f at 0x7fb79d494cf8>}
Python 3:
>>> import loader
>>> e = loader.load ("exports")
>>> e
{'p6': <function _f at 0x7f88ae229ae8>, 'p8': 31}
>>> from exports import *
>>> exports
{'p6': <function _f at 0x7f88ae229ae8>, 'p8': 31}
In the first way proposed, I unfortunately cannot use __all__ in loader.load to filter only listed names from a module being loaded since __getitem__ is not defined for module object.
In the second way proposed I don't get so much control (in that a malicious module can export arbitrary names and manipulate my namespace) and flexibility (in that I cannot assign the module's exported object to arbitrary name anywhere in my code).
So, there is still a bit left to be desired here.
How can I return a JavaScript string from a WebAssembly function?
Can the following module be written in C(++) ?
export function foo() {
return 'Hello World!';
}
Also: Can I pass this to the JS engine to be garbage collected?
WebAssembly doesn't natively support a string type, it rather supports i32 / i64 / f32 / f64 value types as well as i8 / i16 for storage.
You can interact with a WebAssembly instance using:
exports, where from JavaScript you call into WebAssembly, and WebAssembly returns a single value type.
imports where WebAssembly calls into JavaScript, with as many value types as you want (note: the count must be known at Module compilation time, this isn't an array and isn't variadic).
Memory.buffer, which is an ArrayBuffer that can be indexed using (among others) Uint8Array.
It depends on what you want to do, but it seems like accessing the buffer directly is the easiest:
const bin = ...; // WebAssembly binary, I assume below that it imports a memory from module "imports", field "memory".
const module = new WebAssembly.Module(bin);
const memory = new WebAssembly.Memory({ initial: 2 }); // Size is in pages.
const instance = new WebAssembly.Instance(module, { imports: { memory: memory } });
const arrayBuffer = memory.buffer;
const buffer = new Uint8Array(arrayBuffer);
If your module had a start function then it got executed at instantiation time. Otherwise you'll likely have an export which you call, e.g. instance.exports.doIt().
Once that's done, you need to get string size + index in memory, which you would also expose through an export:
const size = instance.exports.myStringSize();
const index = instance.exports.myStringIndex();
You'd then read it out of the buffer:
let s = "";
for (let i = index; i < index + size; ++i)
s += String.fromCharCode(buffer[i]);
Note that I'm reading 8-bit values from the buffer, I'm therefore assuming the strings were ASCII. That's what std::string would give you (index in memory would be what .c_str() returns), but to expose something else such as UTF-8 you'd need to use a C++ library supporting UTF-8, and then read UTF-8 yourself from JavaScript, obtain the codepoints, and use String.fromCodePoint.
You could also rely on the string being null-terminated, which I didn't do here.
You could also use the TextDecoder API once it's available more widely in browsers by creating an ArrayBufferView into the WebAssembly.Memory's buffer (which is an ArrayBuffer).
If, instead, you're doing something like logging from WebAssembly to JavaScript, then you can expose the Memory as above, and then from WebAssembly declare an import which calls JavaScript with size + position. You could instantiate your module as:
const memory = new WebAssembly.Memory({ initial: 2 });
const arrayBuffer = memory.buffer;
const buffer = new Uint8Array(arrayBuffer);
const instance = new WebAssembly.Instance(module, {
imports: {
memory: memory,
logString: (size, index) => {
let s = "";
for (let i = index; i < index + size; ++i)
s += String.fromCharCode(buffer[i]);
console.log(s);
}
}
});
This has the caveat that if you ever grow the memory (either through JavaScript using Memory.prototype.grow, or using the grow_memory opcode) then the ArrayBuffer gets neutered and you need to create it anew.
On garbage collection: WebAssembly.Module / WebAssembly.Instance / WebAssembly.Memory are all garbage collected by the JavaScript engine, but that's a pretty big hammer. You likely want to GC strings, and that's currently not possible for objects which live inside a WebAssembly.Memory. We've discussed adding GC support in the future.
2020 Update
Things have changed since the other answers were posted.
Today I would bet on the WebAssembly Interface Types - see below.
Since you asked specifically about C++, see:
Nbind:
nbind - Magical headers that make your C++ library accessible from JavaScript
nbind is a set of headers that make your C++11 library accessible from JavaScript. With a single #include statement, your C++ compiler generates the necessary bindings without any additional tools. Your library is then usable as a Node.js addon or, if compiled to asm.js with Emscripten, directly in web pages without any plugins.
Embind:
Embind is used to bind C++ functions and classes to JavaScript, so that the compiled code can be used in a natural way by “normal” JavaScript. Embind also supports calling JavaScript classes from C++.
See the following WebAssembly proposals:
JS Types
Reference Types
Interface Types
The proposal adds a new set of interface types to WebAssembly that describe high-level values (like strings, sequences, records and variants) without committing to a single memory representation or sharing scheme. Interface types can only be used in the interfaces of modules and can only be produced or consumed by declarative interface adapters.
For more info and a great explanation, see:
WebAssembly Interface Types: Interoperate with All the Things! by Lin Clark
You can already use it with some experimental features, see:
https://www.youtube.com/watch?v=Qn_4F3foB3Q
For a good real world example using yet another approach, see:
libsodium.js
libsodium.js - The sodium crypto library compiled to WebAssembly and pure JavaScript using Emscripten, with automatically generated wrappers to make it easy to use in web applications.
See also:
Wasmer:
Wasmer is an open-source runtime for executing WebAssembly on the Server. Our mission is make all software universally available. We support running Wasm modules standalone in our runtime, but also can be embedded in multiple languages using our language integrations.
and specifically Wasmer-JS:
Wasmer-JS enables the use of server-side compiled WebAssembly Modules in Node.js and the Browser. The project is set up as mono-repo of multiple JavaScript packages.
There's also some good info in this article on Hacker News.
Given:
mem, the WebAssembly.Memory object (from the module exports)
p, the address of the first character of the string
len, the length of the string (in bytes),
you can read the string using:
let str = (new TextDecoder()).decode(new Uint8Array(mem.buffer, p, len));
This assumes the string is UTF-8 encoded.
I found a hack way just like what we do in the hybird appication way, and it's very easily.
Just inject window.alert function, and then put it back:
let originAlert = window.alert;
window.alert = function(message) {
renderChart(JSON.parse(message))
};
get_data_from_alert();
window.alert = originAlert;
and the native side, just :
// Import the `window.alert` function from the Web.
#[wasm_bindgen]
extern "C" {
fn alert(s: &str);
}
...
pub fn get_data_from_alert() {
alert(CHART_DATA);
}
you can see in examples my GitHub: https://github.com/phodal/rust-wasm-d3js-sample
There's an easier way to do that. First, you need the instance of your binary:
const module = new WebAssembly.Module(bin);
const memory = new WebAssembly.Memory({ initial: 2 });
const instance = new WebAssembly.Instance(module, { imports: { memory: memory } });
Then, if you run console.log(instance), almost at the top of this object you will see function AsciiToString. Pass your function from C++ that returns string and you will see the output. For this case, check out this library.
I have variables in app.js:
var G = {};
module.exports = G;
var DATA = G.DATA = 'DATA';
var F1 = G.F1 = function(val)
{
return val;
};
In this manner, I can export variables under the object G, and at the same time, can access the variable directly writing DATA without G. prefix.
So far so good.
Now, I want to run a test for app.js in test.js:
var G = require('./app.js');
console.log(G.DATA); // -> DATA
This works, but I also want to access the variable directly writing DATA without G. prefix like console.log(DATA); // -> DATA
Surely, I could do like
var DATA = G.DATA; for every variables(property) export&required module G object, but obviously it's a tedious process to add every variable to the test file manually to correspond the G objects.
Is there any way to do this automatically?
So far, I'm pessmistic since
JS function encloses var in the own scope, so in theory there's no way to have a helper function to var for every object property.
Thanks.
PS. I would like to avoid any eval or VM of node solution. I have tried them in past, and too much problems.
I could assign a local variables for every property export&required module G object, but obviously it's a tedious process to add every variable to the test file manually to correspond the G objects.
No, that's how it is supposed to work. You - and only you - are in charge of what local variables exist in your module scope. No changes in the "export variables" of an included module should break your code.
Accessing properties on the imported module (with a self-chosen name) is the way to go. This is quite equivalent to Python's import app or import app as g.
If you want some particular properties as local variables, you will usually choose them manually, as in Python's from app import DATA, F1. In JS, you will need a multiple var statement like the one you've shown in your question. However, there is a syntax feature called destructuring assignment which will make this more fluid. You can use this in JavaScript 1.7+ (Gecko), CoffeeScript, or EcmaScript 6:
var {DATA, F1} = require("app.js");
Is there any way to do this automatically?
Yes and No. You should not do this, but you can - just like Python's frowned-upon from app import *. To cite what they say, which is equally true for JavaScript:
[It] introduces an unknown set of names into the interpreter, possibly
hiding some things you have already defined.
Note that in general the practice of importing * from a module or
package is frowned upon, since it often causes poorly readable code.
However, it is okay to use it to save typing in interactive sessions.
In JavaScript, you can[1] use the with-statement:
with (require("app.js")) {
…
}
[1]: Not in ES5.1 strict mode, though - which is recommended for optimisation