We Keep Coding

Using strings in Nim with Emscripten causes JavaScript error

I'm trying to make a simple Nim app which runs on Emscripten. I'm also using jsbind to call a JavaScript function, namely console.log.
I have the following file, test.nim:
import jsbind
type Window* = ref object of JSObj
type Console* = ref object of JSObj
proc getConsole*(): Console {.jsimportgWithName: "function(){return console;}".}
proc log*(c: Console, a: any) {.jsimport.}
echo("Before log")
getConsole().log("Hello")
echo("After log")
My nim.cfg is:
#if emscripten:
cc = clang
gc = none
clang.exe = "emcc"
clang.linkerexe = "emcc"
clang.options.linker = ""
cpu = "i386"
out = "index.html"
passC = "-Iemscripten"
passL = "-Lemscripten -s TOTAL_MEMORY=335544320"
#end
I then compile with:
nim c -d:emscripten --out=index.html test.nim
When I then open index.htmlin a browser and it gets to the getConsole().log call, I get numerous console errors about how a function UTF8ToString is not defined, and the program terminates:
This function appears to be an Emscripten prelude function - how do I ensure this and any other functions I need are included in my JavaScript output?
This error occurs both with the WebAssembly options enabled and without. I'm using Nim 0.16.0 and emcc 1.35.0.
Here is a JSFiddle of the two Emscripten output files. (They are too large for a Stack Snippet.)

Emscripten 1.35.0 is probably too old to have UTF8ToString function. Your sample works fine for me with Emscripten 1.37.1.
As a side note, I've noticed you're using fixed Emscripten heap size. In case you want dynamically growing heap you can use -s ALLOW_MEMORY_GROWTH=1 flag.

Is it required to grab v8::Locker before making v8::Function::Call?

I'm using V8 to execute some custom javascript code, exposing OnUpdate function to JS world. Overall code works fine but currently I'm concerned about performance of below code - is it required to grab v8::Locker for executing any user defined function? Instruments.app shows code here spends way too much time in v8::Locker constructor and destructor -
90 ms (in actual code execution) vs ~4000ms (by Locker & ~Locker) - this is absurd and I feel I might be doing something wrong.
So my basic question is it really necessary to grab v8::Locker to execute a v8::Function::Call? In current state if I comment out v8::LockerI get below error message:
# Fatal error in HandleScope::HandleScope
# Entering the V8 API without proper locking in place
Code snippet:
int Bucket::send_doc_update_bucket(const char *msg) {
Locker locker(GetIsolate());
Isolate::Scope isolate_scope(GetIsolate());
HandleScope handle_scope(GetIsolate());
Local<Context> context = Local<Context>::New(GetIsolate(), context_);
Context::Scope context_scope(context);
TryCatch try_catch;
Local<Value> args[1];
args[0] = String::NewFromUtf8(GetIsolate(), msg);
assert(!try_catch.HasCaught());
Handle<Value> val;
if(!context->Global()->Get(context,
createUtf8String(GetIsolate(),
"OnUpdate")).ToLocal(&val) ||
!val->IsFunction()) {
return 3;
}
Handle<Function> on_doc_update = Handle<Function>::Cast(val);
on_doc_update->Call(context, context->Global(), 1, args);
if (try_catch.HasCaught()) {
//w->last_exception = ExceptionString(GetIsolate(), &try_catch);
return 2;
}
return 0;
}

If you're not doing any multithreading, you never need to touch v8::Locker. As soon as you do, though, then you have to have it pretty much everywhere.
A v8::Locker is to stop multiple v8::Context's from the same v8::Isolate from running at the same time.
If you want multiple simultaneous threads of execution, each context must be created in a different isolate.

How to run V8 evaluation multiple times?

Maybe it is stupid question (I am newbie to C++, just wanted to use it as library for android), but I am not able to run evaluation of some JS multiple times.
I have started with "hello world" tutorial. But then I have wanted simple thing, re-run main (just wrap content of tutorial code into function and run it twice in newly empty main.
This is what I got:
#
# Fatal error in ../src/isolate.cc, line 1868
# Check failed: thread_data_table_.
#
==== C stack trace ===============================
1: 0xa890b9
2: 0x6a22fc
3: 0x42694f
4: 0x405f66
5: 0x405ec7
6: __libc_start_main
7: 0x405dc9
Illegal instruction (core dumped)
This cames after creating new isolate
Isolate* isolate = Isolate::New(create_params);
Well, what I should do? Am I using wrong construct or so? Should I close/delete/clear something more?
In bigger view I just want to do evaluate function, that can be triggered multiple times, and beside that also run multiple js snipets in same context (how to split this function?).
Any idea?
UPDATE:
Ok, lets say that the main can be split into three logical parts:
init
int main(int argc, char* argv[]) {
// Initialize V8.
V8::InitializeICU();
V8::InitializeExternalStartupData(argv[0]);
Platform* platform = platform::CreateDefaultPlatform();
V8::InitializePlatform(platform);
V8::Initialize();
// Create a new Isolate and make it the current one.
ArrayBufferAllocator allocator;
Isolate::CreateParams create_params;
create_params.array_buffer_allocator = &allocator;
evaluation
Isolate* isolate = Isolate::New(create_params);
{
Isolate::Scope isolate_scope(isolate);
// Create a stack-allocated handle scope.
HandleScope handle_scope(isolate);
// Create a new context.
Local<Context> context = Context::New(isolate);
// Enter the context for compiling and running the hello world script.
Context::Scope context_scope(context);
// Create a string containing the JavaScript source code.
Local<String> source =
String::NewFromUtf8(isolate, "'Hello' + ', World!'",
NewStringType::kNormal).ToLocalChecked();
// Compile the source code.
Local<Script> script = Script::Compile(context, source).ToLocalChecked();
// Run the script to get the result.
Local<Value> result = script->Run(context).ToLocalChecked();
// Convert the result to an UTF8 string and print it.
String::Utf8Value utf8(result);
printf("%s\n", *utf8);
}
isolate->Dispose();
and clean
// Dispose and tear down V8.
V8::Dispose();
V8::ShutdownPlatform();
delete platform;
return 0;
Now as I said before if I run main consists of init->evaluation->clean twice, that mean init->evaluation->clean->init->evaluation->clean, then the error occurs. I have figured out, that if I extract evaluation part into separate function I can run it multiple times e.g. as init->(evaluation){2}->clean
Is that how should it work? Next step is to divide this main into tree separate function that mean I have to have static member with platform? Could it cause leak somehow?
NOTE: that I want to run it from android, that mean e.g. click in UI, propagate js source to C via JNI and then call c++ V8, which is already initialized or not. hm?
Prefered way is to have "blackbox", but if I have to hold platform, so be it. It maybe could be also faster without re-initialization of V8, right?
UPDATE 2:
Well, still have problems with splitting evaluation part to achieve multiple runs in same isolate/context.
I have splitted it after creating context with stored isolate and context, but with no luck. When in second part try to create source string it fails, probably because of using stored isolate (something with isolate scope I guess).
:(

My assumption as I introduced in UPDATE1 was correct. That part works well.
According to UPDATE2 I have splitted evaluation part into two.
First for initialize isolate and context:
mIsolate = Isolate::New(mCreate_params);
Isolate::Scope isolate_scope(mIsolate);
{
// Create a stack-allocated handle scope.
HandleScope handle_scope(mIsolate);
v8::Handle<v8::ObjectTemplate> global = v8::ObjectTemplate::New(mIsolate);
// Bind the global 'print' function to the C++ Print callback.
global->Set(v8::String::NewFromUtf8(mIsolate, "print"), v8::FunctionTemplate::New(mIsolate, Print));
// Create a new context.
mContext = Context::New(mIsolate, NULL, global);
Persistent<Context, CopyablePersistentTraits<Context>> persistent(mIsolate, mContext);
mContext_persistent = persistent;
}
and second that will run js in same context:
Isolate::Scope isolate_scope(mIsolate);
{
HandleScope handle_scope(mIsolate);
mContext = Local<Context>::New(mIsolate, mContext_persistent);
// Enter the context for compiling and running the hello world script.
Context::Scope context_scope(mContext);
{
// Create a string containing the JavaScript source code.
Local<String> source =
String::NewFromUtf8(mIsolate, js_source, NewStringType::kNormal).ToLocalChecked();
// Compile the source code.
Local<Script> script = Script::Compile(mContext, source).ToLocalChecked();
TryCatch trycatch(mIsolate);
// Run the script to get the result.
v8::Local<v8::Value> result;
if(!script->Run(mContext).ToLocal(&result)){
v8::String::Utf8Value exception_str(trycatch.Exception());
dprint(*exception_str);
}else{
if(!result->IsUndefined()){
String::Utf8Value utf8(result);
dprint(*utf8);
}
}
}
}
Well the code works very well on linux, but I still have some issues when I try to run first part for the second time (create new context) on android:
A/art: art/runtime/thread.cc:986] pthread_getschedparam failed for DumpState: No such process
A/art: art/runtime/base/mutex.cc:485] Unexpected state_ 0 in unlock for logging lock
But that's another question I guess. Peace.

Did you initialize v8 more than once?
v8::V8::Initialize() this method should be called once per process.
deep into project source file "v8/src/v8.cc", you will find the prove
bool V8::Initialize() {
InitializeOncePerProcess();
return true;
}

How do I provide source-level debugging for a language like VS does for TypeScript?

I understand the concept of using source maps with JavaScript to make debugging easier with minified scripts, etc. What I don't get is how source-level debugging works like it does for TypeScript in Visual Studio (http://blogs.msdn.com/b/typescript/archive/2012/11/15/announcing-typescript-0-8-1.aspx).
For example, if I create my own language that compiles/translates to JavaScript, how do I interface with browsers to provide this kind of source-level debugging? Is there a standard protocol for this? How does Visual Studio do it?
Update
To clarify even more, let's say I invent a language called Caffeinated Beverage Script that compiles to JavaScript. I build an IDE for my language and I want to be able to set breakpoints, step through code, inspect variables, etc. in my IDE while the JavaScript runs in a browser. How do I get my IDE to communicate with the browser on this level?

You might consider WebKit's remote debugging API:
https://developers.google.com/chrome-developer-tools/docs/protocol/1.0/index
I believe that's what Sublime Web Inspector uses.
https://github.com/sokolovstas/SublimeWebInspector

This is admittedly a bad answer, but I figure its better than nothing. If I were in your shoes, here is what I would do:
Now its obviously possible to communicate from the browser to the IDE using some kind of plugin, but I don't like that for at least 2 reasons: I don't enjoy coding in strongly-typed languages, and any code I produce will be inherently browser-specific. Although its not the lightest weight solution, XHR in blocking mode can get the job done.
The IDE would host a http server that sends the CORS header. The compiled source would contain something like this:
var ___, __INTERRUPT_SIGNAL = {};
(function() {
var oXhr = new XMLHttpRequest();
oXhr.open('POST', 'http://localhost/debugging_port', false); // Force XHR to work synchronously
___ = function(nLineNumber) {
var sState = 'line_number='+nLineNumber;
for(var nOffset = 0; nOffset < arguments.length; nOffset++) sState += '&i=' + escape(arguments[nOffset]);
oXhr.send(sState);
if(oXhr.status !== 200) return ___.apply(this, arguments);
var sCommand = oXhr.responseText;
if(sCommand === 'step_into') {
return;
}else if(sCommand === 'step_out') {
throw __INTERRUPT_SIGNAL;
}else{
return ___.apply(this, arguments);
}
};
})();
Now when the Caffeinated Beverage Script compiler compiles the source, it should place calls to the _ function at every line break of the source. For example,
int nCheese = 0;
char cLetter = 'Q';
customClass productZeroQ = nCheese * cLetter;
might compile to something like
var $0xA1, $0xA2, $0xA3;
___(9, $0xA1, $0xA2, $0xA3);
$0xA1 = 0;
___(10, $0xA1, $0xA2, $0xA3);
$0xA2 = 'Q';
___(11, $0xA1, $0xA2, $0xA3);
try {
$0xA3 = $0x34.$0x21($0xA1, $0xA2);
}catch(oSignal){
if(oSignal !== ___INTERRUPT_SIGNAL) throw oSignal;
}
XHR will block execution while the IDE decides to whether to step_out or step_forward. This could be extended to include the stack tracing, but I don't feel like writing a novel.
I know it's ugly, but I hope this helps a bit!

Haxe for javascript without global namespace pollution?

This question only applies to Haxe version < 2.10
I've known about haxe for a while, but never really played with it until yesterday. Being curious, I decided to port showdown.js, a javascript port of markdown.pl, to haxe. This was pretty straightforward, and the javascript it generates seems to run fine (edit: If you want to see it in action, check it out here).
However, I noticed that the generated code dumps a ton of stuff in the global namespace... and what's worse, it does it by assigning values to undeclared identifiers without using the var keyword, so they're global even if you wrap the whole thing with a closure.
For example...
if(typeof js=='undefined') js = {}
...
Hash = function(p) { if( p === $_ ) return; {
...
EReg = function(r,opt) { if( r === $_ ) return; {
...
I managed to clean most of that up with sed, but I'm also bothered by stuff like this:
{
String.prototype.__class__ = String;
String.__name__ = ["String"];
Array.prototype.__class__ = Array;
Array.__name__ = ["Array"];
Int = { __name__ : ["Int"]}
Dynamic = { __name__ : ["Dynamic"]}
Float = Number;
Float.__name__ = ["Float"];
Bool = { __ename__ : ["Bool"]}
Class = { __name__ : ["Class"]}
Enum = { }
Void = { __ename__ : ["Void"]}
}
{
Math.__name__ = ["Math"];
Math.NaN = Number["NaN"];
Math.NEGATIVE_INFINITY = Number["NEGATIVE_INFINITY"];
Math.POSITIVE_INFINITY = Number["POSITIVE_INFINITY"];
Math.isFinite = function(i) {
return isFinite(i);
}
Math.isNaN = function(i) {
return isNaN(i);
}
}
This is some pretty unsavory javascript.
Questions
Is there a fork or clone of haxe somewhere that doesn't pollute globals? Is it worth it to modify the haxe source to get what I want, or has someone already solved this? Googling hasn't turned up much. I'm open to any suggestions. Meanwhile, I'm dying to see what kind of PHP code this thing's going to produce... :D
Answers?
Here are some of the ideas I've tried:
postprocessing
Here's my humble build script; it does a pretty good job of stripping stuff out, but it doesn't catch everything. I'm hesitant to remove the modifications to the built-in constructor prototypes; I'm sure that would break things. Fixing everything might be a bit of a task, and I don't want to start on it if someone's already done the work...
haxe -cp ~/Projects/wmd-new -main Markdown -js markdown.js
echo "this.Markdown=(function(){ var \$closure, Float;" > markdown.clean.js;
sed "s/^if(typeof js=='undefined') js = {}$/if(typeof js=='undefined') var js = {};/g ;
s/^\([ \x09]*\)\([\$_a-zA-Z0-9]* = \({\|function\)\)/\1var \2/g ;
/^[ \x09]*\(else \)\?null;$/d ;
" markdown.js >> markdown.clean.js
echo "return Markdown}());" >> markdown.clean.js;
java -jar closure/compiler.jar --js markdown.clean.js \
--compilation_level SIMPLE_OPTIMIZATIONS \
> markdown.cc.js
--js-namespace switch saves the day
Thanks to Dean Burge for pointing out the namespace switch. This pretty much solved my problem, with a minor bit of help. Here's my current build script. I think this catches all the global variables...
NS=N\$
haxe -cp ~/Projects/wmd-new -main Markdown --js-namespace $NS -js markdown.js
# export our function and declare some vars
echo "this.markdown=(function(){var \$_,\$Main,\$closure,\$estr,js,"$NS"" > markdown.clean.js;
# strip silly lines containing "null;" or "else null;"
sed "/^[ \x09]*\(else \)\?null;$/d ;" markdown.js >> markdown.clean.js
# finish the closure
echo "return "$NS".Markdown.makeHtml}());" >> markdown.clean.js;

I use the namespace switch on the compiler to clean those global root types up.

Haxe is not meant to be used for writing an isolated reusable component in a javascript web application. This is evidenced by the fact that the compiler emits standard library for every goddamn compilation. Most optimal use of javascript target is to write an application entirely in haxe and call external stuff using untyped blocks hoping it won't break anything. You should treat haxe output like a flash clip, oblivious to the environment it runs in, assumes it is the only thing running.
Or you might try wrapping the code with a with() block.

there's a namespaced (experimental) haxe compiler here http://github.com/webr3/haxe

The JSTM JavaScript generator macro optimizes haxe output in a number of ways:
the javascript output is split into seperate files per type
these files are optimized
a loader script loads the required types asynchronously
only one global variable is used: jstm
only code that is actually required to run your app is downloaded
new types can be loaded at runtime which makes possible highly scalable apps
check out http://code.google.com/p/jstm/ for more info.