I am using NAN for including a c++ library in node.js. I understand how to pass numbers and strings back and forth between the two, but I don't understand how to pass arrays. What I would like to do is something like this:
index.js
var test = require('bindings')('test');
var buffer = [0,0,0,0,0,0.1,0,0,0,0,-0.1];
test.encode(buffer, buffer.length);
test.cc
var encoder = new Encoder();
NAN_METHOD(Encode){
//the next line is incorrect, I want to take the buffer array and pass it as a pointer to the encodeBuffer() function
Local<Number> buffer = args[0].As<Number>();
//get the integer value of the second argument
Local<Number> buffer_length = args[1].As<Number>();
int bl_int = buffer_length->IntegerValue();
//call function
encoder.encodeBuffer(buffer, bl_int);
}
void Init(Handle<Object> exports) {
exports->Set(NanNew("encode"), NanNew<FunctionTemplate>(Encode)->GetFunction());
}
The actual method I would like to use from c++ library is declared:
void encodeBuffer(float *buffer, size_t l);
I tried looking at the documentation but they don't say anything about pointers and arrays.. Am I missing something?
Lets say you have a Buffer, how I normally pass is like this:
var buffer = new Buffer([10, 20, 30, 40, 50]);
Then to Pass it to the extension:
Extension.to_image(buffer, buffer.length
And In my native code:
NAN_METHOD(to_image) {
unsigned char*buf = (unsigned char*) node::Buffer::Data(args[0]->ToObject());
unsigned int size = args[1]->Uint32Value();
As you can see at the end I have a buffer and the buffer length transferred to my c++ code.
Here is a good article: http://luismreis.github.io/node-bindings-guide/docs/arguments.html
And another very good one: http://www.puritys.me/docs-blog/article-286-How-to-pass-the-paramater-of-Node.js-or-io.js-into-native-C/C++-function..html
Related
Currently I'm trying to send values bigger than i8 from JS through wasm-memory to Rust like so:
Rust:
// CREATE WASM MEMORY FOR RUST AND JS DIRECT USE.
const WASM_MEMORY_BUFFER_SIZE: usize = 2; // 2 SLOTS
static mut WASM_MEMORY_BUFFER: [u8; WASM_MEMORY_BUFFER_SIZE] = [0; WASM_MEMORY_BUFFER_SIZE]; // INITIALIZE WITH 0
#[wasm_bindgen] // FOR JS TO GET THE POINTER TO MEMORY
pub fn get_wasm_memory_buffer_pointer() -> *const u8 {
let pointer: *const u8;
unsafe {
pointer = WASM_MEMORY_BUFFER.as_ptr();
}
return pointer;
}
(^ Adjusting all the u8's to u16, u32 .. works fine in Rust)
JS:
// IMPORT THE GENERATED JS MODULE
import init from "../pkg/tem.js";
// GET WASM
const runWasm = async () => {
const rustWasm = await init("../pkg/tem_bg.wasm");
// CREATE THE WASM MEMORY
var wasmMemory = new Uint8Array(rustWasm.memory.buffer);
console.log("wasmMemory", wasmMemory);
// GET THE POINTER TO WASM MEMORY FROM RUST
var mem_pointer = rustWasm.get_wasm_memory_buffer_pointer();
wasmMemory[mem_pointer] = "i8";
};
runWasm();
The problem is that an i8 is quite small, and would need to send bigger numbers through memory.
I can adjust the u8 to f.e. u32 in Rust, than set a value: WASM_MEMORY_BUFFER[0] = "i32"; Getting the pointer as an i32 in JS with the Rust funcion is also still possible.
However, in JS I cannot change var wasmMemory = new Uint8Array(rustWasm.memory.buffer);
to Uint32Array();.
Thus, unlike Rust, setting a value: wasmMemory[mem_pointer] = "i32"; doesn't work.
Can this be resolved? Since I would like to set a value bigger than i8 in JS and read that value in Rust.
I'm playing around with GraalVM (Truffle) in OpenJDK 17, and would like to know what the correct way is to return values to the guest language from method calls? Right now I'm struggling with passing a String[] array back.
Exmaple:
Java (host)
class Services
{
Value message;
#HostAccess.Export
public String[] getArrayString()
{
return new String[]{"s1", "s2", "s3"};
}
#HostAccess.Export
public void setMessage( Value v )
{
message = v;
message.pin();
}
}
...
String jsScript = " ... " // see below
try ( Context context = Context.newBuilder().allowHostAccess(HostAccess.SCOPED).build() )
{
Services s = new Services();
context.getBindings("js").putMember("services", s);
context.eval("js", jsScript);
}
JavaScript (guest)
var a = services.getArrayString();
b = '';
for ( var c in a ) b += c;
services.setMessage('' + a + ' // ' + b)
The final message value is "[object Object] // " (b is blank), however I expected something like "[object Object] // s1s2s3".
I've also tried the return types Object[] and ArrayList<String>. I'm not sure why I can't access the elements of the array, either I'm not passing the array back correctly, or I'm not accessing it correctly within the JavaScript script. The examples I've found in the GraalVM docs are always about passing values directly from the host to the guest, but I'd like to do it via a method call - how is that done?
I've finally found a way to pass an array from the host to the guest - using ProxyArray.
https://www.graalvm.org/22.1/reference-manual/embed-languages/#computed-arrays-using-polyglot-proxies
Reading through the Graalvm documentation and experimenting with the very primative examples they provide, I came up with the following changes to my Services class:
ArrayList<String> array = new ArrayList<String>();
#HostAccess.Export
public void prepareList()
{
array.add("s1");
array.add("s2");
array.add("s3");
}
#HostAccess.Export
public ProxyArray getArray()
{
return new ProxyArray() {
#Override
public Object get( long index )
{
return array.get((int)index);
}
#Override
public void set( long index, Value value )
{
// TODO Auto-generated method stub
}
#Override
public long getSize()
{
return array.size();
}
};
}
And finally the guest call looks like this:
services.prepareList();
var a = services.getArray();
b = '';
for ( i = 0; i < a.length; i++ ) b += a[i];
services.setMessage('' + a + ' // ' + b)
And the value of the message is then as expected "[object Object] // s1s2s3"
Hope someone finds this useful ... and if you have a better way of returning objects (Array, Hashtable, whatever) post your answers here - thanks.
First I would give a straight forward answer to your question. To pass an array from Java scope to the guest language scope, you can just send the Java array/list as input to the execute method. For example:
Context context = Context.newBuilder().allowAllAccess(true).build();
int intArray[] = {1,2,3};
Value ret = context.eval("js", "(function getArray(a) {console.log(a[2]);})").execute(intArray);
The above should print 3 in the JS scope.
Now I decide to add the following paragraphs because after reading your code as well as those in some proposed answers, I think it could be of help.
Say you have a Java code snippet like the one below which returns an array from the JS context.
Value val = context.eval("js", "(function getArray() {return [1,2,3];})").execute();
It is not possible to directly access the whole array object from val in the Java language scope because the memory layout of Java arrays is different from the memory layout of JS arrays. However you can obtain elements at different indices of the JS array from within the Java scope as follows:
int index2 = val.getArrayElement(2).asInt();
System.out.println(index2);
The above should print the value 3.
In order to obtain the full array in the Java scope, you must do an explicit array copy. The classic way to do this is to leverage the technique described here: https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html#as-org.graalvm.polyglot.TypeLiteral-
Using that technique one can define an INTEGER_LIST TypeLiteral as such:
public static final TypeLiteral<List<Integer>> INTEGER_LIST =
new TypeLiteral<List<Integer>>() {};
And then return the polyglot Value as an INTEGER_LIST as follows:
List<Integer> intList = context.eval("js", "(function getArray() {return [1,2,3];})").execute().as(INTEGER_LIST);
System.out.println("JS array in Java scope: " + intList);
The above print should produce: [1,2,3]. This strategy also does an explicit array copy in the back ground but it is neat, more or less.
You could apply exactly the same strategy for your String array, by defining a TypeLiteral like:
public static final TypeLiteral<List<String>> STRING_LIST =
new TypeLiteral<List<STRING>>() {};
And obtaining the String list/array in Java scope via ...execute().as(STRING_LIST);. This should give you your String array in the Java scope.
According to this github comment, I can re-create an Uint8ClampedArray or Uint8Array returned from Rust/wasm by accessing the memory of the wasm instance directly:
const textureRaw = new Uint8ClampedArray(memory.buffer, texture.offset(), texture.size());
The thing is, the js files generated by wasm-bindgen already instantiate a wasm instance, and I'd want to access the memory of this particular instance but it doesn't seem to be exported:
// XXXXX_bg.js
const path = require('path').join(__dirname, 'ed25519_sigs_bg.wasm');
const bytes = require('fs').readFileSync(path);
let imports = {};
imports['./ed25519_sigs.js'] = require('./ed25519_sigs.js');
const wasmModule = new WebAssembly.Module(bytes);
const wasmInstance = new WebAssembly.Instance(wasmModule, imports);
module.exports = wasmInstance.exports;
How would I access the current wasm instance's memory buffer ?
I've tried doing:
import { memory } from "XXXXXX_bg";
// say o is returned as an object with the right offset() and size() accessors. It represents an Uint8Array in memory
let outU8A: Uint8Array = new Uint8Array(
memory.buffer,
o.offset(),
o.size()
);
The output is the expected size but every value is zero. Which makes me think I might be trying to load from a second wasm.memory instance ?
In order to send reference from Rust to Javascript we use as_ptr which is a reference to the beginning of your data type. It is a memory address. An example
// we are returning a pointer type
// *const is a raw pointer and borrowing rules do not apply
pub fn cells(&self)->*const Cell{
// as_ptr is the reference to the first item in the vector
// assume that body:Vec<Cell>
self.body.as_ptr()
}
in the example, I am sending a reference to the beginning of the vector type. You could also write a function to return the lenght of the vector.
// init().then((wasm) => {} initilaization of js code is like this.
const cells = new Uint32Array(
// with memory.buffer you can access to the pointer
wasm.memory.buffer,
o.offset(),
o.size()
);
import { memory } from "XXXXXX_bg";
This memory import should work fine, but I guess you try to access freed memory. Although I do not know why freed memory appears as zeros immediately.
I created a short working example, by using static memory:
#[wasm_bindgen]
pub unsafe fn static_value() -> ByteStream {
static mut values: [u8; 3] = [0; 3];
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
Accessing the freed memory of the following code does not work:
#[wasm_bindgen]
pub fn freed_heap_value() -> ByteStream {
let mut values = Box::new([0; 3]);
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
Depending on your use case, you could also free the heap allocation manually afterwards:
#[wasm_bindgen]
pub fn heap_value() -> ByteStream {
let mut values = Box::new([0; 3]);
let values = Box::leak(values); // has to be freed manually
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
I am at a point where I can pull a single javascript declaration such as:
var cars = ["Saab", "Volvo", "BMW"];
parsed from a page.
I would like to be able to get all the elements of the array ("Saab", "Volvo", "BMW") from this declaration.
Should I be using some javascript engine for this, or what else would be the best way to get javascript variable values from my Java code.
I would hate to reinvent the wheel if something is already out there that is able to do this, so I am just looking for advice on something I can use to do this function.
I assume you found a way to transport that javascript object/array into your Java domain as a String or Stream. What you want now is a JSON parser.
One way is to use json.org or other libraries. Further information about json parsing can be found in this thread:
How to parse JSON in Java
The [org.json][1] library is easy to use. Example code below:
import org.json.*;
JSONObject obj = new JSONObject(" .... ");
String pageName = obj.getJSONObject("pageInfo").getString("pageName");
JSONArray arr = obj.getJSONArray("posts");
for (int i = 0; i < arr.length(); i++)
{
String post_id = arr.getJSONObject(i).getString("post_id");
......
} You may find extra examples from: [Parse JSON in Java][2]
Downloadable jar: http://mvnrepository.com/artifact/org.json/json
[1]: http://www.json.org/java/index.html
[2]: http://theoryapp.com/parse-json-in-java/
You might also want to look into jsonb (https://jcp.org/en/jsr/detail?id=353) that was introduced with Java 7. You can bind an object model and transform JSON objects into java objects and vice versa.
you can iterate through all the values in 'window'
for ( var key in window )
{
if ( typeof window]key] == 'object' && window]key].length > 0 )
{
//this is the array you are looking for
}
}
You can get access to javascript object from java by using httpunit
Method 1: JSON parser, as Alex's answer.
Method 2: Javascript parser for Java
Method 3: Regular Expression (A weird way I figured out!)
First pattern is var\s+([a-zA-Z0-9]+)\s+=\s+\[(.*)\]\s*;*
var + one or more space(s) + variable name($1) + one or more space(s) + equals sign + one or more space(s) + array content($2) + ......
Second pattern is "(.*?)", get the string between two quotation marks.
import java.util.ArrayList;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class JSParser {
public String arrayName;
private String tempValues;
public ArrayList<String> values = new ArrayList<String>();
public boolean parseJSArray(String arrayStr){
String p1 = "var\\s+([a-zA-Z0-9]+)\\s+=\\s+\\[(.*)\\]\\s*;*";
Pattern pattern1 = Pattern.compile(p1);
Matcher matcher = pattern1.matcher(arrayStr);
if(matcher.find()){
arrayName = matcher.group(1);
tempValues = matcher.group(2);
Pattern getVal = Pattern.compile("\"(.*?)\"");
Matcher valMatcher = getVal.matcher(tempValues);
while (valMatcher.find()) { // find next match
String value = valMatcher.group(1);
values.add(value);
}
return true;
}else{
return false;
}
}
}
With JDK 8 the code bellow works :
ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");
String js = "var carsfromjs = [\"Saab\", \"Volvo\", \"BMW\"]";
engine.eval(js);
String[] cars = (String[])engine.eval("Java.to(carsfromjs, \"java.lang.String[]\")");
for(int i=0; i<cars.length; i++){
System.out.println(cars[i]);
}
You can find many ways to access Javascript code throught "nashorn" :
http://winterbe.com/posts/2014/04/05/java8-nashorn-tutorial/
http://www.oracle.com/technetwork/articles/java/jf14-nashorn-2126515.html
http://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/
I'm trying to implement the answer to this question about monitoring a Windows filesystem asynchronously. I'm using js ctypes within a ChomeWorker as part of a XULRunner application but I assume this would be the same if I implemented as a Firefox add-on.
As part of the task, I have tried to declare the function ReadDirectoryChangesW as follows (based on my limited knowledge of js ctypes and the MSDN documentation).
const BOOL = ctypes.bool;
const DWORD = ctypes.uint32_t;
const LPDWORD = ctypes.uint32_t.ptr;
const HANDLE = ctypes.int32_t;
const LPVOID = ctypes.voidptr_t;
var library = self.library = ctypes.open("Kernel32.dll");
ReadDirectoryChangesW = library.declare(
"ReadDirectoryChangesW"
, ctypes.winapi_abi
, BOOL // return type
, HANDLE // hDirectory
, LPVOID // lpBuffer
, DWORD // nBufferLength
, BOOL // bWatchSubtree
, DWORD // dwNotifyFilter
, LPDWORD // lpBytesReturned
);
In addition (not featured here), I have declared function mappings for FindFirstChangeNotification() and WaitForSingleObject() which seem to work fine.
The problem I have is that when a filesystem event occurs, I have no idea what I'm supposed to pass in to the lpBuffer argument, or how to interpret the result.
All of the C++ examples seem to use a DWORD array and then cast out the results. My attempt at that is as follows:
const DWORD_ARRAY = new ctypes.ArrayType(DWORD);
var lBuffer = new DWORD_ARRAY(4000);
var lBufferSize = DWORD.size * 4000;
var lBytesOut = new LPDWORD();
ReadDirectoryChangesW(lHandle, lBuffer.address(), lBufferSize, true, WATCH_ALL, lBytesOut)
This seems to just crash XULRunner every time.
Can anyone suggest what I should pass in for the lpBuffer argument and/or how to get results back from ReadDirectoryChangesW()? All I can find online is C++ examples and they're not a lot of help. Thanks.
Here is a cleaner solution: read the comments, lots of learning there. For type definitions, see here.
var path = OS.Constants.Path.desktopDir; // path to monitor
var hDirectory = ostypes.API('CreateFile')(path, ostypes.CONST.FILE_LIST_DIRECTORY | ostypes.CONST.GENERIC_READ, ostypes.CONST.FILE_SHARE_READ | ostypes.CONST.FILE_SHARE_WRITE, null, ostypes.CONST.OPEN_EXISTING, ostypes.CONST.FILE_FLAG_BACKUP_SEMANTICS | ostypes.CONST.FILE_FLAG_OVERLAPPED, null);
console.info('hDirectory:', hDirectory.toString(), uneval(hDirectory));
if (ctypes.winLastError != 0) { //cutils.jscEqual(hDirectory, ostypes.CONST.INVALID_HANDLE_VALUE)) { // commented this out cuz hDirectory is returned as `ctypes.voidptr_t(ctypes.UInt64("0xb18"))` and i dont know what it will be when it returns -1 but the returend when put through jscEqual gives `"breaking as no targetType.size on obj level:" "ctypes.voidptr_t(ctypes.UInt64("0xb18"))"`
console.error('Failed hDirectory, winLastError:', ctypes.winLastError);
throw new Error({
name: 'os-api-error',
message: 'Failed to CreateFile',
});
}
var dummyForSize = ostypes.TYPE.FILE_NOTIFY_INFORMATION.array(1)(); // accept max of 1 notifications at once (in application you should set this to like 50 or something higher as its very possible for more then 1 notification to be reported in one read/call to ReadDirectoryChangesW)
console.log('dummyForSize.constructor.size:', dummyForSize.constructor.size);
console.log('ostypes.TYPE.DWORD.size:', ostypes.TYPE.DWORD.size);
var dummyForSize_DIVIDED_BY_DwordSize = dummyForSize.constructor.size / ostypes.TYPE.DWORD.size;
console.log('dummyForSize.constructor.size / ostypes.TYPE.DWORD.size:', dummyForSize_DIVIDED_BY_DwordSize, Math.ceil(dummyForSize_DIVIDED_BY_DwordSize)); // should be whole int but lets round up with Math.ceil just in case
var temp_buffer = ostypes.TYPE.DWORD.array(Math.ceil(dummyForSize_DIVIDED_BY_DwordSize))();
var temp_buffer_size = temp_buffer.constructor.size; // obeys length of .array
console.info('temp_buffer.constructor.size:', temp_buffer.constructor.size); // will be Math.ceil(dummyForSize_DIVIDED_BY_DwordSize)
var bytes_returned = ostypes.TYPE.DWORD();
var changes_to_watch = ostypes.CONST.FILE_NOTIFY_CHANGE_LAST_WRITE | ostypes.CONST.FILE_NOTIFY_CHANGE_FILE_NAME | ostypes.CONST.FILE_NOTIFY_CHANGE_DIR_NAME; //ostypes.TYPE.DWORD(ostypes.CONST.FILE_NOTIFY_CHANGE_LAST_WRITE | ostypes.CONST.FILE_NOTIFY_CHANGE_FILE_NAME | ostypes.CONST.FILE_NOTIFY_CHANGE_DIR_NAME);
console.error('start hang');
var rez_RDC = ostypes.API('ReadDirectoryChanges')(hDirectory, temp_buffer.address(), temp_buffer_size, true, changes_to_watch, bytes_returned.address(), null, null);
var cntNotfications = 0;
var cOffset = 0;
while (cOffset < bytes_returned) {
cntNotfications++;
var cNotif = ctypes.cast(temp_buffer.addressOfElement(cOffset), ostypes.TYPE.FILE_NOTIFY_INFORMATION.ptr).contents; // cannot use `temp_buffer[cOffset]` here as this is equivlaent of `temp_buffer.addressOfElement(cOffset).contents` and cast needs a ptr
console.info('cNotif:', cNotif.toString());
cOffset += cNotif.NextEntryOffset; // same as doing cNotif.getAddressOfField('NextEntryoffset').contents // also note that .contents getter makes it get a primaive value so DWORD defined as ctypes.unsigned_long will not be returned as expected ctypes.UInt64 it will be primative (due to the .contents getter), so no need to do the typical stuff with a `var blah = ctypes.unsigned_long(10); var number = blah.value.toString();`
}
console.info('total notifications:', cntNotifications);
I'm working on getting the async version working but having a tricky time.
Here's what I learned as I'm working on doing the same now, still in progress
You have to create a buffer of DWORD so var buf = ctypes.ArrayType(DWORD, BUFSIZE) as it needs to be aligned on DWORD boundary, whatever this means
I don't know what BUFSIZE should be exactly but i have seen 2048 and 4096, I don't know why. I have also seen BUFSIZE of 1024*64, no idea why
Then after succesfully running ReadDirectoryChangesW cast this buffer to FILE_NOTIFY_INFORMATION and then read its contents
Pass null to the final 2 arguments only if you don't want async, we want async so we are going to use the LPOVERLAPPED struct and pass it there.
Edit
Here's solution for sync: This successfully reads one event. If have more you have to move over in temp_buff by next_entry_offset and cast, see here. Install that addon and make a new folder on your desktop or something, and it will log in browser console.
I'm working on async version, having some trouble with that.