According to this github comment, I can re-create an Uint8ClampedArray or Uint8Array returned from Rust/wasm by accessing the memory of the wasm instance directly:
const textureRaw = new Uint8ClampedArray(memory.buffer, texture.offset(), texture.size());
The thing is, the js files generated by wasm-bindgen already instantiate a wasm instance, and I'd want to access the memory of this particular instance but it doesn't seem to be exported:
// XXXXX_bg.js
const path = require('path').join(__dirname, 'ed25519_sigs_bg.wasm');
const bytes = require('fs').readFileSync(path);
let imports = {};
imports['./ed25519_sigs.js'] = require('./ed25519_sigs.js');
const wasmModule = new WebAssembly.Module(bytes);
const wasmInstance = new WebAssembly.Instance(wasmModule, imports);
module.exports = wasmInstance.exports;
How would I access the current wasm instance's memory buffer ?
I've tried doing:
import { memory } from "XXXXXX_bg";
// say o is returned as an object with the right offset() and size() accessors. It represents an Uint8Array in memory
let outU8A: Uint8Array = new Uint8Array(
memory.buffer,
o.offset(),
o.size()
);
The output is the expected size but every value is zero. Which makes me think I might be trying to load from a second wasm.memory instance ?
In order to send reference from Rust to Javascript we use as_ptr which is a reference to the beginning of your data type. It is a memory address. An example
// we are returning a pointer type
// *const is a raw pointer and borrowing rules do not apply
pub fn cells(&self)->*const Cell{
// as_ptr is the reference to the first item in the vector
// assume that body:Vec<Cell>
self.body.as_ptr()
}
in the example, I am sending a reference to the beginning of the vector type. You could also write a function to return the lenght of the vector.
// init().then((wasm) => {} initilaization of js code is like this.
const cells = new Uint32Array(
// with memory.buffer you can access to the pointer
wasm.memory.buffer,
o.offset(),
o.size()
);
import { memory } from "XXXXXX_bg";
This memory import should work fine, but I guess you try to access freed memory. Although I do not know why freed memory appears as zeros immediately.
I created a short working example, by using static memory:
#[wasm_bindgen]
pub unsafe fn static_value() -> ByteStream {
static mut values: [u8; 3] = [0; 3];
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
Accessing the freed memory of the following code does not work:
#[wasm_bindgen]
pub fn freed_heap_value() -> ByteStream {
let mut values = Box::new([0; 3]);
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
Depending on your use case, you could also free the heap allocation manually afterwards:
#[wasm_bindgen]
pub fn heap_value() -> ByteStream {
let mut values = Box::new([0; 3]);
let values = Box::leak(values); // has to be freed manually
let slice = values.as_mut_slice();
slice.copy_from_slice(&[1, 2, 3]); // fill with some data
ByteStream::new(slice)
}
Related
I am delivering data from PLC Siemens S7 through node-red installed on IoT edge device to the cloud storage.
In the node-red I receive data as a JSON object with several properties every second from PLC node. The JSON object (or data) has the same properties every second.
Some properties change every second, some change in five minute, some remain constant for several hours.
To handle data to the cloud the JSON object is handled to the IoT hub node in the node-red.
I would like to process this JSON object before handling to the IoT hub node in a way that the JSON object has only the properties which changed from previous state (a second ago). It means that I want to compare an input object at the moment t with the object at t-1 and generate a new object with properties of the object at t which differ from those of the object at t-1.
The new object will be handled further. The object processing is expected to be done in the function node in the node-red. From PLC node the JSON object is received as the msg.payload.
Could you please help me to script in (javascript) the process I have described?
msg.payload to the function node is json object:
{"Zone0": 200,
"Wire_tension": 2.5}
UPD: Since it was not clear what I have tried, here is the code. Sorry for not providing it, I thought it was clear.
var state = {}; // this state will be messaged further
var prev_state = {
"Zone0": 0,
"Wire_tension": 0
}; // initially zeros are assigned
var current_state = msg.payload; // has these two properties and updated every second
// comparison of properties
if (current_state.Zone0 != prev_state.Zone0) {
state["Zone0"] = current_state.Zone0;
prev_state["Zone0"] = current_state.Zone0;
}
if (current_state.Wire_tension != prev_state.Wire_tension) {
state["Wire_tension"] = current_state.Wire_tension;
prev_state["Wire_tension"] = current_state.Wire_tension;
}
msg.payload = state;
return msg;
It looks like when the flow in the node-red starts the function node runs the code every time. Therefore prev_state is assigned every time. I understand that I need to somehow handle initial state which is updated further.
UPD2:
I also tried using the following code (also with context feature). The example is with my real data:
var state = {}; // this state will be messaged further
flow.set(["Zone0", "Wire_tension"],[0,0]); // initially zeros are assigned
var current_state = msg.payload; // has these two properties and updated every second
// comparison of properties
if (current_state.Zone0 != flow.get("Zone0")) {
state["Zone0"] = current_state.Zone0;
flow.set("Zone0", current_state.Zone0);
}
if (current_state.Wire_tension != flow.get("Wire_tension")) {
state["Wire_tension"] = current_state.Wire_tension;
flow.set("Wire_tension", current_state.Wire_tension);
}
msg.payload = state;
return msg;
best option is use node-red global context variable
var object_to_build = {}
var key_1_val = global.get("key_1");
if (msg.payload.key_1 != key_1_val) {
object_tto_build['key_1'] = msg.payload.key_1
global.set("key_1", msg.payload.key_1);
}
// do whatever you like to do with - object_to_build. I have not validated the code, just the typo
In the "On Start" tab in properties of function node:
flow.set(["Zone0", "Wire_tension"], [0, 0]);
In the "On Message" tab in the properties of function node:
var state = {}; // this state will be messaged further
var current_state = msg.payload; // has these two properties and updated every second
// comparison of properties
if (current_state.Zone0 != context.get("Zone0")) {
state["Zone0"] = current_state.Zone0;
context.set("Zone0", current_state.Zone0);
}
if (current_state.Wire_tension != context.get("Wire_tension")) {
state["Wire_tension"] = current_state.Wire_tension;
context.set("Wire_tension", current_state.Wire_tension);
}
msg.payload = state;
return msg;
Currently I'm trying to send values bigger than i8 from JS through wasm-memory to Rust like so:
Rust:
// CREATE WASM MEMORY FOR RUST AND JS DIRECT USE.
const WASM_MEMORY_BUFFER_SIZE: usize = 2; // 2 SLOTS
static mut WASM_MEMORY_BUFFER: [u8; WASM_MEMORY_BUFFER_SIZE] = [0; WASM_MEMORY_BUFFER_SIZE]; // INITIALIZE WITH 0
#[wasm_bindgen] // FOR JS TO GET THE POINTER TO MEMORY
pub fn get_wasm_memory_buffer_pointer() -> *const u8 {
let pointer: *const u8;
unsafe {
pointer = WASM_MEMORY_BUFFER.as_ptr();
}
return pointer;
}
(^ Adjusting all the u8's to u16, u32 .. works fine in Rust)
JS:
// IMPORT THE GENERATED JS MODULE
import init from "../pkg/tem.js";
// GET WASM
const runWasm = async () => {
const rustWasm = await init("../pkg/tem_bg.wasm");
// CREATE THE WASM MEMORY
var wasmMemory = new Uint8Array(rustWasm.memory.buffer);
console.log("wasmMemory", wasmMemory);
// GET THE POINTER TO WASM MEMORY FROM RUST
var mem_pointer = rustWasm.get_wasm_memory_buffer_pointer();
wasmMemory[mem_pointer] = "i8";
};
runWasm();
The problem is that an i8 is quite small, and would need to send bigger numbers through memory.
I can adjust the u8 to f.e. u32 in Rust, than set a value: WASM_MEMORY_BUFFER[0] = "i32"; Getting the pointer as an i32 in JS with the Rust funcion is also still possible.
However, in JS I cannot change var wasmMemory = new Uint8Array(rustWasm.memory.buffer);
to Uint32Array();.
Thus, unlike Rust, setting a value: wasmMemory[mem_pointer] = "i32"; doesn't work.
Can this be resolved? Since I would like to set a value bigger than i8 in JS and read that value in Rust.
When passing a string to a Rust WASM module, the passed data shows up as blank, as per the pattern matching in the real_code::compute function
The following code is what I've tried. I don't know if it has to do with how its being returned, but when I pass a hardcoded &str, it works fine. However, the JsInteropString shows as blank.
Here is how I encoded the string before sending it to WASM (from Passing a JavaScript string to a Rust function compiled to WebAssembly)
const memory = new WebAssembly.Memory({ initial: 20,
maximum: 100 });
const importObject = {
env: { memory }
};
const memoryManager = (memory) => {
var base = 0;
// NULL is conventionally at address 0, so we "use up" the first 4
// bytes of address space to make our lives a bit simpler.
base += 4;
return {
encodeString: (jsString) => {
// Convert the JS String to UTF-8 data
const encoder = new TextEncoder();
const encodedString = encoder.encode(jsString);
// Organize memory with space for the JsInteropString at the
// beginning, followed by the UTF-8 string bytes.
const asU32 = new Uint32Array(memory.buffer, base, 2);
const asBytes = new Uint8Array(memory.buffer, asU32.byteOffset + asU32.byteLength, encodedString.length);
// Copy the UTF-8 into the WASM memory.
asBytes.set(encodedString);
// Assign the data pointer and length values.
asU32[0] = asBytes.byteOffset;
asU32[1] = asBytes.length;
// Update our memory allocator base address for the next call
const originalBase = base;
base += asBytes.byteOffset + asBytes.byteLength;
return originalBase;
}
};
};
invoking wasm like this:
//...loading and compiling WASM, getting instance from promise (standard)
const testStr = "TEST"
const input = myMemory.encodeString(testStr);
const offset = instance.exports.func_to_call(input);
// A struct with a known memory layout that we can pass string information in
#[repr(C)]
pub struct JsInteropString {
data: *const u8,
len: usize,
}
// Our FFI shim function
#[no_mangle]
pub unsafe extern "C" fn func_to_call(s: *const JsInteropString) -> *mut c_char {
// ... check for nulls etc
/*
THROWS ERROR
error[E0609]: no field `data` on type `*const JsInteropString`
*/
//////let data = std::slice::from_raw_parts(s.data, s.len);
//this fixes the above error
let data = std::slice::from_raw_parts((*s).data, (*s).len);
let dataToPrint: Result<_, _> = std::str::from_utf8(data);
let real_res: &str = match dataToPrint {
Ok(s) => real_code::compute(dataToPrint.unwrap()), //IS BLANK
//Ok(s) => real_code::compute("SUCCESS"), //RETURNS "SUCCESS made it"
Err(_) => real_code::compute("ERROR"),
};
unsafe {
let s = CString::new(real_res).unwrap();
println!("result: {:?}", &s);
s.into_raw()
}
}
mod real_code {
pub fn compute(operator: &str) -> &str {
match operator {
"SUCCESS" => "SUCCESS made it",
"ERROR" => "ERROR made it",
"TEST" => "TEST made it",
_ => operator,
}
}
}
When the function is called from JavaScript, it should return the same string passed. I even went as far as to update my Rust environment with rustup... any ideas?
UPDATE
Using a more representative version of the referenced post:
#[no_mangle]
pub unsafe extern "C" fn compute(s: *const JsInteropString) -> i32 {
let s = match s.as_ref() {
Some(s) => s,
None => return -1,
};
// Convert the pointer and length to a `&[u8]`.
let data = std::slice::from_raw_parts(s.data, s.len);
// Convert the `&[u8]` to a `&str`
match std::str::from_utf8(data) {
Ok(s) => real_code::compute(s),
Err(_) => -2,
}
}
mod real_code {
pub fn compute(operator: &str) -> i32 {
match operator {
"SUCCESS" => 1,
"ERROR" => 2,
"TEST" => 3,
_ => 10,
}
}
}
regardless on the string passed through the js encoding, it still returns the default value from compute. I don't know if the referenced post actually solves the problem.
So yes, the code compiles. It is a runtime issue I am trying to solve. Something is off about how the string is being encoded and passed to WASM.
Update 2
Tried switching my toolchain to Nightly, and what I also found is that the following macros/attributes throw an error regarding unstable attributes
#![feature(wasm_import_memory)]
#![wasm_import_memory]
After realizing that I need to tell rust that memory will be imported, this seems to have fixed the problem of the string being passed as empty. It wasn't empty, it wasnt even passed it seems like.
Inside the .cargo file
[target.wasm32-unknown-unknown]
rustflags = [
"-Clink-args=-s EXPORTED_FUNCTIONS=['_func_to_call'] -s ASSERTIONS=1",
"-C", "link-args=--import-memory",
]
This seems to may have done the trick! #Shepmaster, update your answer from last year for others who stumble across it, as it has good SEO. This is due to changes in the rust environment.
I am using NAN for including a c++ library in node.js. I understand how to pass numbers and strings back and forth between the two, but I don't understand how to pass arrays. What I would like to do is something like this:
index.js
var test = require('bindings')('test');
var buffer = [0,0,0,0,0,0.1,0,0,0,0,-0.1];
test.encode(buffer, buffer.length);
test.cc
var encoder = new Encoder();
NAN_METHOD(Encode){
//the next line is incorrect, I want to take the buffer array and pass it as a pointer to the encodeBuffer() function
Local<Number> buffer = args[0].As<Number>();
//get the integer value of the second argument
Local<Number> buffer_length = args[1].As<Number>();
int bl_int = buffer_length->IntegerValue();
//call function
encoder.encodeBuffer(buffer, bl_int);
}
void Init(Handle<Object> exports) {
exports->Set(NanNew("encode"), NanNew<FunctionTemplate>(Encode)->GetFunction());
}
The actual method I would like to use from c++ library is declared:
void encodeBuffer(float *buffer, size_t l);
I tried looking at the documentation but they don't say anything about pointers and arrays.. Am I missing something?
Lets say you have a Buffer, how I normally pass is like this:
var buffer = new Buffer([10, 20, 30, 40, 50]);
Then to Pass it to the extension:
Extension.to_image(buffer, buffer.length
And In my native code:
NAN_METHOD(to_image) {
unsigned char*buf = (unsigned char*) node::Buffer::Data(args[0]->ToObject());
unsigned int size = args[1]->Uint32Value();
As you can see at the end I have a buffer and the buffer length transferred to my c++ code.
Here is a good article: http://luismreis.github.io/node-bindings-guide/docs/arguments.html
And another very good one: http://www.puritys.me/docs-blog/article-286-How-to-pass-the-paramater-of-Node.js-or-io.js-into-native-C/C++-function..html
I'm trying to implement the answer to this question about monitoring a Windows filesystem asynchronously. I'm using js ctypes within a ChomeWorker as part of a XULRunner application but I assume this would be the same if I implemented as a Firefox add-on.
As part of the task, I have tried to declare the function ReadDirectoryChangesW as follows (based on my limited knowledge of js ctypes and the MSDN documentation).
const BOOL = ctypes.bool;
const DWORD = ctypes.uint32_t;
const LPDWORD = ctypes.uint32_t.ptr;
const HANDLE = ctypes.int32_t;
const LPVOID = ctypes.voidptr_t;
var library = self.library = ctypes.open("Kernel32.dll");
ReadDirectoryChangesW = library.declare(
"ReadDirectoryChangesW"
, ctypes.winapi_abi
, BOOL // return type
, HANDLE // hDirectory
, LPVOID // lpBuffer
, DWORD // nBufferLength
, BOOL // bWatchSubtree
, DWORD // dwNotifyFilter
, LPDWORD // lpBytesReturned
);
In addition (not featured here), I have declared function mappings for FindFirstChangeNotification() and WaitForSingleObject() which seem to work fine.
The problem I have is that when a filesystem event occurs, I have no idea what I'm supposed to pass in to the lpBuffer argument, or how to interpret the result.
All of the C++ examples seem to use a DWORD array and then cast out the results. My attempt at that is as follows:
const DWORD_ARRAY = new ctypes.ArrayType(DWORD);
var lBuffer = new DWORD_ARRAY(4000);
var lBufferSize = DWORD.size * 4000;
var lBytesOut = new LPDWORD();
ReadDirectoryChangesW(lHandle, lBuffer.address(), lBufferSize, true, WATCH_ALL, lBytesOut)
This seems to just crash XULRunner every time.
Can anyone suggest what I should pass in for the lpBuffer argument and/or how to get results back from ReadDirectoryChangesW()? All I can find online is C++ examples and they're not a lot of help. Thanks.
Here is a cleaner solution: read the comments, lots of learning there. For type definitions, see here.
var path = OS.Constants.Path.desktopDir; // path to monitor
var hDirectory = ostypes.API('CreateFile')(path, ostypes.CONST.FILE_LIST_DIRECTORY | ostypes.CONST.GENERIC_READ, ostypes.CONST.FILE_SHARE_READ | ostypes.CONST.FILE_SHARE_WRITE, null, ostypes.CONST.OPEN_EXISTING, ostypes.CONST.FILE_FLAG_BACKUP_SEMANTICS | ostypes.CONST.FILE_FLAG_OVERLAPPED, null);
console.info('hDirectory:', hDirectory.toString(), uneval(hDirectory));
if (ctypes.winLastError != 0) { //cutils.jscEqual(hDirectory, ostypes.CONST.INVALID_HANDLE_VALUE)) { // commented this out cuz hDirectory is returned as `ctypes.voidptr_t(ctypes.UInt64("0xb18"))` and i dont know what it will be when it returns -1 but the returend when put through jscEqual gives `"breaking as no targetType.size on obj level:" "ctypes.voidptr_t(ctypes.UInt64("0xb18"))"`
console.error('Failed hDirectory, winLastError:', ctypes.winLastError);
throw new Error({
name: 'os-api-error',
message: 'Failed to CreateFile',
});
}
var dummyForSize = ostypes.TYPE.FILE_NOTIFY_INFORMATION.array(1)(); // accept max of 1 notifications at once (in application you should set this to like 50 or something higher as its very possible for more then 1 notification to be reported in one read/call to ReadDirectoryChangesW)
console.log('dummyForSize.constructor.size:', dummyForSize.constructor.size);
console.log('ostypes.TYPE.DWORD.size:', ostypes.TYPE.DWORD.size);
var dummyForSize_DIVIDED_BY_DwordSize = dummyForSize.constructor.size / ostypes.TYPE.DWORD.size;
console.log('dummyForSize.constructor.size / ostypes.TYPE.DWORD.size:', dummyForSize_DIVIDED_BY_DwordSize, Math.ceil(dummyForSize_DIVIDED_BY_DwordSize)); // should be whole int but lets round up with Math.ceil just in case
var temp_buffer = ostypes.TYPE.DWORD.array(Math.ceil(dummyForSize_DIVIDED_BY_DwordSize))();
var temp_buffer_size = temp_buffer.constructor.size; // obeys length of .array
console.info('temp_buffer.constructor.size:', temp_buffer.constructor.size); // will be Math.ceil(dummyForSize_DIVIDED_BY_DwordSize)
var bytes_returned = ostypes.TYPE.DWORD();
var changes_to_watch = ostypes.CONST.FILE_NOTIFY_CHANGE_LAST_WRITE | ostypes.CONST.FILE_NOTIFY_CHANGE_FILE_NAME | ostypes.CONST.FILE_NOTIFY_CHANGE_DIR_NAME; //ostypes.TYPE.DWORD(ostypes.CONST.FILE_NOTIFY_CHANGE_LAST_WRITE | ostypes.CONST.FILE_NOTIFY_CHANGE_FILE_NAME | ostypes.CONST.FILE_NOTIFY_CHANGE_DIR_NAME);
console.error('start hang');
var rez_RDC = ostypes.API('ReadDirectoryChanges')(hDirectory, temp_buffer.address(), temp_buffer_size, true, changes_to_watch, bytes_returned.address(), null, null);
var cntNotfications = 0;
var cOffset = 0;
while (cOffset < bytes_returned) {
cntNotfications++;
var cNotif = ctypes.cast(temp_buffer.addressOfElement(cOffset), ostypes.TYPE.FILE_NOTIFY_INFORMATION.ptr).contents; // cannot use `temp_buffer[cOffset]` here as this is equivlaent of `temp_buffer.addressOfElement(cOffset).contents` and cast needs a ptr
console.info('cNotif:', cNotif.toString());
cOffset += cNotif.NextEntryOffset; // same as doing cNotif.getAddressOfField('NextEntryoffset').contents // also note that .contents getter makes it get a primaive value so DWORD defined as ctypes.unsigned_long will not be returned as expected ctypes.UInt64 it will be primative (due to the .contents getter), so no need to do the typical stuff with a `var blah = ctypes.unsigned_long(10); var number = blah.value.toString();`
}
console.info('total notifications:', cntNotifications);
I'm working on getting the async version working but having a tricky time.
Here's what I learned as I'm working on doing the same now, still in progress
You have to create a buffer of DWORD so var buf = ctypes.ArrayType(DWORD, BUFSIZE) as it needs to be aligned on DWORD boundary, whatever this means
I don't know what BUFSIZE should be exactly but i have seen 2048 and 4096, I don't know why. I have also seen BUFSIZE of 1024*64, no idea why
Then after succesfully running ReadDirectoryChangesW cast this buffer to FILE_NOTIFY_INFORMATION and then read its contents
Pass null to the final 2 arguments only if you don't want async, we want async so we are going to use the LPOVERLAPPED struct and pass it there.
Edit
Here's solution for sync: This successfully reads one event. If have more you have to move over in temp_buff by next_entry_offset and cast, see here. Install that addon and make a new folder on your desktop or something, and it will log in browser console.
I'm working on async version, having some trouble with that.