How to build flatbuffers message with existing struct buffer in javascript - javascript

I'm sending an ack to a received message in Node.js server and I want to echo the messageId back to the client. Currently I'm parsing the messageId from a buffer to string and building the ack from the string. Parsing the id to string and back is unnecessary but I'm unable to build the ack directly with the stuct buffer.
This is how it works when messageid is passed in as a string.
function createAck(messageId) {
let builder = new flatbuffers.Builder;
const request = MyServer.MessageAck;
request.startMessageAck(builder);
request.addMessgeId(builder, createUUID(builder, messageId));
const requestMessage = request.endMessageAck(builder);
return builder.finish(requestMessage);
}
function createUUID(builder, messageId) {
let uuidBytes = new Uint8Array(uuidParse.parse(messageId));
let dataview = new DataView(uuidBytes.buffer);
return MyServer.UUID.createUUID(builder,
flatbuffers.Long.create(dataview.getInt32(0, true), dataview.getInt32(4, true)),
flatbuffers.Long.create(dataview.getInt32(8, true), dataview.getInt32(12, true)));
}
I would like to pass in the messageId as a buffer directly taken from the message with
request.addMessgeId(builder, messageId);
But I'm getting an error: 'FlatBuffers: struct must be serialized inline.'
Here is the struct:
struct UUID {
low_bytes: ulong;
high_bytes: ulong;
}

The error refers to the fact that structs must be stored in-line, i.e. their bytes must be written to the buffer in between startMessageAck and endMessageAck. You can't refer to a struct stored elsewhere.
You should be able to copy the existing struct without using the intermediate Uint8Array and DataView however, something along the lines of (not tested):
request.addMessgeId(builder, MyServer.UUID.createUUID(builder,
messageId.low_bytes(), messageId.high_bytes());
Assuming messageId is a reference to an incoming UUID struct, can't tell from your code.
Even better would be if you could copy the struct using the JS equivalent of C memcpy, but that would require some hackery directly on the ByteBuffer in the builder that the current API doesn't directly support, so is probably not worth it for just 2 fields.

Related

Convert JavaScript Uint8Array object into Scala Object

A TypeScript application sends a Uint8Array object through an HTTP POST request to a Scala Play application.
How to convert the Uint8Array object into a Scala object in the Play application?
For example, the TypeScript code sends the following object:
{stuff: new Uint8Array([0, 1, 2])}
Inside the Scala Play controller:
case class StuffData(stuff: List[Byte])
implicit val reads: Reads[StuffData] = Json.reads[StuffData]
def processStuff = Action.async(parse.json[StuffData]) { request =>
val stuffData = request.body
println(stuffData)
}
This does not work... the error message from Play is:
For request 'POST /send-stuff'
[Json validation error List((obj.key,List(JsonValidationError(List(error.expected.jsarray),WrappedArray()))))]
By default, Unit8Array is encoded in JSON as {"0":1,"1":2,"2":3,"3":4}, so you can decode it in Scala as a Map or write your custom reader, that can translate this object into an array type. Or you could make changes from the other side, instead of using Uint8Array you can use an array or a custom stringify function that makes expected JSON.
In my opinion, the easiest one is writing the custom reader. Custom reader example:
implicit val stuffReader = new Reads[StuffData] {
def reads(js: JsValue): JsResult[StuffData] = {
JsSuccess(StuffData(
(js \ "stuff").as[Map[String, Int]].toList.map(_._2)
))
}
}

VB equivalent of Node-Red Buffer.from?

I'll try to summarize the issue the best way I can. I recently purchased a dual interface board that has TCP communication capabilities. In order to get information from the board an array of bytes needs sent to the board in which it will respond with the desired information. In node red I have been able to send the array of bytes using a function and receive a response from the TCP module.
However I would like to use this in an application I am developing that is more user-friendly than node red. Unfortunately, no matter what I have tried I have not been able to receive a response from the device using Visual Basic.
In node red, the array looks something like this:
Msg.payload = Buffer.from([8,121,50,3,100)];
Without buffer.from the device would not respond. I have tried encoding the string in VB into a byte variable and sending via socket, but am having no luck. Any suggestions? Here is the VB code.
Imports System.Net
Public Class Form1
Dim TCPClientz As Sockets.TcpClient
Dim TCPClientStream As Sockets.NetworkStream
Private Sub SendBytesButton_Click(sender As Object, e As EventArgs) Handles SendBytesButton.Click
'Dim intcount As Integer
Dim sendbytes() As Byte = System.Text.Encoding.UTF8.GetBytes(bytestextbox.Text)
TCPClientz = New Sockets.TcpClient(ServerTextBox.Text, PortTextBox.Text) 'Device IP and Port are working. Changing port throws error.
TCPClientStream = TCPClientz.GetStream()
'intcount = TCPClientz.Client.Send(sendbytes)
TCPClientStream.Write(sendbytes, 0, sendbytes.Length)
If TCPClientStream.DataAvailable = True Then 'At this point, I NEVER have gotten the stream to indicate there is available data.
Dim rcvbytes(TCPClientz.ReceiveBufferSize) As Byte
TCPClientStream.Read(rcvbytes, 0, CInt(TCPClientz.ReceiveBufferSize))
replytextbox.Text = System.Text.Encoding.UTF8.GetString(rcvbytes)
End If
End Sub
Does anyone know of any way to capture the bytes being sent by NODE-RED? I can view the payload, but I don't believe this is a representation of the actual bytes being sent. I could try to pair this up with the BYTE array in VB to see if they match.

Parse javascript Blob in golang

In Go, you can read a form sent using Ajax and FormData using r.ParseMultipartForm(), which populates the Form map with form request data.
func form(w http.ResponseWriter, r *http.Request) {
r.ParseMultipartForm(500) //
fmt.Fprintf(w, "This is the value of %+v", r.Form)
}
However, I haven't found a method to parse Blobs. The above code returns an empty map whenever instead of sending a form, I send a Blob. That is, when I send this:
var blob = new Blob([JSON.stringify(someJavascriptObj)]);
//XHR initialization, etc. etc.
xhr.send(blob);
the Go code above doesn't work. Now, when I send this:
var form = new FormData(document.querySelector("form"));
//...
xhr.send(form);
I can read form data without problems.
r.ParseMultipartForm(500)
Perhaps an error is being returned here? Try capturing the error:
if err := r.ParseMultipartForm(500); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
Also, consider raising the 500 byte memory limit as larger blobs will be written to temporary files.
I think javascript treats blob as file, so your can look it in r.MultipartForm.File, get file header, open it, read, decode and parse.
Try for example
r.ParseMultipartForm(500)
fmt.Fprintf(w, "This is the value of %+v", *r.MultipartForm.File)
}
I presume Javascript's Blob is a hex string which can eventually be converted to []byte, which is a standard type for JSON in Go.
// Once you get the blob
blobString := `7b22666f6f223a205b22626172222c202262617a222c2039395d7d`
b, _ := hex.DecodeString(blobString)
json := string(b)
fmt.Println(json) // prints out {"foo": ["bar", "baz", 99]}
You might want to look into encoding/hex and encoding/binary packages to decode your blob acquired from Javascript to type []byte in Go, if it's not already.

Better way to de/serialize an Thrift object from/to JSON using the pure Javascript library?

I have a web server returning an thrift object serialized used the JSON protocol to a client html page using the pure Javascript Thrift library (thrift.js).
The server for example:
from MyThriftFile.ttypes import ThriftClass
from thrift import TSerialization
from thrift.protocol import TJSONProtocol
thrift_obj = new ThriftClass()
result = TSerialization.serialize(
thrift_obj,
protocol_factory=TJSONProtocol.TJSONProtocolFactory())
return result
Now in the C#, Python, Java, and even the node.js Thrift libraries there is some form of this generic TSerialization or TDeserialization utlity and its more or less implemented like so:
def deserialize(base,
buf,
protocol_factory=TBinaryProtocol.TBinaryProtocolFactory()):
transport = TTransport.TMemoryBuffer(buf)
protocol = protocol_factory.getProtocol(transport)
base.read(protocol)
return base
So it gets it data, loads it up into a throw away transport (because we are not going to send this information anywhere), creates a new protocol object for encoding this data, and finally the actual thrift object reads this data to populate itself.
The pure javacript library however seems to lack this functionality. I understand why the client library only support the JSON protocol (web pages don't deal in raw binary data) but why not method for de/serialization from/to JSON?
I made my own method for doing the job but it seems hacky. Anyone have a better trick?
$(document).ready(function() {
$.get("www.mysite.com/thrift_object_i_want/", function(data, status) {
var transport = new Thrift.Transport();
var protocol = new Thrift.Protocol(transport);
// Sets the data we are going to read from.
transport.setRecvBuffer(data);
// This is basically equal to
// rawd = data
rawd = transport.readAll();
// The following is lifited from
// readMessageBegin().
// These params I am setting are private memeber
// vars that protocol needs set in order to read
// data set in setRevBuff()
obj = $.parseJSON(rawd);
protocol.rpos = []
protocol.rstack = []
protocol.rstack.push(obj)
// Now that the protocl has been hacked to function
// populate our object from it
tc = new ThriftClass();
tc.read(protocol);
// u is now a js object equal to the python object
});
});
I haven't tried your code but I assume it is working.
It seems correct and is essentially what the TSerializer et al classes do in those other languages. Granted, it could be wrapped in a more friendly way for the vanilla JS library.
The only thing that I might recommend to make it less "hacky" would be to just create a Thrift service method that returns the object(s) you need... then the serialization/deserialization logic will be automatically wrapped up nicely for you in the generated service client.

Passing objects through Django template to javascript

I am making a website using Django and I want to pass a python object from my view (where it is created) through the Django template and to a Dajax call. The problem is that by the time it gets to dajax it has been turned into type unicode.
In my Template
<script>
var emailer = "{{emailer|safe}}"; <---If I omit the quotes here then I get a javascript error.
sessionStorage.setItem('emailer',emailer);
$(document).ready(function(){
$('.send').on('click', function(e){
var emailer = sessionStorage.getItem('emailer');
Dajaxice.InterfaceApp.sendEmail(submitverify,{'emailer':emailer});
});
});
</script>
The dajax function
#dajaxice_register(method='GET')
def sendEmail(emailer):
logger.warning("type: %s, %s" % (type(emailer),emailer))
email_body = "message"
emailer.addToMessage(email_body)
emailer.send()
message = "Email Sent"
return json.dumps({'message':message})
Here the logger statement returns: type: <type 'unicode'>, <Utils.SIMPL_Emailer instance at 0x103142ab8>. Is there any way to fix this so that I get my emailer object instead of a unicode string?
First try to understand what is happening:
On your template you're trying to save a Python object to a Javascript var:
var emailer = "{{emailer|safe}}";`
But it's not possible. When your template is rendered by Django what you really get is a call to object __str__() method and your Javascript will store the <Utils.SIMPL_Emailer instance at 0x103142ab8> value on your emailer var. And remember: this code run in the client browser. That's why you get an error when you remove the quotes.
To solve it you need to first serialize your emailer object (Turn it into something that could be represented as a String, for example, and then turned back to Python Object). But as pointed by Peter DeGlopper it is a very insecure approach. Never, ever deserialize an whole object that was public accessible. Instead send only the email data to your template. You can create a dictionary with this data, turn it into JSON (it's a serialization too, but this time you are serializating only data) and then pass it to your template.
So do not put your emailer on the template context. Instead create a dictonary and pass it to the template.
Then in your Python sendEmail(emailer) method you'll need to instanciate a new Emailer object and feed it with the data, like:
#dajaxice_register(method='GET')
def sendEmail(email_json):
email = json.loads(email_json) # email_json is a json with your email data only
logger.warning("type: %s, %s" % (type(email_json),email_json))
emailer = Emailer("<with all your params...>")
emailer.addToMessage(email.get('body'))
emailer.send()
message = "Email Sent"
return json.dumps({'message':message})

Categories