We Keep Coding

How to load a protocol buffer binary file in javascript?

I have generated a binary file containing some data using protobuf in Python, and I need to load the binary file now in Javascript and read its content. I tried to use FileReader() but I wasn't able to succeed. How can I do this?

You can make use of protobufjs.
Here's an example in JavaScript:
const fs = require('fs');
const protobuf = require('protobufjs');
// Load the binary data
const binaryData = fs.readFileSync('your-protocol-buffer-file.bin');
// Load the root of the protocol buffer definition
const root = protobuf.Root.fromJSON(yourProtoJSON);
// Look up the desired message type
const MyMessageType = root.lookupType('MyMessageType');
// Decode the binary data using the message type
const message = MyMessageType.decode(binaryData);
// The message object is a JavaScript object that represents the data in the binary file
console.log(message);
You will need to have the Protocol Buffer definition (.proto file) that was used to generate the binary file in order to use protobufjs to decode the data.
For more information on protobufjs, you can visit https://github.com/protobufjs/protobuf.js.

Convert Raw PNG data to base64 in node js

I have an api that gives me raw png data as response, Want to convert the raw data to a file or a base64 string that can be invoked later. Below is the sample of the response that I get from the api. Can someone tell me on how this can be achieved
�PNG
IHDR���(-sRGB���sBIT|d� IDATx���w|SU��'I��i��[(��7(�NDQ� �*����EPq�" ���""�Be�e�M�޻i����(���mҦM�����y�7�{����{���'��"A(A �����������d��h�^k����"���m"""""""���D.�X�P$tt{�������.kB�BNk'""""""r,�#�9t"""""""'��NDDDDDD�Љ�������:�`#'""""""r�DDD���f܊뮽���ha��s1~�dG7����)�8�DDD���? ���ᗭ?5��g�8�X� ��'�ii)�hb�L�vn��.����4=/��zTV�c�>|����h�������g�8�1���Pb^{e9���p���N�GDDt1�:�YS&Omqn�k�������s�c��/���k>����V��՟���S'q㌙�w�C�޾�����,���!��v������ GЉ����T!6�'��c��� pqc��I�����������=��F��Z�F���X�~5�Z-�B#���k���޽�#QQ!�{�-\;m&�����|�i~��W�\.ǃ�b��Q����X��*�T�����s;�����

You can use fs Module
let fs = require('fs')
let writer = fs.createWriteStream('xyz.png')
writer.write(<RAW data here>);
If you getting file directly in body then
let writer = fs.createWriteStream('xyz.png');
let fl = res.body.pipe(writer); (res is your response from api)
fl.on('finish', () => {...Things to do})

Javascript: how to parse base64 encoded csv to fetch the filename and actual csv content

I have a base64 string created by encoding a csv file,
const base64 = 'LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTExNDc2MDgwNjM5MTM4ODk4MTc2NTYwNA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJmaWxlIjsgZmlsZW5hbWU9ImNoYXJ0T2ZBY2NvdW50LmNzdiINCkNvbnRlbnQtVHlwZTogdGV4dC9jc3YNCg0K77u/QWNjb3VudE51bWJlcixBY2NvdW50TmFtZSxEZWR1Y3RhYmlsaXR5DQoxMTExLHRlZWUsMTAwDQoyMjIyMix0ZXN0LDEwMA0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTExNDc2MDgwNjM5MTM4ODk4MTc2NTYwNC0tDQo='
I want to get the name of the file and create the same file back with the filename. What I need to use. I am using node.js here.

If I understand your question right, your base64 variable needs to be decoded. buffer can do this:
const base64 = 'LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTExNDc2MDgwNjM5MTM4ODk4MTc2NTYwNA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJmaWxlIjsgZmlsZW5hbWU9ImNoYXJ0T2ZBY2NvdW50LmNzdiINCkNvbnRlbnQtVHlwZTogdGV4dC9jc3YNCg0K77u/QWNjb3VudE51bWJlcixBY2NvdW50TmFtZSxEZWR1Y3RhYmlsaXR5DQoxMTExLHRlZWUsMTAwDQoyMjIyMix0ZXN0LDEwMA0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTExNDc2MDgwNjM5MTM4ODk4MTc2NTYwNC0tDQo=';
// decode base64
const decodedBase64String = Buffer.from(base64, 'base64').toString();
const filename = decodedBase64String.match(/filename="(.*).csv/)[1];
console.log(decodedBase64String);
With a some regex you can extract your filename ("chartOfAccount.csv") from decodedBase64String. And then use fs (explained here) to create from your content a file.

Piping to same Writable stream twice via different Readable stream

I am trying to concatenate a string and a Readable stream (the readable stream is pointing to a file which may have data in multiple chunks, i.e. the file may be large) into one writable stream so that the writable stream can finally be written to a destination.
I am encrypting the string and content of the file and then applying zlib compression on them, then finally I want to pipe them to the writable stream.
To achieve this, I can:
a) Convert the file content into a string then concatenate both the string then encrypt, do compression and then finally pipe it into the writable stream. But this is not possible because the file may be big in size, thus I can't convert its content to string.
b) I can first encrypt and compress the string then convert the string into a stream then pipe it into the writable stream after that is done completely, pipe the file contents into the same writable stream.
To do so, I have written this:
var crypto = require('crypto'),
algorithm = 'aes-256-ctr',
password = 'd6FAjkdlfAjk';
var stream = require('stream');
var fs = require('fs');
var zlib = require('zlib');
// input file
var r = fs.createReadStream('file.txt');
// zip content
var zip = zlib.createGzip();
// encrypt content
var encrypt = crypto.createCipheriv(algorithm, password, iv);
var w = fs.createWriteStream('file.out');
// the string contents are being converted into a stream so that they can be piped
var Readable = stream.Readable;
var s = new Readable();
s._read = function noop() {};
s.push(hexiv+':');
s.push(null);
s.pipe(zlib.createGzip()).pipe(w);
// start pipe when 'end' event is encountered
s.on('end', function(){
r.pipe(zip).pipe(encrypt).pipe(w);
});
What I observe is:
Only the first pipe is done successfully, i.e. the string is written to the file.out. The second pipe doesn't make any difference on the output destination. At first, I thought that the reason might be due to asynchronous behavior of pipe. So, for this reason, I am piping the file content after the first piping is closed. But still, I didn't get the desired output.
I want to know why is this happening any the appropriate way for doing this.

Found the reason why was the second piping to the writable stream w wasn't working.
When the first .pipe(w) finishes, the writable at w is also automatically closed.
So instead of using s.pipe(zlib.createGzip()).pipe(w); I should have used:
s.pipe(zlib.createGzip()).pipe(w, {end: false});
Where the {end: false} is passed to pipe() as the second argument, stating that the writable should not get closed when the piping is done.

Saving byteArray to pdf file in Titanium

I have a SOAP API that is returning me a file divided in chunks encoded in several base64 strings
i'm not being able to save it to the file system without breaking it
This is the pastebin of a whole file encoded, as is, once i download and chain the responses.
What is the way to save it correctly?
i tried in many ways
var f = Ti.FileSystem.getFile(Ti.FileSystem.tempDirectory, 'test.pdf');
...
var blobStream = Ti.Stream.createStream({ source: fileString, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: fileString.length });
f.write(fileString);
or
var data = Ti.Utils.base64decode(fileString);
var blobStream = Ti.Stream.createStream({ source: data, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: data.length });
var bytes = blobStream.read(newBuffer);
f.write(fileString);
or
var data = Ti.Utils.base64decode(fileString);
var blobStream = Ti.Stream.createStream({ source: data, mode: Ti.Stream.MODE_READ });
var newBuffer = Ti.createBuffer({ length: data.length });
var bytes = blobStream.read(newBuffer);
f.write(bytes);
but i'm not understanding which one is the right path
Do I have to convert back to byteArray the string on my own?
What is the right way to save it?
Do I have to create a buffer from the string or ...?

I think that the base64enc for the file is not valid or incomplete, I've tested it using bash and base64 utils. You can perform these steps.
Copy and paste the base64 string on a file called pdf.base64 then run this command:
cat pdf.base64 | base64 --decode >> out.pdf
the output file is not a valid pdf.
You can try to encode and decode a valid pdf file to take a look at the generated binary:
cat validfile.pdf | base64 | base64 --decode >> anothervalidfile.pdf
Try to check if you are chainig chunks correctly or simply make a phone call with the guy who build the soap api.

Before you start downloading your file you need to create the file stream to write too, writing to a blob is not the way to go:
// Step 1
var outFileStream = Ti.Filesystem.getFile('outfile.bin').open(Ti.Filesystem.MODE_WRITE);
After creating your HTTPClient or socket stream and when you receive a chunk of Base64 data from the serve, you need to put that decoded data into a Titanium.Buffer. This would probably go into your onload or onstream in an HTPPClient, :
// Step 2
var rawDecodedFileChunk = Ti.Utils.base64decode(fileString);
var outBuffer = Ti.createBuffer({
byteOrder : // May need to set this
type : // May also need to set this to match data
value: rawDecodedFileChunk
});
Finally you can write the data out to the file stream:
// Step 3
var bytesWritten = outFileStream.write(outBuffer); // writes entire buffer to stream
Ti.API.info("Bytes written:" + bytesWritten); // should match data length
if(outBuffer.length !== bytesWritten) {
Ti.API.error("Not all bytes written!");
}
Generally errors come from having the wrong byte order or type of data, or writing in the wrong order. Obviously, this all depends on the server sending the data in the correct order and it being valid!
You may also want to consider the pump command version of this, which allows you to transfer from input stream to output file stream, minimizing your load.