We Keep Coding

How to save an array buffer to a local file on disk using javascript

I'm practicing in programming with websocket, i'm building a java websocket server and want to send a binary data to websocket client, then client side can get that data through websocket and save it to a local file on disk, but i don't know what way using javascript to write an arraybuffer into a file.
My example code is shown below
var ws = new WebSocket(URL);
ws.binaryType = "arraybuffer";
ws.onmessage = function(evt) {
if (evt.data instanceOf ArrayBuffer) {
// binary message is riecieved from server and i want to save it as a local file
}
};
Thank you for any help

You can create a textual/string representation that can be saved to and read from a file:
// returns String
function getArrayBufferString(arrayBuffer) {
return new Uint8Array(arrayBuffer).toString()
}
// returns ArrayBuffer
function parseArrayBufferString(string) {
return new Uint8Array(string.split(',')).buffer
}

Save json object as a text file

I am using an API for a Twitch.tv streaming bot called DeepBot.
Here is the link to it on github https://github.com/DeepBot-API/client-websocket
My goal is to create a text document listing all the information pulled from the bot using the command api|get_users|. The bot's response is always a json object. How can I take the json object from the bot and save it as a text file?
Edit: My code
var WebSocket = require('ws');
var ws = new WebSocket('ws://Ip and Port/');
ws.on('open', function () {
console.log('sending API registration');
ws.send('api|register|SECRET');
});
ws.on('close', function close() {
console.log('disconnected');
});
ws.on('message', function (message) {
console.log('Received: ' + message);
});
ws.on('open', function () {
ws.send('api|get_users|');
});

Well that depends on how your setup is? You posted this under javascript. So I guess you are either:
using a browser, to make the websocket connection, in with case there is no direct way to save a file on the client. But in HTML5 you can store key,value pairs with local storage.
using node js (server side javascript) in witch case the code is as below:
some other setup, that I can't guess. in witch case you might tell a little more about it?
In browser with HTML5 capabilities:
// where msg is an object returned from the API
localStorage.setItem('Some key', JSON.stringify(msg));
In Node JS
var fs = require("fs"); // Has to be installed first with “npm install fs”
// where msg is an object returned from the API
fs.writeFile("some-file.json", JSON.stringify(msg), function (err) {
if (err) throw err;
});
Edit: OK, Thanks for clearing it up.
I believe Blag's solution is the way to go.
Good luck with your project!

If it's for a client side JS save :
Create a file in memory for user to download, not through server
and
Convert JS object to JSON string
Is what you need. ( I don't test it, but it'll look like this : )
var j = {"name":"binchen"};
var s = JSON.stringify(j);
window.location = 'data:text/plain;charset=utf-8,'+encodeURIComponent(s);

express request Error: socket hangup while parsing large xml

Im trying to parse a 300MB xml file into json in a worker nodejs app, so the client makes the request to the main web app and the server performs request to the worker server with the file location, after the worker server finished parsing the xml it saves it to a json file and return its location back to the main server.
Everything works fine with xml files under 130MB, however when it encounters a large file the worker server finished parsing the file and saves it, as soon as the response comes to the main server it receives :
{ [Error: socket hang up] code: 'ECONNRESET' }
I have tried to use timeout in the request options, but it still happening.
request
.post({
url:'http://localhost:6666/parsexml',
formData: {filePath:filePath},
json: true
},function(err,httpResponse,jsonResObj){
// throws error here
})
the xml parser worker return 200 message back to the server, and then it crashes.
Anny suggestion on how can i implement this ?

Maybe you have to use different XML parser, for example xml-stream, which can handle large (500+ MB) files.
How can you install it
Install it using the following command:
npm install -g xml-stream
How you can use it in your app
Require it in your project and pass the ReadFile object to initialize it:
var fs = require('fs');
var XmlStream = require('xml-stream');
/*
* Pass the ReadStream object to xml-stream
*/
var stream=fs.createReadStream('file_name.xml');
var xml = new XmlStream(stream);
/*
*Code continues here
*/
If you want to extract only values of, let's say, id from your XML and print them this is a code to do so:
var fs = require('fs')
var XmlStream = require('xml-stream') ;
var stream=fs.createReadStream('some-large-XML.xml');
var xml = new XmlStream(stream);
xml.preserve('id', true);
xml.collect('subitem');
xml.on('endElement: id', function(item) {
console.log(item);
});
More detailed information can be found here.

Want to send images using node.js and socket.io in android

I am Creating a chat app between two users now I can do Simple text chat with different users using node.js and socket.io. Now problem arises here as I have to send image in chat application and after searching for whole long day I am not able to get perfect node.js in which I can send image in chat app. So I want to know is it possible to send image using node.js. Here is my simple node.js file for sending simple text message from one user to another.
socket.on('privateMessage', function(data) {
socket.get('name', function (err, name) {
if(!err) {
// get the user from list by its name to get its socket,
// then emit event privateMessage
// again here we want to make you clear
// that every single client connection has its own
// unique SOcket Object, we need to get this Socket object
// to communicate with every other client. The socket variable
// in this scope is the client who wants to send the private
// message but the socket of the receiver is not know.
// Get it from the saved list when connectMe handlers gets called
// by each user.
onLine[data.to].emit('newPrivateMessage',{from:name, msg:data.msg, type:'Private Msg'})
}
});
});

You can use the Base64 version of your image and send it like this:
onLine[data.to].emit('newPrivateMessage',{from:name, img:data.img.toString('base64'), type:'Private Msg'})
.. and then on the client side receive it and create an image
socket.on("newPrivateMessage", function(data) {
if (data.img) {
var img = new Image();
img.src = 'data:image/jpeg;base64,' + data.img;
// Do whatever you want with your image.
}
});
UPDATE
The following is a snippet taken from the link I've commented below. As you can see it takes the image from the input, reads it and sends to the server. After that you can send the same data from the server to another client.
For the full example, please read the article.
JavaScript (client)
...
$('#imageFile').on('change', function(e) {
var file = e.originalEvent.target.files[0],
reader = new FileReader();
reader.onload = function(evt) {
var jsonObject = {
'imageData': evt.target.result
}
// send a custom socket message to server
socket.emit('user image', jsonObject);
};
reader.readAsDataURL(file);
});
...
HTML
...
Image file: <input type="file" id="imageFile" /><br/>
...
UPDATE 2
Here is one example I have found:
Java (client)
File file = new File("path/to/the/image");
try {
FileInputStream imageInFile = new FileInputStream(file);
byte imageData[] = new byte[(int) file.length()];
imageInFile.read(imageData);
// Converting Image byte array into Base64 String
String imageDataString = Base64.encodeBase64URLSafeString(imageData);
} catch (...) {
...
}
The above snippet shows how to read the file and encode the data into a base64 string. So then you can send it just like a string (I assume).
Here is the complete example: How to convert Image to String and String to Image in Java?
Also I have found encodeToString function of Base64.Encoder (java.util package), which you can use.

The easiest way I can think of is to simply Base64 encode the image and send it through the text pipe. You would need to distinguish text and image messages with header information (Maybe send a JSON object?).

Large file upload with WebSocket

I'm trying to upload large files (at least 500MB, preferably up to a few GB) using the WebSocket API. The problem is that I can't figure out how to write "send this slice of the file, release the resources used then repeat". I was hoping I could avoid using something like Flash/Silverlight for this.
Currently, I'm working with something along the lines of:
function FileSlicer(file) {
// randomly picked 1MB slices,
// I don't think this size is important for this experiment
this.sliceSize = 1024*1024;
this.slices = Math.ceil(file.size / this.sliceSize);
this.currentSlice = 0;
this.getNextSlice = function() {
var start = this.currentSlice * this.sliceSize;
var end = Math.min((this.currentSlice+1) * this.sliceSize, file.size);
++this.currentSlice;
return file.slice(start, end);
}
}
Then, I would upload using:
function Uploader(url, file) {
var fs = new FileSlicer(file);
var socket = new WebSocket(url);
socket.onopen = function() {
for(var i = 0; i < fs.slices; ++i) {
socket.send(fs.getNextSlice()); // see below
}
}
}
Basically this returns immediately, bufferedAmount is unchanged (0) and it keeps iterating and adding all the slices to the queue before attempting to send it; there's no socket.afterSend to allow me to queue it properly, which is where I'm stuck.

Use web workers for large files processing instead doing it in main thread and upload chunks of file data using file.slice().
This article helps you to handle large files in workers. change XHR send to Websocket in main thread.
//Messages from worker
function onmessage(blobOrFile) {
ws.send(blobOrFile);
}
//construct file on server side based on blob or chunk information.

I believe the send() method is asynchronous which is why it will return immediately. To make it queue, you'd need the server to send a message back to the client after each slice is uploaded; the client can then decide whether it needs to send the next slice or a "upload complete" message back to the server.
This sort of thing would probably be easier using XMLHttpRequest(2); it has callback support built-in and is also more widely supported than the WebSocket API.

In order to serialize this operation you need the server to send you a signal every time a slice is received & written (or an error occurs), this way you could send the next slice in response to the onmessage event, pretty much like this:
function Uploader(url, file) {
var fs = new FileSlicer(file);
var socket = new WebSocket(url);
socket.onopen = function() {
socket.send(fs.getNextSlice());
}
socket.onmessage = function(ms){
if(ms.data=="ok"){
fs.slices--;
if(fs.slices>0) socket.send(fs.getNextSlice());
}else{
// handle the error code here.
}
}
}

You could use https://github.com/binaryjs/binaryjs or https://github.com/liamks/Delivery.js if you can run node.js on the server.

EDIT : The web world, browsers, firewalls, proxies, changed a lot since this answer was made. Right now, sending files using websockets
can be done efficiently, especially on local area networks.
Websockets are very efficient for bidirectional communication, especially when you're interested in pushing information (preferably small) from the server. They act as bidirectional sockets (hence their name).
Websockets don't look like the right technology to use in this situation. Especially given that using them adds incompatibilities with some proxies, browsers (IE) or even firewalls.
On the other end, uploading a file is simply sending a POST request to a server with the file in the body. Browsers are very good at that and the overhead for a big file is really near nothing. Don't use websockets for that task.

I think this socket.io project has a lot of potential:
https://github.com/sffc/socketio-file-upload
It supports chunked upload, progress tracking and seems fairly easy to use.