We Keep Coding

How to Obtain Image Bytes as String in JavaScript?

All I have is <input type="file" name="myFileUpload" />.
After the user chooses a file (most likely an image), how do I obtain the actual contents of the file as a string? (If possible, please tell me anything about base 64 and url encoding/decoding.)
I was asked to obtain such a string and set it as a value of a JSON object, then such a JSON object would be posted to the server "as is", that is, application/json; charset=utf-8.
I'm not sure if the above is a common practice since I'm accustomed to just posting such data as multipart/form-data which I was told not to use.
The receiver of this gigantic JSON object is an ASP.net Web API Controller. I suppose there would be a problem with deserialization if such an object is potentially multi-megabytes large.
So again, how to obtain the image bytes as a string and what problems may I encounter if I try to post such a large JSON object especially when it's received the server-side.

Is there a logical limit to variables in JavaScript?

I have a script that fetches a JSON object with images URLs from a server via AJAX. The number of image fetched is random and can range from 60 to 300. The script is getting the JSON from the server at a fixed interval.
To prevent network issues, the server returns a base64 of one image at random so the front-end script can store it in a "cache" object. Eventually, the "cache" will fill up with enough material to take over if an AJAX request has failed.
The "cache" also checks how many base64 images it contains before adding a new one. If it reaches its limit, it overwrites the oldest base64, effectively pruning the data.
I've installed a monitor application on the machine I was testing with and Chrome hangs after 21 hours or so. That means approximatively 260 AJAX requests, potentially up to 78k images and a "cache" JS object storing 260 images in base64. Is something wrong with my process?
To recap, here's the whole thing:
Front-end send AJAX request to server
Server replies with a JSON object filled with images URLs and one JPG in base64
Front-end creates <img> tags with the URLs and stores the base64
If the server fails to reply, the front-end use its stored base64 images
Repeat process to infinity and beyond
Chrome crashes after 21 hours
Any help appreciated!

how to parse both data & binary websocket messages?

I've found this great page that shows how to stream video.
His code assumes that the onmessage data is jpg like so
ws.onmessage = function (e) {
$("#image").attr('src', 'data:image/jpg;base64,'+e.data);
}
but I'd like to have audio and other page data as well.
How can WebSockets be made to parse message types and binary types?

The data in Websocket messages can be either string (text messages) or binary data. In your case it's string data, where the string content is the base64 encoding of a binary image.
In a better optimized program the image could be also transferred binary.
Depending on the message type, e.data will be of another type.
If the message is a text message then e.data will be of type string.
If the message is a binary message then e.data will contain either an ArrayBuffer or a Blob object. You can by yourself decide which representation of binary data you want to receive by setting the WebSocket.binaryType property (e.g. to "arraybuffer").
You can use instanceof to check if you received a binary or text message.
If you want to transfer different types of binary data you can use a header at the start of the message to tell the client what the message contains. E.g. declare the first byte of the binary data as header. If the first byte contains 1 then the remaining data is a picture, if the first byte contains 2 then the remaining data is a short audio sample, etc.
However it's a little more difficult on both sides since you need to split or combine the binary data array.
Alternatively you could use multiple parallel websocket connections between client and server. On for video data, one for audio data, one for general messages, etc.

Upload binary data to AppEngine Blobstore via HTTP request

I'm trying to figure out the lowest data-overhead way to upload/download binary data to Google AppEngine's Blobstore from a JavaScript initiated HTTP request. Ideally, I would like to submit the binary data directly, i.e. as unencoded 8-bit values; maybe in a POST request that looks something like this:
...
Content-Type: multipart/form-data; boundary=boundary;
--boundary
Content-Disposition: form-data; name="a"; filename="b"
Content-Type: application/octet-stream
##^%(^Qtr...
--boundary--
Here, ##^%(^Qtr... ideally represents arbitrary 8-bit binary data.
Specifically, I am trying to understand the following:
Is it possible to directly upload 8-bit binary data, or would I need to encode the data somehow, like a base-64 MIME encoding?
If I use a different encoding, would Blobstore save the data as 8-bit binary internally or in the encoded format? I.e. would a base-64 encoding increase my storage cost by 33%?
Along the same lines: Does encoding overhead increase outgoing bandwidth cost?
Is there a better way to format the POST request so I don't need to come up with a boundary that doesn't appear in my binary data? E.g. is there a way to specify a Content-Length rather than a boundary?
In the GET request to retrieve the data, can I simply expect to have binary data end up in the return string, or is the server going to automatically encode the data somehow?
If I need to use some encoding, which one would be the best choice among the supported options for essentially random 8-bit data? (base-64, UTF-8, someting else?)

Even though I received the Tumbleweed Badge for this question, let me report on my progress anyways in case somebody out there does care:
This question turned out to pose 3 independent problems:
Uploading data to BlobStore efficiently
Making sure BlobStore saves it in the smallest possible format
Finding a way to reliably download the data
Let's start with (3), because this ends up posing the biggest issue:
So far I have not been able to find a way to download true 8-bit data to the browser via XHR. Using mime-types like application/octet-stream leads to only 7 bits reaching the client reliably, unless the data is downloaded to a file. The best solution I found, is using the following mime-type for the data:
text/plain; charset=ISO-8859-1
This seems to be supported in all browsers that I've tested: IE 8, Chrome 21, FF 12.0, Opera 11.61, Safari 5.1.2 under Windows, and Android 2.3.3.
With this, it is possible to transfer almost any 8-bit value, with the following restrictions/caveats:
Character 0x00 is interpreted as the end of the input string in IE8 and must therefore be avoided.
Most browsers interpret charset ISO-8859-1 as Windows-1252 instead, leading to characters 0x80 through 0x9F being changed accordingly. This can be fixed, though, as the changes are unambiguous. (see http://en.wikipedia.org/wiki/Windows-1252#Codepage_layout)
Characters 0x81, 0x8D, 0x8F, 0x90, 0x9D are reserved in the Windows-1252 charset and Opera returns an error code for these, therefore these need to be avoided as well.
Overall, this leaves us with 250 out of the 256 characters which we can use. With the required basis-change for the data, this means an outgoing-data-overhead of under 0.5%, which I guess I'm ok with.
So, now to problem (1) and (2):
As incoming bandwidth is free, I've decided to reduce the priority of solving problem (1) in favor of problems (2) and (3). Turns out, using the following POST request does the trick then:
...
Content-Type: multipart/form-data; boundary=-
---
Content-Disposition: form-data; name="a"; filename="b"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: base64
abcd==
-----
Here, abcd== is the base64-MIME-encoded data consisting of the above described 250 allowed characters (see http://en.wikipedia.org/wiki/Base64#Examples, GAE uses + and / as the last 2 characters). The encoding is necessary (correct me if I'm wrong) as calling the XHR send() function with String data will result in UTF-8 encoding of the string, which screws up the data received by the server. Unfortunately passing ArrayBuffers and Blobs to the send() function isn't available in all browsers yet to circumvent this issue more elegantly.
Now the good news: The AppEngine BlobStore decodes this data automatically and correctly and stores it without overhead! Therefore, using the base64-encoding only leads to slower data-uploads from the client, but does not result in additional hosting cost (unless maybe a couple CPU cycles for the decoding).
Aside: The AppEngine development-server will report the encoded size (i.e. 33% larger) for the stored blob, both in the admin console and in a retrieved BlobInfo record. The production servers do not have this issue, though, and report the correct blob size.
Conclusion:
Using Content-Transfer-Encoding base64 for uploading binary data of Content-Type text/plain; charset=ISO-8859-1, which may not contain characters 0x00, 0x81, 0x8D, 0x8F, 0x90, and 0x9D, leads to reliable data transfer for many tested browsers with a storage/outgoing-bandwidth overhead of less than half a percent. The upload-overhead of the base64-encoded data is 33%, which is better than the expected 50% for UTF-8 (for random 8-bit data), but still far from desirable.
What I don't know is: Is this the optimal solution, or could one do better? Anyone up for the challenge?

How would I send and receive packets over a WebSocket in Javascript

I want to send data from Javascript to a WebSocket server and also from a WebSocket server to Javascript.
I want to send this:
Headers
-------
Field 1: 2 byte hex
Field 2: 2 byte hex
Field 3: 4 byte hex
Data
----
Field1 : 2 byte hex
Field1 : 8 byte hex
From Javascript, I can send a two-byte value via
socket = new WebSocket(host);
...
socket.send(0xEF);
But I want to send multiple fields, together...let's say 0xEF, 0x60, and 0x0042.
How do I do this?
And, how to I interpret via Javascript data containing multiple fields coming from the WebSocket server?

You can send data as a string. For example:
socket.send("hello world");
I recommend you to use JSON as data format. You can convert JSON strings directly into objects and vice versa. It's so simple and useful!

You can send data as JSON objects.
socket.send(JSON.stringify({field1:'0xEF', field2:'0x60',field3: '0x0042'}));

Sound like what you are asking is how to send binary data over a WebSocket connection.
This is largely answered here:
Send and receive binary data over web sockets in Javascript?
A bit of extra info not covered in that answer:
The current WebSocket protocol and API only permits strings to be sent (or anything that can be coerced/type-cast to a string) and received messages are strings. The next iteration of the protocol (HyBi-07) supports binary data is currently being implemented in browsers.
Javascript strings are UTF-16 which is 2 bytes for every character internally. The current WebSockets payload is limited to UTF-8. In UTF-8 character values below 128 take 1 byte to encode. Values 128 and above take 2 or more bytes to encode. When you send a Javascript string, it gets converted from UTF-16 to UTF-8. To send and receive binary data (until the protocol and API natively support it), you need to encode your data into something compatible with UTF-8. For example, base64. This is covered in more detail in the answer linked above.