We Keep Coding

How do I efficiently hash an image for indexed search in the browser?

I'm writing a chrome extension that saves images from websites. In addition to saving the files themselves, I'd like to turn the images into some type of hash.
The objective is to index the images in a database so that I can easily determine if an image is a duplicate (independent of size, i.e., a thumbnail and a full-size image would be considered duplicates). I'm not really worried about images with slight differences (besides size).
I've tried to work with this library, but it's large, a bit slower than I'd like, and (ostensibly) not supported anymore.
I've also tried a number of phash algorithm implementations, but as near as I can tell, they're all intended for server-side use. I'm using webpack, which was unable to bundle any of the libs I tried (very possible this is user-error, i'm no webpack-pro).
Lastly, I tried converting the image to base64, but the results are 10k+ characters, and it's not clear to me this would work for images of different sizes.

I would just implement a fast string hash in javascript. Convert the image to base64, then run a string hash on it:
https://www.npmjs.com/package/non-crypto-hash
(these work in both node and the browser, you could bring this in with browserify)
or an algorithm you can convert:
http://landman-code.blogspot.ca/2008/06/superfasthash-from-paul-hsieh.html
Assuming you don't need a cryptographically secure hash, these will probably be your speediest options.

Render RGBA to PNG in pure JavaScript?

Let's say I have a canvas element, and I need to turn the image on the canvas into a PNG or JPEG. Of course, I can simply use canvas.toDataURL, but the problem is that I need to do this a twenty times a second, and canvas.toDataURL is extremely slow -- so slow that the capturing process misses frames because the browser is busy converting to PNG.
My idea is to call context.getImageData(...), which evidently is much faster, and send the returned CanvasPixelArray to a Web Worker, which will then process the raw image data into a PNG or JPEG. The problem is that I can't access the native canvas.toDataURL from within the Web Worker, so instead, I'll need to resort to pure JavaScript. I tried searching for libraries intended for Node.js, but those are written in C++. Are there any libraries in pure JavaScript that will render raw image data into a PNG or JPEG?

There have been several JavaScript ports of libpng, including pnglets (very old), data:demo, and PNGlib.
The PNG spec itself is pretty straightforward – the most difficult part you may have is with implementing a simple PNG encoder yourself is getting ZLIB right (for which there are also many independent JavaScript implementations out there).

There's actually a C++ to JavaScript compiler called Emscripten.
Someone made a port of libpng, which you might want to check.

I was able to write my own PNG encoder, which supports both RGB and palette depending on how many colors there are. It's intended to be run as a Web Worker. I've open-sourced it as usain-png.

How to load <img> tag SRC into a javascript variable to be re-used across multiple images?

So here is my plan for fast web page downloading...
Place all images into a single png-24 image sprite.
Encode that image as base64 and include it in the webpage HTML code.
Duplicate the SRC of the original image sprite and re-use it for the logo, share buttons, other images, etc..
The only problem I can foresee is the duplication of the base64 encoded image source.
Can I readily extract the image source with jQuery and just re-insert it into all of my blank images (the images that need the sprite to be created)?
EDIT: Some people are mentioning base64 images are not cached, but wouldn't my entire webpage (containing the base64 images) be cached if I told it to be?

That is a common technique with CSS icons / reusable images.
You can get the image src using $(element).attr('src');.
However, I don't see the advantage of encoding the image binary (I'm assuming you meant the image file itself) to base64 to include with the HTML markup. You may be over-thinking this a bit.
I don't think you can "save" bytes by re-encoding the image data into base 64, primarily because base 64 is a narrower character set than the encoding used in the original data (think binary 111 = decimal 7), so I expect a larger output actually. (But that's just me theorycrafting, so correct me if I'm wrong.)
However, if you do manage, for example, to re-encode to at most an equal size of markup, then you're not making any headway with "faster downloading". You're still downloading the same amount of data. Most probably more.
If you do manage a smaller payload, is the performance hit of encoding / re-encoding worth it? Not to mention the cross-browser compatibility.
A better technique would be to package the images into a single image file (which is the spirit of your exercise), and just let the browser download that as normal. Once one copy of an image is downloaded, as long as its cached by the browser, it won't download it anymore.
EDIT
To answer your edit regarding caching of the web pages, yes, your web pages will be cached. So will your base-64 encoded images. But since your images are effectively part of the HTML markup, they're going to be cached with the HTML pages.
e.g. If I download foo.html (which includes my encoded sprite file), I'm definitely going to get my markup as normal. That page is cached.
Now, I download bar.html (which uses my sprite file too). I expect that your image won't be cache-accessible from bar.html, because as far as the browser is concerned, that image is part of the HTML markup of foo.html. It probably won't even realize that there's an image wedged in there.
The way caching works (as best I can understand it) is URL matching. That's the reason why if I download facepalm.jpg in one page, and request facepalm.jpg again in another, the browser recognizes that I've already downloaded it, so it doesn't.
With your encoding plan, I'm not going to be requesting foo.html (or part of it) from bar.html, so I expect that your image caching won't work as you expect it to in your question.
If I visit foo.html again though, I'd get all benefits of caching for that page, as I've "downloaded that before".

I agree, a better plan is to use a single image with many sprites, then use the
background: url(...)
background-position: 0px 0px;
attributes in CSS. That way you only ever load 1 image, which DOES cache.

you can use var imgSrc = $("#yourImage").attr("src"); and then use $("img").attr("src", imgSrc); to accomplish what you're asking for.
I'm not sure encoding it to base64 is the best option, as base64 images don't get cached, so it would be reloaded every page visit. And it's pretty widely accepted now that base64 encoding adds about 33% to file size.
Read more: http://davidbcalhoun.com/2011/when-to-base64-encode-images-and-when-not-to

I think the base64 thing is a bad idea. Just load the sprite normally.
Don't blindly look to eliminate all possible HTTP requests at any expense. (Even if "it's what Google does.") Achieving good performance is a balancing act, and given the differences between browsers, it's slightly more of an art than it is science.
Reasons you don't want to do this:
Your image sprite will not be cached across pages – in other words, you must transfer the entire image on every page. Assuming a separately served image has proper caching headers, subsequent page requests will be slower and waste bandwidth.
Base 64 encoding is by its nature wasteful of bandwidth. A base64-encoded file will be 33% larger than the original. This doesn't matter so much for files a couple kilobytes large, but this will be a big problem for large files, like your image sprite.
As you've realized, you'd have to duplicate the base64-encoded sprite in every <img> tag on your page, which is extremely wasteful, and fixing this via script brings us to...
Your images become dependent on JavaScript, which is a bad idea. What if a script doesn't load? JavaScript is turned off?
Eliminating this image HTTP request doesn't actually accomplish much in terms of how long before your page is displayed, because images don't block DOM ready. What's important is reducing HTTP requests for content that does block rendering: scripts and CSS. (In most cases, one request each.) Your page does not render at all until script and CSS is fully loaded. However, the page does render (albeit with empty space) before images download.
You need to balance the type of images with number HTTP request. Icons and such generally should be sprited in a PNG so that as many images as possible download in one HTTP request. Anything photographic in nature needs to go into its own JPEG. Do not use PNG for pictures. It is the wrong compression algorithm, and will result in files 1000% larger than a comparable JPEG.

Javascript lossless image compression

I am looking for a way to encoding a bmp image as a tiff or some other standard, compressed but lossless format in a browser (javascript). I guess if worst comes to worst I can write a library myself (getting the data from a canvas element) but I was hoping either: there's a better way to do it or someone else has already written something similar (I can't find a library).
I have users wanting to upload ~ 4mb (8-bit monochrome) bmp files, which I can losslessly compress to ~700kb tiff files with LZW (or even better ~300kb lossless JPEG-2000). I'd like to do this before the files are uploaded to save transfer costs/time.
Before you ask, I'm not being anal about the lossless encoding instead of just using high bitrate JPEG. These are astronomy photos that are used for analysis so they can't handle any compression artifacts being introduced.
Thanks,
Jonny

Use PNG. It's lossless and uses zlib compression. There are even programs like pngcrush that will optimize the image for size (only problem is it takes a while for this).
Is there any reason you're using JavaScript of all things for this process? Wouldn't it be easier in the long run if you did it in a Java applet (using a library that interfaces with the java.awt.Image class), or uploaded it to a server that did the image processing and returned the compressed image?
Edit: Please don't use a Java applet; the technology isn't well-supported anymore.

If you are willing to experiment something new, you could try a new lossless compression algorithm I created; through a Java applet it is possible to visualize the compressed images in a browser. You could use Javascript to trigger the loading of the applet, or manipulate directly the Java code (the program is open source). For many images, the compression ratio will be better than lossless Jpeg 2000. The address of the website is http://www.researchandtechnology.net/bcif/ .
If instead you want to use some well-known standard, then I'd agree with amphetamachine: using PNG would be the best choice.
So long,
Stefano

Compressing plaintext in JavaScript?

I have a simple Notepad-like web application I'm making for fun. When you save a document, the contents of a <textarea> are sent to the server via Ajax and persisted in a database.
Let's just say for shits and giggles that we need to compress the contents of the <textarea> before sending it because we're on a 2800 baud modem.
Are there JavaScript libraries to do this? How well does plain text compress in the first place?

Simple 7 bit compression might work if you're only using the 7 bit ascii character set. A google search yielded this: http://www.iamcal.com/png-store/
Or you could use LZW
http://rosettacode.org/wiki/LZW_compression#JavaScript
As far as compression ratio; according to Dr. Dobbs:
It is somewhat difficult to characterize the results of any data compression technique. The level of compression achieved varies quite a bit, depending on several factors. LZW compression excels when confronted with data streams that have any type of repeated strings. Because of this, it does extremely well when compressing English text. Compression levels of 50 percent or better can be expected.

Well, you couldn't use gzip comppression. See here: Why can't browser send gzip request?
I suppose you could strip whitespace, but that would prove unsustainable. I'm not sure if this is an itch that needs scratching.
I did find this with a google search: http://rumkin.com/tools/compression/compress_huff.php That will eventually yield a smaller set of text, if the text is large enough. It actually inflates the text if the text is short.
I also found this: http://www.sean.co.uk/a/webdesign/javascript_string_compression.shtm

First, run the LZW compression, this yields compressed data in binary format.
Next then do base-64 encoding on the the compressed binary data. This will yield a text version of the compressed data that you can store in your database.
To restore the contents, do the base-64 decode. Then the LZW decompression.
There are Java libraries to do both. Just search on "LZW compression Java" and on "base-64 encode Java".

It varies heavily on the algorithm and the text.
I'm making my own compression algorithm here, as of writing its not done but it already works extremely well for English plaintext compression. ~50% compression for both small and large messages. It wouldn't be useful to share a code snippet because I'm using experimental dictionary compression, but heres my project: https://github.com/j-stodd/SMOL
I also tried the LZW compression shared by Suirtimed but it doesn't seem to perform that well, it will decrease length but bytes stay mostly the same. Compressing "aaaaaaaa" with LZW will save you only one byte. My algorithm would save you 5 bytes.