We Keep Coding

Decompressing bzip2 data in Javascript

I ultimately have to consume some data from a Javascript file that looks as follows:
Note: The base64 is illustrative only.
function GetTripsDataCompressed() { return 'QlpoOTFBWSZTWdXoWuEDCAgfgBAHf/.....=='; }
GetTripsDataCompressed() returns a base64 string that is derived as an array of objects converted to JSON using JSON.NET and the resulting string then compressed to bzip2 using SharpCompress with the resulting memory stream Base64 encoded.
This is what I have and cannot change it.
I am struggling to find a bzip2 JavaScript implementation that will take the result of:
var rawBzip2Data = atob(GetTripsDataCompressed());
and convert rawBzip2Data back into the string that is the JSON array. I cannot use something like compressjs as I need to support IE 10 and as it uses typed arrays that means IE10 support is out.
So it appears that my best option is https://github.com/antimatter15/bzip2.js however because I have not created an archive and only bzip2 a string it raises an error of Uncaught No magic number found after doing:
var c = GetTripsDataCompressed();
c = atob(c);
var arr = new Uint8Array(c);
var bitstream = bzip2.array(arr);
bzip2.simple(bitstream);
So can anyone help me here to decompress a BZip2, Base64 encoded string from JavaScript using script that is IE 10 compliant? Ultimately I don't care whether it uses https://github.com/antimatter15/bzip2.js or some other native JavaScript implementation.

It seems to me the answer is in the readme:
decompress(bitstream, size[, len]) does the main decompression of a single block. It'll return -1 if it detects that it's the final block, otherwise it returns a string with the decompressed data. If you want to cap the output to a certain number of bytes, set the len argument.
Also, keep in mind the repository doesn't have a license attached. You'll need to reach out to the author if you want to use the code. That might be tricky given that the repository is eight years old.
On the other hand, the Bzip2 algorithm itself is open-source (BSD-like license), so you can just reimplement it yourself in Javascript. It's just a few hundred lines of relatively straight-forward code.

RSA sign in python and verify in JS

I am using pyCrypto PKCS1_v1_5 signature to sign the message. (Original code)
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import *
from Crypto.PublicKey import RSA
from Crypto import Random
import base64
import codecs
private_key = """-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQDtmBQTFLfaYOsio48B0Sbw4qMnaXJ7Qk4i1gzI+lgLIOPV0YyU
fWMKgl24iJsGN6W5Vk+4GfDE7jKGkti9ZsswaJ/1Un+zFpXlVCOjEe9CZz3npIEK
G8jJL1rwnBc2qPY3YD8jWBhLaPUgchbxUbe1Q2eKDmlRF7B8BzeNG4aVdQIDAQAB
AoGBAK+x6Q5fIMHVL6vyxRqz3pb9EWdgm664Tm2GWff44EiCbti717gqrKWl18ZR
Bkpnth5CzDq4vAn0ltpbFvmgXRmnUL8WsxigrL6tf2mcx4QAgcqr4B6er7X738lB
UDSaEiSLmT/hr9Rg6H/P8IoP01tLKnj/gYQd5CVLkh/VD+dVAkEA9InBNnM5ViAM
Aq7lL/+RWc+jfKnT9kWo+X0v7K8fD9/188K45G6zK/9X0J6Qj83dk1uni5QGx1i1
VVBXIlUnqwJBAPi7AEl8nmmohrSsq8YHX6lTTy6EybSgQS4Qu3SBcVl2RoG89BLx
kReqyTRGupZz/fNS9VfGFZU9VOHcFCeVl18CQEOiLgwvRjZp2qiLUtw5pSvf3+nE
1tkQXzHRzAV8Ue0EFnR68MRNUcTjdJhAot8DIzt0aByUrmNIR67274KRZs0CQC1X
kZ7T2+Dw+tV24L1x3Kt2Z2nYhRirWhZ2sGV1r18ao5HFC01kwglpddJUznDc5j90
MQt2LbsN+ipOP1JT/8sCQE4qVh+TeYVd8aXSqlJaTsLZNzDMjmREYfuodmyZp5WR
q1R6SaZoxQltHgTtK85QqhDxmmWPkR6jdNLDlIR0Bx4=
-----END RSA PRIVATE KEY-----"""
public_key = """"-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDtmBQTFLfaYOsio48B0Sbw4qMn
aXJ7Qk4i1gzI+lgLIOPV0YyUfWMKgl24iJsGN6W5Vk+4GfDE7jKGkti9ZsswaJ/1
Un+zFpXlVCOjEe9CZz3npIEKG8jJL1rwnBc2qPY3YD8jWBhLaPUgchbxUbe1Q2eK
DmlRF7B8BzeNG4aVdQIDAQAB
-----END PUBLIC KEY-----"""
message = 'To be signed'
key = RSA.generate(1024)
pub_key = key.publickey().exportKey(public_key)
priv_key = key.exportKey(private_key)
h = SHA256.new(message)
print '-----------------'
signer = PKCS1_v1_5.new(key)
signature = signer.sign(h)
print signature
hexlify = codecs.getencoder('hex')
m = hexlify(signature)[0] #<--- I am sending this hex signature to JS.
#--------------Code below is to test the verify in python, n it works !
h = SHA256.new('To be signed')
verifier = PKCS1_v1_5.new(key.publickey())
if verifier.verify(h, signature):
print "The signature is authentic."
else:
print "The signature is not authentic."
for testing to verify in JS, I am using this demo sample page. Its written on the page that Signature value should be hexa decimal encoded 'RSASSA-PKCS1-v1_5' signature, so thats why I am converting my signature to hex value in python, but still the signature doesn't match in JS.
I've noticed one strange thing in python code, that is, when ever I generate a signature with fixed private key, the signature is not the same. Is this normal ? but in JS it gives me fixed signature every time.
Note: There is a little bug in JS DEMO sample page,go to the source code and change rsa.signString to rsa.sign. And also, change rsa.verifyString to rsa.verify. I've raised this issue in his Github, he said he'll gona fix it shortly.

when ever I generate a signature with fixed private key, the signature is not the same. Is this normal ?
Not normal. With PKCS#1 v1.5 the signature should always be the same with the same private key.
Check the following:
The SHA256 hash is exactly the same on both ends
I'm not a Python expert, but it also seems weird that you are calling RSA.generate() when you already have key
Given that your Python code is giving different signature every time, I would concentrate my investigation on that part. If a single byte is different in the input data the produced hash is different - so check with your Python code if the hash is changing with the same input data.

Someone posted an answer, suggesting me to check python code and told that its NOT normal to get different signature for fixed private key. I don't know for what reason he deleted the answer, but his observation was right. Actually the way I was getting the key pairs in python is totally wrong. I am generating new key pair every time and exporting it to local variable pub_key & priv_key. Instead I should do something like this;
#key = RSA.generate(2048)
#pub_key = RSA.publickey().exportKey(public_key)
priv_key = RSA.importKey(private_key)
pub_key = RSA.importKey(public_key)
In this way, I get same signature for this fixed private key, and it really matches the signature of JS code :D ...

JavaScript RSA encryption to PHP

I need to authenticate myself via PHP script on remote website, and website uses JS-based RSA encryption for passwords. Here's the code from website:
function rsa_encrypt(strPlainText) {
var strModulus = "some_random_string";
var strExponent = "10001";
var rsa = new RSAKey();
rsa.setPublic(strModulus, strExponent);
var res = rsa.encrypt(strPlainText);
if (res) {
return res;
}
return false;
}
Browsed a lot of topics on this website, and found that the recommended way is to use phpseclib (if there's another one, let me know). However, using basic example from http://phpseclib.sourceforge.net/rsa/examples.html#encrypt,enc2 I get just an empty page. I entered some_random_string into $rsa->loadKey('...'); - not sure if I did it right? However, I can't see a place to enter strExponent (which is 10001) in this example.
So I tried another solution - Encrypt and Decrypt text with RSA in PHP and modified my code to look the following:
include('Crypt/RSA.php');
$privatekey = "some_random_string";
$rsa = new Crypt_RSA();
$rsa->loadKey($privatekey);
$plaintext = new Math_BigInteger('10001');
echo $rsa->_exponentiate($plaintext)->toBytes();
However, I get this error:
Fatal error: Call to a member function abs() on null in Math\BigInteger.php on line 1675
The solution was posted some time ago, so I guess something got changed in phpseclib library during this time, and I'm just not sure how to re-modify my code.

Popular formats for RSA keys typically contain both the exponent and the modulus within them. See, for example, my answer to I understand the mathematics of RSA encryption: How are the files in ~/.ssh related to the theory? for a more detailed discussion of one particular type of key format.
If you have the exponent and modulo as distinct values try doing this:
$rsa->loadKey([
'e' => new Math_BigInteger('10001', 16),
'n' => new Math_BigInteger('some_random_string', 16);
]);
Note the , 16 bit. 65537 (10001 in hex) is a common RSA exponent. Math_BigInteger assumes, by default, that the number being passed to it is in base-10, unless you specifically tell it otherwise. One requirement of RSA is that e be coprime to either phi(n) or lcm(n). 65537 is trivially coprime because it is prime. 10001 is not prime. It can be factored into 73*137.

Base64 HMAC SHA1 String in VBA

I'm trying to convert an ASP/VBScript OAuth library to VBA. One of the challenges is this line of code:
Get_Signature = b64_hmac_sha1(strSecret, strBaseSignature)
This function, b64_hmac_sha1 is actually a function contained in a JavaScript library. It appears to me that calling a JavaScript function from VBA is fairly impractical.
Because I know so little about encryption, it's not even clear to me what this b64_hmac_sha1 function does. Is HMAC SHA1 different from SHA1?
I half suspect I might be able to find some VBA code online to do what I need to do if I just understood what this function is actually doing. If I do not find an existing function, I could possibly write one that would use the .NET Cryptography library (you can actually call the .NET cryptography libraries from VBA if you know how).
I'm not looking for someone to convert this JavaScript to VBA. I'm only trying to understand what it is that this b64_hmac_sha1 function is outputting so I can try to find ways to achieve the same output in VBA if possible.
A copy of this JavaScript library is visible on this website. You'll have to scroll down past the VBScript to the JavaScript section.
http://solstice.washington.edu/solstice/ASP_Signing_REST_Example
Edit1:
OK, so here's the functions I ended up writing and using:
Public Function Base64_HMACSHA1(ByVal sTextToHash As String, ByVal sSharedSecretKey As String)
Dim asc As Object, enc As Object
Dim TextToHash() As Byte
Dim SharedSecretKey() As Byte
Set asc = CreateObject("System.Text.UTF8Encoding")
Set enc = CreateObject("System.Security.Cryptography.HMACSHA1")
TextToHash = asc.Getbytes_4(sTextToHash)
SharedSecretKey = asc.Getbytes_4(sSharedSecretKey)
enc.Key = SharedSecretKey
Dim bytes() As Byte
bytes = enc.ComputeHash_2((TextToHash))
Base64_HMACSHA1 = EncodeBase64(bytes)
Set asc = Nothing
Set enc = Nothing
End Function
Private Function EncodeBase64(ByRef arrData() As Byte) As String
Dim objXML As MSXML2.DOMDocument
Dim objNode As MSXML2.IXMLDOMElement
Set objXML = New MSXML2.DOMDocument
' byte array to base64
Set objNode = objXML.createElement("b64")
objNode.DataType = "bin.base64"
objNode.nodeTypedValue = arrData
EncodeBase64 = objNode.Text
Set objNode = Nothing
Set objXML = Nothing
End Function
Using this function:
Debug.Print Base64_HMACSHA1("abc", "123")
VAsMU9SSWDe9krP3Gr56nXC2dsQ=

HMAC is a construct for turning a hash function, like SHA1, into a Message Authentication Code (MAC).
Normal hash functions don't have any secret data associated with it. This means that anyone can compute the digest, assuming they have the original input. HMAC uses a secret key, so that only those in possession of the key can compute outputs.
Suppose I have a file, file.txt. I want to send this to you, and we need to make sure nobody tampers with it. Sorry, I have no clever way to represent this with just text.
me -> file.txt -> you
me -> SHA1(file.txt) -> you
Then you verify the result by computing your own SHA1 digest, and verifying it matches what I sent you.
Now suppose an attacker was in the middle. Unfortunately, because there is no secret involved, the attacker can modify the file, and compute his own file/digest pair. When you compute your version, it'll match what he sent you, and you'll be none the wiser.
me -> file.txt -> attacker -> modified.txt -> you
me -> SHA1(file.txt) -> attacker -> SHA1(modified.txt) -> you
With HMAC, we add a secret key to the computation.
me -> file.txt -> you
me -> SHA1_HMAC(file.txt, our_secret) -> you
When you compute your version, you apply the secret key as well, and the result matches. The attacker, without knowledge of the key, can't replace the digest.
me -> file.txt -> attacker -> modified.txt -> you
me -> SHA1(file.txt) -> attacker -> SHA1_HMAC(modified.txt, // DOESN'T KNOW KEY) -> you
HMAC is a very specific way of adding the secret key. Unfortunately, simple methods of just concatenating a key to the end of the file, or pre-pending it before hashing, are vulnerable to different attacks (length extension attacks, for example).
The B64 is Base64 encoding the output, to make it pretty.
What this code is ultimately doing is taking some input, and some secret key, and computing a 160-bit digest, and base64 encoding the result.
There is an implementation of SHA1 HMAC in .NET
This looks like an implementation of Base64 for VBA
I hope this answers it well enough, or clear enough. If the text is confusing, please let me know. I tried a couple routes of how to express it, and none of them seemed that clear.

You have written:
It appears to me that calling a JavaScript function from VBA is fairly impractical.
That is a misjudgment.
Javascript can be easily packaged as a Windows Script Component (WSC) and then invokved via COM, from VBA, Perl, VB6, or what-have-you.
Here's an example of packaging Javascript as a WSC and invoking it: https://stackoverflow.com/a/849970/48082
Therefore, your problem should be easily solvable.

Encrypting data with ruby decrypting with node

I want to encrypt some data in a ruby app and then decode it in a nodejs app. I have been trying to get this to work and now I am just trying to encrypt the same piece of data in both languages to get the same result but I can't seem to do it.
//js
var crypto = require('crypto');
var key = crypto.createHash('sha1').update('key').digest('hex');
console.log(key); // a62f2225bf70bfaccbc7f1ef2a397836717377de
var encrypted = "";
var cipher = crypto.createCipher('bf-cbc', key);
encrypted += cipher.update('text');
encrypted += cipher.final('hex');
console.log(encrypted); //outputs 4eafd5542875bd3c
So it looks like I get a hexadecimal string from the encoding.
#ruby
require 'openssl'
require 'digest/sha1'
c = OpenSSL::Cipher::Cipher.new("bf-cbc")
c.encrypt
# your pass is what is used to encrypt/decrypt
c.key = key = Digest::SHA1.hexdigest("key")
p key # a62f2225bf70bfaccbc7f1ef2a397836717377de
e = c.update("text")
e << c.final
p e # 皋?;??
Is there some sort of encoding issue that I am missing. I tried to base64 decode e but that didn't produce the same result as the node app. Any pointers?
UPDATE: So this is as close as a friend and I can get: https://gist.github.com/a880ea13d3b65a21a99d. Sheesh, I just want to encrypt something in ruby and decrypt it in node.
UPDATE2: Alright, the code in this issue gets me a lot of the way there: https://github.com/joyent/node/issues/1395

There are several subtle things that make this fail. The most important one - you are not specifying an IV in your code, so a random value will be generated for you. You would notice that you couldn't even decrypt your ciphertext within the same programming language this way.
So you need to provide an explicit IV to both implementations. But before I show you the code, some advice:
Key generation:
Blowfish operates on 64 bit blocks, its key size varies, but OpenSSL (which currently powers both Ruby's and node.js' cipher implementation) uses 128 bit by default, that is 16 bytes.
So your key violates two principles - the first: it's simply too long. It's the hex representation of a SHA-1 hash, which is 20 bytes * 2 = 40 bytes instead of 16. Most of the time this is fine, because the implementation truncates the values appropriately, but that is something you should not depend on.
The second mistake, much more severe, is that you use the hex representation instead of the raw bytes: big security issue! Hex characters are not random at all, so in effect you reduce the entropy of your input to half the length (because the underlying bytes were random).
A secure way to generate random keys is using OpenSSL::Random
key = OpenSSL::Random.random_bytes(cipher_key_len)
A third mistake is to keep your key hard-coded in the sources. It's a bad idea. The least you should do is to store it elsewhere on the file system, where access is tightly restricted. See also my answer to another question. The key should be stored out-of-band and only loaded dynamically within the application.
Cipher:
Blowfish grows old. It's still considered unbroken in the sense that brute-forcing it is the only way to break it. But a search space of 2^64 is not out of reach for resourceful attackers. So you should indeed move on to AES.
Padding:
OpenSSL pads using PKCS5Padding (also known as PKCS7Padding) by default. Ruby profits from this and my bet is node.js utilizes this, too - so you should be safe on this.
Now to the working solution. We need to generate an IV, Blowfish requires it to be 64 bit - 8 bytes. You will need rbytes to get secure random numbers in node. The IV may be hardcoded in your sources (it's public information, no security impact) - but it must be the same on both sides. You should pregenerate a value and use it for both node.js and Ruby.
/*node.js*/
var rbytes = require('rbytes');
var iv = rbytes.randomBytes(8);
/*see advice above - this should be out-of-band*/
var key = rbytes.randomBytes(16);
var encrypted = "";
var cipher = crypto.createCipheriv('bf-cbc', key, iv);
encrypted += cipher.update('text');
encrypted += cipher.final('hex');
Now the Ruby part:
require 'openssl'
c = OpenSSL::Cipher::Cipher.new("bf-cbc")
c.encrypt
# should be out-of-band again
c.key = OpenSSL::Random.random_bytes(16)
# may be public but has to be the same for Ruby and node
iv = OpenSSL::Random.random_bytes(8)
c.iv = iv
e = c.update("text")
e << c.final
puts e.unpack('H*')[0]

Your cyphertext will be some random looking bytes. Those bytes can be expressed as hex, Base64 or in other ways. It looks as if your ruby code is outputting the raw bytes. I suggest that you convert those raw bytes to hex to make your comparison.
Looking at your code, you should also change from Blowfish ("bf") to AES. Blowfish has a 64-bit block size and is now obsolete.
You would do well to explicitly specify padding, PKCS7 is common

OK. I want to thank everyone for helping me out. Basically this thread here answers my question: https://github.com/joyent/node/issues/1395. I am going to go ahead and post the two programs in case anyone else has to go through this rigamarole. Keep in mind this isn't mean to be hardcore secure, this is a stepping stone for ruby encrypting data and node decrypting it. You will have to take more steps to make sure higher security measures are taken.
The code is located at this gist: https://gist.github.com/799d6021890f34734470
These were run on ruby 1.9.2p290 and node 0.4.10