Is there some library (preferably in javascript) that will classify a cookie value into some sensible data type?
When I look at various cookie values, I see various types such as:
plain english
numbers (hex, dec)
base64
some combination of above
It would be even more awesome if in addition to guessing the data-type, the library can also guess the type of encryption, or hashing used.
I remember experimenting with a python library a while ago but that did not seem to guess even simple hashes such as shasum, sha256sum, sha256sum, md5sum etc
There's no way to do this since all cookies are stored as string values. A workaround can be to classify them into general classes by putting them through regular expressions.
/^[0-9A-Fa-f]{8}$/ Signifying that could be an Adler-32 or CRC-32 Checksum
/^[0-9A-Fa-f]{32}$/ Signifying that could be an MD2, MD4, MD5, or Haval Sum
/^[0-9A-Fa-f]{64}$/ Signifying that could be a SHA-256 Sum
/^[0-9A-Fa-f]{96}$/ Signifying that could be a SHA-384 Sum
/^[0-9A-Fa-f]{128}$/ Signifying that could be a SHA-512 Sum
/^[^0-9A-Za-z+/]+={0,2}$/ Signifying that could be a Base-64 Encode
There's a chance that any one of these could just be regular numbers or plain test too (like "DEADBEEF"). If you're in charge of that data, I would specify the type in another cookie.
In summary, there's just no guarantee, unless you know what to expect.
Related
I would like to generate a unique number from string. The string is a combination of username and password. I would like to generate a unique number id (not string) from this combination. I first md5 the combination and then convert it to number. The number length needs to be 10. Any suggestions?
It would be best if you can provide more details about the third-party you're trying to interface with, because this is a very odd request and it contains a fundamental flaw. You ask for the number to be unique, but you are allowing for only 10 decimal ("number id") digits, or ~10 billion possible values.
This sounds like an awful lot but it's really not. This gives you a hash of just over 33 bits. The simple hash collision probability calculator at http://davidjohnstone.net/pages/hash-collision-probability puts this at a 44% chance of a collision at just 100,000 entries. But that assumes full usage of all the available input characters. Since username and password combinations are almost always limited to alphabetic and numeric characters, the real collision chance is much worse at far fewer entries (can't be calculated without knowing the characters you allow for these fields - but it's bad).
NodeJS provides numerous crypto functions in the crypto module. A whole set of hashing functions is available, including the ideal-case SHA* options. These can be used to provide safe, irreversible hashes with astronomically collision probabilities.
If these options are not usable for you, I would suggest you have a fundamental design flaw. You're almost certainly mapping a user/pass combination to a userID in a remote system in a way that an attacker would find easy to compromise with a simple brute-force attack, given the high collision risk in your model.
If you are doing what I think you are doing, the "right" way to do this would be to have a simple database on a server somewhere. The user/pass would be assigned a unique ID in there, and it doesn't matter what this is - it could be an auto-increment ID field in a single MySQL table. The server would then contact this remote service with the ID value for any API calls necessary, and return the results to the user. This eliminates the security risk because the username/password are not actually hashed, just stored, and can be checked 100% on every call.
Never use a hash as a primary data value. It's a simplification, not a real value on its own.
I would like to get Both CryptoJS's SHA256 and php's Crypt SHA256 output to match.
PHP crypt has a salt and a number of rounds. E.g. for 5000 rounds and salt of "usesomesillystringforsalt" it would be;
$hash = crypt('Clear Text String', '$5$rounds=5000$usesomesillystringforsalt$');
I hope im not blind, but i cant find how to reproduce this behaviour in crypto-js. Its syntax doesn't seem to allow for rounds or salt.
Is it possible or should i just resort to using the basic PHP hash instead of crypt?
The CryptoJS API doesn't provide a way to specify a salt or the number of rounds for SHA256. You could add a salt manually if necessary, and specifying rounds doesn't make sense since "plain" SHA256 always uses a fixed number of rounds (64).
The number of rounds in PHP's crypt() actually defines how often the SHA256 algorithm is applied consecutively, in order to increase the complexity of brute force attacks. The PHP source code comments on this as follows: "Repeatedly run the collected hash value through SHA256 to burn CPU cycles".
As you can see in the source code (here and here), crypt() is actually a key derivation function that only makes use of SHA256 to generate cryptographically secure, salted hashes. So it also doesn't simply append the given salt to the key, instead it's a more elaborate process.
Therefore, it is not possible to get the same results with the algorithms provided by CryptoJS.
crypt() is mainly intended for password hashing. So if you need the hashes for another purpose, hash() is a good alternative (and of course creates exactly the same results as CryptoJS.SHA256()).
However, bear in mind that any cryptography with JavaScript is generally considered harmful. Hence, you should better use SSL in your application, if possible, and generate the hashes server side. If this is an option, have a look at bcrypt.
Let's say I have am creating a webapp, where users can create a nested tree of strings (with sensitive information). These strings are presumably quite short. I want to encrypt both keys and values in this tree before saving it. All values in the tree will be encrypted client-side using a symmetric key supplied by the user. Likewise they will be decrypted client-side, when reading.
The tree is persisted in a Mongo database.
I can't decide whether I should serialize the tree and encrypt it has a whole string or whether to encrypt values individually, considering that all data in the tree will be encrypted using the same key.
What are the pros and cons of either?
From what I can tell, AES uses a block size of 128 bits, meaning that any string can grow up to 15 characters in length when encoded, which speaks in favor of encoding a serialized string (if you want to avoid overhead)
Note: Although the webapp will use both HTTPS, IP whitelisting and multifactor authentication, I want to make an effort to prevent data breach in the event the Mongo database is stolen. That's what I'm going for here. Advice or thoughts on how to accomplish this is appreciated.
Update
Furthermore, I also want my service to inspire trust. Sending data in the clear (although over HTTPS) means the user must trust me to encrypt it before persisting it. Encrypting client-side allows me to emphasize that I don't know (or need to know) what I'm saving.
I can't think of a reason why these approaches would be different in terms of security of the actual strings (assuming they are both implemented correctly). Encrypting the strings individually obviously means that the structure of the tree will not be secret, but I'm not sure if you are concerned with that or not. For example, if you encrypt each string individually, someone seeing the ciphertexts could find out how many keys there are in the tree, and he could also learn something about the length of each key and value. If you encrypt the tree as a whole serialized blob, then someone seeing the ciphertext can tell roughly how much data is in the tree but nothing about the lengths or number of individual keys/values.
In terms of overhead, the padding would be a consideration, as you mentioned. A bigger source of storage overhead is IVs: if you are using a block cipher mode such as CTR, you need to use a distinct IV for each ciphertext. This means if you are encrypting each string individually, you need to store an IV for each string. If you encrypt the whole serialized tree, then you just need to store the one IV for that one ciphertext.
Before you implement this in Javascript, though, you should make sure that you're actually getting a real improvement in security from doing client-side encryption. This article is a classic: http://www.matasano.com/articles/javascript-cryptography/ One important point is to remember that the server is providing the Javascript encryption code, so encrypting data on the client doesn't protect it from the server. If your main concern is a stolen database, you could achieve the same security by just encrypting the data on the server before inserting it in the database.
First of all, I am not a security expert ;-)
I can't decide whether I should serialize the tree and encrypt it has a whole string or whether to encrypt values individually, considering that all data in the tree will be encrypted using the same key.
I would say serializing the tree first and encrypting the result of that has the biggest con.
What plays a huge role in successfully cracking encryption is often the knowledge about certain characters that appear quite often in the original text – for example the letters e and n in English language – and doing statistical analysis based on that on the encrypted text.
Now lets say you use for example JSON to serialize your tree client-side before encrypting it. As the attacker, I would easily know that, since I can analyze your client-side script at my leisure. So I also know already that the “letters” {, }, [, ], : and " will have a high percentage of occurrence in every “text” that you encrypt … and that the first letter of every text will have been either a { or a [ (based upon whether your tree is an object or an array) – that’s already quite a bit of potentially very useful knowledge about the texts that get encrypted by your app.
I am currently working on a project that will involve credit card swipes for admissions based on database rows. Like a will call system, the SHA-256 hash of the CC number must match the hash in the DB row in order to be considered the "proper pickup".
However, because the box office system is based in the browser, the CC number on pickup must be hashed client-side, using Javascript, and then compared to the previously downloaded will call data.
However when trying to hash the numbers, the hash always ends up different than what was hashed when the DB row was created (using VB.NET and SQL Server 2008 R2). For example, if a CC number in the database happened to be 4444333322221111, then the resulting hash from .NET would become xU6sVelMEme0N8aEcCKlNl5cG25kl8Mo5pzTowExenM=.
However, when using any SHA-256 hash library for Javascript I could find, the resulting hash would always be NbjuSagE7lHVQzKSZG096bHtQoMLscYAXyuCXX0Wtw0=.
I'm assuming this is some kind of Unicode/UTF-8 issue, but no matter what I try I cannot get the hashes to come out the same and it's starting to drive me crazy. Can anyone offer any advice?
Here's something that may provide some insight. Please go to http://www.insidepro.com/hashes.php?lang=eng and insert "4444333322221111" without quotes into the Password box. Afterwards, scroll down to the SHA-256 section.
You can see that there are four results, two of them are the hash codes I posted (the second from the top being the Javascript hash and the bottom one being the SQL hash). According to that page, the bottom hash result is generated using a base 64 string, as well as making the password into unicode format.
I've investigated this and tried many different functions to encode the password into unicode format, but no matter what little tweaks I try or other functions I make, I could never get it to match the hash code I need.
I am currently investigating the parameters used when calling the SHA-256 function on the server side.
UPDATE:
So just to make sure I wasn't crazy, I ran the Hash method I'm using for the CC numbers in the immediate window while debugging. Again, the result remains the same as before. You can see a screenshot here: http://i.imgur.com/raEyX.png
According to online SHA-256 hash calculator and a base-64 to hex decoder, it is the .NET implementation that has not calculated the hash correctly. You may want to double check the parameters you pass to the hashing functions.
When you are dealing with two untrusted implementations, it is always a good idea to find another independent implementation, and choose the one that matches the third one as correct. Either that, or find some test vectors, and validate the implementations individually.
EDIT:
A quick experiment shows that the SHA-256 hash you get from .NET matches the hext string 3400340034003400330033003300330032003200320032003100310031003100 - little endian 16-bit characters. Make sure you pass in ASCII.
Adam Liss had it right when he mentioned the byte arrays between strings in .NET/SQL Server are different than strings in Javascript. The array in .NET for the string 4444333322221111 would look like [52 0 52 0 52 0 52 0 51 0 51 0... etc.] and the same thing in Javascript would just look like [52 52 52 52 51 51 51 51...]. Thus, with different byte arrays, different hashes were generated.
I was able to remedy this for my application by modifying the base 64 SHA-256 hashing algorithm from here, where each character is pulled from the string one at a time in order to generate the hash.
Rather than having it do it this way, I first converted the string into a unicode-like byte array (like the .NET example above, 52 0 52 0 etc), fed that array to the hashing algorithm instead of the string, and did some very minor tweaks in order for it to grab each array member to generate the hash. Low and behold, it worked and now I have a very convenient method of hashing CC numbers in the same fashion as the .NET framework for quick and easy order lookup.
Are you sure about your JavaScript SHA256 function ?
And your firstly generated hash ?
SHA-256("4444333322221111"); // 35b8ee49a804ee51d5433292646d3de9b1ed42830bb1c6005f2b825d7d16b70d
hex: 35b8ee49a804ee51d5433292646d3de9b1ed42830bb1c6005f2b825d7d16b70d
HEX: 35B8EE49A804EE51D5433292646D3DE9B1ED42830BB1C6005F2B825D7D16B70D
h:e:x: 35:b8:ee:49:a8:04:ee:51:d5:43:32:92:64:6d:3d:e9:b1:ed:42:83:0b:b1:c6:00:5f:2b:82:5d:7d:16:b7:0d
base64: NbjuSagE7lHVQzKSZG096bHtQoMLscYAXyuCXX0Wtw0=
I want to encode a number which would be maximal 10 digits long using a key with ASCII chars.
The encoded string should be decodable with the same key , which should return the decoded number.
10 => encoding_with("secret_pass") => hash => decoding_with("secret_pass") => 10
Both operations should work the same way in Javascript and Ruby.
What algorythm should I use for this purpose ?
All data should be in ASCII, no multibyte data for input, hash, and the key.
A simple XOR should be sufficient.
JavaScript
Ruby
Yes, you can go overboard and break out the full crypto suite, but since one part of the algorithm will run in-browser, in JS (completely untrusted environment), any attempt at Serious Cryptography™ will give you a false sense of security (in other words, will actually make your system less secure).
If you're trying to protect the data in transit, use the tool that's made for the job; in this case, HTTPS.
I would look into some for of a Symmetric Key Encryption, and the most prominent one AES. AES is pretty much the standard and is implemented in both languages. Just make sure to use the same key and salt for encryption and decryption.
Javascript AES
Ruby AES
If you want it to be simple, and security isn't a great concern (since the javascript... observation in the comments), the easiest way would be to simple generate any random value, bigger (more digits) than the number, and simple XOR it with the number. This assumes the key (=random number) would be previously shared with the other program.
1. generate random number with 10 hex digits -> KEY
2. take the number then do (number XOR key ) -> result
3. send result
4. get the result and do (result XOR key) -> number
something better (stronger) would be to use any public-private key system. Exchange keys, crypt with public on one side, decrypt with private on the other side.