We Keep Coding

How to make SJCL and Python hashlib generate identical pdkdf2 output

I have a JavaScript app and a Python app that communicate using a key derived from a password using pbdkf2. The problem is, the generated keys don't match. I've produced a minimal test case for each.
Python
import hashlib, binascii
bytes = hashlib.pbkdf2_hmac('sha256', "password".encode(), b'', 100000)
print(binascii.hexlify(bytes).decode())
Generates: 64a868d4b23af696d3734d0b814d04cdd1ac280128e97653a05f32b49c13a29a
JavaScript
<script src="lib/sjcl.js"></script>
<script>
var hmacSHA256 = function (key) {
var hasher = new sjcl.misc.hmac(key, sjcl.hash.sha256);
this.encrypt = function () {
return hasher.encrypt.apply(hasher, arguments);
};
};
hash = sjcl.misc.pbkdf2("password", [0], 100000, 256, hmacSHA256);
console.log(sjcl.codec.hex.fromBits(hash));
</script>
Generates: 41c04f824d843d5be0ae66b3f621d3f05db7d47e7c46ee0e9171b5cbff7f3631
I'm scratching my head a lot now. I think b'' and [0] are equivalent salts, but I'm not sure. I think they both use utf-8 to encode the password, but I'm not sure. And I'm not convinced the JavaScript hmacSHA256 function exactly matches what Python is doing. Or it could be something else still.

Off the top of my head, have you checked if
hash = sjcl.misc.pbkdf2("password", "", 100000, 256);
gives the correct result?
As far as I can tell from the docs, SJCL's PBKDF2 implementation defaults to HMAC-SHA256 if you don't explicitly give it a PRF. If making that change fixes the bug, then there's probably something wrong with your hmacSHA256 wrapper.
Also, I'm not sure if specifying an empty salt as [0] really works (or is guaranteed to work in future versions, given that the format of SJCL's bitArrays is explicitly subject to change), but "" definitely should work.

This is sharing for the rest based on my recent experience.
My objective:
To generate PBKDF2 password using Python. The client will be Android (Java), and the back end will be on Flask (Python).
Issue:
While testing, I discovered that both versions (Java vs Python) produced different hashing output (all other parameters were equal - SHA256, 1000 iterations, similar SALT)
What I have found out:
Using PBKDF2 generation tools available on the internet, the Android result was an exact match, while the Python was not. So there is a good chance the Python result was somehow skewed .....
Problem solved:
While looking for possible explanation in SO, I discovered that the way I converted String to bytes in Python was somehow not entirely correct:
Original code:
dk = hashlib.pbkdf2_hmac('sha256', b'base64_message', b'salt', 1000)
Working code:
dk = hashlib.pbkdf2_hmac('sha256', base64_message.encode(), salt.encode(), 1000)
This is probably due to my lack of experience in Python. Hope this note will be of use to others, especially those who are new to Python!

How to set locale in JavaScript, for example for toLocaleUpperCase()?

I'd like to use the JavaScript toLocaleUpperCase() method to make sure that the capitalization works correctly for the Turkish language. I cannot be sure, however, that Turkish will be set as the user's locale.
Is there a way in modern browsers to set the locale in run time, if I know for sure that the string is in Turkish?
(I ran into this problem while thinking about Turkish, but actually it can be any other language.)

There isn't really anything much out there but I came across this JavaScript setlocale function script that you might find useful.

You unfortunately cannot set locale during runtime. All hope is not lost though, there are many good libraries on npm for you to use. Check out https://www.npmjs.com/package/upper-case and https://www.npmjs.com/package/lower-case for example, it will work for many other languages too.
If that's too much, you can roll your own simple library:
var ALL_LETTERS_LOWERCASE = 'abcçdefgğhıijklmnoöprsştuüvyz';
var ALL_LETTERS_UPPERCASE = 'ABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZ';
function toLowerCaseLetter(letter) {
letter_index = ALL_LETTERS_UPPERCASE.indexOf(letter);
return ALL_LETTERS_LOWERCASE[letter_index];
}
function toLowerCase(my_str) {
var lower_cased = ''
for (letter of my_str) {
lower_cased += toLowerCaseLetter(letter);
}
return lower_cased;
}
console.log(toLowerCase('ÇDEFGĞHIİJKLMNOÖPRSŞTUÜ'))
Very similar for upper case version.

This option may not have existed back in 2013 but may help new visitors on this topic:
According to MDN (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/toLocaleUpperCase) the function toLocaleUpperCase takes an optional parameter 'locale'.
Setting the right language tag is a topic on its own (https://www.w3.org/International/articles/language-tags/). Simplest example looks like this
'selam dünya'.toLocaleUpperCase('tr'); // SELAM DÜNYA

is there any builtin javascript string hash function in newest browsers?

Everytime a new versions of browsers show up I hear about new stuff being added, like say webGL and other technologies that no one really knows if they catch up.
But I wonder if someone ever thought about such basic stuff in JS like hashing functions (MD5,SHA1 and the like).
By newest browsers I mean today's development versions too like Opera 12, Chrome 17 or Firefox 10.
Looking now for solution I found this comment on another thread here: https://stackoverflow.com/questions/7204097/short-hashing-function-for-javascript (Do you know that javascript objects already are hashtables ?). So what are these 'hashtables' ? Does it mean that I can make any string into a hash, but not an established one like md5 or sha1 but some JS build in specific ?
basically what I need to do is:
var txt="Hello world!";
var hash = txt.toSha1();

For anybody still looking for this information. There is a WebCrypto API which appears to have been finalised at the beginning of 2017.
To use it in a browser, you can find it at window.crypto.subtle which contains methods for encryption, digests etc. Documentation on the available functions here.

Paul Johnston has implemented the following algorithms in javascript
MD5, RIPEMD-160, SHA-1, SHA-256 and sha-512
You can find the source code and some examples here:
http://pajhome.org.uk/crypt/md5/
I hope this is what you were looking for.

async function sha256(source) {
const sourceBytes = new TextEncoder().encode(source);
const digest = await crypto.subtle.digest("SHA-256", sourceBytes);
const resultBytes = [...new Uint8Array(digest)];
return resultBytes.map(x => x.toString(16).padStart(2, '0')).join("");
}

Note: This answer was written in 2014 when the Web Cryptography API was not available. Do not use this in the context where cryptographic security is needed. This may be useful when you need a simple reversible encryption with "builtin" support.
When I need simple client side hashing without external libraries I use the browsers' built in atob() and btoa() functions.
window.btoa() creates a base-64 encoded ASCII string from a "string" of binary data.
function utf8_to_b64( str ) {
return window.btoa(encodeURIComponent( escape( str )));
}
The window.atob() function decodes a string of data which has been encoded using base-64 encoding.
function b64_to_utf8( str ) {
return unescape(decodeURIComponent(window.atob( str )));
}
http://caniuse.com/#search=btoa and http://caniuse.com/#search=atob shows it is hugely supported by the modern browsers
Example taken from https://developer.mozilla.org/en-US/docs/Web/API/window.btoa

Using Xpath selectors to extract / combine data from Javascript Objects?

I'm quite fond of CakePHP Set class, it comes with a few awesome tools i now love. I use it mainly to extract or combine arrays.
I was wondering if there was a way to do just that with javascripts Objects/Arrays of data.
// Desired Usage:
var users = {User: {0:{id:1,name:'A'},1:{id:2,name:'B'}}}
var results = $.extract('/User/id', users);
// results returns:
// {0:1,1:2};
// /User[id>2][<5] Selects all Users with an id > 2 but < 5
It could support on jQuery or maybe just Sizzle.
Do i have to develop thoses functions from zero or is there already some native/plugin xpath selector/extractor support out there ? Can Sizzle do this ?
Thanks a lot!

Look for that library http://code.google.com/p/jsonpath/
Not sure, whether it could process request exactly in this syntax: User[id>2][<5],
but it's rather powerful library and should have similar feature.
Even if there's no function for request parts like "[<5]", you may call
.slice(0, 5);

What JavaScript library should I use for parsing URL parameters?

How do I parse URL parameters in JavaScript? (These are the parameters I would ordinarily call GET parameters or CGI parameters, but in this case the page is basically submitting to itself, not a server, so there is no GET request and definitely no CGI program.)
I've seen a number of routines on the net that I can copy, but I have no idea how robust any of them are. I'm used to other languages like Perl and Java where I can rely on an extremely well-tested and robust library that I know will handle millions of little edge-cases in the standard. I would like the same here, rather than just cutting and pasting an example.

jQuery URL Utils or jQuery URL Parser.

Here's are two simple functions that do the job : http://adamv.com/dev/javascript/querystring
Here is a sample of the API Reference :
var qs = new Querystring();
// Parse a given querystring
var qs2 = new Querystring("name1=value1&name2=value2");
var v1 = qs2.get("name1");
var v3 = qs2.get("name3", "default value");

If it's "submitting to itself," do you need to do GET parameters?
But if you do, most browsers now have the decodeURIComponent function which will handle individual parameters; your job is to split them on & (String#split will do that nicely). If you want a library, jQuery and Prototype are both well-used and tested.

The best way I have found is to simply do it yourself and funnel the params into a global key/value object.
Getting quer params is simple...
just take a couple of .split()'s
var myquery = thewholeurl.split("?")[1]; //will get the whole querystring with the ?
then you can do a
myparams = myquery.split("&")
then you can do
for each param in myparams
{
temp = param.split("=");
mykeys.push(temp[0]);
myvalues.push(temp[1]);
OR
myObject[temp[0]] = temp[1];
}
It's just a matter of style.
This is not perfect code, just psuedo stuff to give you the idea.

I use the parseUri library available here:
http://stevenlevithan.com/demo/parseuri/js/
It allows you to do exactly what you are asking for:
var uri = 'http://google.com/?q=stackoverflow';
var q = uri.queryKey['q'];
// q = 'stackoverflow'
I've been using it for a while so far and haven't had any problems.

I found this useful for simple url parsing, modifying url (like adding new query params): https://github.com/derek-watson/jsUri

I think this library would work quite well, it is independent so you can use it with JQuery or with YAHOO or Dojo, another advantage is that it is pretty well documented.
http://www.openjsan.org/doc/t/th/theory/HTTP/Query/0.03/lib/HTTP/Query.html
You can use HTTP.Query to do all of the work for you in this case. It is only like 1.2 KB compressed so you could even include it in a bigger library if you wanted.

I recommend query-string library
Installing:
npm install query-string
Usage:
import queryString from 'query-string';
console.log(location.search);
//=> '?foo=bar'
const parsed = queryString.parse(location.search);
console.log(parsed);
//=> {foo: 'bar'}
parsed.foo = 'unicorn';
parsed.ilike = 'pizza';
const stringified = queryString.stringify(parsed);
//=> 'foo=unicorn&ilike=pizza'
location.search = stringified;
// note that `location.search` automatically prepends a question mark
console.log(location.search);
//=> '?foo=unicorn&ilike=pizza'
https://www.npmjs.com/package/query-string

Javascript has no built in support for URL parameters.
Anyway, the location.search property returns the portion of current page URL starting from the question mark ('?').
From this, you can write your own parameter parser or you can make use of one of those available in most common Javascript frameworks, such as JQuery and similar.