We Keep Coding

How to encrypt JSON object with JS-NaCl and decrypt with php Libsodium

I managed to find a Libsodium js library (JS-NaCl) for front end encryption and has setup my PHP backend for Libsodium encrypt/decrypt also. When I encrypt a JSON object like below
const key = "827ccb0eea8a706c4c34a16891f84e7b";
const nonce = "0123456789abcdefghijvbnm";
var credentials = {
"zip":"265",
"account_number":"10028979739",
"passcode":"1234",
"account_type":"personal",
"request":"login",
"device":"iPhone 11"
};
function encrypt(data){
return sodium.crypto_secretbox(sodium.encode_utf8(data),nonce,key);
};
function decrypt(data){
return sodium.decode_utf8(sodium.crypto_secretbox_open(data, nonce, key));
}
function login(data){
$.ajax({
url:baseURL+"account/account.php",
method:"POST",
contentType:"application/x-www-form-urlencoded",
dataType:"json",
data:"datax="+JSON.stringify(encrypt(credentials)),
beforeSend:()=>{
console.log(credentials);
},success:(response)=>{
console.log(response);
},error:(e)=>{
swal("Connection Error","Failed to connect to the server!","error");
}
});
}
When I fire the login method with it encrypts the JSON object using the encrypt method hence I send something like this:
datax: {"0":191,"1":118,"2":248,"3":134,"4":45,"5":163,"6":3,"7":157,"8":78,"9":73,"10":157,"11":137,"12":178,"13":6,"14":68,"15":91,"16":217,"17":219,"18":50,"19":11,"20":127,"21":177,"22":130,"23":25,"24":209,"25":254,"26":210,"27":44,"28":119,"29":13,"30":144}
at the php backend code I am doing this:
<?php
function decrypt($data){
$key = "e9897cea109576c2f8088c277125d553e4f83afbc0abbb92cfb1f7b776b4fee0";
$nonce = "0123456789abcdefghijvbnm";
return sodium_crypto_secretbox_open($data,$nonce,$key);
}
function encrypt($data){
$data = utf8_encode($data);
$key = "e9897cea109576c2f8088c277125d553e4f83afbc0abbb92cfb1f7b776b4fee0";
$nonce = "0123456789abcdefghijvbnm";
return sodium_crypto_secretbox($data,$nonce,$key);
}
$credentials = $_POST["datax"];
echo decrypt($credentials);
?>
Same Key, Same nonce but it doesn't echo back anything. How to I decrypt this??

The code needs some changes. On the JavaScript side (frontend):
The JavaScript object must be converted into a string.
Besides the data, nonce and key must also be encoded using Utf8. Although the key could also be hexadecimal encoded to a 16 bytes key, in this context it must be Utf8 encoded to a 32 bytes key, because sodium.crypto_secretbox expects a 32 bytes key. The expected nonce must be 24 bytes in size.
Now the data can be encrypted.
sodium.crypto_secretbox returns the data as Uint8Array, which must therefore be encoded for transfer into a suitable format, e.g. hexadecimal.
The corresponding code is:
nacl_factory.instantiate(function (sodium) {
var credentials = {
"zip":"265",
"account_number":"10028979739",
"passcode":"1234",
"account_type":"personal",
"request":"login",
"device":"iPhone 11"
};
// Convert JavaScript object to string
var data = JSON.stringify(credentials);
// Utf8 encode key, nonce and data
var keyUtf8 = sodium.encode_utf8("827ccb0eea8a706c4c34a16891f84e7b");
var nonceUtf8 = sodium.encode_utf8("0123456789abcdefghijvbnm");
var dataUtf8 = sodium.encode_utf8(data);
// Encrypt
var encrypted = sodium.crypto_secretbox(dataUtf8, nonceUtf8, keyUtf8);
// Hex encode encrypted data for transfer
var encryptedHex = sodium.to_hex(encrypted);
console.log("Ciphertext (hex):\n" + encryptedHex.replace(/(.{64})/g, "$1\n"));
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/js-nacl/1.3.2/nacl_factory.js"></script>
On the PHP side (backend):
The hexadecimal string $encryptedHex must be decoded.
The decoded data are to be decrypted. For this, key and nonce of the encryption must be used. In the posted code a different key is used, which is not possible in the context of crypto_secretbox (symmetric encryption), i.e. both sides use the same key. For asymmetric encryption there is crypto_box.
The result can be decoded into a JavaScript object whose objects can be accessed as usual.
The corresponding code is:
// Hex decode
$encrypted = sodium_hex2bin($encryptedHex);
// Decrypt
$nonce = "0123456789abcdefghijvbnm";
$key = "827ccb0eea8a706c4c34a16891f84e7b";
$decrypted = sodium_crypto_secretbox_open($encrypted, $nonce, $key);
// Convert to JavaScript object
$decryptedJSON = json_decode($decrypted);
echo "Zip: " . $decryptedJSON->zip . "\n";
echo "Account number: " . $decryptedJSON->account_number . "\n";
echo "Passcode: " . $decryptedJSON->passcode . "\n";
echo "Account type: " . $decryptedJSON->account_type . "\n";
echo "Request: " . $decryptedJSON->request . "\n";
echo "Device: " . $decryptedJSON->device . "\n";

php base64_encode result not similar to javascript encode result

Code snippets:
php:
$shaVal = '59bc125840733ea828f42e276661b01e177f1414';
$enc = base64_encode(pack('H*', $shaVal));
echo $enc;
//prints => WbwSWEBzPqgo9C4nZmGwHhd/FBQ=
and in Javascipt I used buffer npm module
let Buffer = require('buffer').Buffer;
let shaVal = '59bc125840733ea828f42e276661b01e177f1414';
//function similar to php's pack() and returns binary data
let bData = Buffer.from(shaVal, 'hex').toString();
console.log('bData ', bData)
//encode with base64
let val64 = Buffer.from(bData, 'binary').toString('base64');
console.log('base 64 encode ', val64)
//prints => Wf0SWEBzPv0o/S4nZmH9Hhd/FBQ=
How can I get the exact same output printed by php?
Note: Both options showing binary data as Y�X#s>�(�.'fa�

It's because PHP pack returns string, where as javascript buffer returns Array.
This answer might help : https://stackoverflow.com/a/41962257/3086531

Decrypt blowfish in php

I'm using blowfish.js on my clientside.
To encrypt on the clientside I use
var encryptedData = blowfish.encrypt(
JSON.stringify(myData), myEncryptionKey, {cipherMode: 0, outputType: 0}
);
That way encryptedData is Base64 encoded blowfish string. It is OK.
To decrypt on the clientside I use
var decryptedData = blowfish.decrypt(
encryptedData , myEncryptionKey, {cipherMode: 0, outputType: 0}
);
So decryptedData equals to JSON.stringify(myData). It is OK.
Then I send my encryptedData to the .php script that contains
$data = $_POST['data'];
$data = mcrypt_decrypt(MCRYPT_BLOWFISH, $myEncryptionKey, pack('H*',$data), MCRYPT_MODE_ECB);
And when I try to print $data I'm getting something unreadable like
�Nv��Xd�̿�:q6��A����,!v��c�O¡1�%>��ҭ� {0�� � ���g�u�����t�z3q$����T��/Ҧ.j-�/!���;�lS���Z�X
��&����{j�m�����F�`7��.......and so on
It isnt OK.
Does anyone know what I'm doing wrong on the server-side?

Why do you pack('H*',$data) as you wrote the data is a base64 encoded string. Just base64_decode the data and pass it to the decrypt function.
$data = $_POST['data'];
$decryptedData = base64_decode($data);
$data = mcrypt_decrypt(MCRYPT_BLOWFISH, $myEncryptionKey, $decryptedData, MCRYPT_MODE_ECB);

Japanese/korean into MySQL table

i am trying to store some info in my db. One of the fields are Japanese names. I am getting a error:
Warning: #1366 Incorrect string value: '\xE3\x83\xA9\xE3\x83\x87...' for column 'japan-name' at row 1
So i cannot chnage the charset of my db. Can i use PHP or Javascript to convert Japanese/Korean to something else, and them when i go read it, reconvert to Japanese/Korean?

PHP offers the base64_encode() and base64_decode() functions. They are fast, and impose a storage penalty of about 33%. You can use the first to convert your utf-8 east Asian text to what looks like gibberish in ASCII before you store it in your table. The second will convert it back after you retrieve it.
Here's an example:
$jp = " 私はガラスを食べられます。それは私を傷つけません。";
$jpe = base64_encode ($jp);
$jpd = base64_decode ($jpe);
After you run these lines, the $jpe variable has the value
IOengeOBr+OCrOODqeOCueOCkumjn+OBueOCieOCjOOBvuOBmeOAguOBneOCjOOBr+engeOCkuWCt+OBpOOBkeOBvuOBm+OCk+OAgg==
That stores just fine in an ASCII or Latin-1 column.

utf-8 saves the unicode data in table... but other way is to encode and save and then decode and display
update:
searched on web and found answer at How do you Encrypt and Decrypt a PHP String?
define("ENCRYPTION_KEY", "!##$%^&*");
$string = "This is the original data string!";
echo $encrypted = encrypt($string, ENCRYPTION_KEY);
echo "<br />";
echo $decrypted = decrypt($encrypted, ENCRYPTION_KEY);
/**
* Returns an encrypted & utf8-encoded
*/
function encrypt($pure_string, $encryption_key) {
$iv_size = mcrypt_get_iv_size(MCRYPT_BLOWFISH, MCRYPT_MODE_ECB);
$iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
$encrypted_string = mcrypt_encrypt(MCRYPT_BLOWFISH, $encryption_key, utf8_encode($pure_string), MCRYPT_MODE_ECB, $iv);
return $encrypted_string;
}
/**
* Returns decrypted original string
*/
function decrypt($encrypted_string, $encryption_key) {
$iv_size = mcrypt_get_iv_size(MCRYPT_BLOWFISH, MCRYPT_MODE_ECB);
$iv = mcrypt_create_iv($iv_size, MCRYPT_RAND);
$decrypted_string = mcrypt_decrypt(MCRYPT_BLOWFISH, $encryption_key, $encrypted_string, MCRYPT_MODE_ECB, $iv);
return $decrypted_string;
}

Chunking MySQL array with json encode

I know there are existing some Questions about Chunking a mysql array in php, but my problem is, that I want to keep the output in JSON.
Scenario:
I want to get data from mysql, do some stuff with it ( like time formatting ) and output it in JSON.
The JSON data is parsed in the browser and visualized over a javascript chart.
Problem:
All of the above is working, but because of the huge amount of data, I'm getting an out of memory error, when I select bigger date ranges to output.
The Idea of directly sending out each x-lines of data is not working because of the JSON format it needs to be. Several JSON chunks won't work, it needs to be one for the chart.
So in the end I need to chunk the data but keep it as one big JSON.
(And setting up the memory limit is not really a solution.)
Ideas:
One Idea would be, to let the browser chunk the date range and ask the data as chunks & then put them together.
Of course this would work, but if there is a way to do this server side, it would be better.
Code:
private function getDB($date1, $date2){
$query = 'SELECT * FROM `db1`.`'.$table.'` WHERE `date` BETWEEN "'.$date1.'" AND "'.$date2.'" order by `date`;';
// date = datetime !
$result = $this->db->query($query);
$array = array();
while ( $row = $result->fetch_assoc () ) {
$array[] = array( strtotime($row[ 'date' ])*1000 , (float)$row[ 'var' ] );
// the formatting needs to be done, so the chart accepts it..
}
$result->close();
return json_encode($array);
}

Since this is not an option,
ini_set("memory_limit","32M")
perhaps you can add LIMIT to the function paramaters and query:
private function getDB($date1, $date2, $start, $pageSize){
$query = 'SELECT * FROM `db1`.`'.$table.'` WHERE `date` BETWEEN "'.$date1.'" AND "'.$date2.'" order by `date` LIMIT $start, $pageSize;';
// date = datetime !
$result = $this->db->query($query);
$array = array();
while ( $row = $result->fetch_assoc () ) {
$array[] = array( strtotime($row[ 'date' ])*1000 , (float)$row[ 'var' ] );
// the formatting needs to be done, so the chart accepts it..
}
$result->close();
return json_encode($array);
}
Then setup a for loop in javascript, call this with Ajax, incrementing the $start variable each time.
Store each responseText.substr(1).substr(-1) in an array.
When the responseText is "", all of the records have been returned.
.join the array with a comma, then add a new opening and closing "{ }", and you should have a JSON equivalent to all records.
Minimal parsing, and you'll be using built-in functions for most of it.
var startRec=0;
var pageSize=50;
var xmlhttp=new XMLHttpRequest();
var aryJSON=[];
var JSON;
xmlhttp.onreadystatechange=function(){
if (xmlhttp.readyState==4 && xmlhttp.status==200){
if(xmlhttp.responseText==""){ //Might need to check for "{}" here instead of ""
//All records are received
JSON="{" + aryJSON.join(",") + "}";
aryJSON=[];
startRec=0
}else{
aryJSON.push(xmlhttp.responseText.substr(1).substr(-1));
startRec+=pageSize;
getNextPage();
}
}
}
function getNextPage(){
xmlhttp.open("GET","your.php?start=" + startRec + "&pageSize=" + pageSize,true);
xmlhttp.send();
}

I would recommend that you have the server send the browser exactly what it needs to create the table. Parsing can be a heavy task, so why have the client do that lifting?
I would have your backend send the browser some kind of data structure that represents the table (i.e. list of lists), with all the formatting already done. Rendering the table should be faster and less memory-intensive.

One way of answer would be, to do the chunking on the server, by giving out the JSON, removing the leading [ & ].
#apache_setenv('no-gzip', 1);
#ini_set('zlib.output_compression', 0);
#ini_set('implicit_flush', 1);
$array = array();
echo '[';
$started = false;
while ( $row = $result->fetch_assoc () ) {
$array[] = [ strtotime($row[ 'datetime' ])*1000 , (float)$row[ 'var' ] ];
if(sizeof($array) == 1000){
if($started){
echo ',';
}else{
$started = true;
}
echo substr(substr(json_encode($array),1), 0, -1);
// converting [[datetime1, value1],[datetime2, value2]]
// to [datetime1, value1],[datetime2, value2]
ob_flush();
$array = array();
}
}
if($started)echo ',';
$this->flushJSON($array);
echo ']';
flush();
$result->close();
This is working and reducing the ram usage to 40%.
Still it seems that Apache is buffering something, so the ram usage increases over the time, the script is running. (Yeah, the flush is working, I debugged that, that's not the problem.)
But because of the remaining increase, the fastest way to achieve a clean chunking is to do this like alfadog67 pointed it out.
Also, to mention it, I had to disable the output compression, otherwise apache wouldn't flush it directly..