We Keep Coding

Search in 1 million <string Name, int score> pairs using sub-linear time

I have a JSON object which contains 1 million pairs.
var student = {[
{
name: "govi",
score: "65"
},
{
name: "dharti",
score: "80"
},
{
name: "Akash",
score: "75"
},............. till 1 million
]
};
Now My Concern as below.
I want to build a server program which accepts a user query such that for each query, it will responds with the top 10 names (ranked by score) that start with s or contains '_s' (so for example, both "revenue" and "yearly_revenue" match the prefix "rev"). it's too easy with normal Jquery and json program but there is a condition.
Condition
Query answering should run in sub-linear time (in terms of the number of names in the input).

1) Add the Newtonsoft library from the nuget rep
2) Add the following references
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
3) use this code
//JObjectString is your string that contains the values
JArray ValuesArray = JArray.Parse(JObjectString);
Dictionary<string, int> SearchDict = new Dictionary<string, int>();
//here the search term is the query from the user input
string searchTerm = "govi";
foreach (var rec in ValuesArray)
{
SearchDict.Add(rec["name"].ToString(), Int32.Parse(rec["score"].ToString()));
}
//here is the result in javascript array format, return it
string ResultString = JsonConvert.SerializeObject(SearchDict.Where(o => o.Key == searchTerm | o.Key.Contains(searchTerm)).
Select(o => o).OrderByDescending(o => o.Value).Take(10).Select(o => o));

sort your Array descending by score. this ensures that the next step creates subsets wich are already sorted, and is faster than sorting (often overlapping) subsets.
Create a dictionary by char. Go through each char of each name and add the item to the subset.
When searching for a substring (like "rev"), you can check the Arrays for each letter, take the shortest and can iterate through that, instead of the whole data.
an example:
//building the map
let map = Object.create(null);
data
//.slice() /if you can't rearrange the items in the original Array
.sort((a,b) => b.score - a.score)
.forEach(item => {
for(let char of new Set(item.name)){
var arr = map[char] ||| (map[char] = []);
arr.push(item);
}
});
function findSubset(str){
let best;
for(let char of new Set(str)){
//there can be no entry for "rev", if there is no subset for "v" for example
if(!(char in map)) return [];
let arr = map[char];
if(!best || arr.length < best.length)
best = arr;
}
return best || [];
}
function contains(str, limit=Infinity){
let subset = findSubset(str);
let results = [];
for(let i=0; i<subset.length && results.length < limit; ++i){
let item = subset[i];
if(item.name.includes(str))
results.push(item);
}
return results;
}

Calling StudentIndex.GetTop10(string s) will be O(n) where is n is the length of string s. This will consume a lot of memory, but there are a number of ways to cut this down. (Such as doing a linear search once Node.Top10 has less than 10 values).
I leave you to write the tree building.
public class Student
{
public string Name { get; set; }
public double Score { get; set; }
}
public class Node
{
public Dictionary<char, Node> Children { get; set; }
public List<Student> Top10 { get; set; }
}
public class StudentIndex
{
private Node _root;
public StudentIndex(IEnumerable<Student> students)
{
Node root = new Node();
foreach(var student in students)
{
var parts = student.Name.Split(new[] {'_'});
foreach(var part in parts)
{
//you'll add each student to the tree using each part of the name
}
}
//set _root
}
public IEnumerable<Student> GetTop10(string s)
{
return GetTop10(s.ToLower(), _root);
}
private IEnumerable<Student> GetTop10(string s, Node node)
{
if (node.Children == null) return node.Top10;
if (s.Length == 0) return node.Top10;
var c = s[0];
Node n;
if (node.Children.TryGetValue(c, out n))
{
return GetTop10(s.Substring(1), n);
}
else
{
return Enumerable.Empty<Student>();
}
}
}

Generate CHAP response in java

Currently I'm trying to generate a chap response in java. It works in php... but our backend needs it done in java.
var challenge = "cda9af6faa83d0883d694fa58d2e88wh";
var password = "password123";
var hexchal = challenge.packHex();
var newchal = hexchal;
var response = md5('\0' + password + newchal)
I've managed to find some javascript code, but the response is off by a few characters.
function getPapPassword(){
//function to add value padding to a string of x length default : null padding
String.prototype.pad = function (length, padding) {
var padding = typeof padding === 'string' && padding.length > 0 ? padding[0] : '\x00'
, length = isNaN(length) ? 0 : ~~length;
return this.length < length ? this + Array(length - this.length + 1).join(padding) : this;
};
//function to convert hex to ascii characters
String.prototype.packHex = function () {
var source = this.length % 2 ? this + '0' : this
, result = '';
for (var i = 0; i < source.length; i = i + 2) {
result += String.fromCharCode(parseInt(source.substr(i, 2), 16));
}
return result;
};
var challenge = "cda9af6faa83d0883d694fa58d2e88wh";
var password = "password123";
var hexchal = challenge.packHex();
var newchal = hexchal;
var response = md5('\0' + password + newchal);
}
return response;
}
Can anyone point me in the right direction or assist?
Thanks
Okay so what I've done is tried to convert from hex to ascii in java... here is my code.
public class App{
public String convertHexToString(String hex){
StringBuilder sb = new StringBuilder();
StringBuilder temp = new StringBuilder();
//49204c6f7665204a617661 split into two characters 49, 20, 4c...
for( int i=0; i<hex.length()-1; i+=2 ){
//grab the hex in pairs
String output = hex.substring(i, (i + 2));
//convert hex to decimal
int decimal = Integer.parseInt(output, 16);
//convert the decimal to character
sb.append((char)decimal);
temp.append(decimal);
}
System.out.println("Decimal : " + temp.toString());
return sb.toString();
}
public static void main(String[] args) {
App app = new App();
System.out.println("ASCII : " + app.convertHexToString("9f9585f4e88305fde280c762925f37af"));
}
}
The php of the hex challenge output is: Ÿ•…ôèƒýâ€Çb’_7¯
The java output of the hex challenge is: ôèýâÇb_7¯
So I've found that the javascript is exactly the same as the java. Something is happening in php that is changing it. The php version works perfectly when I connect to the UAM with that chap response.

Array with Apache Thrift using server Java & Javascript Client

im using Javascript to show info generated in Java.
Im talking of 10,000 datas, all of them generated in Java and for porpouse of testing im using random values. Well i want to see this 10,000 values in my javascript page.
I already have a simple Server Java and a Client Javascript to share 2 doubles.
This is the Thrift file for 2 doubles:
namespace java test_thrift
service test{
double number(1:double n1, 2:double n2)
}
This is the code of my Javascript Client
function calc() {
var transport = new Thrift.Transport("/service");
var protocol = new Thrift.Protocol(transport);
var client = new testClient(protocol);
var workbench = Math.random()*1000;
try {
result = client.number(workbench);
$('#result').val(result);
$('#result').css('color', 'black');
document.getElementById("demo").innerHTML = result;
} catch(ouch){
$('#result').val(ouch.why);
$('#result').css('color', 'red');
}
}
Im sending a Random only to get the range of the return. Example: 1 return a value from 3 to 9, 2 return a 9 to 15 value, etc.
And in java i have a testHandler class:
public double number(double n1, double n2) throws TException {
//System.out.println(n1 + " - " + n2);
Random rnd = new Random();
n1 = rnd.nextDouble() * 10 + 1;
n2 = rnd.nextDouble() * 15 + 10;
return n2;
}
Well this returns 1 value. And i want to see all in my Javascript Page. But with 10,000 elements. How can i do this?
Also i want to add that the final struct to share is like this:
dis[10000][3]={
ABC,12.5,13.5,
ACD,14.4,11.5,
.....ETC......}
Im stuck
Found this, but i dont know how to get it work :/
namespace java test_thrift
typedef list<double> Vector
struct test
{
1:i32 rows,
2:i32 cols,
3:list<Vector> data,
}

Use this IDL file
namespace java test_thrift
struct MyPair {
1: double one
2: double two
}
service test{
list<double> number( 1 : list<MyPair> data)
}
The function is then called like so:
var list = [];
for( var i = 0; i < 10000; ++i) {
list.push({
one : Math.random()*1000,
two : Math.random()*1000
});
}
result = client.number(list);
Result should then be your list of returned values, given of course the server end is implemented accordingly.

So, i managed to fix it. Here's a Client JavasCript that make a reequest and a Java server Give back a 10000 x 18 Matrix.
Thrift file:
namespace java test_thrift
struct Cell {
1: string did
2: double l_x
3: double l_y
4: double l_z
5: double m_x
6: double m_y
7: double m_z
8: double a_x
9: double a_y
10: double a_z
11: double g_x
12: double g_y
13: double g_z
14: string d_t
15: double tp
16: double r_q
17: string o_m
18: double b_v
}
service test{
list<Cell> number( 1 : i16 req)
}
Then in Javascript i send only the request:
function calc() {
var transport = new Thrift.Transport("/service");
var protocol = new Thrift.Protocol(transport);
var client = new pruebaClient(protocol);
var workbench = Math.random()*1000;
var div = document.getElementById('deltat');
try {
result = client.number(1);
div.innerHTML = div.innerHTML + '\nReady';
} catch(ouch){
$('#result').val("ERROR");
$('#result').css('color', 'red');
}
}
And in result y have the [10000][18] answer and print it like this:
for (var i = result.length - 1; i >= 0; i--) {
//div.innerHTML = div.innerHTML + '\n' + result[i];
div.insertAdjacentHTML('beforeend', '\n'+result[i].did+' '+result[i].l_x+' '+result[i].l_y+' '+result[i].l_z+' '+result[i].m_x+' '+result[i].m_y+' '+result[i].m_z+' '+result[i].a_x+' '+result[i].a_y+' '+result[i].a_z+' '+result[i].g_x+' '+result[i].g_y+' '+result[i].g_z+' '+result[i].d_t+' '+result[i].tp+' '+result[i].r_q+' '+result[i].o_m+' '+result[i].b_v);
};
}
Finally the handler in my Java Server is like this:
public class TestHandler implements test.Iface {
public Random rnd = new Random();
public static List<Cell> p = new ArrayList<Cell>();
public void test() {
for (int i = 0; i < 10000; i++) {
Cell a = new Cell();
a.did = "SomeString";
a.l_x = rnd.nextDouble()*10+1;
a.l_y = rnd.nextDouble()*10+1;
a.l_z = 0.0;
a.m_x = 0.0;
a.m_y = 0.0;
a.m_z = 0.0;
a.a_x = 0.0;
a.a_y = 0.0;
a.a_z = 0.0;
a.g_x = 0.0;
a.g_y = 0.0;
a.g_z = 0.0;
a.d_t = "String here";
a.tp = 0.0;
a.r_q = 0.0;
a.o_m = "A";
a.b_v = 0.0;
p.add(a);
}
}
#Override
public List<Cell> number(short req) throws TException {
test();
return ips.ReminderBeep.list_d;
}
I hope this will be useful for someone

How to get the string like this remove repeating in String Java

I am sending the arrays from jquery to controller, and this arrays I am getting as String in controller String getIdVal = ParamUtil.getString(resourceRequest, "getId");
The Values in String getIds are like getIdVal ="1_ABC,2_ABC,3_ABC,4_NMO,5_NMO,6_XYZ";
I am trying to get the result but no successes.
(Considering 1 is key ABC is value).
I wanted to remove ABC(ie repeated values) from all keys and append only once at the end. And at the same time I want the keys of repeated values should be like this (1-2-3_ABC). Finally the String should look like this "1-2-3_ABC,4-5_NMO,6_XYZ"
here I am try8ing to split based on comma, but I dont know how to solve.
List<String> keysVals = new ArrayList<String>(Arrays.asList(getIdVal .split(",")));
String getKeyVal;
String[] getKeys;
String key;
String value;
for (String getKeysVals : keysVals) {
getKeyVal = getKeysVals;
getKeys = getKeyVal.split("\\_");
key = getKeys[0];
value = getKeyVal.substring(getKeyVal.lastIndexOf("_") + 1) .trim();
// i am not getting how to check for dublicate
}

Did you mean something like this?
private static String mergeKeys(String input) {
Map<String, StringBuilder> valKeys = new TreeMap<>();
if (! input.isEmpty())
for (String keyVal : input.split(",")) {
int idx = keyVal.indexOf('_');
String key = keyVal.substring(0, idx);
String val = keyVal.substring(idx + 1);
StringBuilder builder = valKeys.get(val);
if (builder == null)
valKeys.put(val, new StringBuilder(key));
else
builder.append('-').append(key);
}
StringJoiner result = new StringJoiner(",");
for (Entry<String, StringBuilder> entry : valKeys.entrySet())
result.add(entry.getValue().append('_').append(entry.getKey()).toString());
return result.toString();
}
Or this slow version of the same logic, using string = string + string (Yikes!):
private static String mergeKeys(String input) {
Map<String, String> valKeys = new LinkedHashMap<>();
if (! input.isEmpty())
for (String keyVal : input.split(",")) {
int idx = keyVal.indexOf('_');
String key = keyVal.substring(0, idx);
String val = keyVal.substring(idx + 1);
String prevKeys = valKeys.get(val);
valKeys.put(val, prevKeys == null ? key : prevKeys + "-" + key);
}
String result = "";
for (Entry<String, String> entry : valKeys.entrySet())
result += "," + entry.getValue() + "_" + entry.getKey();
return result.substring(1); // skip leading comma
}
TEST
System.out.println(mergeKeys("1_ABC,2_ABC,3_ABC,4_NMO,5_NMO,6_XYZ"));
OUTPUT
1-2-3_ABC,4-5_NMO,6_XYZ

Extract RSA private key from Cryptico.js

I believe this is a pretty basic question, but I'm starting the studies in JavaScript and in RSA, so I'm a little bit lost. I just downloaded the library Cryptico, which gives me an easy to use RSA key gen/encryption/decryption. The public part of the generated RSA Key, can be extracted easily just using the command:
publicKeyString(RsaKey)
Which is:
my.publicKeyString = function(rsakey)
{
pubkey = my.b16to64(rsakey.n.toString(16));
return pubkey;
}
The rsakey.n is defined while generating the key in the function:
function RSAGenerate(B, E)
{
var rng = new SeededRandom();
var qs = B >> 1;
this.e = parseInt(E, 16);
var ee = new BigInteger(E, 16);
for (;;)
{
for (;;)
{
this.p = new BigInteger(B - qs, 1, rng);
if (this.p.subtract(BigInteger.ONE).gcd(ee).compareTo(BigInteger.ONE) == 0 && this.p.isProbablePrime(10)) break;
}
for (;;)
{
this.q = new BigInteger(qs, 1, rng);
if (this.q.subtract(BigInteger.ONE).gcd(ee).compareTo(BigInteger.ONE) == 0 && this.q.isProbablePrime(10)) break;
}
if (this.p.compareTo(this.q) <= 0)
{
var t = this.p;
this.p = this.q;
this.q = t;
}
var p1 = this.p.subtract(BigInteger.ONE);
var q1 = this.q.subtract(BigInteger.ONE);
var phi = p1.multiply(q1);
if (phi.gcd(ee).compareTo(BigInteger.ONE) == 0)
{
this.n = this.p.multiply(this.q);
this.d = ee.modInverse(phi);
this.dmp1 = this.d.mod(p1);
this.dmq1 = this.d.mod(q1);
this.coeff = this.q.modInverse(this.p);
break;
}
}
}
But the private part of the key, I just can't understand how to extract, so I'll be able to save public/private key parts and available for later using.
Library Documentation:
https://github.com/wwwtyro/cryptico

RSA is defined in such a way that the values contained in a public key make up a subset of the values contained in the private key. So your private key is already rsakey. Other public key schemes work differently where public and private key values are completely distinct.
Furthermore rsakey.n doesn't fully define the public key. You need at least the public exponent e. It is often simply set to 65537, so this might be hardcoded if you never need to change the key or upgrade the scheme. It's the E in RSAGenerate. It is ignored in this case, because
A (seeded) random RSA key is generated with Tom Wu's RSA key generator with 3 as a hard-coded public exponent.
You can choose a similar encoding of the private key as the public key, but since it has to hold multiple values, I opted for JSON serialization:
(function(c){
var parametersBigint = ["n", "d", "p", "q", "dmp1", "dmq1", "coeff"];
c.privateKeyString = function(rsakey) {
var keyObj = {};
parametersBigint.forEach(function(parameter){
keyObj[parameter] = c.b16to64(rsakey[parameter].toString(16));
});
// e is 3 implicitly
return JSON.stringify(keyObj);
}
c.privateKeyFromString = function(string) {
var keyObj = JSON.parse(string);
var rsa = new RSAKey();
parametersBigint.forEach(function(parameter){
rsa[parameter] = parseBigInt(c.b64to16(keyObj[parameter].split("|")[0]), 16);
});
rsa.e = parseInt("03", 16);
return rsa
}
})(cryptico)