I have a (hypothetical) question and I think the solution would be to dynamically generate code.
I want to quickly evaluate an arbitrary mathematical function that a user has entered, say to find the sum i=1 to N of i^3+2i^2+6i+1. N is arbitrary and i^3+2i^2+6i+1 is arbitrary too (it need not be a polynomial, and it might contain trigonometric functions and other functions too). Suppose N can be very large. I want to know how I can evaluate the answer quickly, assuming that I have already parsed the user input to some bytecode or something else my program can understand.
If possible, I would also like my code to be easily compiled and run on different operating systems (including mobile).
I have thought of a few ways:
1) Write an interpreter that interprets and executes each command in my bytecode. This makes me free to use any language, but it's slow.
2) Write in Java/C# and use the dynamic code generation (e.g. Is it possible to dynamically compile and execute C# code fragments?). This would execute as as fast as if I had written the function directly in my source code, with a only a slight slowdown as C#/Java are both JIT-compiled to machine code. The limitation is that Java isn't widely supported on mobile, and C# is Windows-only.
3) Embed an assembler/C++ compiler/compiler for whatever compiled language that I use. The limitation is that it won't work on mobile either - it won't let me execute a data file.
4) Write HTML/Javascript then embed it in a web browser control and put it in an application (I think this is the way some people use to make a universal app that would run anywhere). But it's slow too and writing real applications in Javascript is a pain.
Which option do you think is most suitable? Or perhaps I should go with a mix, maybe my application code will create and execute a generated Javascript function?
The fastest and simplest way to perform these calculations on large values of N are with raw maths instead of repeated summation.
Here's a formula to calculate each individual item in the expression, perform this for all items in the expression and you are done:
H[n] is the nth Harmonic number.
There are multiple approaches to calculating H[n]. Some calculate the largest required number and generate all up to that number, saving any other values required...
Alternately store every 10,000th item in the series in a file and calculate H[n] from the nearest entry.
Related
I developed a web server in node.js and a web page (see Picture) hosted on this web server. The web server is embedded in the raspberry pi and I used a HAT with a D/A converter in order to generate voltages to an external circuit.
In the web page can be selected, if the user wants to use one channel or two channel of the DAC. Likewise, the parameters for the ramp generation from 0.0 to 5.0 V (incremental) or 5.0 V to 0.0 V (decremental). These settings are stored as a string in a vector which is passed to the web server using socket.io.
For the specific case of selecting two channels, the program executes a nested loop like the shown below. Where:
//i= Start Value (DAC0); val_max=Final value (DAC0); forSign="<=" or ">=" ;
//j= Start Value (DAC1); val_max=Final value (DAC1);
//incr = i='i+step' or i='i-step' --> step= Step Value (DAC0)
//incr1 = i='i+step' or i='i-step' --> step= Step Value (DAC1)
for(i; eval(i+forSign+val_max); i=eval(incr)){
for(j; eval(j+forSign+val_max1); j=eval(incr1)){
execSync(com_dac.replace(/dac0/g,j).replace(/dac1/g,i));
spawnSync("sudo",['./ads1256_test', 'adc.txt']);
ADC_read = fs.readFileSync("adc.txt").toString()
}
}
However, the performance of the for loop is not the best. Sometimes it is executed, but sometimes not. There is any suggestion to improve passing the parameters to the for loop? Taking into an account that i and j are values, while step, val_max(1) and forSign are string variables. I would appreciate any feedback :)
Is there any particular reason you used eval()? It should be avoided unless there is a very valid rationale behind the use of it.
First, if you're using eval() for arithmetic expression, JavaScript does this automatically. Needless to use eval().
Second, eval() lacks the performance boost of caching compiled codes that modern JavaScript engine offers. With limited ram such as a Pi, this will prove to be a performance issue.
Hi basically what I want to do is passing a JavaScript array to a c module function, then the function modify the array in place, then JavaScript reads the modified array.
Current approach is use carrays.i and array_functions, create and converting Array to and from doubleArray and due to copying array, its giving me result worse than native JS. My array have about 41000 items.
C module: ~10ms(actual C function running time ~0.1ms)
JS module: ~3ms
For me, it's not possible to use doubleArray from very beginning (as this is a part of a larger process). So the question is how can I improve it? Is it possible to use TypedArray/ArrayBuffer? If yes then how?
following is my pseudo code
let cArray = MyCModule.new_doubleArray(array.length),
outArray = new Array(array.length);
arrayCopyJS2C(cArray, array);//written in JS and use a lot of time
MyCModule.MyCFunction(cArray, array.length);
arrayCopyC2JS(cArray, outArray);//also written in JS and use a lot of time
Yes, using an ArrayBuffer (with externalized backing store) is an efficient way to share a (number) array between JavaScript and C, because it doesn't require you to copy things around. That's assuming that you can use a TypedArray "from the beginning" on the JavaScript side; if the same limitation applies as to using a doubleArray from the beginning and you'd still have to copy, then the benefit will be smaller or nonexistent (depending on how fast you've made accesses to your doubleArray).
That said, V8 generates highly efficient code for operations on number arrays. I'm finding it hard to believe that the same function takes either 3ms in JS or 0.1ms in C. Can you share your JS implementation? If a C implementation is 30x as fast, then I bet the JS implementation could be improved a lot to get pretty close to that. Array operations are usually dominated by the time it takes to actually get the elements from memory, and no language has a particular advantage at that.
Short: I need a way to save Formulas, so that I can execute them when I need it
Details: I am writing something for a eccommerce-system, so that the price of a product can be calculated by volume of the product. I want the backend-user (admin, seller of the product) to be able to set custom formulas for different ways of calculating the voluma. [e.g. (A x B - C x D) * E; A x B x (C - D);]. They differ in operations used (*,-,/,+) and in the amount of variables used in the formula.
I need a way to save this formulas (string is obviously a bad idea) in PHP, so that I can use them when I need them (set A,B,C,D,E to values and get the result) and also pass them by to Javascript and use them there too.
I appreciate any input on how this could be done.
Wow, nice problem! Here are some ideas.
a) Meta programming (is it the right word?)
In JS you can define a formula with a simple statement:
var f = new Function('a', 'alert(a)');
You can write a startup script which reads all the formula and load then dinamically (querying a webservice for example). This function could be called as expected:
f('Here I am');
There are many years since i learnt PHP but I think you could try dynamic includes. When user defines a formula you could generate a file on a special folder containing the PHP code representing it. If PHP allows function reference, the formulas could be accessed like:
$formulaBag['MyFormula']('Here I am')
Since you're allowing the user to insert code in your system, you should avoid "direct programming". Offer your user a small language for formula definition. When it is done, you trigger a process parse the formula-script and generate the corresponding JS and PHP code.
b) Dynamic parsing
It seems you have some performance concerns... if you define a simple language, parsing it in PHP would be not that difficult. Do you expect to run many formulas on a long-time-running process? If not, maybe this is a better option since you will not need to generate dynamic includes (security risk?). Calculating the formula would be something like this:
$res = ExecuteFormula($formula_name, $array_of_parameters)
[Added]
Parsing is not that difficult. You will need a Finity State Machine and a Stack to "reserve" temporary values when leading parentheses.
Well, I hope it helps.
I am working on a system which needs to keep constant (and I mean constant) track of browser side changes to a textarea via AJAX requests to the server. Ideally, every character would be stored as a row in the database, along with the position it was inserted and a timestamp.
I am hoping that there is either a good Javascript library that I have somehow missed which will make it trivial to do this all in the browser, but I think that inconsistencies in the DOM prevents one from doing so in any way which will be resource-reasonable. I'm a jQuery user, if that makes a difference.
The documents being created can get very large, so it is inefficient to send the entire document back and perform a diff on the server, but I think that I can work out a way to only send back the lines which are affected by an edit. Unfortunately, I do need a way to get per-character as opposed to per-line diffs calculated once it reaches the server.
I would like to use Ruby, so if there is a Ruby library that can do that, awesome. If not, is there a generic algorithm to calculating actual deltas between two strings that someone can suggest?
Summary:
Javascript Library for very tightly monitored textarea OR
Ruby library for calculating deltas OR
Generic delta calculation algorithm
In that order. Thank you in advance.
I think you can try having Ruby to call diff via the command line, then return those diff back to user ;)
I would like to write a JavaScript function that validates a zip code, by checking if the zip code actually exists. Here is a list of all zip codes:
http://www.census.gov/tiger/tms/gazetteer/zips.txt (I only care about the 2nd column)
This is really a compression problem. I would like to do this for fun. OK, now that's out of the way, here is a list of optimizations over a straight hashtable that I can think of, feel free to add anything I have not thought of:
Break zipcode into 2 parts, first 2 digits and last 3 digits.
Make a giant if-else statement first checking the first 2 digits, then checking ranges within the last 3 digits.
Or, covert the zips into hex, and see if I can do the same thing using smaller groups.
Find out if within the range of all valid zip codes there are more valid zip codes vs invalid zip codes. Write the above code targeting the smaller group.
Break up the hash into separate files, and load them via Ajax as user types in the zipcode. So perhaps break into 2 parts, first for first 2 digits, second for last 3.
Lastly, I plan to generate the JavaScript files using another program, not by hand.
Edit: performance matters here. I do want to use this, if it doesn't suck. Performance of the JavaScript code execution + download time.
Edit 2: JavaScript only solutions please. I don't have access to the application server, plus, that would make this into a whole other problem =)
You could do the unthinkable and treat the code as a number (remember that it's not actually a number). Convert your list into a series of ranges, for example:
zips = [10000, 10001, 10002, 10003, 23001, 23002, 23003, 36001]
// becomes
zips = [[10000,10003], [23001,23003], [36001,36001]]
// make sure to keep this sorted
then to test:
myzip = 23002;
for (i = 0, l = zips.length; i < l; ++i) {
if (myzip >= zips[i][0] && myzip <= zips[i][1]) {
return true;
}
}
return false;
this is just using a very naive linear search (O(n)). If you kept the list sorted and used binary searching, you could achieve O(log n).
I would like to write a JavaScript function that validates a zip code
Might be more effort than it's worth, keeping it updated so that at no point someone's real valid ZIP code is rejected. You could also try an external service, or do what everyone else does and just accept any 5-digit number!
here is a list of optimizations over a straight hashtable that I can think of
Sorry to spoil the potential Fun, but you're probably not going to manage much better actual performance than JavaScript's Object gives you when used as a hashtable. Object member access is one of the most common operations in JS and will be super-optimised; building your own data structures is unlikely to beat it even if they are potentially better structures from a computer science point of view. In particular, anything using ‘Array’ is not going to perform as well as you think because Array is actually implemented as an Object (hashtable) itself.
Having said that, a possible space compression tool if you only need to know 'valid or not' would be to use a 100000-bit bitfield, packed into a string. For example for a space of only 100 ZIP codes, where codes 032-043 are ‘valid’:
var zipfield= '\x00\x00\x00\x00\xFF\x0F\x00\x00\x00\x00\x00\x00\x00';
function isvalid(zip) {
if (!zip.match('[0-9]{3}'))
return false;
var z= parseInt(zip, 10);
return !!( zipfield.charCodeAt(Math.floor(z/8)) & (1<<(z%8)) );
}
Now we just have to work out the most efficient way to get the bitfield to the script. The naive '\x00'-filled version above is pretty inefficient. Conventional approaches to reducing that would be eg. to base64-encode it:
var zipfield= atob('AAAAAP8PAAAAAAAAAA==');
That would get the 100000 flags down to 16.6kB. Unfortunately atob is Mozilla-only, so an additional base64 decoder would be needed for other browsers. (It's not too hard, but it's a bit more startup time to decode.) It might also be possible to use an AJAX request to transfer a direct binary string (encoded in ISO-8859-1 text to responseText). That would get it down to 12.5kB.
But in reality probably anything, even the naive version, would do as long as you served the script using mod_deflate, which would compress away a lot of that redundancy, and also the repetition of '\x00' for all the long ranges of ‘invalid’ codes.
I use Google Maps API to check whether a zipcode exists.
It's more accurate.
Assuming you've got the zips in a sorted array (seems fair if you're controlling the generation of the datastructure), see if a simple binary search is fast enough.
So... You're doing client side validation and want to optimize for file size? you probably cannot beat general compression. Fortunately, most browsers support gzip for you, so you can use that much for free.
How about a simple json coded dict or list with the zip codes in sorted order and do a look up on the dict. it'll compress well, since its a predictable sequence, import easily since it's json, using the browsers in-built parser, and lookup will probably be very fast also, since that's a javascript primitive.
This might be useful:
PHP Zip Code Range and Distance Calculation
As well as List of postal codes.