Javascript Textarea Monitoring / Ruby Delta Calculation - javascript

I am working on a system which needs to keep constant (and I mean constant) track of browser side changes to a textarea via AJAX requests to the server. Ideally, every character would be stored as a row in the database, along with the position it was inserted and a timestamp.
I am hoping that there is either a good Javascript library that I have somehow missed which will make it trivial to do this all in the browser, but I think that inconsistencies in the DOM prevents one from doing so in any way which will be resource-reasonable. I'm a jQuery user, if that makes a difference.
The documents being created can get very large, so it is inefficient to send the entire document back and perform a diff on the server, but I think that I can work out a way to only send back the lines which are affected by an edit. Unfortunately, I do need a way to get per-character as opposed to per-line diffs calculated once it reaches the server.
I would like to use Ruby, so if there is a Ruby library that can do that, awesome. If not, is there a generic algorithm to calculating actual deltas between two strings that someone can suggest?
Summary:
Javascript Library for very tightly monitored textarea OR
Ruby library for calculating deltas OR
Generic delta calculation algorithm
In that order. Thank you in advance.

I think you can try having Ruby to call diff via the command line, then return those diff back to user ;)

Related

Options for dynamic code generation

I have a (hypothetical) question and I think the solution would be to dynamically generate code.
I want to quickly evaluate an arbitrary mathematical function that a user has entered, say to find the sum i=1 to N of i^3+2i^2+6i+1. N is arbitrary and i^3+2i^2+6i+1 is arbitrary too (it need not be a polynomial, and it might contain trigonometric functions and other functions too). Suppose N can be very large. I want to know how I can evaluate the answer quickly, assuming that I have already parsed the user input to some bytecode or something else my program can understand.
If possible, I would also like my code to be easily compiled and run on different operating systems (including mobile).
I have thought of a few ways:
1) Write an interpreter that interprets and executes each command in my bytecode. This makes me free to use any language, but it's slow.
2) Write in Java/C# and use the dynamic code generation (e.g. Is it possible to dynamically compile and execute C# code fragments?). This would execute as as fast as if I had written the function directly in my source code, with a only a slight slowdown as C#/Java are both JIT-compiled to machine code. The limitation is that Java isn't widely supported on mobile, and C# is Windows-only.
3) Embed an assembler/C++ compiler/compiler for whatever compiled language that I use. The limitation is that it won't work on mobile either - it won't let me execute a data file.
4) Write HTML/Javascript then embed it in a web browser control and put it in an application (I think this is the way some people use to make a universal app that would run anywhere). But it's slow too and writing real applications in Javascript is a pain.
Which option do you think is most suitable? Or perhaps I should go with a mix, maybe my application code will create and execute a generated Javascript function?
The fastest and simplest way to perform these calculations on large values of N are with raw maths instead of repeated summation.
Here's a formula to calculate each individual item in the expression, perform this for all items in the expression and you are done:
H[n] is the nth Harmonic number.
There are multiple approaches to calculating H[n]. Some calculate the largest required number and generate all up to that number, saving any other values required...
Alternately store every 10,000th item in the series in a file and calculate H[n] from the nearest entry.

Calculate the distance within php or js?

First of all, I am not sure this question should be in stackoverflow or other stackexchange network. So if wrong place just move for me.
It is better to calculate the distance between 2 points of lat/lng in DB or js?
Since Google maps provide computeDistanceBetween(). This is really easy to use, but i am thinking if i have 10000 rows and only 5 rows are within the distance and to display into the map.
Any idea?
If your question is only about where it should be calculated then I prefer client side. It is simple, fast, more flexible and dinamic, and you unload your server.
Without knowing much about your problem, I'd probably go with doing calculation-intensive tasks on the client side (or "in JS"). This will never put too much load on your server-side application, by distributing it among clients.
Of course there are many variables you have to take into account to choose the best approach.
Other things you may consider:
doing it server-side and caching the results,
using google-api and also caching it on your server,
many many more...
It depends to the frequency of calculations.
If your points are permanent then you can calculate all required distances only once before inserting the row into table and save the calculated result to provide it to clients later.
But if calculations are repeatable and coordinates of points will be changed eventually then it's better to use client-side calculations.
Really it's not that hard task for browser to execute some JavaScript which will calculate distances even for a lot of points.
Also if you have really tons of input data you can consider some background pre-calculations on client-side and caching of results in localStorage* so your users will never wait.
* store.js could help you to ensure cross-browser compatibility.
Where it is better to calculate distance is completely dependent on your application. If you the client has the information then it is better in JS.
In this scenario you appear to be trying to calculate the nearest points stored in your database to a given point. This is most certainly better handled by the database using geospatial indexes.
The typical algorithm narrows down the result set with a unsophisticated approach. Within x & y +/- 10KM and then with that result set do a full calculation.
I would say if your information is in the database then look into using built in geospatial tools for your DBMS.

create html elements on the serverside VS get data as JSON and create tags with javascript

I want to create a AJAX search to find and list topics in a forum (just topic link and subject).
The question is: Which one of the methods is better and faster?
GET threads list as a JSON string and convert it to an object, then loop over items and create a <li/> or <tr>, write data (link, subject) and append it to threads list. (jQuery Powered)
GET threads list which it wrapped in HTML tags and print it (or use innerHTML and $(e).html())
Thanks...
I prefer the second method.
I figure server-side you have to either convert your data to JSON or html format so why not go directly to the one the browser understands and avoid having to reprocess it client-side. Also you can easily adapt the second method to degrade gracefully for users who have disabled JavaScript (such that they still see the results via standard non-JS links.)
I'm not sure which way is better (I assume the second method is better as it would seem to touch the data less) but a definitive way to found out is try both ways and measure which one does better.
'Faster' is probably the second method.
'Better' is probably subjective.
For example, I've been in situations (as a front end dev) where I couldn't alter the html the server was returning and i wished they would have just delivered a json object so i could design the page how i wanted.
Also, (perhaps not specific to your use case), serving up all the html on initial page load could increase the page size and load time.
Server generated HTML is certainly faster if the javascript takes long time to process the JSON and populate the html.
However, for maintainability, JS is better. You can change HTML generation just by changing JS, not having to update server side code, making a delta release etc etc.
Best is to measure how slow it really is. Sometimes we think it is slow, but then you try it out in real world and you don't really see a big difference. You might have the major delay in transmitting the JSON object. That delay will still be there and infact increase if you send an html representation from the server.
So, if you bottleneck really is parsing JSON and generating html, not the transmission from server, then sending html from server makes sense.
However, you can do a lot of optimization in producing the html and parsing JSON. There are so many tricks to make that faster. Best if you show me the code and I can help you make a fast JS based implementation or can tell you to do it on the server.

JavaScript Games and Security

Let's say I'm making an HTML5 game using JavaScript and the <canvas> The varaibles are stored in the DOM such as level, exp, current_map, and the like.
Obviously, they can be edited client-side using Firebug. What would I have to do to maximize security, so it would be really hard to edit (and cheat)?
Don't store the variables in the DOM if you wish a reasonable level of security. JavaScript, even if obfuscated, can easily be reverse engineered. That defeats any local encryption mechanisms.
Store key variables server-side and use https to maximize security. Even so, the client code (JavaScript) is quite vulnerable to hacking.
You can use Object.freeze or a polyfill or a framework which does the hiding for you.
Check out http://netjs.codeplex.com/
You could also optionally implement some type of signing system but nothing is really impenetrable. For instance objects locked with Object.freeze or Object.watch can still be manually modified in memory.
What are you really trying to accomplish in the end?
What you could do is send a representation of the matrix of the game or the game itself or a special hash or a combination of both and tally the score at the server... causing the user to not only have to modify the score but to correctly modify the state of the game.
Server-side game logic
You need to keep the sensitive data on the server and a local copy on the browser for display purposes only. Then for every action that changes these values the server should be the one responsible for verifying them. For example if the player needs to solve a puzzle you should never verify the solution client side, but take for example the hash value of the ordered pieces represented as a string and send it to the server to verify that the hash value is correct. Then increase the xp/level of the player and send the information back to the client.
Anything that is living in the client can be modified. That is because in MMORPG the character's data is living on the server, so players can't hack their characters using any memory tools, hex editor, etc (they actually "can", but because the server keep the correct version of the character's data is useless).
A good example was Diablo 2: you have actually two different characters: one for single player (and Network playing with other players where one was the server), and one for Battle.net. In the first case, people could "hack" the character's level and points just editing the memory on the fly or the character file with an hex editor. But that wasn't possible with the character you was using on Battle.net.
Another simple example could be a quiz where you have a limited time to answer. If you handle everything on client side, players could hack it and modify the elapsed time and always get the best score: so you need to store the timestamp on the server as well, and use that value as comparison when you get the answer.
To sum up, it doesn't matter if it's JavaScript, C++ or Assembly: the rule is always "Don't rely on client". If you need security for you game data, you have to use something where the clients have no access: the server.

Ajax "Is there new content? If so, update page" - How to do this without breaking the server?

It's a simple case of a javascript that continuously asks "are there yet?" Like a four year old on a car drive.. But, much like parents, if you do this too often or, with too many kids at once, the server will buckle under pressure..
How do you solve the issue of having a webpage that looks for new content in the order of every 5 seconds and that allows for a larger number of visitors?
stackoverflow does it some way, don't know how though.
The more standard way would indeed be the javascript that looks for new content every few seconds.
A more advanced way would use a push-like technique, by using Comet techniques (long-polling and such). There's a lot of interesting stuff under that link.
I'm still waiting for a good opportunity to use it myself...
Oh, and here's a link from stackoverflow about it:
Is there some way to PUSH data from web server to browser?
In Java I used Ajax library (DWR) using Comet technology - I think you should search for library in PHP using it.
The idea is that server is sending one very long Http response and when it has something to send to the client it ends it and send new response with updated data.
Using it client doens't have to ping server every x seconds to get new data - I think it could help you.
You could make the poll time variable depending on the number of clients. Using your metaphor, the kid asks "Are we there yet?" and the driver responds "No, but maybe in an hour". Thankfully, Javascript isn't a stubborn kid so you can be sure he won't bug you until then.
You could consider polling every 5 seconds to start with, but after a while start to increase the poll interval time - perhaps up to some upper limit (1 minute, 5 minute - whatever seems optimal for your usage). The increase doesn't have to be linear.
A more sophisticated spin (which could incorporate monzee's suggestion to vary by number of clients), would be to allow the server to dictate the interval before next poll. The server could then increase the intervale over time, and you can even change the algorithm on the fly, or in response to network load.
You could take a look at the 'Twisted' framework in python. It's event-driven network programming framework that might satisfy what you are looking for. It can be used to push messages from the server.
Perhaps you can send a query to a real simple script, that doesn't need to make a real db-query, but only uses a simple timestamp to tell if there is anything new.
And then, if the answer is true, you can do a real query, where the server has to do real work !-)
I would have a single instance calling the DB and if a newer timestamp exists, put that new timestamp in a application variable. Then let all sessions check against that application variable. Or something like that. That way only one innstance are calling the sql-server and the number of clients does'nt matter.
I havent tried this and its just the first idéa on the top of the head but I think that cashe the timestamp and let the clients check the cashe is a way to do it, and how to implement the cashe (sql-server-cashe, application variable and so on) I dont know whats best.
Regarding how SO does it, note that it doesn't check for new answers continuously, only when you're typing into the "Your Answer" box.
The key then, is to first do a computationally cheap operation to weed out common "no update needed" cases (e.g., entering a new answer or checking a timestamp) before initiating a more expensive process to actually retrieve any changes.
Alternately, depending on your application, you may be able to resolve this by optimizing your change-publishing mechanism. For example, perhaps it might be feasible for changes (or summaries of them) to be put onto an RSS feed and have clients watch the feed instead of the real application. We can assume that this would be fairly efficient, as it's exactly the sort of thing RSS is designed and optimized for, plus it would have the additional benefit of making your application much more interoperable with the rest of the world at little or no cost to you.
I believe the approach shd be based on a combination of server-side sockets and client-side ajax/comet. Like:
Assume a chat application with several logged on users, and that each of them is listening via a slow-load AJAX call to the server-side listener script.
Whatever browser gets the just-entered data submits it to the server with an ajax call to a writer script. That server updates the database (or storage system) and posts a sockets write to noted listener script. The latter then gets the fresh data and posts it back to the client browser.
Now I haven't yet written this, and right now I dunno whether/how the browser limit of two concurrent connections screws up the above logic.
Will appreciate hearing fm anyone with thoughts here.
AS

Categories