proper ways to reduce loading big hashtable - javascript

i have a JavaScript app that contains big hashtable (1 megabyte). What's the proper way to load it to reduce loading time?
the hash table is this:
function unicodeTable (num) {
// returns the unicode name of a given unicode num in decimal
var unicodedata =
{
0x0020:"SPACE",
// tens of thousand entries here
}
...
}
the app is a Unicode browser. When user hovers over a Unicode, it displays its name and codepoint.
PS I'd like the solution not involving some js lib. Thanks.
Addendum:
the page is here
http://xahlee.info/comp/unicode_6_emoticons_list.html
this is the JavaScript
http://xahlee.info/comp/unicode_popup.js
on desktop/laptop it's fine, but i noticed on a Blackberry tablet, it froze browser for 5 or so minutes. I'm not sure how to use ajax, or perhaps worker is the answer?

Load it asynchronously, like using AJAX
Use a CDN if possible
Define it in a scope that does not terminate. This avoids JS from creating and destroying it everytime.
function foo(){
//creating bar
var bar = 'someValue';
//using bar here
//bar will be destroyed as soon as foo is done
}
But in this one, bar lives since it's outside foo:
//creating bar
var bar = 'someValue';
function foo(){
//using bar here
//bar still lives after foo is done
}
Offload processing (search, accessing, traversal) to a WebWorker or a simulated worker using timers.

first, make sure you compress the responses with gzip. that will save a lot of space since texts compress really well. Second, there is no need to have a hashtable. you can just have an array with the names at their correct location and when someone hovers the space element in the page, you go in this array at [20] and retrieve the name. as i see there, that will save 7 bytes per char. so a bit from here and a bit from there it can add up. but that's about all i can think of.

Related

How to cut off function from global scope

I have an idea for a game where people can type in some simple instructions for their character like player.goLeft() or player.attackInFront() and for that I have people type their code into a text box and then I parse it into eval(). This works well but it also allows people to change their own character object by typing things like player.health = Infinity; or something similar. I have a list of functions I want to allow people to use, but I am unsure how to restrict it to only use them.
I understand that the whole point of not letting people use eval is to avoid accidental cross-site scripting but I am unsure on how else to do this. If you have a suggestion please leave a comment about that.
I asked some people around on what to do and most suggested somehow changing scope(which is something I was not able to figure out) or to add some odd parameter to each function in my code that would be required to be a specific string to execute any function, but that seems hacky and since I am making the game in browser with p5js it would be easy to just inspect element and see what the password is.
basically every character has variable called "instruction" which is just a string of javascript. Then every frame of the game I execute it by doing eval(playerList[i].instruction);
tl;dr, how can I only allow specific function to be executed and not allow any others?
EDIT: I forgot to mention that I also am planning to provide player with information so that people can made code that would adapt to the situation. For example there will be parameter called vision that has vision.front and vision.left etc. These variables would just say if there is an enemy, wall, flower, etc around them in a grid. Some people suggested that I just replace some functions with key words but then it compromises the idea of using if statements and making it act differently.
EDIT 2: Sorry for lack of code in this post, but because of the way I am making it, half of the logic is written on server side and half of it works on client side. It will be a little large and to be completely honest I am not sure how readable my code is, still so far I am getting great help and I am very thankful for it. Thank you to everybody who is answering
Do NOT use eval() to execute arbitrary user input as code! There's no way to allow your code to run a function but prevent eval() from doing the same.
Instead, what you should do is make a map of commands the player can use, mapping them to functions. That way, you run the function based on the map lookup, but if it's not in the map, it can't be run. You can even allow arguments by splitting the string at spaces and spreading the array over the function parameters. Something like this:
const instructions = {
goLeft: player.goLeft.bind(player),
goRight: player.goRight.bind(player),
attackInFront: player.attackInFront.bind(player)
};
function processInstruction(instruction_string) {
const pieces = instruction_string.split(' ');
const command = pieces[0];
const args = pieces.slice(1);
if (instructions[command]) {
instructions[command](...args);
} else {
// Notify the user their command is not recognized.
}
};
With that, the player can enter things like goLeft 5 6 and it will call player.goLeft(5,6), but if they try to enter otherFunction 20 40 it will just say it's unrecognized, since otherFunction isn't in the map.
This issue sounds similar to the SQL Injection problem. I suggest you use a similar solution. Create an abstraction layer between the users input and your execution, similar to using parameters with stored procedures.
Let the users type keywords such as 'ATTACK FRONT', then pass that input to a function which parses the string, looks for keywords, then passes back 'player.attackInFront()' to be evaluated.
With this approach you simplify the syntax for the users, and limit the possible actions to those you allow.
I hope this isn't too vague. Good luck!
From your edit, it sounds like you're looking for an object-oriented approach to players. I'm not sure of your existing implementation needs, but it would look like this.
function Player() {
this.vision = {
left: '',
// and so on
}
}
Player.prototype.updateVisibilities = function() {
// to modify the values of this.visibility for each player
}
Player.prototype.moveLeft = function() {
}
Don't give the user an arbitrary interface (such as an input textfield that uses eval) to modify their attributes. Make a UI layer to control this logic. Things like buttons, inputs which explicitly run functions/methods that operate on the player. It shouldn't be up to the player as to what attributes they should have.

Replace text using an array of placeholder : replacement pairs in JavaScript

This is a simple problem (I am new to JavaScript and have a limited knowledge of the syntax and using arrays etc.), so I am sure someone more knowledgeable will be able to advise the simplest solution fairly easily!
I would like to replace a number of text placeholders in an existing Google Doc template with variable text inputs, which I ultimately plan to populate from one or more external sources via APIs (such as a form).
function replaceAllPlaceholders() {
var body = DocumentApp.getActiveDocument().getBody(); //defines the range within which to replace text
body.replaceText('placeholder1', 'replacement1');
body.replaceText('placeholder2', 'replacement2');
body.replaceText('placeholder3', 'replacement3');
// ...
body.replaceText('placeholder98', 'replacement98');
body.replaceText('placeholder99', 'replacement99'); }
Rather than repeat the replaceText( function for each replacement as I have done above, how can I instead layout the information out as an array of placeholder:replacement pairs, and then loop through each?
// for example something like this (pseudo):
//
// var obj = {
// 'placeholder1': 'replacement1' // I would like to keep open the option to retrieve this array from an external source instead
// 'placeholder2': 'replacement2'
// 'placeholder3': 'replacement3' };
//
// body.replaceText(*all placeholders*,*all replacements*);
I imagine this would allow greater flexibility in editing the set of placeholders and or replacements going forward, either directly within Google Apps Script or by replacing the whole array to one retrieved from an external source (as well as reducing the code required). The problem is I have not been able to figure out the correct method to do this. Any suggestions?
Alternatively, is there a better way to achieve my goal?
I am open to all recommendations!
Try this
var placeholders = [
['placeholder1', 'replacement1'],
['placeholder2', 'replacement2'],
['placeholder3', 'replacement3']
];
placeholders.forEach(function(pair) {
body.replaceText(pair[0], pair[1]);
});

What is the maximum file size javascript can process? [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Javascript Memory Limit
I'm working on creating html page using client side javascript which will load around 150mb of XML data file on page load. When the file size was around 5 mb, it took 30 seconds to load whole data into an array. But when I changed file to 140 mb, the page is not responding in firefox and crashing abruptly in chrome. My snippet to load data will process on every individual tag in xml. My question is, is there any limited heap size for javascript? Any academic article resource is preferable to emphasize my research.
$(document).ready(function () {
// Open the xml file
$.get("xyz.xml", {}, function (xml) {
// Run the function for each in the XML file
$('abc', xml).each(function (i) {
a = $(this).find("a").text();
b = $(this).find("b").text();
c = $(this).find("c").text();
ab = $(this).find("ab").text();
bc = $(this).find("bc").text();
cd = $(this).find("cd").text();
de = $(this).find("de").text();
// process data
dosomething(a,b,c,ab,bc,cd,de);
}); }); });
I don't know of any limits. I've been able to load even a 1Gb file. Yes, it was slow to load initially and everything ran slowly because most of the memory will be paged.
However, there are problems with trying to load a single JavaScript object that is that big, mostly because the parsers can't parse an object that is too big. See Have I reached the limits of the size of objects JavaScript in my browser can handle?
For that case, the solution was to break up the creation of the JavaScript object into multiple stages rather than using a single literal statement.
Because it's to much to improve. First of all i'd like to recommend you a post 76 bytes for faster jQuery. So, relying on that replace your $(this) on $_(this).
it will save you a lot memory and time!!
If you don't want to use single jQuery object, please cashe you variable like that:
$('abc', xml).each(function (i) {
var $this = $(this);
a = $this.find("a").text();
....
});
and you can provide your dosomething function to try to improve it

jQuery: add() performance; is there a better way?

What I want to do: Group all the like elements on a page (of a certain kind) into an object which I can later iterate on -- or apply sweeping changes to every element within.
My code is successful at accomplishing the given task but when the number of elements grows to 200-300+ then the performance drastically drops off and users have noticed. I have isolated the offending lines of code and want to know if there is another way of accomplishing the same problem.
The add() function appears to be the problematic operation based on timers I have placed around them. At first the time required to perform the operation is .001 but grows until the number of elements reaches 300 and it takes ~.1 of a second for each additional element AND continues slowing down.
I have researched (and more) for jQuery performance enhancing abilities and have implemented a few of them (namely 3) but they have not given me any meaningful performance increases. Amazingly, this code performs within 1 second (!) for Firefox (300+ calls to add()) while Chrome and IE take roughly 10-20x longer or more...
Here is my code:
rowsToChange = $([]);
// Grab all the ids greater than whichever one I'm currently looking at:
var arr = $.makeArray($("[id^=stackLocatorLinkFillUp]:gt("+(uniqueID-1)+")"));
for (var i=0; i<arr.length; i++) {
$this = arr[i];
// <<< VARIOUS CONDITIONALS that make this as selective as possible REMOVED >>>
startTimer = new Date().getTime();
// **************************
// PROBLEMATIC LINE FOLLOWS when 200+ records:
rowsToChange = rowsToChange.add($this);
// Grows from .001 to .1xx after 300 iterations
console.log("innertiming:"+(new Date().getTime() - startTimer)/1000);
// **************************
}
The end result looks like this (via Chrome Inspector):
[<div style=​"display:​none" id=​"stackLocatorLinkFillUp1">​itemType=BOUND&ccLocale=PERIODICAL​</div>​,
<div style=​"display:​none" id=​"stackLocatorLinkFillUp2">​itemType=BOUND&ccLocale=PERIODICAL​</div>​,
...
]
Eventually I process all these as follows (which I love the simplicity of!):
var superlink = "...new <a> goodness to display for all elements...";
rowsToChange.html(superlink).css("display","block");
This looked like it could be a valid solution (different add method?) but I would prefer to continue gathering a list of objects together so that the last line can work its magic.
(am not i am pointed out that the following is not true -- regarding concatenation; thanks 'am not i am')
It seems like the add() operation must be concatenating strings since that appears to be one of the main problems others face. But transforming my add() statement into += doesn't look like it works.
Thanks for checking this out;
Chrome: 18.0.1025.142 m
Firefox: 11.0
IE: 8.0.7600.16385
First observation: add saves the previous element set. Try rowsToChange = jQuery.merge(rowsToChange, [$this]); instead.
Second observation: it seems as though rowsToChange will end up being the exact same element set as the one you called $.makeArray on. Why not just save the original set?
DCoder shows how to appropriately merge the information together if you are using a for loop. However, if you come here and are using a .each() loop, use what follows.
The main difference is that brackets are unnecessary / necessary depending on the structure of 'this'. It also seems to be generally accepted that .each() is at least slightly slower than the native javascript for loop. (evidence from 2009) (timing test_copied from question above)
var $this, rowsToChange = $([]);
// slower than a for loop
$("[id^=stackLocatorLinkFillUp]:gt("+(uniqueID-1)+")").each( function() {
// If statements <removed> that decide whether or not to include in the new container
$this = $(this); // probably unnecessary under most situations
rowsToChange = jQuery.merge(rowsToChange, $this);
});
Operate on every piece of the new sub-group decided upon by the removed if statements!
rowsToChange.html("...");
Thanks to everyone who viewed the question, took the time to answer, voted it up, etc.!

JS 1-2k variables make page load slow

On one page of my website the user has the ability to choose and remove up to 2000 items through selecting multiple string representations of them in a dropdown list.
On page load, the objects are loaded onto the page from a previous session into 7 different drop-down lists.
In the window.onload event, the function looping through the items in the drop-downs makes an internal collection of the objects by adding them to a global array - This makes the page ridiculously slow to load, so, I'm fairly certain probably doing it wrong!
How else am I supposed to store these variables?
This is their internal representation:
function Permission(PName, DCID, ID) {
this.PName = PName;
this.DCID = DCID;
this.ID = ID;
}
where: PName is string. DCID is int. ID is int.
EDIT:
Thanks for the quick replies! I appreciate the help, I'm not great with JS! Here is more information:
'selectChangeEvent' is added to the Change and Click event of the Drop down list.
function selectChangeEvent(e) {
//...
addListItem(id);
//...
}
'addListItem(id)' sets up the visual representation of the objects and then calls :
function addListObject(x, idOfCaller) {
var arIDOfCaller = idOfCaller.toString().split('-');
if (arIDOfCaller[0] == "selLocs") {
var loc = new AccessLocation(x, arIDOfCaller[1]);
arrayLocations[GlobalIndexLocations] = loc;
GlobalIndexLocations++;
totalLocations++;
}
else {
var perm = new Permission(x, arIDOfCaller[1], arIDOfCaller[2]);
arrayPermissions[GlobalIndexPermissions] = perm;
GlobalIndexPermissions++;
totalPermissions++;
}
}
Still not enough to go on, but there are some small improvements I can see.
Instead of this pattern:
var loc = new AccessLocation(x, arIDOfCaller[1]);
arrayLocations[GlobalIndexLocations] = loc;
GlobalIndexLocations++;
totalLocations++;
which seems to involve redundant counters and has surplus assignment operations, try:
arrayLocations[arrayLocations.length] = new AccessLocation(x, arIDOfCaller[1]);
and just use arrayLocations.length where you would refer to GlobalIndexLocations or totalLocations (which fromt he code above would seem to always be the same value).
That should gain you a little boost, but this is not your main problem. I suggest you add some debugging Date objects to work out where the bottleneck is.
You may want to consider a design change to support the load. Some sort of paged result set or similar, to cut down on the number of concurrent records being modified.
As much as we desperately want them to be, browsers aren't quite there yet in terms of script execution speed that allow us to do certain types of heavy lifting on the client.
While I haven't tested this idea, I figured I'd throw it out there - might it be faster to return a JSON string from the server side, where your array is fully calculated on that side?
From that point, I'd wager that eval()'ing it (as evil as this may be) might be fast enough to where you could then write the contents onto the page, and your array setup would already be taken care of.
Then again, I suppose the amount of work it'd take the browser to construct the 2k new objects and inject them into the DOM wouldn't necessarily help the speed side of things in the end. At the end of the day, a design change is probably necessary, but sometimes we're stuck with what we've got, eh?

Categories