Memory allocation of a Javascript array? - javascript

If I add a value to the 1000th element of a Javascript array, then is there any difference to adding that value to the 0th element assuming those positions are open?
I'm speaking in terms of memory consumption.
Example:
arr[1000] = 'asdf';
versus
arr[0] = 'asdf';

Due to JavaScript arrays actually being objects, memory is not contiguous. Thus, by your example, accessing array[1000] without ever storing anything elsewhere will take the same amount of memory as whatever you're storing (not 1000 * size of value).
In fact, doing var arr = new Array(1000); per Danny's answer does not create an array of 1000 empty slots. It creates an array with a length property of the value 1000.
Think about it this way: How is JavaScript to know how much memory to set aside if it doesn't know the type and size of what it's storing?
For example:
var arr = [];
arr[1000] = 'String value';
What's saying I can't come by and store an integer at another index?
arr[0] = 2;
Source:
https://stackoverflow.com/a/20323491/2506594

If you have an array of size 1000, you are taking up 1000 slots in memory.
var arr = new Array(1000);
The time complexity is still constant for array lookups, so that is not going to slow down your application.
However, you are explicitly asking for 1000 slots in memory, so it's still a sizeable piece of space. While memory, as hardware, is cheap, you should still aim to keep your arrays dynamic in size whenever possible to prevent your program from taking up unnecessary space in memory.

JavaScript objects don't really have an ordering, per se. Whether they're Arrays or Objects, the order of the keys isn't considered important; an implementation can put them in whatever order it wants. Of course, you have to store them in some kind of order, and so most implementations keep them in the order you inserted them in. But this is only a coincidence: the specs do not mandate it, and so you can't count on it always being true.
Because of this, arr[1000] isn't necessarily the 1000th element of the array: it's just a member with a key that happens to be 1000. It might indeed be the 1000th element, if you inserted 999 elements before it and your runtime keeps them in insertion order. But it could just as easily be, say, the 42nd element. It might even be the very first element, depending on both the runtime's implementation and your own code.
And since Arrays don't have an ordering, assigning to arr[1000] doesn't take up any more memory than assigning to arr[0], assuming that nothing else has been set yet. If you've already set 999 elements, then obviously setting arr[1000] will cause the array to take up a little more space than before, because you've got to store that element somewhere. But it doesn't take up any additional space just because its key is 1000.

Related

How does JavaScript(or NodeJS) handle memory allocation?

Assume V8 context.
Lets say an element of Number type has a size of 4 bytes.
Now, if I have an array,
let a = [1,2,3,4,5];
Logically speaking, the starting addresses of blocks for each element should be 1000,1004,1008,1012,1016
Now, lets say we have a string TestString which takes 10 bytes, and I do:
a[2] = 'TestString';
Such that the array becomes [1,2,'TestString',4,5].
How does JS handle the memory allocation of TestString and managing the address space of blocks in the array?
An array in JavaScript is really just a special type of object. As such, the "indexes" of the array are really just properties storing an integer value, and the data being stored are just pointers to the allocated memory blocks-- very much like a linked list.
This is why you can use array methods like push or pop on the existing array without re-allocating the block of memory set aside for the array.
I don't know the exact details of what V8 is doing, but I'm going to assume because of JavaScript's loosely/dynamically typed nature, it's probably not allocating memory contiguously like you used in your example-- there are just too many potential drawbacks to that for such a weak and dynamically typed language.
JavaScript basically makes everything an object. An array is an object, the index values of the array, just pointers to objects. Which is why every data type is a variable and has access to properties and methods.
In reality, the address spaces of:
let a = [1, 2, 3, 4, 5];
Would be pointers to the allocated memory blocks such as 548995, 48885, 3889282, 093838, 7883344. Or something like that. When you re-allocate any of them, JavaScript will find a block of memory in the heap and set the array index value to the allocated block's pointer.

Storage and 'weight' of Arrays and Object variables

So...
var testArray=new Array("hello");
testArray.length=100;
console.log(testArray.length);
I believe the above means I have created an array that has 100 elements. The first element contains the word hello, the others are null, but their position reserved. While small, I suspect reserving these "slots" uses memory.
What about
var myObj={ testArray: new Array(), testVar: "elvis", anotherArray: new Array() };
myObj.testArray.length=1000;
What is the impact or weight of this setup within javascript. Does the engine reserve three containers, each of similar size for testArray, testVar and anotherArray since they fall under myObj?
I tend to create a single global object (called DATA), and then I create variables under that. These variables contain temporary session data within an intranet app which is used several hours a day. Some of the variables are arrays, some are strings. I do it this way because with a single command DATA=null I can empty everything which would not be the case if I were to have several global variables.
I'm just wondering if my thinking is poor or acceptable/understandable. The answer will also help me to better understand how javascript stores data.
Comments welcome...
I believe the above means I have created an array that has 100
elements. The first element contains the word hello, the others are
null, but their position reserved. While small, I suspect reserving
these "slots" uses memory.
You created an array with 1 element, then you changed the length of that array to 100. However, that does not mean that you have 99 null elements; you only have 1 element in the array. In other words, the length property of the array does not necessary tell you the number of defined elements. The process to reserve this does take memory, but is negligible for a small number of elements. However, I do not recommend using the .length property as an assignment; you are really introducing the potential for some unexpected behavior in your code, as well as mismanaging resources. Instead,You should allow the array to grow and shrink as needed, using functions; .push() and .pop(), .splice(). By doing this, you are minimizing the amount of space required by the array and improving performance.
What is the impact or weight of this setup within JavaScript. Does the engine reserve three containers, each of similar size for testArray, testVar and anotherArray since they fall under myObj?
There are 3 containers that are created for the object.
1)DATA object gets a container2)testArray gets a container because you used the constructor approach (not best-practice)3)anotherArray container because you used the constructor approach (not best-practice)
In your example, DATA is the container for all of the name:value pairs that exist within this container. The "weight" is exactly the same as your first approach (except you are allocating 1000 slots, instead of 100).
I highly suggest that you do not have statements that assign a value to the length of the array. The array should only use as much space as is needed, no more and no less.
You should also create arrays with the literal approach: var array = []; NOT with the new keyword, constructor approach, as you are doing. By using the new keyword, you raise the potential to have bugs in your code. Also, JavaScript variables declared using the new keyword are always created as objects. See the below example.
var data = new Array(2, 10); // Creates an array with two elements (2 and 10)
var data = new Array(2); // Creates an array with 2 undefined elements
Avoid polluting the global namespace!!!! Using an object is a good approach in your situation, but it's very important to keep the global namespace clean and free of clutter as much as possible.
Further supporting resources:
MDN Arrays
MDN array.length
MDN Objects
Writing Fast, Memory-Efficient JavaScript

Can values be stored without allocating memory?

I have been reading here
https://developer.mozilla.org/en-US/docs/JavaScript/Memory_Management#Allocation_via_function_calls
and these lines confused me a bit:
var s = "azerty";
var s2 = s.substr(0, 3); // s2 is a new string
// Since strings are immutable value, JavaScript may decide
// to not allocate memory, but just store the [0, 3] range.
So the comments say, JavaScript may decide to not allocate memory and just store the range [0,3] , Now doesnt memory have to be allocated before storing? in case not, as the comment implies, what exactly happens so that the stored range, gets stored in free space, not occupied already by other values.
The comment means that s2 does not have to allocate memory specifically for the three characters "aze". Instead, it can use the memory already allocated for "azerty" by s and remember that its own length is just three characters.
The immutability part is also important: if s were not immutable then it could decide to change its value from "azerty" to "foobar" without telling anyone, thus also indirectly changing the value of s2 to "foo" -- which would be disastrous.
Of course this does not mean that no memory at all will be allocated; we still need to allocate memory for storing the location where the string's contents can be found, and for the string's length.
The comment is talking about allocating space for the character data of the string.
When you create a new variable it naturally has to allocate space for it, but that doesn't have to be something like a new object on the heap, it could be an item in a preallocated array of variables.
The benefit of having two variables point to the same character data is that getting a substring uses very little memory even if the length of the substring is long, and the drawback is that the original string will remain in memory as long as any substrings from it are used.

If I set only a high index in an array, does it waste memory?

In Javascript, if I do something like
var alpha = [];
alpha[1000000] = 2;
does this waste memory somehow? I remember reading something about Javascript arrays still setting values for unspecified indices (maybe sets them to undefined?), but I think this may have had something to do with delete. I can't really remember.
See this topic:
are-javascript-arrays-sparse
In most implementations of Javascript (probably all modern ones) arrays are sparse. That means no, it's not going to allocate memory up to the maximum index.
If it's anything like a Lua implementation there is actually an internal array and dictionary. Densely populated parts from the starting index will be stored in the array, sparse portions in the dictionary.
This is an old myth. The other indexes on the array will not be assigned.
When you assign a property name that is an "array index" (e.g. alpha[10] = 'foo', a name that represents an unsigned 32-bit integer) and it is greater than the current value of the length property of an Array object, two things will happen:
The "index named" property will be created on the object.
The length will be incremented to be that index + 1.
Proof of concept:
var alpha = [];
alpha[10] = 2;
alpha.hasOwnProperty(0); // false, the property doesn't exist
alpha.hasOwnProperty(9); // false
alpha.hasOwnProperty(10); // true, the property exist
alpha.length; // 11
As you can see, the hasOwnProperty method returns false when we test the presence of the 0 or 9 properties, because they don't exist physically on the object, whereas it returns true for 10, the property was created.
This misconception probably comes from popular JS consoles, like Firebug, because when they detect that the object being printed is an array-like one, they will simply make a loop, showing each of the index values from 0 to length - 1.
For example, Firebug detects array-like objects simply by looking if they have a length property whose its value is an unsigned 32-bit integer (less than 2^32 - 1), and if they have a splice property that is a function:
console.log({length:3, splice:function(){}});
// Firebug will log: `[undefined, undefined, undefined]`
In the above case, Firebug will internally make a sequential loop, to show each of the property values, but no one of the indexes really exist and showing [undefined, undefined, undefined] will give you the false sensation that those properties exist, or that they were "allocated", but that's not the case...
This has been like that since ever, it's specified even of the ECMAScript 1st Edition Specification (as of 1997), you shouldn't worry to have implementation differences.
About a year ago, I did some testing on how browsers handle arrays (obligatory self-promotional link to my blog post.) My testing was aimed more at CPU performance than at memory consumption, which is much harder to measure. The bottom line, though, was that every browser I tested with seemed to treat sparse arrays as hash tables. That is, unless you initialized the array from the get-go by putting values in consecutive indexes (starting from 0), the array would be implemented in a way that seemed to optimize for space.
So while there's no guarantee, I don't think that setting array[100000] will take any more room than setting array[1] -- unless you also set all the indexes leading up to those.
I dont think so because javascript treats arrays kinda like dictionaries, but with integer keys.
alpha[1000000] = alpha["1000000"]
I don't really know javascript, but it would be pretty odd behaviour if it DIDN'T allocate space for the entire array. Why would you think it wouldn't take up space? You're asking for a huge array. If it didn't give it to you, that would be a specific optimisation.
This obviously ignores OS optimisations such as memory overcommit and other kernel and implementation specifics.

How do arrays in JavaScript work ? (ie without a starting dimension size)

I was reading this post the other night about the inner workings of the Array, and learned a lot from the answers posted, especially from Jonathan Holland's one.
So the reason you give a size to an array beforehand is so that space will need to be reserved beforehand, so that elements in the array will be placed next each other in memory, and thus providing O(1) access time, because of the pointer + offset traversal.
But in JavaScript, you can initialize an array like such:
var anArray = []; //Initialize an empty array, without a dimension
So my question is, since in JavaScript you can initialize an array Without specifying a dimension before hand, how is memory allocated for an array to still provide a O(1) access time since the 'amount' of memory locations is not specified beforehand ?
Hmm. You should distinguish between arrays and associative arrays.
arrays:
A=[0,1,4,9,16];
associative arrays:
B={a:'ha',b:27,c:30};
The former has a length, the latter does not. When I run this in a javascript shell, I get:
js>A=[0,1,4,9,16];
0,1,4,9,16
js>A instanceof Array
true
js>A.length
5
js>B={a:'ha',b:27,c:30};
[object Object]
js>B instanceof Array
false
js>B.length
js>
How arrays "work" in Javascript is implementation-dependent. (Firefox and Microsoft and Opera and Google Chrome would all use different methods) My guess is they (arrays, not associative arrays) use something like STL's std::vector. Your question:
how is memory allocated for an array
to still provide a O(1) access time
since the 'amount' of memory locations
is not specified beforehand ?
is more along the lines of how std::vector (or similar resizable arrays) works. It reallocates to a larger array as necessary. Insertions at the end take amortized O(1) time, namely if you insert N elements where N is large, the total time takes N*O(1). Those individual inserts where it does have to resize the array may take longer, but on the average it takes O(1).
Arrays in Javascript are "fake". They are implemented as hash maps. So in the worst case their access time is not O(1). They also need more memory and you can use any string as an array index. You think that's weird? It is.
As I understand, it's like this:
There are two different things in JavaScript: Arrays and Objects. They both act as hashtables, although the underlying implementation is specific to each runtime. The difference between the two is that an Array has an implicit length property, while an object does not. Otherwise you can use [] or . syntaxes for both of them. Yes, that means that objects can have numerical properties and arrays have string indices. No problem. Although the length property might not be what you expect when using such tricks or sparse arrays. You should rely on it only if the array is not sparse and indices start from 0.
As for the performance - sorry, it's not the O(1) you'd expect. As stated before - it's actually implementation specific. But in general case it's not possible to ensure that there will be O(1) performance for all operations in a hashtable. That said, I'd expect that decent implementations should have quite a few optimizations in place for standard cases, which would make the performance quite close to O(1) under most scenarios. But at any rate - storing huge volumes of data in JavaScript is not a wise idea.
It is the same in PHP. I came from a PHP/Javascript background and dimensionalizing arrays really got me when i moved onto other languages.
javascript has no real arrays.
Elements are allocated as you define them.
It is a flexible tool. You can use the for many purposes but as a general purpose tool is not as efficient as special purpose arrays.
As Jason said, unless it is explicitly specified by the ECMAScript standard (unlikely), it is implementation dependent. The article shown by Feet shows that IE's implementation was poor (until IE8?), which is confirmed by JavaScript loop performance.
Other JS engines probably takes a more pragmatic approach. For example, they can do like in Lua, having a true array part and an associative array part: if the array is dense, it lives in the true array (which can still be extended at the cost of re-allocation), and you can still have sparse high indices living in the associative part.
Thus you have the best of two worlds, speed of access to dense parts and low memory use for sparse parts.
You can even do:
var asocArray = {key: 'val', 0: 'one', 1: 'two'}
and
var array = []; array['key'] = 'val'; array[0] = 'one'; array[1] = 'two';
When looping, you can use them in the same way using a for i in object loop, instead of using that for the asocArray and using a for var i=0; i<array.length; i++ loop for "array". The only difference here is that if you're expecting the indices in "array" to be a typeof i === 'number' or (i).constructor === Number then it will only be in the "for var i=0; i<array.length; i++ loop"; the for i in object loop makes all the keys (even indices) a String.

Categories