Confused about _.memoize() implementation - javascript

I'm learning different implementations of the memoize function, and something is confusing me. Here is one version of the code:
_.memoize = function(func) {
let cache = {};
return function() {
let arg = JSON.stringify(arguments); //arg is an stringified array
if(!(arg in cache)) {
cache[arg] = func.apply(this, arguments);
}
return cache[arg];
};
};
My understanding of cahce[arg] is that the cache object has a key of arg, or if it doesn't, the function call of func.apply(this, arguments) becomes the property at an arg key. If arg is already a key in cache, than the function returns the property at cache[arg].
Ok, but isn't arg an array with a string in it? Assigned to a stringified version of arguments? Or is it a super long, single string formed by JSON.stringify?
How are the different arguments accessed? To me, this seems like the cache has a single key of either an array or a long string. Which means the identity of the individual arguments would be lost in a single piece of data, and I don't see how they are being accessed. I guess I would assume a for loop or forEach would loop through to access each argument, but that's not happening, so I don't really know how to read this.

Ok, but isn't arg an array with a string in it?
No, it is actually:
Or is it a super long, single string formed by JSON.stringify?
Yes. For example, if you memoize a function and then call it with
foo(1, 2)
foo(3, 4, 5)
foo('str')
the cache keys will then be composed of the following strings:
{"0":1,"1":2}
{"0":3,"1":4,"2":5}
{"0":"str"}
Each such line uniquely identifies a particular combination of arguments; there is a one-to-one mapping between a unique ordered number of arguments and the resulting stringified key. No information is lost; the fact that the arguments can be converted into a single string representing those arguments does not mean that any information about the arguments (their order or value) goes away; it's just put into a different format, that of a string, for the cache to work. (One-to-one correspondence is an incredibly useful technique in mathematics and algorithms. This is just one instance of it.)
I guess I would assume a for loop or forEach would loop through to access each argument, but that's not happening
That loop is happening, it's just implemented in native code, under the hood with JSON.stringify. But it only has to be done one way. Once an ordered set of arguments has been turned into a string, that string can then be used to uniquely identify the arguments, so the key can then be either looked up on the cache, or set on the cache.
function foo() {
console.dir(JSON.stringify(arguments));
}
foo(1, 2);

Related

How can I mitigate injection/exfiltration attacks from dynamic property accesses (i.e. square bracket notation) in JavaScript?

After setting up eslint-plugin-security, I went on to attempt to address nearly 400 uses of square brackets in our JavaScript codebase (flagged by the rule security/detect-object-injection). Although this plugin could be a lot more intelligent, any uses of square brackets could possibly be an opportunity for a malicious agent to inject their own code.
To understand how, and to understand the whole context of my question, you need to read this documentation: https://github.com/nodesecurity/eslint-plugin-security/blob/master/docs/the-dangers-of-square-bracket-notation.md
I generally tried to use Object.prototype.hasOwnProperty.call(someObject, someProperty) where I could to mitigate the chance that someProperty is maliciously set to constructor. Lot of situations were simply dereferencing an array index in for loops (for (let i=0;i<arr.length;i++) { arr[i] }) If i is always a number, this is obviously always safe.
One situation I don't think I have handled perfectly, are square bracket assignments like this:
someObject[somePropertyPotentiallyDefinedFromBackend] = someStringPotentiallyMaliciouslyDefinedString
I think the easiest way to solve this issue is with a simple util, safeKey defined as such:
// use window.safeKey = for easy tinkering in the console.
const safeKey = (() => {
// Safely allocate plainObject's inside iife
// Since this function may get called very frequently -
// I think it's important to have plainObject's
// statically defined
const obj = {};
const arr = [];
// ...if for some reason you ever use square brackets on these types...
// const fun = function() {}
// const bol = true;
// const num = 0;
// const str = '';
return key => {
// eslint-disable-next-line security/detect-object-injection
if (obj[key] !== undefined || arr[key] !== undefined
// ||
// fun[key] !== undefined ||
// bol[key] !== undefined ||
// num[key] !== undefined ||
// str[key] !== undefined
) {
return 'SAFE_'+key;
} else {
return key;
}
};
})();
You'd then use it like so:
someObject[safeKey(somePropertyPotentiallyDefinedFromBackend)] = someStringPotentiallyMaliciouslyDefinedString
This means if the backend incidentally sends JSON with a key somewhere of constructor we don't choke on it, and instead just use the key SAFE_constructor (lol). Also applies for any other pre-defined method/property, so now the backend doesn't have to worry about JSON keys colliding with natively defined JS properties/methods.
This utility function is nothing without a series of passing unit tests. As I've commented not all the tests are passing. I'm not sure which object(s) natively define toJSON - and this means it may need to be part of a hardcoded list of method/property names that have to be blacklisted. But I'm not sure how to find out every one of these property methods that needs to be blacklisted. So we need to know the best way anyone can generate this list, and keep it updated.
I did find that using Object.freeze(Object.prototype) helps, but I don't think methods like toJSON exist on the prototype.
How can we make sure the property being set is essentially not already defined on vanilla objects? (i.e. constructor)
It is more important to prevent a key from being accessed on the wrong object than to validate/protect object keys themselves. Designating certain object keys as ‘unsafe’ and avoiding accessing just these regardless of the circumstances is just another form of the ‘sanitizing’ anti-pattern. If the object doesn’t contain sensitive data in the first place, there is no risk of it being exfiltrated or modified by untrusted inputs. You don’t need to worry about accessing src or innerHTML if you don’t access it on a DOM node; you don’t need to worry about exposing eval if you don’t perform lookups on the global object. As such:
Only use bracket notation with objects that either are arrays or specifically contain nothing other than a mapping from arbitrary strings to other values (those usually constructed by object literal notation); this kind of object I’m going to call map-like below. When using map-like objects, also ensure the following:
that you never store functions (or, in later versions of ECMAScript, classes or proxies to either) directly in a map-like object. This is to avoid problems when keys like 'toJSON' or 'then' map to functions that the language may then interpret as methods that modify the behaviour of the object. If, for some reason, you need to store functions in a map-like object, put the function in a wrapper like { _: function () { /* ... */ } }.
that you never coerce map-like objects to strings using built-in language mechanisms: the toString method, the String constructor (with or without new), the + operator or Array.prototype.join. This is to avoid triggering problems when the 'toString' key is set on a map-like object, as even a non-function value will prevent the default coercion behaviour from occurring and will instead throw a TypeError.
When accessing arrays, make sure the index is indeed an integer. Consider also using built-in methods like push, forEach, map or filter that avoid explicit indexing altogether; this will reduce the number of places you will need to audit.
If you ever need to associate arbitrary data with an object with a relatively fixed set of keys, e.g. a DOM node, window or an object you defined with class (all of which I’ll call class-like below), and for some reason WeakMap is not available, put the data it on a hardcoded key; if you have more than one such data item, put it in a map-like object a stored on a hardcoded key.
Even when following the above though, you may still fall victim to an injection or exfiltration attack by inadvertently accessing a property of Object.prototype. Particularly worrying are constructor and various built-in methods (which can be leveraged to access the Function object, and ultimately perform arbitrary code execution), and __proto__ (which can be used to modify the prototype chain of an object). To protect against those threats, you may try some of the following strategies. They are not mutually exclusive, but for the sake of consistency it may be preferable to stick with just one.
Mangle all keys: this is probably the (conceptually) simplest option, portable even to engines from the days of ECMAScript 3, and robust even against future additions to Object.prototype (as unlikely as they are). Simply prepend a single non-identifier character to all keys in map-like objects; this will safely namespace away untrusted keys from all reasonably conceivable JavaScript built-ins (which presumably should have names that are valid identifiers). When accessing map-like objects, check for this character and strip it as appropriate. Following this strategy will even make concerns about methods like toJSON or toString mostly irrelevant.
// replacement for (key in maplike)
function maplikeContains(maplike, key) {
return ('.' + key) in maplike;
}
// replacement for (maplike[key])
function maplikeGet(maplike, key) {
return maplike['.' + key];
}
// replacement for (maplike[key] = value)
function maplikeSet(maplike, key, value) {
return maplike['.' + key] = value;
}
// replacement for the for-in loop
function maplikeEach(maplike, visit) {
for (var key in maplike) {
if (key.charAt(0) !== '.')
continue;
if (visit(key.substr(1), maplike[key]))
break;
}
}
Mangling all keys indiscriminately ensures that you will not end up confusing unmangled keys for mangled keys or vice versa. For example if, like in the question, you mangle 'constructor' into 'SAFE_constructor', but leave 'SAFE_constructor' itself as-is, then after mangling both keys will end up referring to the same data, which may be a security vulnerability in itself.
A drawback of this approach is that the prefix character is going to end up in JSON, if you ever serialise such a map-like object.
Enforce direct property access. Read accesses can be guarded with Object.prototype.hasOwnProperty, which will stop exfiltration vulnerabilities, but will not protect you from inadvertently writing to __proto__. If you never mutate such a map-like object, this should not be a problem. You can even enforce immutability using Object.seal. If don’t want that though, you may perform property writes via Object.defineProperty, available since ECMAScript 5, which can create properties directly on the object, bypassing getters and setters.
// replacement for (key in maplike)
function maplikeContains(maplike, key) {
return Object.prototype.hasOwnProperty.call(maplike, key);
}
// replacement for (maplike[key])
function maplikeGet(maplike, key) {
if (Object.prototype.hasOwnProperty.call(maplike, key))
return maplike[key];
}
// replacement for (maplike[key] = value)
function maplikeSet(maplike, key, value) {
Object.defineProperty(maplike, key, {
value: value,
writable: true,
enumerable: true,
configurable: true
});
return value;
}
// replacement for the for-in loop
function maplikeEach(maplike, visit) {
for (var key in maplike) {
if (!Object.prototype.hasOwnProperty.call(maplike, key))
continue;
if (visit(key, maplike[key]))
break;
}
}
Clear the prototype chain: make sure map-like objects have an empty prototype chain. Create them via Object.create(null) (available since ECMAScript 5) instead of {}. If you created them via direct object literals before, you can wrap them in Object.assign(Object.create(null), { /* ... */ }) (Object.assign is available since ECMAScript 6, but easily shimmable to earlier versions). If you follow this approach, you can use the bracket notation as usual; the only code you need to check is where you construct the map-like object.
Objects created by JSON.parse will by default will still inherit from Object.prototype (although modern engines at least add a JSON key like __proto__ directly on the constructed object itself, bypassing the setter from the prototype’s descriptor). You can either treat such objects as read-only, and guard read accesses by hasOwnProperty (as above), or strip away their prototypes by writing a reviver function that calls Object.setPrototypeOf. A reviver function can also use Object.seal to make an object immutable.
function maplikeNew(maplike) {
return Object.assign(Object.create(null), maplike);
}
function jsonParse(json) {
return JSON.parse(json, function (key, value) {
if (typeof value === 'object' && value !== null && !Array.isArray(value))
Object.setPrototypeOf(value, null);
return value;
});
}
Use Maps instead of map-like objects: using Map (available since ECMAScript 6) allows you to use keys other than strings, which isn’t possible with plain objects; but even just with string keys you have the benefit that entries of the map are completely isolated from the prototype chain of the map object itself. Items in Maps are accessed by .get and .set methods instead of the bracket notation and cannot clash with properties at all: the keys exist in a separate namespace.
There is however the problem that a Map cannot be directly serialized into JSON. You can remedy this by writing a replacer function for JSON.stringify that converts Maps into plain, prototype-free map-like objects, and a reviver function for JSON.parse that turns plain objects back into Maps. Then again, naïvely reviving every JSON object into a Map is also going to cover structures I called ‘class-like’ above, which you probably don’t want. To distinguish between them, you might need to add some kind of schema parameter to your JSON-parsing function.
function jsonParse(json) {
return JSON.parse(json, function (key, value) {
if (typeof value === 'object' && value !== null && !Array.isArray(value))
return new Map(Object.entries(value));
return value;
});
}
function jsonStringify(value) {
return JSON.stringify(value, function (key, value) {
if (value instanceof Map)
return Object.fromEntries(value.entries());
return value;
});
}
If you ask me for my preference: use Map if you don’t need to worry about pre-ES6 engines or JSON serialisation; otherwise use Object.create(null); and if you need to work with legacy JS engines where neither is possible, mangle keys (first option) and hope for the best.
Now, can all this discipline be enforced mechanically? Yes, and it’s called static typing. With good enough type definitions, TypeScript should be able to catch cases where class-like objects are accessed in map-like fashion and vice versa. It can even catch some cases where an object with an unwanted prototype appears:
type Maplike<T> = {
[K: string]: T|undefined;
constructor?: T;
propertyIsEnumerable?: T;
hasOwnProperty?: T;
toString?: T;
toLocaleString?: T;
valueOf?: T;
};
const plain: { [K: string]: string } = {};
const naked: Maplike<string> = Object.create(null); // OK
const error: Maplike<string> = {}; // type error
function f(k: string) {
naked[k] = 'yay'; // OK
plain[k] = 'yay'; // OK
document.body[k] = 'yay'; // type error
}
console.info(plain.toString()); // OK
console.info(naked.toString()); // type error
Bear in mind this is no panacea, however. The above type definition may catch the most obvious mistakes, but it’s not too hard to come up with a case which it will not detect.

How to determine operator placement

I am confused about placement of operators:
Given:
var a = [0, 1, 2];
So far as I can tell, each of the following is correct:
var len = a.length;
var lastElt = a.pop();
var str = String(a);
var typeStr = typeof a;
Is there an easy way to remember/think about whether the operator goes before or after the operand and whether it requires parentheses? Or is it simply rote memorization?
Thanks.
Let's go:
case #1
var len = a.length;
In this case you're calling the method lenght of the array store in the variable a.
case #2
var lastElt = a.pop();
As before, here you're calling the method pop of the array.
case #3
var str = String(a);
You aren't not calling any array's method. Here you're casting the array. i.e. you are stringifying the array.
case #4
var typeStr = typeof a;
Here you also aren't calling any array's method. You are calling the typeof method of the window object and passing the array a as an argument.
As you can see. In the first two cases, you are calling array's methods. In the last two cases, you are calling window object's methods and passing your array as an argument.
Your list there mostly consists of things that are not operators:
length is a property belonging to Array objects. It returns the number of items in the Array.
pop() is a method belonging to Array objects. It removes the last item from the Array and retuns it.
String() is a global object which is the constructor for a String object. It takes any object as a parameter and converts it to a String
typeof is an operator. It returns a string that indicates the type of the subsequent operand.
For more reference, here is some information about JavaScript operators.
I highly recommend searching Google for some beginning JavaScript tutorials so you can learn the basic concepts.
Thanks for all the prompt replies. In reply to the question 'What do you mean by "operator" here?' I come from a background of mathematics and (lately) c programming. By "operator" I wanted to speak abstractly about anything that mapped its argument to something useful, using it to include methods, properties, etc., without enumerating them. My attempt to abstract these is probably the source of my confusion. I now understand that what is required is rote memorization, something like declensions in Latin ;-).
Operators refer to +,=*,/ etc.
I don't think there is an easy way to remember. If it is a 'property', there will be no parentheses. If it is a function, there will be parentheses. I always remember that length is a property of an array, and that push and pop are actions you can do to the array. Type casting always starts with the 'type' capitalized followed by what you want casted in the parenthesis. typeof is just a weird one.

Rewrite following piece of javascript code

I am trying to create a function that mimics Array.prototype.push.
It takes a variable number of arguments and pushes them into a specific array.
I have managed to do this with the following code:
var array=[];
function append(){
for(var i=0;i<arguments.length;i++)
array.push(arguments[i]);
}
Now my question is:Can I rewrite the append function without using "for loop"?
Thanks in advance.
If you need to get arguments array, you should use Array's slice function on an arguments object, and it will convert it into a standard JavaScript array:
var array = Array.prototype.slice.call(arguments);
You could use Array.prototype.push.apply
function append(){
// make arguments an array
var args = Array.prototype.slice.call(arguments);
// return the number of elements pushed in the array
return Array.prototype.push.apply(array, args);
}
So, what's happening here with args? We use Array.prototype.slice.call with arguments, the purpose being to make arguments an array, because it is a special object. Function.prototype.call is used to call a function with a specific context (aka this), and then the arguments to call the function with (comma separated). Conveniently, it appears that slice() looks at the length property of the this context, and arguments has one too, and when not empty, has properties from 0 to length -1, which allows slice to copy arguments in a new array.
You can rewrite this without a for loop, but you have to use a loop of some sort (you're working with multiple items, it's a necessity).
If you have access to ES6 or Babel, I would use something like:
function append(...args) {
return array.concat(args);
}
Without ES6, you need to work around the fact that arguments isn't a real array. You can still apply most of the array methods to it, by accessing them through the Array prototype. Converting arguments into an array is easy enough, then you can concat the two:
function append() {
var args = Array.prototype.map.call(arguments, function (it) {
return it;
});
return array.concat(args);
}
Bear in mind that neither of these will modify the global array, but will return a new array with the combined values that can be used on its own or assigned back to array. This is somewhat easier and more robust than trying to work with push, if you're willing to array = append(...).
Actually i honestly believe that push must be redefined for the functional JS since it's returning value is the length of the resulting array and it's most of the time useless. Such as when it's needed to push a value and pass an array as a parameter to a function you cant do it inline and things get messy. Instead i would like it to return a reference to the array it's called upon or even a new array from where i can get the length information anyway. My new push proposal would be as follows;
Array.prototype.push = function(...args) {
return args.reduce(function(p,c) {
p[p.length] = c;
return p
}, this)
};
It returns a perfect reference to the array it's called upon.

Javascript How to overload a method and handle keys

I'm trying to implement a dictionary much like Python. So, I would like to have a keys() method that returns keys added to the subclass Dict, but not properties such as the Object's method "keys"
EDIT AGAIN
Basically, I'm making a class to pass settings to a function like function(arg1, arg2, myObj) where my object is {map: texMap, alphaMap: aTexMap}. It's for Three.js, and I have to wait on images to download before I can create settings on 3D objects. So, interface like one would expect with d in var d = { a: aData b: bData }, but hide the methods etc that are not added by the user.
ie don't return this.prototype.propertyName when own is passedHere's what I have so far:
function Dict(){
this.prototype = {};
var _keys = this.prototype.keys;
this.keys = function(own){
if(typeof own === 'undefined') { return _keys(); }
var ownKeys = [];
for(var key in _keys()){
if(this.hasOwnProperty(key)) {
ownKeys.push(key);
}
}
return ownKeys;
}
}
Will this work as follows? Is there a better or already existent way to do it?
save the overloaded keys() method to a private var
return everything as usual, unless own is something that resolves to true.
if own == true, get the usual keys and filter out those
belonging to the superclass.
On the subject, I'm likely most concerned about saving back the prototype method as a way to get all of the keys and filter out proto keys.
Also, I've read overloading isn't built into Javascript. But, much of what I've found deals with standalone functions such as this Q&A on best practices. I don't need a built in way, but I'll take advantage of whatever's available (Hence, using Object as a Dict).
Any feedback is appreciated!
EDIT
In Python, we get this:
In[2]: d = {}
In[3]: 'has_key' in d.keys()
Out[3]: False
In[7]: 'has_key' in d.__class__.__dict__.keys()
Out[7]: True
In[8]: d.has_key('has_key')
Out[8]: False
In[9]: d['newKey'] = 5
In[10]: d.newKey # ERROR
Python has a dict attribute contained in its class where the functions are accessed via a dot (see In[8]...). So, those standard {} or dict() functions and operators are hidden (not private) while keys/data are added to the user's dict are accessed via []. d['newKey'] = 5 adds a new key or overwrites the old and sets the data to 5.
I don't need all of that to work, though it would be great. keys() returning Python-like keys would be fine for now.
There seem to be multiple issues here.
You seem to want to pass variable arguments to a function:
I'm making a class to pass settings to a function like function(arg1, arg2, myObj) where my object is {map: texMap, alphaMap: aTexMap}.
JS function arguments are very flexible.
You can either set up names for every one of them:
function foo(arg1, arg2, map, alphaMap)
and pass values directly. This style is preferred for functions that work on a fixed set of arguments.
Or you can set up an "options" object that collects keys and values:
function foo(options)
and pass {arg1: val1, arg2: val2, map: valMap, alphaMap: valAlphaMap}. This style often occurs on constructor functions that initialize objects with a certain set configuration options.
Or you can set up an empty function signature
function foo()
and work with the arguments collection inside the function. This is found in functions that work with a variable number of uniform arguments (imagine add(1, 2, 3, 4, 6)) or strictly positional arguments instead of named ones.
In any case, passing arguments to a function is optional in JavaScript, even when there is an argument list in the function signature. You are free to pass none, less or more arguments. Of course all these approaches can be combined if it suits you.
It's for Three.js, and I have to wait on images to download before I can create settings on 3D objects.
This is a problem caused by the asynchronous nature of the web. The solution is to use event handlers. These are either callbacks or - as an abstraction over callbacks - promises.
So, interface like one would expect with d in var d = { a: aData b: bData }, but hide the methods etc that are not added by the user.
This can be solved by not adding methods etc to data objects, or at least not directly. Add them to the prototype if your data objects must have behavior.
The direct equivalent to a Python Dict is a plain object in JavaScript.
var dict = {};
The direct equivalent of Python's keys() method is the Object.keys() static method in JavaScript.
var keys = Object.keys(dict);
To iterate the keys you can either use an imperative approach:
var i, key;
for (i = 0; i < keys.length; i++) {
key = keys[i];
doSomething(key, dict[key]);
}
or a functional one
keys.forEach(function (key) {
doSomething(key, dict[key]);
});
The direct equivalent of Python's in is .hasOwnProperty() in JavaScript:
if ( dict.hasOwnProperty('foo') ) ...
or, if it is a pure data object with no prototype chain, you can use in as well.
if ('foo' in dict)
Using in in for loops is not recommendable because it iterates the prototype properties as well. The way to guard against this is by using Object.keys() instead or by combining it with .hasOwnProperty(), as you did.
var key;
for (key in dict) {
if ( dict.hasOwnProperty(key) ) ...
}
Your question indicates that you are missing basic puzzle pieces about JS and try to substitute them with more familiar Python constructs. I would recommend not doing that.
I also suspect that you try to shoehorn Python's class-based inhertiance pattern into JS' prototype-based inheritance pattern. I strongly recommend that you don't do that, either.

what happens when javascript array is treated like an object?

the index variable below is incorrectly initialized because f() will be returning stuff other than numbers, like strings. So what's the worst that can happen here? My testing seems to indicate that it has no effect, but now I am wondering...
function index(o, f) {
var index = []; // should be index = {};
each(o, function(k, v, o) { index[f(k, v, o)] = v; });
return index;
}
Javascript arrays are special objects that have an automatically set length property and inherit Array.prototype.
Unless you use a length property, there is no harm in treating an array as an object.
An array is an object, thus it can be treated as such without much side effects. Doing so however might result in some confusion, as the length property does not count non-numeric keys, and all the array prototype functions will likewise ignore them.
Just change [] to {}
You'll be creating an associative array, which is a valid JavaScript structure. Although, it is technically different than an object, you can interact with the array just like you would an object (for ... in to iterate, myarray[key] to fetch values). You may want to consider returning an object instead of an array if you suspect some keys will be strings.
"f() will be returning stuff other than numbers, like strings"
If f() only returns strings, then you're good to go, you're just using your array as an object and adding properties. The only downside is that the array itself remains empty, so for example you cannot count how many items you have added.
If f() can return both strings and numbers, it's going to create a mess. The loop will populate sometimes the array, sometimes the object properties.
I am not sure what you mean by "like strings", but if what f() returns is neither a number nor a string then it's not going to work.

Categories