Related
I have an array of objects containing world countries with some additional information e.g.
countries = [
{
flag: 'assets/flags/angola.svg',
code: 'AGO',
name: 'Angola',
regions: [{
name: 'Luanda'
}]
},
{
flag: 'assets/flags/albania.svg',
code: 'ALB',
name: 'Albania',
regions: [{
name: 'Korça'
}, {
name: 'Tirana'
}, {
name: 'Gjirokastër'
}]
}...
I want to extract three of my favorite countries into a new array while removing them from the original array so I end up with two arrays one for my favorite countries and one for the rest of the countries.
I managed to achieve this the following way:
public createCountriesList(allCountries: Country[]) {
let topCountries: Country[] = [];
let restOfCountries: Country[];
allCountries.forEach((element) => {
switch (element.code) {
case 'HRV':
topCountries.push(element);
break;
case 'AT':
topCountries.push(element);
break;
case 'GER':
topCountries.push(element);
break;
}
});
restOfCountries = allCountries.filter((c) => {
return !topCountries.includes(c);
});}
It works, but I was wondering if there is a more elegant way to do this?
Everything seems fine according to me...
Obviously you need two arrays one for the extracted ones and second for rest of countries.
One thing we can work on is the switch case.
Instead of switch case you can use .includes function.
Store the name of countries you want to extract in an array.
const arr = ['HRV','AT','GR']
now you can do,
if(arr.includes(element.code)){
//push into new array
} else{
//push into another
}
One more thing you can do is save restOfTheCountries using .filter function.
Just return true for the countries which fails your above if case.
You can just use regular filter to split the array:
const isTop = ({code}) => ['HRV','AT','GR'].includes(code);
const topCountries = allCountries.filter(isTop);
const restOfCountries = allCountries.filter((contry) => !isTop(contry));
Another way, you can add a property that shows whether this country is top or not, and filter by this key
const withTop = countries.map((e) => ({...e, top: ['AGO','AT','GR'].includes(e.code)}));
// {
// code: "AGO"
// flag: "assets/flags/angola.svg"
// name: "Angola"
// regions: [{…}]
// top: true
// }
I would probably create a separate generic function for splitting array based on the criteria (using ts since you are)
const splitArray = <T>(array: Array<T>, matchFunction: (el: T) => boolean) => {
const matching: T[] = [], nonMatching: T[] = []
array.forEach(el => matchFunction(el) ? matching.push(el) : nonMatching.push(el))
return [matching, nonMatching]
}
then you can call it with the array and a function
const [topCountries, restOfCountries] = splitArray(countries, c => ["HRV", "AT", "GER"].includes(c.code))
that would be a bit more readable. a more elegant solution is to extend Array with that functionality (Array.prototype.split) then using countries.split(c => ["HRV", "AT", "GER"].includes(c.code))
All suggestions so far seem valid, I ended up using #Alexandr Belan answer as it was the most straightforward to implement in my case, I don't know why I used switch case instead of filter like for the topCountries 🤷♂️. Final code (added alphabetical sorting of the countries as well)
const suggestedCountriesList = ['Brazil', 'France', 'Germany'];
const suggested = ({ name }) => suggestedCountriesList.includes(name);
const suggestedCountries = allCountries.filter(suggested);
const otherCountries = allCountries.filter((country) => !suggested(country));
// sort alphabetically
otherCountries.sort((a, b) => a.name.toLowerCase().localeCompare(b.name.toLowerCase()));
suggestedCountries.sort((a, b) => a.name.toLowerCase().localeCompare(b.name.toLowerCase()));
return [suggestedCountries, otherCountries];
I have multiple options and apparently, the third party api doesn't like when I try to pass variables that aren't defined. It'll give me a signature error!
what's required is side, symbol, order_type, qty and time_in_force
and that's what I'm passing in the example below
mutation {
setActiveOrder(
side: "Buy",
symbol: "BTCUSD",
order_type: "Market",
qty: 1,
time_in_force: "GoodTillCancel"
) {
data
}
}
but sometimes (order_type === "Limit") and price is required
sometimes there will be either or both take_profit and stop_loss for either of the two possibilities above.
that being said I'm trying to figure out the best way to optionally pass variables without writing if statements for every possibility the hard way
I'd love another perspective
settActiveOrder: async (
_,
{
side,
symbol,
order_type,
qty,
price,
time_in_force,
take_profit,
stop_loss,
reduce_only,
close_on_trigger,
order_link_id,
},
ctx
) => {
const setActiveOrder = async () => {
try {
return await cclient.placeActiveOrder({
side: side,
symbol: symbol,
order_type: order_type,
qty: qty,
// price: price,
time_in_force: time_in_force,
// take_profit: take_profit,
// stop_loss: stop_loss,
// reduce_only: reduce_only,
// close_on_trigger: close_on_trigger,
// order_link_id: order_link_id,
});
} catch (error) {
console.error(error);
}
};
const data = await setActiveOrder();
return { data: data };
},
If you need to build a new object (as you do in your example), and the api doesn't handle 'undefined' values itself, then you do need a way of checking the input and build the object conditionally, but you can do it in a loop.
To clarify, here your code again, with comments:
settActiveOrder: async (
_,
// here you are using "object destructuring".
// Not existing properties will become a valid,
// existing variable with the value >>undefined<<
{ side, symbol, order_type, qty, /* ... */ },
ctx
) => {
/* ... */
// here your are building a new object,
// possibly having existing properties with the value >>undefined<<,
// but you want these properties to not exist
return await cclient.placeActiveOrder({
side: side,
symbol: symbol,
order_type: order_type,
qty: qty,
/* ... */
});
/* ... */
};
Solution:
you could reference the original object, instead of using object destructuring, and loop over its properties, e.g.:
settActiveOrder: async ( _, originalParams, ctx ) => { // <-- no object destructuring
/* ... */
// build new object in a loop, but skip undefined properties
const passedParams = Object.keys( originalParams ).reduce(( acc, key ) => {
if( typeof originalParams[ key ] === 'undefined' ){
return acc;
} else {
return {
...acc,
[key]: originalParams[ key ]
}
}
}, {});
return await cclient.placeActiveOrder( passedParams );
/* ... */
});
Alternative solution:
In case you just need to pass all defined properties, and do not really have to build the object new, then you could simply pass the whole object as it is, instead of using the object destructuring.
Then the not-existing-properties will stay not-existing-properties, instead of becoming existing-propterties-with-value-undefined.
settActiveOrder: async ( _, originalParams, ctx ) => {
/* ... */
return await cclient.placeActiveOrder( originalParams );
/* ... */
});
I'm afraid you won't avoid if statements,
but better approach than writing all combinations would be to declare dictionary and append prefered variables for request
Something like this:
var dictForRequest = {
side: side,
symbol: symbol,
order_type: order_type,
qty: qty,
time_in_force: time_in_force,
};
if(wantPrice)
dictForRequest["price"] = price;
if(wantTakeProfit)
dictForRequest["take_profit"] = take_profit;
....
Than pass dictForRequest to
...
const setActiveOrder = async () => {
try {
return await cclient.placeActiveOrder(dictForRequest);
} catch (error) {
console.error(error);
}
You can use ternary operators. which are if else statements but shorter and easier to write.
{
price: price ? price : 0,
}
// The above code translates to:
if(price) {
price = price
} else {
price = 0
}
Also, with ES6 you don't necessarily have to write it like this:
{
side: side,
symbol: symbol,
order_type: order_type,
}
// These two are the same. You can write it like below.
{
side,
symbol,
order_type
}
The above code is valid as long as the name of the key and the variable name passed to the key as value are the same. JS can figure out and map the right value to the right key, but they must be named exactly the same. Also, this rule is case sensitive. So, price: Price is not the same. it has to be price: price for this rule to hold true.
If you are looking for another option apart from ternary operators, then you can initialize your variables when you declare them. But, initialize them with the let keyword instead of const. Since, const variables are immutable -> let price = 0;
Consider that we have an 'api_fetchData' which fetch data from server, and based on route, it could be table or a tree, also we have two state that we should update them based on the received data. and i should note that if the data was a table we should sort it's records by its priority. i wonder how should i do that in functionl programming way(like use ramda or either ...)
const tree=null;
and
const table = null:
//table
{
type: "table",
records: [
{
id: 1,
priority: 15
},
{
id: 2,
priority: 3
}
]
}
//tree
{
type: "tree",
children: [
{
type: "table",
records: [
{
id: 1,
priority: 15
},
{
id: 2,
priority: 3
}
]
}
]
}
This is what i did:
//define utils
const Right = x => ({
map: f => Right(f(x)),
fold: (f, g) => g(x),
inspect: x => `Right(${x})`
});
const Left = x => ({
map: f => Left(x),
fold: (f, g) => f(x),
inspect: x => `Left(${x})`
});
const fromNullable = x => (x != null ? Right(x) : Left(null));
const state = reactive({
path: context.root.$route.path,
tree: null,
table: null
});
// final code:
const fetchData = () => {
return fromNullable(mocked_firewall[state.path])
.map(data =>
data.type === "tree"
? (state.tree = data)
: (state.table = R.mergeRight(data, {
records: prioritySort(data.records)
}))
)
.fold(
() => console.log("there is no data based on selected route"),
x => console.log(x)
);
};
You have several problems here -
safely accessing deeply nested data on an uncertain object
sorting an array of complex data using functional techniques
safely and immutably updating deeply nested data on an uncertain object
1. Using .map and .chain is low-level and additional abstraction should be added when syntax becomes painful –
const state =
{ tree:
{ type: "sequoia"
, children: [ "a", "b", "c" ]
}
}
recSafeProp(state, [ "tree", "type" ])
.getOrElse("?") // "sequoia"
recSafeProp(state, [ "tree", "foobar" ])
.getOrElse("?") // "?"
recSafeProp(state, [ "tree", "children", 0 ])
.getOrElse("?") // "a"
recSafeProp(state, [ "tree", "children", 1 ])
.getOrElse("?") // "b"
recSafeProp(state, [ "tree", "children", 99 ])
.getOrElse("?") // "?"
We can implement recSafeProp easily, inventing our own convenience function, safeProp, along the way –
const { Nothing, fromNullable } =
require("data.maybe")
const recSafeProp = (o = {}, props = []) =>
props.reduce // for each props as p
( (r, p) => // safely lookup p on child
r.chain(child => safeProp(child, p))
, fromNullable(o) // init with Maybe o
)
const safeProp = (o = {}, p = "") =>
Object(o) === o // if o is an object
? fromNullable(o[p]) // wrap o[p] Maybe
: Nothing() // o is not an object, return Nothing
2. The Comparison module. You seem to have familiarity with functional modules so let's dive right in –
const { empty, map } =
Comparison
const prioritySort =
map(empty, record => record.priority || 0)
// or...
const prioritySort =
map(empty, ({ priority = 0}) => priority)
myarr.sort(prioritySort)
If you want immutable sort –
const isort = ([ ...copy ], compare = empty) =>
copy.sort(compare)
const newArr =
isort(origArr, proritySort)
Here's the Comparison module. It is introduced in this Q&A. Check it out there if you're interested to see how to make complex sorters in a functional way –
const Comparison =
{ empty: (a, b) =>
a < b ? -1
: a > b ? 1
: 0
, map: (m, f) =>
(a, b) => m(f(a), f(b))
, concat: (m, n) =>
(a, b) => Ordered.concat(m(a, b), n(a, b))
, reverse: (m) =>
(a, b) => m(b, a)
, nsort: (...m) =>
m.reduce(Comparison.concat, Comparison.empty)
}
const Ordered =
{ empty: 0
, concat: (a, b) =>
a === 0 ? b : a
}
Combining sorters –
const { empty, map, concat } =
Comparison
const sortByProp = (prop = "") =>
map(empty, (o = {}) => o[prop])
const sortByFullName =
concat
( sortByProp("lastName") // primary: sort by obj.lastName
, sortByProp("firstName") // secondary: sort by obj.firstName
)
data.sort(sortByFullName) // ...
Composable sorters –
const { ..., reverse } =
Comparison
// sort by `name` then reverse sort by `age` –
data.sort(concat(sortByName, reverse(sortByAge)))
Functional prinicples –
// this...
concat(reverse(sortByName), reverse(sortByAge))
// is the same as...
reverse(concat(sortByName, sortByAge))
As well as –
const { ..., nsort } =
Comparison
// this...
concat(sortByYear, concat(sortByMonth, sortByDay))
// is the same as...
concat(concat(sortByYear, sortByMonth), sortByDay)
// is the same as...
nsort(sortByYear, sortByMonth, sortByDay)
3. The lowest hanging fruit to reach for here is Immutable.js. If I find more time later, I'll show a lo-fi approach to the specific problem.
Your solution seems overkill to me.
Functional programming is about many things, and there is no concise definition, but if your main unit of work is the pure function and if you don't mutate data, you're well on the way to being a functional programmer. Using an Either monad has some powerful benefits at times. But this code looks more like an attempt to fit Either into a place where it makes little sense.
Below is one suggested way to write this code. But I've had to make a few assumptions along the way. First, as you discuss fetching data from a server, I'm assuming that this has to run in an asynchronous mode, using Promises, async-await or a nicer alternative like Tasks or Futures. I further assumed that where you mention mocked_firewall is where you are doing the actual asynchronous call. (Your sample code treated it as an object where you could look up results from a path; but I can't really make sense of that for mimicking a real service.) And one more assumption: the fold(() => log(...), x => log(x)) bit was nothing essential, only a demonstration that your code did what it was supposed to do.
With all that in mind, I wrote a version with an object mapping type to functions, each one taking data and state and returning a new state, and with the central function fetchData that takes something like your mocked_firewall (or really like my alteration of that to return a Promise) and returns a function that accepts a state and returns a new state.
It looks like this:
// State-handling functions
const handlers = {
tree: (data, state) => ({... state, tree: data}),
table: (data, state) => ({
... state,
table: {... data, records: prioritySort (data .records)}
})
}
// Main function
const fetchData = (firewall) => (state) =>
firewall (state)
.then ((data) => (handlers [data .type] || ((data, state) => state)) (data, state))
// Demonstrations
fetchData (mocked_firewall) ({path: 'foo', table: null, tree: null})
.then (console .log) // updates the tree
fetchData (mocked_firewall) ({path: 'bar', table: null, tree: null})
.then (console .log) // updates the table
fetchData (mocked_firewall) ({path: 'baz', table: null, tree: null})
.then (console .log) // server returns type we can't handle; no update
fetchData (mocked_firewall) ({path: 'qux', table: null, tree: null})
.then (console .log) // server returns no `type`; no update
.as-console-wrapper {min-height: 100% !important; top: 0}
<script>
// Dummy functions -- only for demonstration purposes
const prioritySort = (records) =>
records .slice (0) .sort (({priority: p1}, {priority: p2}) => p1 - p2)
const mocked_firewall = ({path}) =>
Promise .resolve ({
foo: {
type: "tree",
children: [{
type: "table",
records: [{id: 1, priority: 15}, {id: 2, priority: 3}]
}]
},
bar: {
type: 'table',
records: [{id: 1, priority: 7}, {id: 2, priority: 1}, {id: 3, priority: 4}]
},
baz: {type: 'unknown', z: 45},
} [path] || {})
</script>
You will notice that this does not alter the state; instead it returns a new object for the state. I see that this is tagged vue, and as I understand it, that is not how Vue works. (This is one of the reasons I haven't really used Vue, in fact.) You could easily change the handlers to update the state in place, with something like tree: (data, state) => {state.tree = data; return state}, or even skipping the return. But don't let any FP zealots catch you doing this; remember that the functional programmers are well versed in "key algorithmic techniques such as recursion and condescension." 1.
You also tagged this ramda.js. I'm one of the founders of Ramda, and a big fan, but I see Ramda helping here only around the edges. I included for instance, a naive version of the prioritySort that you mentioned but didn't supply. A Ramda version would probably be nicer, something like
const prioritySort = sortBy (prop ('priority'))
Similarly, if you don't want to mutate the state, we could probably rewrite the handler functions with Ramda versions, possibly simplifying a bit. But that would be minor. For the main function, I don't see anything that would be improved by using Ramda functions.
There is a good testability argument to be made that we should pass not just the firewall to the main function but also the handlers object. That's entirely up to you, but it does make it easier to mock and test the parts independently. If you don't want to do that, it is entirely possible to inline them in the main function like this:
const fetchData = (firewall) => (state) =>
firewall (state)
.then ((data) => (({
tree: (data, state) => ({... state, tree: data}),
table: (data, state) => ({
... state,
table: {...data, records: prioritySort(data .records)}
})
}) [data .type] || ((data, state) => state)) (data, state))
But in the end, I find the original easier to read, either as is, or with the handlers supplied as another parameter of the main function.
1 The original quote is from Verity Stob, but I know it best from James Iry's wonderful A Brief, Incomplete, and Mostly Wrong History of Programming Languages.
🎈 Short and simple way
In my example I use ramda, you need to compose some functions and voila ✨:
const prioritySort = R.compose(
R.when(
R.propEq('type', 'tree'),
R.over(
R.lensProp('children'),
R.map(child => prioritySort(child))
)
),
R.when(
R.propEq('type', 'table'),
R.over(
R.lensProp('records'),
R.sortBy(R.prop('priority')),
)
),
)
const fetchData = R.pipe(
endpoint => fetch(`https://your-api.com/${endpoint}`, opts).then(res => res.json()),
R.andThen(prioritySort),
)
fetchData(`tree`).then(console.log)
fetchData(`table`).then(console.log)
Check the demo
For inspection you can simple use function const log = value => console.log(value) || value
R.pipe(
// some code
log,
// some code
)
it will log the piping value.
There is a question about for await ... of loop.
I have the following code:
for await (let V of reagentItems.map(objectElement => getPricing(objectElement))) {
console.log(objectElement) //I'd like to have access to the current iteration.
//or
console.log(reagentItems[i]) //current fulfilled element from reagent.map
}
So the problem is that this array (from .map) of functions (V as a single response of getPricing) doesn't return the objectElement itself but I need to have access to this objectElement inside for await ... of loop.
Does it possible somehow or not? And if it doesn't, using of Promise.all handles this problem somehow? Or I should modify the original getPricing and return current objectElement with other results?
From what I can understand from your question and your comments, you want to be able to access both object_element and the result of getPricing(object_element) within the for loop.
Your problem right now is that as a result of map, you only have the result of getPricing, but not the original object_element. As a solution, simply return both from map in an object:
// just a random getPricing function
function getPricing(objectElement) {
return {
item_id: objectElement.id,
price: Math.random()
};
}
const reagent_items = [
{
id: 'a'
},
{
id: 'b'
}
];
for (let V of reagent_items.map(object_element => ({object_element, pricing: getPricing(object_element)}))) {
console.log(`object_element: ${JSON.stringify(V.object_element)}, pricing: ${JSON.stringify(V.pricing)}`);
}
You should not be using for await here at all - that is meant for asynchronous generators. Instead, do
for (const object_element of reagent_items) {
const V = await getPricing(object_element);
console.log(object_element, V)
…
}
With Promise.all, the code would look like this:
Promise.all(reagent_items.map(object_element =>
getPricing(object_element).then(V => {
console.log(object_element, V);
…
})
)).then(…)
// or:
await Promise.all(reagent_items.map(async object_element => {
const V = await getPricing(object_element)
console.log(object_element, V);
…
}));
I'm trying to improve my knowledge regarding memoization in javascript. I have created a memoization function(I think..)
I've got an array of changes(a change log) made to items. Each item in the array contains a reference-id(employeeId) to whom made the edit. Looking something like this.
const changeLog = [
{
id: 1,
employeeId: 1,
field: 'someField',
oldValue: '0',
newValue: '100',
},
{
id: 2,
employeeId: 2,
field: 'anotherField',
oldValue: '20',
newValue: '100',
},
...
]
I've also got an array containing each employee, looking something like this:
const employees = [
{
name: 'Joel Abero',
id: 1
},
{
name: 'John Doe',
id: 2
},
{
name: 'Dear John',
id: 3
}
]
To find the employee who made the change I map over each item in the changeLog and find where employeeId equals id in the employees-array.
Both of these arrays contains 500+ items, I've just pasted fragments.
Below I pasted my memoize helper.
1) How can I perform a test to see which of these two run the fastest?
2) Is this a proper way to use memoization?
3) Is there a rule of thumb when to use memoization? Or should I use it as often as I can?
const employees = [
{
name: 'Joel Abero',
id: 1
},
{
name: 'John Doe',
id: 2
},
{
name: 'Dear John',
id: 3
}
]
const changeLog = [
{
id: 1,
employeeId: 1,
field: 'someField',
oldValue: '0',
newValue: '100',
},
{
id: 2,
employeeId: 2,
field: 'anotherField',
oldValue: '0',
newValue: '100',
},
{
id: 3,
employeeId: 3,
field: 'someField',
oldValue: '0',
newValue: '100',
},
{
id: 4,
employeeId: 3,
field: 'someField',
oldValue: '0',
newValue: '100',
},
{
id: 5,
employeeId: 3,
field: 'someField',
oldValue: '0',
newValue: '100',
}
]
function findEditedByEmployee (employeeId) {
return employees.find(({ id }) => id === employeeId)
}
function editedByWithMemoize () {
let employeesSavedInMemory = {}
return function(employeeId) {
if(employeeId in employeesSavedInMemory) {
console.log("from memory")
return employeesSavedInMemory[employeeId]
}
console.log("not from memory")
const findEditedBy = findEditedByEmployee(employeeId)
employeesSavedInMemory[findEditedBy.id] = {name: findEditedBy.name }
return findEditedBy
}
}
const memoizedEmployee = editedByWithMemoize();
// with memoization
const changeLogWithEmployeesMemoized = changeLog.map( log => {
const employeeName = memoizedEmployee(log.employeeId);
return {
...log,
employeeName: employeeName.name
}
})
// without memoization
const changeLogWithEmployees = changeLog.map( log => {
const editedBy = findEditedByEmployee(log.employeeId);
return {
...log,
employeeName: editedBy.name
}
})
console.log('memoized', changeLogWithEmployeesMemoized)
console.log('not memoized', changeLogWithEmployees)
I'll try to answer each of your questions:
1) How can I perform a test to see which of these two run the fastest?
The best way is just a simple for loop. Take for example a fake API request:
const fakeAPIRequest = id => new Promise(r => setTimeout(r, 100, {id}))
This will take 100ms to complete on request. Using memoization, we can avoid making this 100ms request by checking if we've made this request before:
const cache = {}
const memoizedRequest = async (id) => {
if (id in cache) return Promise.resolve(cache[id])
return cache[id] = await fakeAPIRequest(id)
}
Here's a working example:
const fakeAPIRequest = id => new Promise(r => setTimeout(r, 100, {id}))
const cache = {}
const memoizedRequest = async (id) => {
if (id in cache) return Promise.resolve(cache[id])
return cache[id] = await fakeAPIRequest(id)
}
const request = async (id) => await fakeAPIRequest(id)
const test = async (name, cb) => {
console.time(name)
for (let i = 50; i--;) {
await cb()
}
console.timeEnd(name)
}
test('memoized', async () => await memoizedRequest('test'))
test('normal', async () => await request('test'))
2) Is this a proper way to use memoization?
I'm not entirely sure what you mean by this, but think of it as short-term caching.
Should your memo call include an API request, it could be great for non-changing data, saving plenty of time, but on the other hand, if the data is subject to change within a short period of time, then memoization can be a bad idea, meaning it will shortly be outdated.
If you are making many many calls to this function, it could eat up memory depending on how big the return data is, but this factor is down to implementation, not "a proper way".
3) Is there a rule of thumb when to use memoization? Or should I use it as often as I can?
More often than not, memoization is overkill - since computers are extremely fast, it can often boil down to just saving milliseconds - If you are only calling the function even just a few times, memoization provides little to no benefit. But I do keep emphasising API requests, which can take long periods of time. If you start using a memoized function, you should strive to use it everywhere where possible. Like mentioned before, though, it can eat up memory quickly depending on the return data.
One last point about memoization is that if the data is already client side, using a map like Nina suggested is definitely a much better and more efficient approach. Instead of looping each time to find the object, it loops once to transform the array into an object (or map), allowing you to access the data in O(1) time. Take an example, using find this time instead of the fakeAPI function I made earlier:
const data = [{"id":0},{"id":1},{"id":2},{"id":3},{"id":4},{"id":5},{"id":6},{"id":7},{"id":8},{"id":9},{"id":10},{"id":11},{"id":12},{"id":13},{"id":14},{"id":15},{"id":16},{"id":17},{"id":18},{"id":19},{"id":20},{"id":21},{"id":22},{"id":23},{"id":24},{"id":25},{"id":26},{"id":27},{"id":28},{"id":29},{"id":30},{"id":31},{"id":32},{"id":33},{"id":34},{"id":35},{"id":36},{"id":37},{"id":38},{"id":39},{"id":40},{"id":41},{"id":42},{"id":43},{"id":44},{"id":45},{"id":46},{"id":47},{"id":48},{"id":49},{"id":50},{"id":51},{"id":52},{"id":53},{"id":54},{"id":55},{"id":56},{"id":57},{"id":58},{"id":59},{"id":60},{"id":61},{"id":62},{"id":63},{"id":64},{"id":65},{"id":66},{"id":67},{"id":68},{"id":69},{"id":70},{"id":71},{"id":72},{"id":73},{"id":74},{"id":75},{"id":76},{"id":77},{"id":78},{"id":79},{"id":80},{"id":81},{"id":82},{"id":83},{"id":84},{"id":85},{"id":86},{"id":87},{"id":88},{"id":89},{"id":90},{"id":91},{"id":92},{"id":93},{"id":94},{"id":95},{"id":96},{"id":97},{"id":98},{"id":99}]
const cache = {}
const findObject = id => data.find(o => o.id === id)
const memoizedFindObject = id => id in cache ? cache[id] : cache[id] = findObject(id)
const map = new Map(data.map(o => [o.id, o]))
const findObjectByMap = id => map.get(id)
const list = Array(50000).fill(0).map(() => Math.floor(Math.random() * 100))
const test = (name, cb) => {
console.time(name)
for (let i = 50000; i--;) {
cb(list[i])
}
console.timeEnd(name)
}
test('memoized', memoizedFindObject)
test('normal', findObject)
test('map', findObjectByMap)
All in all, memoization is a great tool, very similar to caching. It provides a big speed up on heavy calculations or long network requests, but can prove ineffective if used infrequently.
I would create a Map in advance and get the object from the map for an update.
If map does not contain a wanted id, create a new object and add it to employees and to the map.
const
employees = [{ name: 'Joel Abero', id: 1 }, { name: 'John Doe', id: 2 }, { name: 'Dear John', id: 3 }],
changeLog = [{ id: 1, employeeId: 1, field: 'someField', oldValue: '0', newValue: '100' }, { id: 2, employeeId: 2, field: 'anotherField', oldValue: '20', newValue: '100' }],
map = employees.reduce((map, o) => map.set(o.id, o), new Map);
for (const { id, field, newValue } of changeLog) {
let object = map.get(id);
if (object) {
object[field] = newValue;
} else {
let temp = { id, [field]: newValue };
employees.push(temp)
map.set(id, temp);
}
}
console.log(employees);
.as-console-wrapper { max-height: 100% !important; top: 0; }
Your memoization process is faulty!
You don't return objects with the same shape
When you don't find an employee in the cache, then you look it up and return the entire object, however, you only memoize part of the object:
employeesSavedInMemory[findEditedBy.id] = {name: findEditedBy.name }
So, when you find the employee in cache, you return a cut-down version of the data:
const employees = [ { name: 'Joel Abero', id: 1 }, { name: 'John Doe', id: 2 }, { name: 'Dear John', id: 3 } ]
function findEditedByEmployee (employeeId) {
return employees.find(({ id }) => id === employeeId)
}
function editedByWithMemoize () {
let employeesSavedInMemory = {}
return function(employeeId) {
if(employeeId in employeesSavedInMemory) {
console.log("from memory")
return employeesSavedInMemory[employeeId]
}
console.log("not from memory")
const findEditedBy = findEditedByEmployee(employeeId)
employeesSavedInMemory[findEditedBy.id] = {name: findEditedBy.name }
return findEditedBy
}
}
const memoizedEmployee = editedByWithMemoize();
const found = memoizedEmployee(1);
const fromCache = memoizedEmployee(1);
console.log("found:", found); //id + name
console.log("fromCache:", fromCache);//name
You get different data back when calling the same function with the same parameters.
You don't return the same objects
Another big problem is that you create a new object - even if you change to get a complete copy, the memoization is not transparent:
const employees = [ { name: 'Joel Abero', id: 1 }, { name: 'John Doe', id: 2 }, { name: 'Dear John', id: 3 } ]
function findEditedByEmployee (employeeId) {
return employees.find(({ id }) => id === employeeId)
}
function editedByWithMemoize () {
let employeesSavedInMemory = {}
return function(employeeId) {
if(employeeId in employeesSavedInMemory) {
console.log("from memory")
return employeesSavedInMemory[employeeId]
}
console.log("not from memory")
const findEditedBy = findEditedByEmployee(employeeId)
employeesSavedInMemory[findEditedBy.id] = { ...findEditedBy } //make a copy of all object properties
return findEditedBy
}
}
const memoizedEmployee = editedByWithMemoize();
memoizedEmployee(1)
const found = memoizedEmployee(1);
const fromCache = memoizedEmployee(1);
console.log("found:", found); //id + name
console.log("fromCache:", fromCache); //id + name
console.log("found === fromCache :", found === fromCache); // false
The result is basically the same you get "different" data, in that the objects are not the same one. So, for example, if you do:
const employees = [ { name: 'Joel Abero', id: 1 }, { name: 'John Doe', id: 2 }, { name: 'Dear John', id: 3 } ]
function findEditedByEmployee (employeeId) {
return employees.find(({ id }) => id === employeeId)
}
function editedByWithMemoize () {
let employeesSavedInMemory = {}
return function(employeeId) {
if(employeeId in employeesSavedInMemory) {
console.log("from memory")
return employeesSavedInMemory[employeeId]
}
console.log("not from memory")
const findEditedBy = findEditedByEmployee(employeeId)
employeesSavedInMemory[findEditedBy.id] = { ...findEditedBy } //make a copy of all object properties
return findEditedBy
}
}
const memoizedEmployee = editedByWithMemoize();
const original = employees[0];
const found = memoizedEmployee(1);
found.foo = "hello";
console.log("found:", found); //id + name + foo
const fromCache = memoizedEmployee(1);
console.log("fromCache 1:", fromCache); //id + name
fromCache.bar = "world";
console.log("fromCache 2:", fromCache); //id + name + bar
console.log("original:", original); //id + name + foo
Compare with a normal implementation
I'll use memoize from Lodash but there are many other generic implementations and they still work the same way, so this is only for reference:
const { memoize } = _;
const employees = [ { name: 'Joel Abero', id: 1 }, { name: 'John Doe', id: 2 }, { name: 'Dear John', id: 3 } ]
function findEditedByEmployee (employeeId) {
return employees.find(({ id }) => id === employeeId)
}
const memoizedEmployee = memoize(findEditedByEmployee);
const original = employees[0];
const found = memoizedEmployee(1);
console.log("found 1:", found); //id + name
found.foo = "hello";
console.log("found 2:", found); //id + name + foo
const fromCache = memoizedEmployee(1);
console.log("fromCache 1:", fromCache); //id + name + foo
fromCache.bar = "world";
console.log("fromCache 2:", fromCache); //id + name + foo + bar
console.log("original:", original); //id + name + foo + bar
console.log("found === fromCache :", found === fromCache); //true
<script src="https://cdn.jsdelivr.net/npm/lodash#4.17.15/lodash.min.js"></script>
Just a demonstration that the memoization is completely transparent and does not produce any odd or unusual behaviour. Using the memoized function is exactly the same as the normal function in terms of effects. The only difference is the caching but there is no impact on how the function behaves.
Onto the actual questions:
How can I perform a test to see which of these two run the fastest?
Honestly, and personally - you shouldn't. A correct implementation of memoization has known properties. A linear search also has known properties. So, testing for speed is testing two known properties of both algorithms.
Let's dip into pure logic here - we have two things to consider:
the implementation is correct (let's call this P)
properties of implementation are correct (let's call this Q)
We can definitely say that "If the implementation is correct, then properties of implementation are correct", transformable to "if P, then Q" or formally P -> Q. Were we to go in the opposite direction Q -> P and try to test the known properties to confirm the implementation is correct, then we are committing the fallacy of affirming the consequent.
We can indeed observe that testing the speed is not even testing the solution for correctness. You could have incorrect implementation of memoization yet it would exhibit the same speed property of O(n) lookup once and O(1) on repeat reads as correct memoization. So, the test Q -> P will fail.
Instead, you should test the implementation for correctness, if you can prove that, then you can deduce that you'd have constant speed on repeat reads. And O(1) access is going to be (in most cases, especially this one), faster than O(n) lookup.
Consequently, if you don't need to prove correctness, then you can take the rest for granted. And if you use a known implementation of memoization, then you don't need to test your library.
With all that said, there is something you might still need to be aware of. The caching during memoization relies on creating a correct key for the cached item. And this could potentially have a big, even if constant, overhead cost depending on how the key is being derived. So, for example, a lookup for something near the start of the array might take 10ms yet creating the key for the cache might take 15ms, which means that O(1) would be slower. Slower than some cases.
The correct test to verify speed would normally be to check how much time it takes (on average) to lookup the first item in the array, the last item in the array, something from the middle of the array then check how much time it takes to fetch something from cache. Each of these has to be ran several times to ensure you don't get a random spike of speed either up or down.
But I'd have more to say later*
2) Is this a proper way to use memoization?
Yes. Again, assuming proper implementation, this is how you'd do it - memoize a function and then you get a lot of benefits for caching.
With that said, you can see from the Lodash implementation that you can just generalise the memoization implementation and apply it to any function, instead of writing a memoized version of each. This is quite a big benefit, since you only need to test one memoization function. Instead, if you have something like findEmployee(), findDepartment(), and findAddress() functions which you want to cache the results of, then you need to test each memoization implementation.
3) Is there a rule of thumb when to use memoization? Or should I use it as often as I can?
Yes, you should use it as often as you can* (with a huge asterisk)
* (huge asterisk)
This is the biggest asterisk I know how to make using markdown (outside just embedding images). I could go for a slightly bigger one but alas.
You have to determine when you can use it, in order to use it when you can. I'm not just saying this to be confusing - you shouldn't just be slapping memoized functions everywhere. There are situations when you cannot use them. And that's what I alluded to at the end of answering the first question - I wanted to talk about this in a single place:
You have to tread very carefully to verify what your actual usage is. If you have a million items in an array and only the first 10 are looked up faster than being fetched from cache, then there is 0.001% of items that would have no benefit from caching. In that case - you get a benefit from caching...or do you? If you only do a couple of reads per item, and you're only looking up less than a quarter of the items, then perhaps caching doesn't give you a good long term benefit. And if you look up each item exactly two times, then you're doubling your memory consumption for honestly quite trivial improvement of speed overall. Yet, what if you're not doing in-memory lookup from an array but something like a network request (e.g., database read)? In that case caching even for a single use could be very valuable.
You can see how a single detail can swing wildly whether you should use memoization or not. And often times it's not even that clear when you're initially writing the application, since you don't even know how often you might end up calling a function, what value you'd feed it, nor how often you'd call it with the same values over and over again. Even if you have an idea of what the typical usage might be, you still will need a real environment to test with, instead of just calling a non-memoized and a memoized version of a function in isolation.
Eric Lippert has an an amazing piece on performance testing that mostly boils down to - when performance matters, try to test real application with real data, and real usage. Otherwise your benchmark might be off for all sorts of reasons.
Even if memoization is clearly "faster" you have to consider memory usage. Here is a slightly silly example to illustrate memoization eating up more memory than necessary:
const { memoize } = _;
//stupid recursive function that removes 1 from `b` and
//adds 1 to `a` until it finds the total sum of the two
function sum (a, b) {
if(b)
return sum(a + 1, b - 1)
//only log once to avoid spamming the logs but show if it's called or not
console.log("sum() finished");
return a;
}
//memoize the function
sum = memoize(sum);
const result = sum(1, 999);
console.log("result:", result);
const resultFromCache1 = sum(1, 999); //no logs as it's cached
console.log("resultFromCache1:", resultFromCache1);
const resultFromCache2 = sum(999, 1); //no logs as it's cached
console.log("resultFromCache2:", resultFromCache2);
const resultFromCache3 = sum(450, 550); //no logs as it's cached
console.log("resultFromCache3:", resultFromCache3);
const resultFromCache4 = sum(42, 958); //no logs as it's cached
console.log("resultFromCache4:", resultFromCache4);
<script src="https://cdn.jsdelivr.net/npm/lodash#4.17.15/lodash.min.js"></script>
This will put one thousand cached results in memory. Yes, the function memoized is silly and doing a lot of unnecessary calls, which means there is a lot of memory overhead. Yet at the same time, if we re-call it with any arguments that sum up to 1000, then we immediately get the result, without having to do any recursion.
You can easily have similar real code, even if there is no recursion involved - you might end up calling some function a lot of times with a lot of different inputs. This will populate the cache with all results and yet whether that is useful or not is still up in the air.
So, if you can you should be using memoization. The biggest problem is finding out if you can.