I'm working on a Vue/Vuetify app that has a performance problem. I've created a custom component that wraps around a standard Vuetify v-data-table component. It works fine for small amounts of data, but giving it moderate to large amounts of data causes Firefox to hang and Chrome to crash.
Here's a slightly simplified version of my code:
<script>
import ...
export default {
props: {
theValues: Array,
// other props
},
computed: {
theKeys: function() {
return this.schema.map(col => col.col);
},
schema: function() {
return this.theValues[0].schema;
},
dataForDataTable: function() {
console.time('test');
let result = [];
for (let i = 0; i < theValues[0].data.length; i++) {
let resultObj = {};
for (let j = 0; j < theKeys.length; j++) {
// The "real" logic; this causes the browser to hang/crash
// resultObj[theKeys[j]] = theValues[0].data[i][j];
// Test operation to diagnose the problem
resultObj[theKeys[j]] = Math.floor(Math.random() * Math.floor(99999));
}
result.push(resultObj);
}
console.timeEnd('test');
// For ~30k rows, timer reports that:
// Real values can take over 250,000 ms
// Randomly generated fake values take only 7 ms
return result;
},
// other computed
},
// other Vue stuff
</script>
And here's an example of what theValues actually looks like:
[
{
data: [
[25389, 24890, 49021, ...] <-- 30,000 elements
],
schema: [
{
col: "id_number",
type: "integer"
}
]
}
]
The only meaningful difference I see between the fast code and the slow code is that the slow code accesses the prop theValues on each iteration whereas the fast code doesn't touch any complicated part of Vue. (It does use theKeys, but the performance doesn't change even if I create a local deep copy of theKeys inside the function.)
Based on this, it seems like the problem is not that the data table component can't handle the amount of data I'm sending, or that the nested loops are inherently too inefficient. My best guess is that reading from the prop so much is somehow slowing Vue itself down, but I'm not 100% sure of that.
But I do ultimately need to get the information from the prop into the table. What can I do to make this load at a reasonable speed?
The performance problem is actually a symptom of your loop code rather than Vue. The most expensive data access is in your inner loop in dataForDataTable():
for (i...) {
for (j...) {
theValues[0].data[i][j] // ~50 ms average (expensive)
}
}
// => long hang for 32K items
An optimization would be to cache the array outside your loop, which dramatically improves the loop execution time and resolves the hang:
const myData = theValues[0].data
for (i...) {
for (j...) {
myData[i][j] // ~0.00145 ms average
}
}
// => ~39 ms for 32K items
demo 1
Note the same result can be computed without loops, using JavaScript APIs. This affords readability and reduced lines of code at a slight performance cost (~1ms). Specifically, use Array.prototype.map to map each value from data to an object property, obtained by Array.prototype.reduce on theKeys:
theValues[0].data
.map(values => theKeys.reduce((obj,key,i) => {
obj[key] = values[i]
return obj
}, {}))
// => ~40 ms for 32K items
demo 2
Times above measured on 2016 MacBook Pro - 2.7GHz i7, Chrome 87. Codesandbox demos might show a vast variance from the above.
Tip 1
Original text:
Accessing the prop (data) should not be an issue. Yes, data are reactive but reading it should be very efficient (Vue is just "making notes" that you are using that data for rendering)
Well it seems I was clearly wrong here...
Your component is getting data by prop but it is very probable that the data is reactive in parent component (coming from Vuex or stored in parent's data). Problem with Vue 2 reactivity system is it is based on Object.defineProperty and this system does not allow to intercept indexed array access (Vue is not able to detect code like arr[1] as an template dependency). To workaround this, if object property (theValues[0].data in your code) is accessed, it checks whether the value is array and if yes it iterates the whole array (plus all nested arrays) to mark the items as dependencies - you can read more in depth explanation here
One solution to this problem is to create local variable let data = theValues[0].data as tony19 suggests. Now the .data Vue getter is not called every time and the performance is fixed...
But if your data is immutable (never change), just use Object.freeze() and Vue will not try to detect changes of such data. This will not only make your code faster but also saves a ton of memory in case of large lists of objects
Note that this problem is fixed in Vue 3 as it uses very different reactivity system based on ES6 proxies...
Tip 2
Although Vue computed properties are highly optimized, there is still some code running every time you access the property (checking whether underlying data is dirty and computed prop needs reevaluate) and this work adds up if you use it in a tight loop as in your case...
Try to make local copy of the theKeys computed prop before executing the loop (shallow copy is enough, no need for a deep copy)
See this really good video from Vue core member
Of course the same issue applies to accessing the dataForDataTable computed prop from the template. I encourage You to try to use watcher instead of computed to implement same logic as dataForDataTable and store it's result in data to see if it makes any difference...
Related
Let's say we have those two similar es-modules:
The first one exports a big static array of e.g. dummy data:
export default const dummyData = [ /* big data set */ ];
The other one exports a function that returns the dummy data:
export default function getDummyData() {
const dummyData = [ /* big data set */ ];
return dummyData;
}
Is one approache preferable to the other one e.g. because of memory usage at runtime?
First consider whether the data is actually large enough for the amount of resources it consumes to be worth worrying about. Often times, it isn't; many modern phones and computers are capable of a lot without noticeably impacting the user's experience.
If the amount of data is large enough to worry about, then: exporting a function will result in the data being put on the heap only on demand, but (your current implementation) will also result in data being created every time the function runs. So
If, in many cases, the data never gets requested, it could be better to use a function because then the data doesn't get loaded into memory unnecessarily
If multiple other modules use this one, calling the function for each one could be an issue, because then you'd be loading a new deep copy of the data into memory for each call. (This could be mitigated by saving the data into a persistent outer variable the first time the function is called, but that'll also prevent garbage collection.)
If the data is always used, and it's only used once, it doesn't matter. May as well export the plain object.
If this code runs on the frontend, some might consider a nicer approach to be to call an API that returns the data instead of hard-coding it into the source.
i am doing an app in Vuejs and i am stuck with a problem in replacing one object for another that comes from an API every 10 seconds. I have this at the moment:
watch: {
myObj() {
for (let [key, tuner] of Object.entries(myObj)) {
---- some code ----
}
}
},
},
created() {
setInterval(this.callMyApi(), 10000);
},
I am watching the object and then use it in a for loop that does some logic. I saw the documentation mention using 'this.$set()' but this only adds a property to the object and i want replace one for another. Thanks for the help.
I would use:
this.myObj = JSON.parse(JSON.stringify(newObj))
to make a deep copy of the object inside this.callMyApi().
To watch for changes on this object do the following: create a new bool variable and use it on the component you want to update:
<componenttoupdate :reactto=objupdated/>
data: {
...
objupdated: false
...
}
and switch the objupdated bool state anytime you do the deep copy on your function, and so the component will update.
This is a cheaper solution than keeping watchers over large objects that may get updated very often as yours.
I'm a fan of Vue which a try to use on some occasions. Anyway, there is something I always found not so handy with it: reactivity lies within $data. Well not always, as external data can be tracked by Vue, as in computed properties, in templates… But I found this way uncomfortable and not always consistent (see another question about it, here Reactivity on Variables Not Associated With Data, Computed, etc). So my decision now is use $data as the main source of reactivity and stop trying to find other ways.
However, reactivity within $data poses me a problem in what is a common case for me: many pieces of data here and there in other imported objects. This makes even more sense as I consider Vue as the View end not the business logic. Those imported objects are sometimes complex and within Vue components, I found no way to cherry pick pieces of information and kind of ask Vue to bind to them. The only way was to declare entire objects in the $data section which makes tracking very heavy: loads of setters/getters when only one would be enough in a simple component, for example.
So I designed a class called 'Reactor' whose instances role is to install getter/setters on any piece data of my wish in a complex object (or more than one). Those instances are imported into Vue components and then $watchers of Reactor instances have properties which can contain as many functions as I wish which are called when pieces of data are altered through the Reactor. To make things simple by default is filled with the same property name as the data it bounds to. This precisely those function which will update $data when external data change.
class Reactor {
constructor() {
this.$watchers = {};
}
addProperty(originalObject, keyString, aliasKeyString) {
if(aliasKeyString === undefined) {
aliasKeyString = keyString;
}
if(this[aliasKeyString] !== undefined || originalObject[keyString] === undefined) {
const errorMessage = `Reactor: cannot add property '${aliasKeyString}'!`;
console.error(errorMessage);
throw errorMessage;
}
this.$watchers[aliasKeyString] = [];
Object.defineProperty(this, aliasKeyString, {
set(newValue) {
const oldValue = originalObject[keyString];
originalObject[keyString] = newValue;
this.$watchers[aliasKeyString].forEach((f) => {
if(typeof f === "function") {
f(newValue, oldValue, aliasKeyString);
}
});
},
get() {
return originalObject[keyString];
},
});
}
}
An example can be seen in the codepen here: https://codepen.io/Djee/pen/gyVZMG
So it's sort of an 'inverted' Vue which allows updating $data on external conditions.
This pattern also helped me resolve a case which was rather difficult before: have a double-bind on an input with a filter in-between which will set the input and its attached external value straight upon #change event only. This can be seen in the same codepen given above.
I was a little surprised to have found nothing taking this in charge in Vue itself. Did I miss something obvious? This is mainly the purpose of this somewhat long introduction. I had no time to check whether Vuex would solve this nicely.
Thanks for any comments as well.
I'm building a node.js app and I'm evaluating Sequelize.js for persistent objects. One thing I need to do is publish new values when objects are modified. The most sensible place to do this would seem to be using the afterUpdate hook.
It almost works perfectly, but when I save an object the hook is passed ALL the values of the saved object. Normally this is desirable, but to keep the publish/subscribe chatter down, I would rather not republish fields that weren't saved.
So for instance, running the following
tasks[0].updateAttributes({assignee: 10}, ['assignee']);
Would automagically publish the new value for the assignee for that task on the appropriate channel, but not republish any of the other fields, which didn't change.
The closest I've come is with an afterUpdate hook:
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
which is pretty straightforward, but since the 'task' object has all the fields, I'm being unnecessarily noisy. (The pubSub system is ignorant of previous values and I'd like to keep it that way.)
I could override the setters in the task object (and all my other objects), but I would prefer not to publish until the object is saved. The object to be saved doesn't seem to have the old values (that I can find), so I can't base my publish on that.
So far the best answer I've come up with from a design standpoint is to tweak one line of dao.js to add the saved values to the returned object, and use that in the hook:
self.__factory.runHooks('after' + hook, _.extend({}, result.values, {savedVals: args[2]} ), function(err, newValues) {
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task.savedVals).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
Obviously changing the Sequelize library is not ideal from a maintenance standpoint.
So my question is twofold: is there a better way to get the needed information to my hook without modifying dao.js, or is there a better way to attack my fundamental requirement?
Thanks in advance!
There is not currently. In the implementation for exactly what you describe we simply had to implement logic to compare old and new values, and if they differed, assume that they have changed.
I'm analyzing the basic todo application.
Why is it that when I delete the StatsView (from the main todos.js and from todos.handlebars) the remaining method (property) of the todoListController stops updating itself?
Todos.todoListController = SC.ArrayController.create({
...
remaining: function() {
console.log('remaining');//doesn't apear in the console
return this.filterProperty('isDone', false).get('length');
}.property('#each.isDone').cacheable(),
...
});
I can imagine, that this is because with the StatsView I deleted the binding. But shouldn't it be, that the #each keeps an eye on the changes?
SproutCore optimizes to do as little work as possible. So, when you deleted the StatsView, you deleted the thing that cares about the .remaining property. Since nothing is asking for it, SproutCore doesn't compute it. This is why you should always use the get() and set() methods when accessing properties so that they can decide whether to use the cached version or to actually compute the property.