Let's say we have those two similar es-modules:
The first one exports a big static array of e.g. dummy data:
export default const dummyData = [ /* big data set */ ];
The other one exports a function that returns the dummy data:
export default function getDummyData() {
const dummyData = [ /* big data set */ ];
return dummyData;
}
Is one approache preferable to the other one e.g. because of memory usage at runtime?
First consider whether the data is actually large enough for the amount of resources it consumes to be worth worrying about. Often times, it isn't; many modern phones and computers are capable of a lot without noticeably impacting the user's experience.
If the amount of data is large enough to worry about, then: exporting a function will result in the data being put on the heap only on demand, but (your current implementation) will also result in data being created every time the function runs. So
If, in many cases, the data never gets requested, it could be better to use a function because then the data doesn't get loaded into memory unnecessarily
If multiple other modules use this one, calling the function for each one could be an issue, because then you'd be loading a new deep copy of the data into memory for each call. (This could be mitigated by saving the data into a persistent outer variable the first time the function is called, but that'll also prevent garbage collection.)
If the data is always used, and it's only used once, it doesn't matter. May as well export the plain object.
If this code runs on the frontend, some might consider a nicer approach to be to call an API that returns the data instead of hard-coding it into the source.
Related
Disclaimer: I am really not well experienced with neither svelte, nor D3, nor general JavaScript patterns. But I really like it, so I really want to learn it and already invested quite some time. Still, this feels like a super basic question that annoys me a lot. I hope it is not too confusing and someone might has an idea.
It is basically about how to setup a simple graph (let it be a bar chart) in an efficient, reproducible and "best-practice" way. I guess my main concern is on how to pass around the data and use it for different tasks. E.g. I think it might be a good idea to separate out the construction of the scales (using d3) in a separate component. However, this component needs access to the data (and probably also access to the, in the best case resposive width, of the chart-container).
However, also the bars, which are in another component, need access to the data in order to know how do draw the rectangles.
A general misunderstanding (i guess that is the right word) I have with JavaScript is that I do not understand how to fetch data asynchronously (using e.g. the browsers fetchor D3's csvmethod). I simply can not fetch the data and then pass it as prop to another component. Because what I would be passing would be a promise...
So I have this very basic REPL that kind of shows a bit this know I have in my head: https://svelte.dev/repl/398f4c21b7a9409a9811fd8e38703a36?version=3.44.1
It looks like this. In the App.html I fetch the data that I want to use for multiple purposes. However I cannot "get it out" of that component.
<script>
import Chart from "./Chart.svelte"
const url = "https://api.github.com/search/repositories?q=stars:>100000";
async function getData(){
let response = await fetch(url)
let data = await response.json()
console.log(data)
}
//async function getDataLocal(){
// let data = await d3.csv(<path_to_data>)
// return await data
// }
let data = await getData()
</script>
<Chart {data}>Do Something with the data. Make the chart, build the scales, ....</Chart>
So the main questions are:
Are there any ressources on how to learn building sustainable graphics with remote data, svelte and a bit of D3. I already watched many many youtube videos and I guess I will rewatch the one from Matthias Stahl;)
Is it a good idea to use stores in such a case to store the data
And a little more specific: As the data is (probably) fixed, however the dimension arent't: What is a good way/place to let the app know to recalculate the scales etc.
There are 3 separate concerns here:
fetching, storing and retrieving data (aka the data source layer)
manipulating/transforming data (aka the business logic layer)
displaying data (aka the presentation layer)
I will leave the last part aside as it solely concerns D3 (if that is your visualization library of choice) and there are plenty of resources available online on this topic, and I will instead focus on what seems to be the heart of your question, i.e. how to fetch data in Svelte, where to store it, how to pass it around to components, and how to manipulate the data.
1. Asynchronous queries in Svelte
Your first inquiry is about how to deal with asynchronous requests. You cannot use await statements at the root level of the <script> section of a Svelte file, meaning the following reactive statement would generate an error:
// will fail
$: data = await getData(url)
However, you can call an asynchronous function that will handle the assignment. Reactivity will still work and your component will re-render when the url is changed and the new data retrieved:
// will work
$: updateData(url)
async function updateData(url) {
data = await getData(url)
}
Here is a working example based on the REPL in your question
2. Using stores
As you could see from the above example, you had to pass the data to your <Header> and <Chart> components for it to be used in either:
<Header {data}>GitHub Lookup</Header>
<Chart {data}/>
But what if you want to use your Chart somewhere else in your application? What if you have another component that wants to make use of the same data?
Obviously you do not want to fetch the same data over & over (unless the request itself has changed). You also want to avoid passing the data around as a prop everywhere in your app. You will want to make the data available only to these components that will use it.
This is where stores come in handy. Stores can be subscribed to by any component. A writable store will allow its contents to be updated, while a readable store will be -as the name implies- read-only.
A store need not be complex. The following is a very basic writable store:
import { writable } from 'svelte/store'
export const githubStore = writable(null) // initialized with a null value
All you have to do then is interact with your store.
Updating the store in your App component:
import { githubStore as data } from './githubStore.js' // import the store we defined above under the name 'data'
.
.
.
async function updateData(url) {
$data = await getData(url) // using the $ shorthand to access the store (assigning a new value will update the store content)
}
Using (i.e. subscribing to) the store in your components:
import { githubStore as data } from './githubStore.js' // import the store we defined above under the name 'data'
.
.
.
// using the $ shorthand to access the store
{#each $data.items as item (item.id)}
<li><a href={item.html_url}>{item.full_name}</a> [{item.stargazers_count}⭐]</li>
{/each}
Read here for details on using the $ reactive syntax with stores
Now that your child components are subscribing to the store where you stored your data, you do not need to pass that data as a prop any more:
<Header>GitHub Lookup</Header>
<Chart />
Here is an updated version of the REPL above, using stores
3. Further considerations
When you want to start manipulating or transforming data that has been put into a store, derived stores come in handy. When the data in your original store is updated, the derived store will automatically update itself based on the changes to the original.
You can also build on the provided readable/writable stores by adding your own functionality and custom methods. These are slightly more advanced topics but would come in handy where data manipulation is concerned.
Finally, D3 will provide its own data manipulation methods, so it will be up to you to decide how much manipulation you handle directly in Svelte, and how much you delegate to D3. I would probably leave everything connected to visualization (scaling, zooming, etc.) on the D3 side, and have the actual pre-visualization manipulation of data (i.e. the business logic) on the Svelte side (or better yet, directly on the back-end if you have access to that!).
Is it allowed to store non serializable object in React context ?
I'm unable to find answer on this question in the docs
I know it is considered as a bad practice in redux store, see here.
Could you please provide any arguments or link to the docs about using non serializable objects in context?
Consider next example:
class A {
foo=()=> void 0
bar=()=> void 0
}
const context = React.createContext(new A()) // is this ok ?
I think it is perfectly fine to store non serializable object in React context. And here are some arguments.
First of all, I do not know (correct me if I'm wrong) any internal react mechanics that serializes or unserializes arbitrary context and / or assumes it to be serializable.
Second, react context now is a stable feature (in contrast like it was unstable some time ago). It is unlikely that something will change here, so we do not need to be proactively careful.
Finally, react context is just a way to pass something implicitly deep down into the components tree, not more, not less. React context is actually not about storing something, it is about providing and consuming something. You do not store things in context, you store it somewhere else, maybe in state, maybe somewhere, and then just provide it to consumers. So, why not to provide function to consumers if they need it? How it differs from passing things through props (where passing function is for sure not an antipattern)?
It is more about concrete scenario, and no one knows better then you, how and when to restrict particular context shape. For example, in our project we have some internal redux-like library that synchronizes stores between electron windows via rpc calls. It uses context and we specifically restricted (with typescript) context shape to be serializable (to be able to transfer it via rpc). On the other hand we have many cases where we pass non serializable stuff via context.
I tried using class to solve this problem. The context worked fine.
Sample code:
class Stuff {
data :any = {}
getData() {
return this.data
}
setData(_data) {
this.data = _data
}
}
const StuffInst = new Stuff();
// import this and use where you want
Please comment for suggestions/errors.
I'm working on a Vue/Vuetify app that has a performance problem. I've created a custom component that wraps around a standard Vuetify v-data-table component. It works fine for small amounts of data, but giving it moderate to large amounts of data causes Firefox to hang and Chrome to crash.
Here's a slightly simplified version of my code:
<script>
import ...
export default {
props: {
theValues: Array,
// other props
},
computed: {
theKeys: function() {
return this.schema.map(col => col.col);
},
schema: function() {
return this.theValues[0].schema;
},
dataForDataTable: function() {
console.time('test');
let result = [];
for (let i = 0; i < theValues[0].data.length; i++) {
let resultObj = {};
for (let j = 0; j < theKeys.length; j++) {
// The "real" logic; this causes the browser to hang/crash
// resultObj[theKeys[j]] = theValues[0].data[i][j];
// Test operation to diagnose the problem
resultObj[theKeys[j]] = Math.floor(Math.random() * Math.floor(99999));
}
result.push(resultObj);
}
console.timeEnd('test');
// For ~30k rows, timer reports that:
// Real values can take over 250,000 ms
// Randomly generated fake values take only 7 ms
return result;
},
// other computed
},
// other Vue stuff
</script>
And here's an example of what theValues actually looks like:
[
{
data: [
[25389, 24890, 49021, ...] <-- 30,000 elements
],
schema: [
{
col: "id_number",
type: "integer"
}
]
}
]
The only meaningful difference I see between the fast code and the slow code is that the slow code accesses the prop theValues on each iteration whereas the fast code doesn't touch any complicated part of Vue. (It does use theKeys, but the performance doesn't change even if I create a local deep copy of theKeys inside the function.)
Based on this, it seems like the problem is not that the data table component can't handle the amount of data I'm sending, or that the nested loops are inherently too inefficient. My best guess is that reading from the prop so much is somehow slowing Vue itself down, but I'm not 100% sure of that.
But I do ultimately need to get the information from the prop into the table. What can I do to make this load at a reasonable speed?
The performance problem is actually a symptom of your loop code rather than Vue. The most expensive data access is in your inner loop in dataForDataTable():
for (i...) {
for (j...) {
theValues[0].data[i][j] // ~50 ms average (expensive)
}
}
// => long hang for 32K items
An optimization would be to cache the array outside your loop, which dramatically improves the loop execution time and resolves the hang:
const myData = theValues[0].data
for (i...) {
for (j...) {
myData[i][j] // ~0.00145 ms average
}
}
// => ~39 ms for 32K items
demo 1
Note the same result can be computed without loops, using JavaScript APIs. This affords readability and reduced lines of code at a slight performance cost (~1ms). Specifically, use Array.prototype.map to map each value from data to an object property, obtained by Array.prototype.reduce on theKeys:
theValues[0].data
.map(values => theKeys.reduce((obj,key,i) => {
obj[key] = values[i]
return obj
}, {}))
// => ~40 ms for 32K items
demo 2
Times above measured on 2016 MacBook Pro - 2.7GHz i7, Chrome 87. Codesandbox demos might show a vast variance from the above.
Tip 1
Original text:
Accessing the prop (data) should not be an issue. Yes, data are reactive but reading it should be very efficient (Vue is just "making notes" that you are using that data for rendering)
Well it seems I was clearly wrong here...
Your component is getting data by prop but it is very probable that the data is reactive in parent component (coming from Vuex or stored in parent's data). Problem with Vue 2 reactivity system is it is based on Object.defineProperty and this system does not allow to intercept indexed array access (Vue is not able to detect code like arr[1] as an template dependency). To workaround this, if object property (theValues[0].data in your code) is accessed, it checks whether the value is array and if yes it iterates the whole array (plus all nested arrays) to mark the items as dependencies - you can read more in depth explanation here
One solution to this problem is to create local variable let data = theValues[0].data as tony19 suggests. Now the .data Vue getter is not called every time and the performance is fixed...
But if your data is immutable (never change), just use Object.freeze() and Vue will not try to detect changes of such data. This will not only make your code faster but also saves a ton of memory in case of large lists of objects
Note that this problem is fixed in Vue 3 as it uses very different reactivity system based on ES6 proxies...
Tip 2
Although Vue computed properties are highly optimized, there is still some code running every time you access the property (checking whether underlying data is dirty and computed prop needs reevaluate) and this work adds up if you use it in a tight loop as in your case...
Try to make local copy of the theKeys computed prop before executing the loop (shallow copy is enough, no need for a deep copy)
See this really good video from Vue core member
Of course the same issue applies to accessing the dataForDataTable computed prop from the template. I encourage You to try to use watcher instead of computed to implement same logic as dataForDataTable and store it's result in data to see if it makes any difference...
I'm building a node.js app and I'm evaluating Sequelize.js for persistent objects. One thing I need to do is publish new values when objects are modified. The most sensible place to do this would seem to be using the afterUpdate hook.
It almost works perfectly, but when I save an object the hook is passed ALL the values of the saved object. Normally this is desirable, but to keep the publish/subscribe chatter down, I would rather not republish fields that weren't saved.
So for instance, running the following
tasks[0].updateAttributes({assignee: 10}, ['assignee']);
Would automagically publish the new value for the assignee for that task on the appropriate channel, but not republish any of the other fields, which didn't change.
The closest I've come is with an afterUpdate hook:
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
which is pretty straightforward, but since the 'task' object has all the fields, I'm being unnecessarily noisy. (The pubSub system is ignorant of previous values and I'd like to keep it that way.)
I could override the setters in the task object (and all my other objects), but I would prefer not to publish until the object is saved. The object to be saved doesn't seem to have the old values (that I can find), so I can't base my publish on that.
So far the best answer I've come up with from a design standpoint is to tweak one line of dao.js to add the saved values to the returned object, and use that in the hook:
self.__factory.runHooks('after' + hook, _.extend({}, result.values, {savedVals: args[2]} ), function(err, newValues) {
Task.hook('afterUpdate', function(task, fn) {
Object.keys(task.savedVals).forEach(function publishValue(key) {
pubSub.publish('Task:'+task.id+'#'+key, task[key]);
});
return fn();
});
Obviously changing the Sequelize library is not ideal from a maintenance standpoint.
So my question is twofold: is there a better way to get the needed information to my hook without modifying dao.js, or is there a better way to attack my fundamental requirement?
Thanks in advance!
There is not currently. In the implementation for exactly what you describe we simply had to implement logic to compare old and new values, and if they differed, assume that they have changed.
Please consider the Javascript code excerpt at the bottom. Roughly it consists of two modules, one for handling messages. What is the benefit of the filtersUpdateSuccess method within the messages module?
Currently it merely delegates to the overwriteAll method of the tplPanels module. One idea that strikes me is that within the filtersUpdateSuccess method, the call to tplPanels.overwriteAll could be wrapped in a try/catch. Could this benefit me, and are there any other benefits to the extra level of indirection?
PS .... I am familiar with the following question and have consulted it and followed the links within it, but now I want an answer in a specific context as opposed to the more general: Level of Indirection solves every Problem
function msgHandlers() {
var scope
,msgs;
return {
SET_CONTEXT: function(context) {
scope = context.scope;
msgs = context.messages;
}
,SUBSCRIBE_ALL: function() {
me.subscribe(scope, msgs.FILTERS_UPDATE_SUCCESS, this.filtersUpdateSuccess);
//me.subscribe(scope, msgs.FILTERS_UPDATE_SUCCESS, tpl.overwriteAll);
}
,filtersUpdateSuccess: function(filterValues) {
tplPanels().overwriteAll(filterValues);
}
}
}
function tplPanels() {
var instances = {};
return {
locationInstance: function() {
if (!instances.locationPanel) {
instances.locationPanel = locationPanel();
}
return instances.locationPanel;
}
//more instances, etcetera
,overwriteAll: function(filterValues) {
//do something useful
}
}
//etcetera
The case here seems to be one of abstraction of interface, not extra layers of reference indirection.
Interface abstraction serves several purposes:
To cause a piece of code to fit an interface required by one generic client or one client whose shape is out of your control.
To cause a piece of code to fit an interface shared by multiple clients.
To cause a piece of code to fit a maintainer's mental model by abstracting away implementation details.
To cause a piece of code to fit an interface required by a planned (not just possible) future use-case.
If another module in your system already takes different inputs with the context-setting-subscribing-filtering interface then it can be valuable because it allows your code to plug in there: (1) or (2) above.
If you have concreate plans to have multiple providers or consumers of this interface, then it can be valuable to just make it conform to the interface now since you've already done the work to page the code into memory: (4) above.
If other modules are already structured this way, or if filtering is a common operation other modules use, then code maintainers can leverage that to learn your code quickly: (3) above.
If none of those apply, then, since filtersUpdateSuccess doesn't close over anything that tplPanels doesn't, there's no value in the interface abstraction.