I'm new to Sequelize and try to achieve the following:
Assume I have a very simple database with 3 Models/Tables:
Person, Group and Category.
Person has a Many-To-One relation to Group (1 Person can be in 1 Group, 1 Group holds multiple people) & Group has a Many-To-One relation to Category (1 Group has 1 Category, 1 Category can be applied to multiple Groups).
Because I don't want to save the whole Category in my database, but only a short string, I have a mapper in the backend in my app.
Let's say my Category-Mapper looks like this:
//category.mapper.js
module.exports = Object.freeze({
cat1: "Here is the String that should be sent to and displayed by the FrontEnd",
cat2: ".....",
});
So basically, in my database I save "cat1" as the category and every time I get one or more Categories via Sequelize from the database, I want to go into my mapper, resolve the short string to the long string and send it to the Frontend, so I wrote the following code:
//category.model.js
const categoryMapper = require("../mapper/category.mapper");
Category.afterFind((models) => {
if(!Array.isArray(models)) {
models = [models];
}
models.forEach(model => {
model.name = categoryMapper[model.name];
});
});
This works great when I call Category.findAll()..., but does not trigger when I include the Category as in this example:
Group.findAll({
include: [Category]
})
There is this rather old GitHub Issue referencing this behavior, where someone published some code to make sure the hooks run on include. See here.
I tried implementing the referenced code into my project, but when I do, the hook for Category runs twice in my following code:
Person.findAll({
include: [{
model: Group,
include: [Category]
}]
})
My assumption is, that, with the code from the GitHub-Issue comment, my hook gets triggered every time the relationship is detected and the code runs. Therefore the hook runs once after including Group, because Group has a relationship to Category and a second time when Category is actually included, which breaks my mapping function because the second time it tries to resolve the long string, which doesn't work.
I'm looking for a solution that basically runs my hooks once and only once, namely when the actual include for my model triggers, regardless of on what level the include happens.
Sorry for the lengthy post, but I did not find any solution to my problem online, but don't believe what I am trying to achieve is very exotic or specific to my project only.
If there is a better solution I am not seeing, I'm open to suggestions and new approaches.
Thanx in advance!
Related
I'm to get an answer on whether or not the way my database is setup will cause too many recursive reads and therefore exponentially increase the number of read operations.
I currently have a collection of users, inside of each user doc, I have 3 other catalogs, goods, bundles and parts. Each user has a list of parts and a list of bundles, for example.
Each doc in the bundles catalog has an array of maps with a reference to a doc in the parts catalog in each map.
When I query the bundles, I want to also get the details of each part in the bundle. Does this require that I run another onSnapshot?
Here's an example:
database:
users (catalog)
userID
parts
partID1
partID2
partID3
bundles
bundleID1
title: "string",
parts: [
part:"/users/userID/parts/partID1,
qty: 1
]
parts: [
part:"/users/userID/parts/partID2,
qty: 1
]
parts: [
part:"/users/userID/parts/partID3,
qty: 1
]
getting the bundles
initBundle(bid) {
const path = this.database.collection('users').doc('userID').collection('bundles').doc(bid);
path.ref.onSnapshot(bundle => {
const partsArr = [];
bundle.data().parts.forEach(part => {
part.part.onSnapshot(partRef => {
const partObj = {
data: partRef.data(),
qty: part.qty
};
partsArr.push(partObj);
});
});
const bundleObj = {
title: bundle.data().title,
parts: partsArr
};
this.bundle.next(bundleObj);
});
return this.bundle;
}
I'm using Ionic/Angular for this, so when I return the item, it needs to be an array of objects. I'm sort of recreating the object to include each part on this init. As you can see, for each part in the bundles return, I am doing another onSnapshot. This seems incorrect to me.
Something that is dawning on me is that I probably should be making a single call to the user, which in turn returns everything? But how do I get the sub catalogs at that point? I'm not sure how to proceed without racking up a bill!
If you do a nested part.part.onSnapshot(partRef => { listener, be sure to manage those listeners. I know of three common approaches:
Once your outer onSnapshot listener disappears, the nested ones should probably also be stopped (as their data likely isn't needed anymore). This is a fairly simple approach, since you just need one list of listeners for the entire bundle
Alternatively you can manage each "part" listener based on the status of that part in the outer listener, removing the listener for "part1" when that disappears from the bundle. This can be made into a highly efficient solution, but does require (quite some) additional code.
Many developers use get()s for the nested document reads, as that means there is nothing to manage.
I will try to be as clear as possible but please ask if I am missing some detail.
So I have an application that I've inherited and it's running slow. It was built with Angular and Typescript on the front end, Loopback handling our CRUD operations and a Mongo DB. We isolated one of the reasons for the slowness as excess REST calls (potentially tens of thousands at times) so I am looking at reducing this. The first one I have found is a relatively small one so I hoped it'd help me to learn but I've already hit a challenge. Descriptions, table names and finer detail has been removed for simplicity.
On the first page, we present to the user a list of links to areas that they have access to. This is stored in an "Areas" table. We pull back all of the areas in a function similar to this:
this.ourApi.areas.find({
"filter": {
"include": ["generalInfo"]
/*
This table is related to most tables and
holds things like version numbers - it
is not relevant to this question
*/
}
}).$promise.then((returnedAreas) => {
/* Stuff */
});
After we have all the areas, we then have to get some configuration values for these area links (icons, mostly). Our configuration table holds configuration data for a number of items, not just Areas. So to get the relevant config information, we loop through the returnedAreas and, for each area, get the config where the relatedId matches the areaId. A bit like this:
return this.ourApi.configs.find({
"filter": {
"where": {
"relatedId": areaId
},
"include": ["generalInfo"]
}
}).$promise.then((areaConfigurations) => {
/* Stuff */
});
Ideally, I'd like to compress this down to just one call (and then mimic this pattern throughout the application). The only thing we really care about (and the only objects we actually use) are the configuration objects so we really don't need the "Areas" objects at all. The challenge I am unable to solve is how I adjust that second call above (that gets the config) so that it first gets the Areas and then uses the areaId in the query that gets the data that I care about. In SQl, this would be quite easy but I can't figure it out from the docs.
Essentially, I am looking for the Loopback equivalent of:
SELECT * FROM configs WHERE relatedId IN (SELECT areaId FROM areas)
Is this possible?
installed versions
ember-cli 2.14.2
ember-data 2.14.10
A little perspective:
I have a service called menu that performs store queries inside computed properties. One of these store queries is behaving rather weird. It fetches all records under the model name product-segment from a fully functional JSON API. This model has a n-n relationship to a model called product, referenced through hasMany DS objects:
models/product-segment.js
export default DS.Model.extend({
products: DS.hasMany('product'),
// ...
});
models/product.js
export default DS.Model.extend({
productSegments: DS.hasMany('product-segment'),
// ...
})
The problem:
Now, when I fetch these product-segment models, I instruct the API to { include: 'products' }, and the API does as is requested. The response includes 15 related product models for a particular product-segment, which is correct.
(let's call this particular product-segment segment x, it's the subject for all my debugging info below)
However, accessing the relationship collection on segment x from any context, at any time, only returns me 12 models, so 3 are missing. I witnessed similar issues with other product-segment models, so I don't think it's an issue with one specific model.
More perspective
I initially thought I was dealing with a race condition of some kind, and to be sure I created a computed property - test - to find out, and I dumped {{menu.test}} into my view to tickle the computed prop.
Here's the bare minimum info inside services/menu.js
export default Service.extend({
store: inject(),
activeProductSegment: null,
// As a note: this does not trigger an infinite loop
productSegments: computed('store.product.[]', 'store.product-segment.[]', function() {
return get(this, 'store').findAll('product-segment', { include: 'products' });
}),
test: computed('activeProductSegment', function() {
let segment = get(this, 'activeProductSegment');
if (segment) {
console.log(segment.hasMany('products').ids());
console.log(get(segment, 'products').mapBy('id'));
}
}),
});
The property activeProductSegment is being set to different product-segment model instances through an action of a component , which looks like this:
export default Component.extend({
menu: inject(), // menu service is injected here...
actions: {
setProductSegment(segment) {
get(this, 'menu').set('activeProductSegment', segment);
}
}
});
The action itself works as expected, and is actually quite unrelated to my problem. activeProductSegment is never updated in any other way. The view passes this action product-segment model objects:
{{#each menu.productSegments as |segment|}}
<li {{action 'setProductSegment' segment}}>{{segment.name}}</li>
{{/each}}
Trouble starts here
I set menu.activeProductSegment to segment x by clicking its associated <li> element.
When I now try to get all related product models of segment x, only 12 of 15 models are present within the returned collection. To be sure that the JSON response was really fine (i.e. the type definitions etc. are right) I checked the amount of product IDs that were registered at segment x. I logged the following line (the context of the logs below is in the Ember.Service snippet above):
console.log(segment.hasMany('products').ids());
That returned me an array with 15 correct IDs, so segment x has all id's as it is supposed to have. All product models of those id's have been included in the response, so I suppose there should be no issue with async data of some kind. Still, the following line gave me back an array of 12 id's:
console.log(get(segment, 'products').mapBy('id'));
I tried putting the 2 logs into a 2-second setTimeout, but the result stayed identical:
I'm starting to think this is a bug, since I noticed that the first time that an id was not accompanied by a model, is when for the first time the next ID is lower than the preceding ID.
Update on the above I tried a different order in the response, and note the second and the third id's: "7", "6". Guess this is not the problem:
Unless I misunderstand, the models should be live, so any relationship is supposed to update as data becomes available. I think it is very unlikely this has anything to do with malformed data.
What could be the cause for the missing models in the hasMany relationship collection, despite the fact that all necessary ids are properly registered at the hasMany relationship object, and we're not required to await arrival of any async/network data at this point? And what might be a suitable solution to the problem?
I know it does not appear to be related to async issues, but I would still try defining the hasMany as not async:
products: DS.hasMany('product', {async: true}),
I am using node.js with bookshelf as an ORM. I am a serious novice with this technology.
I have a situation where I have several columns in a database table. For the sake of this question, these columns shall be named 'sold_by_id', 'signed_off_by_id' and 'lead_developer_id', and are all columns that will reference a User table with an ID.
In other words, different User's in the system would at any point be associated with three different roles, not necessarily uniquely.
Going forward, I would need to be able to retrieve information in such ways as:
let soldByLastName = JobTicket.soldBy.get('last_name');
I've tried searching around and reading the documentation but I'm still very uncertain about how to achieve this. Obviously the below doesn't work and I'm aware that the second parameter is meant for the target table, but it illustrates the concept of what I'm trying to achieve.
// JobTicket.js
soldBy: function() {
return this.belongsTo(User, 'sold_by_id');
},
signedOffBy: function() {
return this.belongsTo(User, 'signed_off_by_id');
},
leadDeveloper: function() {
return this.belongsTo(User, 'lead_developer_id');
}
Obviously I would need a corresponding set of methods in User.js
I'm not sure where to start, can anyone point me in the right direction??
Or am I just a total idiot? ^_^
Your definitions look right. For using them it will be something like:
new JobTicket({ id: 33 })
.fetch({ withRelated: [ 'soldBy', 'signedOffBy' ] })
.then(jobTicket => {
console.log(jobTicket.related('soldBy').get('last_name');
});
Besides that I would recommend you to use the Registry plugin for referencing other models. That eases the pains of referencing models not yet loaded.
I am relatively new to Meteor, and I'm trying to create a web store for my sister-in-law that takes data from her existing Etsy store and puts a custom skin on it. I've defined all of my Meteor.methods to retrieve the data, and I've proofed the data with a series of console.log statements... So, the data is there, but it won't render on the screen. Here is an example of some of the code on the server side:
Meteor.methods({
...
'getShopSections': function() {
this.unblock();
var URL = baseURL + "/sections?api_key="+apiKey;
var response = Meteor.http.get(URL).data.results;
return response;
}
...
});
This method returns an array of Object. A sample bit of JSON string from one of the returned Objects from the array:
{
active_listing_count: 20,
rank: 2,
shop_section_id: 1******0,
title: "Example Title",
user_id: 2******7
}
After fetching this data without a hitch, I was ready to make the call from the client side, and I tried and failed in several different ways before a Google search landed me at this tutorial here: https://dzone.com/articles/integrating-external-apis-your
On the client side, I have a nav.js file with the following bit of code, adapted from the above tutorial:
Template.nav.rendered = function() {
Meteor.call('getShopSections', function(err, res) {
Session.set('sections', res);
return res;
});
};
Template.nav.helpers({
category: function() {
var sections = Session.get('sections');
return sections;
}
});
And a sample call from inside my nav.html template...
<ul>
{{#each category}}
<li>{{category.title}}</li>
{{/each}}
</ul>
So, there's a few things going on here that I'm unsure of. First and foremost, the DOM is not rendering any of the category.title String despite showing the appropriate number of li placeholders. Secondly, before I followed the above tutorial, I didn't define a Session variable. Considering that the list of shop categories should remain static once the template is loaded, I didn't think it was necessary from what I understand about Session variables... but for some reason this was the difference between the template displaying a single empty <li> tag versus a number of empty <li>'s equal to category.length --- so, even though I can't comprehend why the Session variable is needed in this instance, it did bring me one perceived step closer to my goal... I have tried a number of console.log statements on the client side, and I am 100% sure the data is defined and available, but when I check the source code in my Developer Tools window, the DOM just shows a number of empty li brackets.
Can any Meteor gurus explain why 1) the DOM is not rendering any of the titles, and 2) if the Session variable indeed necessary? Please let me know if more information is needed, and I'll be very happy to provide it. Thanks!
You set the data context when you use #each, so simply use:
<li>{{title}}</li>
If a Session is the right type of reactive variable to use here or not is hard to determine without knowing what you are doing but my rough guess is that a Mini Mongo collection may be better suited for what it appears you are doing.
To get you started on deciding the correct type of reactive variable to use for this head over to the full Meteor documentation and investigate: collections, sessions, and reactive vars.
Edit: To step back and clarify a bit, a Template helper is called a reactive computation. Reactive computations inside of helpers will only execute if they are used in their respective templates AND if you use a reactive variable inside of the computation. There are multiple types of reactive variable, each with their own attributes. Your code likely didn't work at all before you used Session because you were not using a reactive variable.