So using a list of ids, I'm trying to get data for each element in the database and add that element to a list. At the end, I'm displaying it on an HTML page. How would I do that? Right now, since it executes the queries async, the list will be empty after running the code to execute the query and add the result to a list.
Lets assume you have a table called teams that contains a row for each team, with columns like id, games, total_points, wins, and losses.
Lets assume you have an array of team id. You want to query the database for those records, then do some things with them before returning them to the front end as json.
This is how i understand your problem. If this isn't the case, please clarify in a comment on this answer.
Possible solution:
Do it all in one query. I'm going to use my favorite mysql library, knexjs.
var columns = ['id', 'games', 'total_points', 'wins', 'loses'];
function getTeams(ids){
return knex.select(columns).where('id', 'in', ids)
.then((teams) => {
//do whatever with array of teams
return teams;
}
}
It's pretty much as simple as that.
Related
Hi guys I have a problem with a multiple query in the firebase list
This query filters requests by these queries
But it brings me three results and correct there are only two results
this.all_being_executed = this.fdb.list('main_requests', ref => {
let query = ref.orderByChild('requests_receiving_status').equalTo(false);
query = ref.orderByChild('requests_status').equalTo(true);
return query;
}).valueChanges();
return this.all_being_executed;
Is there any help to filter the results correctly
You can use only one orderByChild() clause in your query, so it seems like your second clause is overwritting the first one and you're just getting the ones with request_status===true.
From the documentation:
Queries can only order by one key at a time. Calling orderByChild()
multiple times on the same query throws an error
My suggestion is to either refactor your current data structure to allow for a simpler query, or use a new third field which concatenates both requests_receiveing_status and request_status, them do your query on this concatenated field. Something like this:
{
"requests_receiving_status_and_status": "false_true";
}
and query it like this:
let query = ref.orderByChild('requests_receiving_status_and_status').equalTo('false_true');
Now, I know that this seems hacky and non-performant, but it is actually a valid approach for Firebase.
But, if you prefer a cleaner solution, try to separate your objects on two separated lists, one for requests_receiving_status true and the other for the false ones, them search only on the false ones using query = ref.orderByChild('requests_status').equalTo(true);. It adds redundancy but it is another valid approach on NoSql databases.
Also, check out this video and see if it helps you to tackle this issue.
Dataloader is able to batch and cache requests, but it can only be used by either calling load(key) or loadMany(keys).
The problem I am having is that sometimes I do not know they keys of the items I want to load in advance.
I am using an sql database and this works fine when the current object has a foreign key from a belongsTo relation with another model.
For example a user that belongs to a group and so has a groupId. To resolve the group you would just call groupLoader.load(groupId).
On the other hand, if I wanted to resolve the users within a group, of which there could be many I would want a query such as
SELECT * from users where user.groupId = theParticularGroupId
but a query such as this doesn't use the keys of the users and so I am not sure how make use of dataloader.
I could do another request to get the keys like
SELECT id from users where user.groupId = theParticularGroupId
and then call loadMany with those keys... But I could have just requested the data directly instead.
I noticed that dataloader has a prime(key, value) function which can be used to prime the cache, however that can only be done once the data is already fetched. At which point many queries would already have been sent, and duplicate data could have been fetched.
Another example would be the following query
query {
groups(limit: 10) {
id
...
users {
id
name
...
}
}
}
I cannot know the keys if I am searching for say the first or last 10 groups. Then once I have these 10 groups. I cannot know the keys of their users, and if each resolver would resolve the users using a query such as
SELECT * from users where user.groupId = theParticularGroupId
that query will be executed 10 times. Once the data is loaded I could now prime the cache, but the 10 requests have already been made.
Is there any way around this issue? Perhaps a different pattern or database structure or maybe dataloader isn't even the right solution.
You'll want a dataloader instance for the lookup you can do, in this case you have a group ID and you want the users:
import DataLoader from 'dataloader';
const userIdsForGroupLoader = new DataLoader(groupIds => batchGetUsersIdsForGroups(groupIds));
Now your batchGetUsersForGroups function is essentially has to convert an array of group IDs to an array of arrays of users (one array of user IDs for each group).
You'd start off with an IN query:
SELECT id from users where user.groupId in (...groupIds)
This will give you a single result set of users, which you'll have to manipulate, by grouping them by their groupId, the array should be ordered according to the original array of groupIds. Make sure you return an empty array for groupIds that don't have any users.
Note that in this we're only returning the user ids, but you can batch fetch the users in one go once you have them. You could tweak it slightly to return the users themselves, you'll have to decide for yourself if that's the right approach.
Everything I mention in this article can be achieved using clever use of Dataloader. But the key takeaway is that the values you pass to the load/loadMany functions don't have to correspond to the IDs of the objects you're trying to return.
I'm working with EmberJS and SailsJS. Now I've been asked to make a statistics page and handle the filtering process in the SailsJS.
I have a model for departments and another model for requests the relationship between these models is (request belongsTo department). For some reason my manager prevented me to make a (hasMany) relationship.
Now what I want to do is to loop through all the departments and store them in new Object, inside that loop I want to loop through all the requests by Using Request.count({where : { department : department.id }}) and get the number of requests for each department in the departments Array Of Object.
I tried to do it as I explained but the problem is when I log the department inside the request loop it gives me the result as I imagined but when I add (.id) it shows me 'undefined'.
Here is my code:
Department.find().then((departments) => {
report.departments = departments;
Request.count({ department : departments.id}).exec(function countMe(err, count) {
console.log(count);
})
})
PS: if there's any other approach for this task please tell me, I'm kind of a beginner.
Your departments object in the .then callback is an array of department objects. To get an id you'd need to do something like departments[0].id.
I might not recommend using .count because that means a separate trip to the database for each department. Sails waterline may have some way for you to count up Requests grouped by department id, but just to get a steamroller working example, I would first just get everything and do some processing in your code:
Department.find().then((departments) => {
Request.find().then((requests) => {
var requestCounts = {}; // we'll store the counts in this object
for (var idx = 0; idx < requests.length; idx++) {
if (!requestCounts[requests[idx].department]) {
requestCounts[requests[idx].department] = 0;
}
requestCounts[requests[idx].department]++;
}
// use requestCounts...
});
});
Creating a separate object like that may not be what you want to do, but something like this should serve whatever purpose you have. Notice, the code I wrote did not require finding all the department objects, but whatever you are doing likely will...
If you're interested in efficiency, then once you get this working, you can see if there is some way you can query the database to directly get the counts you want instead of this in-code processing. But I would start with something simple like this.
EDIT:
It sounds like you may be able to use something like:
Request.find().groupBy('department').sum('count').exec(function (err, results){
console.log(results);
});
But I'm finding conflicting reports on whether this works with sails-mongo, so take this as a "maybe this will work" recommendation.
I assume Request has a property department which is an ObjectId related to departments.
The quick one is to correct the following:
the 2nd line does not make sense as reports is not used
do
let departmentIds = _.map(departments, 'id');
and use departmentIds in the count query object
However, you are anyway finding all departments in the initial query, I would assume you count all the Requests anyway. Finding the departments first only makes sense in case you have query object there limiting the number of returned departments.
In addition, if you know the departmentIds in question you might not need to query them. Otherwise you might use a projection to just return the departmentIds instead of all the properties of all the departments
Ok, so I've been reading and reading and searching and searching and strangely it doesn't seem like my scenario has been really covered anywhere.
I have an app that creates a list of products. I want a simple view that can sort the products and page through them.
Fore reference here is a simple representation of the data in Firebase.
app
stock
unique_id
name
url
imageUrl
price
When creating the list I have multiple threads using the push method on my firebase references:
new Firebase(firebaseUrl).child('stock').push({
name: "name",
price: 123
});
This gives me a lovely "hash" collection on the stock property of the app.
So what I'd now like to do is have a table to sort and page through the records that were placed in the stock hash.
I make a GET request to my server to a url like /stock?limit=10&skip=10&sort=name%20asc. This particular url would be the second page where the table contained 10 records per page and was sorted by the name property in ascending order.
Currently in my query handler I have this:
var firebaseRef = new Firebase(firebaseUrl).child('stock');
if (this.sortDesc) {
firebaseRef = firebaseRef
.orderByChild(this.sortProperty)
.endAt()
.limitToFirst(this.limitAmount);
} else {
firebaseRef = firebaseRef
.orderByChild(this.sortProperty)
.limitToFirst(this.limitAmount);
if (this.skipAmount > 0) {
firebaseRef = firebaseRef.startAt(this.skipAmount);
}
}
firebaseRef.once('value', function (snapshot) {
var results = [];
snapshot.forEach(function (childSnapshot) {
results.push(childSnapshot.val());
});
callback(null, results);
});
I'm running into a couple of problems. I'm going to split this into two cases, ascending and descending queries.
Ascending query
The orderByChild and limitToFirst seems to work correctly in the sorting ascending case. This means I can change which property has an ascending sort and how many results to return. What I am not able to get to work is skipping n records for paging to work. In the example query above I'm going to the second page. I do not get results 11-20, but I instead get the same 10 records as the first page.
Descending query
In this case I cannot begin to figure out how to tell Firebase to order by a property of the object identified by the unique key in a descending fashion. The closest I've read is to use endAt() and then limit. Docs say the limit is deprecated plus this still doesn't help me with any paging.
I tired to do doodles picturing how this would work. I came up with: order by the property, start at the 'end' of the collection, and then limit back to the page size. While this still wouldn't solve paging I would expect it to give me the last n records where n was the size of the page. I get no results.
I suppose I could say use firebaseRef = firebaseRef .orderByChild(this.sortProperty).limitToLast(this.limitAmount + this.skipAmount); and in the result callback use the forEach loop to take the first (or would it be the last; I'm not sure how that iteration would work) n records where n=this.limitAmount. This just seems inefficient. Wouldn't it be better to limit the query instead of using CPU cycles to limit data that had come over the wire or is this the relational DB query thought pattern overriding the correct thought process for NoSQL?
Further Confusion
After posting this I've still been working on a solution. I've had some things get close, but I'm also running into this filtering issue. How could I filter a set of items to one property by still sorting on another? Jeez! I want to have the ability for a user to get all the stock that isn't sold out and order it by price.
Finally
Why hasn't this basic example been fleshed out on any of the Firebase "Getting Started" pages? Being able to show tabular data, page through it, sort, and filter seem like something that EVERY web developer would come across. I'm using ng-table in an Angular app to drive the view, but it still seems that regardless of platform that the queries that I'm trying to generate would be practical on any platform that Firebase supports. Perhaps I'm missing something! Please educate me!
Firebase and NoSQL
I've come up with this simple scenario that I often run into with web applications. I want to show tabular data, filter, page, and sort it. Very simple. Very common. Writing a SQL statement for this would be dead easy. Why is the query so complicated for something like Firebase. Is this common with all NoSQL solutions? There is no relational data being stored thus the need for a relational database seems unnecessary. Yet, it seems like I could hack together a little flat file to do this storage since the ability to make Firebase do these simple tasks is not made clear in its API or Docs. FRUSTRATED!!!
I want to query object from Parse DB through javascript, that has only 1 of some specific relation object. How can this criteria be achieved?
So I tried something like this, the equalTo() acts as a "contains" and it's not what I'm looking for, my code so far, which doesn't work:
var query = new Parse.Query("Item");
query.equalTo("relatedItems", someItem);
query.lessThan("relatedItems", 2);
It seems Parse do not provide a easy way to do this.
Without any other fields, if you know all the items then you could do the following:
var innerQuery = new Parse.Query('Item');
innerQuery.containedIn('relatedItems', [all items except someItem]);
var query = new Parse.Query('Item');
query.equalTo('relatedItems', someItem);
query.doesNotMatchKeyInQuery('objectId', 'objectId', innerQuery);
...
Otherwise, you might need to get all records and do filtering.
Update
Because of the data type relation, there are no ways to include the relation content into the results, you need to do another query to get the relation content.
The workaround might add a itemCount column and keep it updated whenever the item relation is modified and do:
query.equalTo('relatedItems', someItem);
query.equalTo('itemCount', 1);
There are a couple of ways you could do this.
I'm working on a project now where I have cells composed of users.
I currently have an afterSave trigger that does this:
const count = await cell.relation("members").query().count();
cell.put("memberCount",count);
This works pretty well.
There are other ways that I've considered in theory, but I've not used
them yet.
The right way would be to hack the ability to use select with dot
notation to grab a virtual field called relatedItems.length in the
query, but that would probably only work for me because I use PostGres
... mongo seems to be extremely limited in its ability to do this sort
of thing, which is why I would never make a database out of blobs of
json in the first place.
You could do a similar thing with an afterFind trigger. I'm experimenting with that now. I'm not sure if it will confuse
parse to get an attribute back which does not exist in its schema, but
I'll find out, by the end of today. I have found that if I jam an artificial attribute into the objects in the trigger, they are returned
along with the other data. What I'm not sure about is whether Parse will decide that the object is dirty, or, worse, decide that I'm creating a new attribute and store it to the database ... which could be filtered out with a beforeSave trigger, but not until after the data had all been sent to the cloud.
There is also a place where i had to do several queries from several
tables, and would have ended up with a lot of redundant data. So I wrote a cloud function which did the queries, and then returned a couple of lists of objects, and a few lists of objectId strings which
served as indexes. This worked pretty well for me. And tracking the
last load time and sending it back when I needed up update my data allowed me to limit myself to objects which had changed since my last query.