Firebase query taking forever - javascript

My firebase data looks like this...
In JSON down below...
I've been trying to use what I've found in their docs to query the db so that I end up with of all the terms and their keys. How do I query just the term property and key for every node under vocab? I'm using this to populate an autosuggest. Also, the 0 and 1 under the vocab node are keys... they showed up that way when I imported the data as a JSON file. Any new vocab goes in with unique long key values though.
I've tried both .on() and .equalTo()... But I have had no luck whatsoever. It always seems to be grabbing everything in the entire vocab node of the database and taking on the order of 7 whole seconds to do it each time. Here is an example of one of my attempts where I try to put just the terms into an array...
let termList = [];
const termsRef = firebase.database().ref('vocab/');
termsRef.on('child_added', snap => termList.push(snap.val().term));
This doesn't even get the keys (which I need) and it still takes around 7 seconds to do it. I've seen videos where people do things like this in one line and it goes quick... not my luck. Any help would be much appreciated.
As requested here is a JSON sample of one of the database nodes... there are over 400 like this... the relevant property is term... all the way at the bottom. I need an array of terms and firebase database keys. How do I get them with a query?
{
"date" : "2016-07-26T14:50:10.906Z",
"defs" : [ {
"AddedBy" : "",
"date" : "2016-07-26T14:50:10.906Z",
"def" : "it's your need to fulfill or to obtain your full potential, and have meaningful goals.",
"image" : ""
}, {
"AddedBy" : "",
"date" : "2016-07-26T14:50:10.906Z",
"def" : "the very last stage in priority in order to be happy.",
"image" : ""
}, {
"AddedBy" : "",
"date" : "2016-07-26T14:50:10.906Z",
"def" : "the need to be able to set goals and reach them and be able to accept one's self.",
"image" : ""
} ],
"examples" : [ {
"AddedBy" : "",
"addDate" : "2016-07-26T14:50:10.906Z",
"approved" : true,
"checkDate" : "2016-07-26T14:50:10.906Z",
"example" : "a self-actualizing person is a person who accepts themselves, sets realistic and meaningful goals and accepts change.",
"feedback" : {
"comment" : "",
"grammarTrouble" : false,
"incorrect" : false,
"moreSpecific" : false,
"noContext" : false
},
"nonExample" : false,
"seenRecently" : [ {
"sawOn" : "",
"user" : ""
} ]
}, {
"AddedBy" : "",
"addDate" : "2016-07-26T14:50:10.906Z",
"approved" : true,
"checkDate" : "2016-07-26T14:50:10.906Z",
"example" : "who is not self-actualizing is a person who dislikes themselves, has no goals in life, and refuses to accept change.",
"feedback" : {
"comment" : "",
"grammarTrouble" : false,
"incorrect" : false,
"moreSpecific" : false,
"noContext" : false
},
"nonExample" : false,
"seenRecently" : [ {
"sawOn" : "",
"user" : ""
} ]
}, {
"AddedBy" : "",
"addDate" : "2016-07-26T14:50:10.906Z",
"approved" : true,
"checkDate" : "2016-07-26T14:50:10.906Z",
"example" : "Someone who achieves to their best personal ability and faces every situation head on.",
"feedback" : {
"comment" : "",
"grammarTrouble" : false,
"incorrect" : false,
"moreSpecific" : false,
"noContext" : false
},
"nonExample" : false,
"seenRecently" : [ {
"sawOn" : "",
"user" : ""
} ]
} ],
"term" : "Self Actualization",
"unit" : 0
},

Related

Firebase realtime database - orderByChild not returning any result

I am using the Firebase Realtime Database functions .orderByChild() and .equalTo() to fetch data nodes to be updated. However, I cannot seem to get any results back from my queries.
My database structure
{
"4QEg0TWDbESiMX8Cu8cvUCm17so2" : {
"-LmGtXsgJbAvVS8gv5-E" : {
"createdAt" : 1565815876803,
"message" : "Hello",
"isSender" : false,
"sender" : {
"alias" : "",
"name" : "Person A",
"sid" : "mUO3DtYY2yRw3zkv4EmTlfldB3S2"
},
"sysStatus" : 0
},
"-LmGtt4nyuygG9B4s6__" : {
"createdAt" : 1565815967746,
"message" : "Hej!",
"isSender" : true,
"sender" : {
"alias" : "",
"name" : "Person B",
"sid" : "4QEg0TWDbESiMX8Cu8cvUCm17so2"
},
"sysStatus" : 0
},
"-LmJxcvL_Y7JmojxPiiK" : {
"createdAt" : 1565867281849,
"isSender" : true,
"message" : "111",
"sender" : {
"alias" : "",
"name" : "Person B",
"sid" : "4QEg0TWDbESiMX8Cu8cvUCm17so2"
},
"sysStatus" : 0
}
},
"mUO3DtYY2yRw3zkv4EmTlfldB3S2" : {
"222" : {
"createdAt" : 1565867281849,
"isSender" : true,
"message" : "Test",
"sender" : {
"alias" : "",
"name" : "Person A",
"sid" : "mUO3DtYY2yRw3zkv4EmTlfldB3S2"
},
"sysStatus" : 0
},
"333" : { <-- This is one node I wish to fetch
"createdAt" : 1565815967746,
"isSender" : false,
"message" : "123",
"sender" : {
"alias" : "",
"name" : "Person B",
"sid" : "4QEg0TWDbESiMX8Cu8cvUCm17so2" <-- This is the value I am trying to match
},
"sysStatus" : 0
}
},
"rKNUGgdKqdP68T0ne6wmgJzcCE82" : {
"-Lm_GTmNHyxqCqQl4D4Z" : { <-- This is one node I wish to fetch
"createdAt" : 1566140917160,
"isSender" : false,
"message" : "Sesame",
"sender" : {
"alias" : "",
"name" : "Person B",
"sid" : "4QEg0TWDbESiMX8Cu8cvUCm17so2" <-- This is the value I am trying to match
},
"sysStatus" : 5
}
}
}
My code
dbRoot.child('messages')
.orderByChild('sid')
.equalTo(userId)
.once('value', (senderSnapshot) => {
console.log('senderSnapshot', senderSnapshot.val())
console.log('senderSnapshot amount', senderSnapshot.numChildren())
senderSnapshot.forEach((sender)=>{
//Do the work!
})
})
The code logs
senderSnapshot null
senderSnapshot amount 0
I have manually checked that there are several nodes where "sid" is set to the "userId" I am looking for.
Why am I not getting any results back from my query?
It seems like I have to search dbRoot.child('messages/rKNUGgdKqdP68T0ne6wmgJzcCE82') to get my value. :/ (And then repeat the search for each user)
How much extra data overhead would it be to download/collect all contacts and then loop thru each users contact?
Firebase Database queries allow you to search through the direct child nodes at a certain location for a value at a fixed path under each child. So: if you know the user whose message you want to search through, you can do:
dbRoot
.child('messages')
.child('mUO3DtYY2yRw3zkv4EmTlfldB3S2') // the user ID
.orderByChild('sender/sid')
.equalTo(userId)
Note the two changes I made here from your code:
We now start the search from messages/mUO3DtYY2yRw3zkv4EmTlfldB3S2, so that it only searches the messages for that one user.
We now order/filter on the sender/sid property for each child under messages/mUO3DtYY2yRw3zkv4EmTlfldB3S2.
By combining these we are essentially searching a flat list of child nodes, for a specific matching value at a path under each child node.
There is no way in your current data structure to find all messages for a specific sender/sid value across all users. To allow that you'll have to add an additional data structure, where you essentially invert the current data.
"sender_messages": {
"mUO3DtYY2yRw3zkv4EmTlfldB3S2": {
"4QEg0TWDbESiMX8Cu8cvUCm17so2/-LmGtXsgJbAvVS8gv5-E": true,
},
"4QEg0TWDbESiMX8Cu8cvUCm17so2": {
"4QEg0TWDbESiMX8Cu8cvUCm17so2/-LmGtXsgJbAvVS8gv5-E": true,
"4QEg0TWDbESiMX8Cu8cvUCm17so2/-LmGtt4nyuygG9B4s6__": true,
"4QEg0TWDbESiMX8Cu8cvUCm17so2/-LmJxcvL_Y7JmojxPiiK": true,
"mUO3DtYY2yRw3zkv4EmTlfldB3S2/333": true,
"rKNUGgdKqdP68T0ne6wmgJzcCE82/Lm_GTmNHyxqCqQl4D4Z": true
},
"mUO3DtYY2yRw3zkv4EmTlfldB3S2": {
"mUO3DtYY2yRw3zkv4EmTlfldB3S2/222": true,
}
}
Now you can find the messages for a specific sender/sid by reading:
dbRoot
.child('sender_messages')
.child('4QEg0TWDbESiMX8Cu8cvUCm17so2') // the sender ID
And then looping over the results, and loading the individual messages from each path as needed.
This is quite common in NoSQL databases: you'll often have to modify your data structure to allow the use-cases you want to add to your app.
See also:
Firebase query if child of child contains a value
Firebase Query Double Nested
Speed up fetching posts for my social network app by using query instead of observing a single event repeatedly (to learn why loading the additional messages is not nearly as slow as you may initially think)

MongoDB converting from int32 to double itself

I am getting a weird problem when I run in shell for mongoDB.
My collection structure is like below in mongoDB.The type of ServiceList items must be integer.
{
"_id" : ObjectId("56565"),
"Contact" : "",
"Email" : "",
"AppList" : [
{
"Code" : "",
"Usage" : NumberInt(542),
"LicenceType" : "Monthly",
"CreatedDate" : ISODate("2015-07-29T06:15:20.520+0300"),
"ServiceList" : [
1,5,8
]
}
]
}
My aim to run this code is converting Usage type from int32 to int64.It works fine.When I run the code below. something weird occours, and the type of the items inside AppList.ServiceList is converted to double from int32 itself.
db.CustomerInfo.find({}).forEach( function (item) {
if(item.AppList != undefined){
item.AppList.forEach(function (x){
x.Usage = NumberLong(x.Usage);
});
db.CustomerInfo.save(item);
}
});
and it seems like this,
{
"_id" : ObjectId("56565"),
"Contact" : "",
"Email" : "",
"AppList" : [
{
"Code" : "",
"Usage" : NumberLong(542),
"LicenceType" : "Monthly",
"CreatedDate" : ISODate("2015-07-29T06:15:20.520+0300"),
"ServiceList" : [
1.0,5.0,8.0
]
}
]
}
How can I solve this problem ?
MongoDB version:3.4.9
MongoShell version:v3.4.9

Apply an expression to each array element in document

I've got a sample document that I'm trying to project within a MongoDB aggregate pipeline. I'm testing with a single document that looks roughly like this:
{
"_id" : "",
"title" : "Questions",
"sortIndex" : 0,
"topics" : [
{
"_id" : "",
"title" : "Creating a Question",
"sortIndex" : 1,
"thumbnail" : "CreatingAQuestion.jpg",
"seenBy" : [ "user101", "user202" ],
"pages" : [
{
"visual" : "SelectPlanets.gif",
"text" : "Some Markdown"
}
]
},
{
"_id" : "",
"title" : "Deleting a Question",
"sortIndex" : 0,
"thumbnail" : "DeletingAQuestion.jpg",
"seenBy" : [ "user101" ],
"pages" : [
{
"visual" : "SelectCard.gif",
"text" : "Some Markdown"
}
]
}
]
}
The output I'm trying to obtain is something along these lines:
{
"_id" : "",
"title" : "Questions",
"topics" : [
{
"title" : "Creating a Question",
"thumbnail" : "CreatingAQuestion.jpg",
"seen" : true
},
{
"title" : "Deleting a Question",
"thumbnail" : "DeletingAQuestion.jpg",
"seen" : false
}
]
}
Specifically the bit I'm struggling with is the seen flag.
I've read the docs which state:
When projecting or adding/resetting a field within an embedded document...
... Or you can nest the fields:
contact: { address: { country: <1 or 0 or expression> } }
I wish to use an expression and I took note of the following:
When nesting the fields, you cannot use dot notation inside the embedded document to specify the field, e.g. contact: { "address.country": <1 or 0 or expression> } is invalid.
So I'm trying to work out how to "reference" a subdocument field within an expression. That quote suggests I can't use dot notation but when I can't seem to get it working with nested notation either. Here's what I've got so far:
db
.getCollection('chapters')
.aggregate([
{
$project: {
title: 1,
topics: {
title: 1,
thumbnail: 1,
publishedAt: 1,
test: "$seenBy",
seen: { $in: ["user202", "$seenBy"] },
}
}
}
])
So I've hard coded user202 into my query for now, and expected to see true and false for the 2 documents. I've also put in a test field to map out the seenBy field from the sub-document. What this produces is:
{
"_id" : "",
"title" : "Questions",
"topics" : [
{
"title" : "Creating a Question",
"thumbnail" : "CreatingAQuestion.jpg",
"test" : [
"user101",
"user202"
],
"seen" : true
},
{
"title" : "Deleting a Question",
"thumbnail" : "DeletingAQuestion.jpg",
"test" : [
"user101",
"user202"
],
"seen" : true
}
]
}
So obviously my "$seenBy" isn't accessing the correct topic because the test field contains the data from the 1st document.
So ultimately my question is, how can I access the seenBy field within a subdocument, referring to the current subdocument so I can create an expression?
Note: I have got this working with multiple $project and an $unwind but wanted to try compress/clean it up a bit.
You really need to use $map here. Simply notating the array in projection ( which is a bit of a boon since MongoDB 3.2 ) does not really cut it when you need a localized value for the current element. That is what you need and it's what $map provides:
db.getCollection('chapters').aggregate([
{ $project: {
title: 1,
topics: {
$map: {
input: "$topics",
as: "t",
in: {
title: "$$t.title",
thumbnail: "$$t.thumbnail",
publishedAt: "$$t.publishedAt",
test: "$$t.seenBy",
seen: { $in: ["user202", "$$t.seenBy"] },
}
}
}}
])
So for each element the current value of "seenBy" as a property is being tested by the expression. Without the $map that is not possible, and you can only really notate the "whole" array. Which is really not what you want to test here.

Display relational data in Firebase using AngularJS

I want to display content using a flat relational data structure (similar to that of Firefeed). Everything is working the way it should, except I can't figure out how to display the actual content. I'm struggling for a while now and I think I'm almost there actually, but something is still missing.
I have two references:
var userFeedRef = new Firebase(FIREBASE_URL).child("users").child($rootScope.currentUser.$id).child("feed");
var uploadsRef = new Firebase(FIREBASE_URL).child("uploads");
The JSON looks like this:
{
"uploads" : {
"-KGkxzQj0FM2CMAXo5Em" : {
"author" : "87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d",
"date" : 1462184179564,
"name" : "Zizazorro",
"reasonUpload" : "Test",
"startSec" : 80,
"uploadTitle" : "Kid Ink - Promise (Audio) ft. Fetty Wap"
},
"-KGlCoD7kEa1k3DaAtlG" : {
"author" : "87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d",
"date" : 1462188328130,
"name" : "Zizazorro",
"reasonUpload" : "Test2",
"startSec" : 80,
"uploadTitle" : "Kid Ink - Show Me ft. Chris Brown"
}
},
"users" : {
"87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d" : {
"date" : 1459369248532,
"feed" : {
"-KGkxzQj0FM2CMAXo5Em" : true,
"-KGlCoD7kEa1k3DaAtlG" : true
},
"firstname" : "Bob",
"followers" : {
"e3de536b-03a1-4fb4-b637-ebcebaae55c6" : true
},
"following" : {
"e3de536b-03a1-4fb4-b637-ebcebaae55c6" : true
},
"regUser" : "87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d",
"uploads" : {
"-KGkxzQj0FM2CMAXo5Em" : true,
"-KGlCoD7kEa1k3DaAtlG" : true
},
"username" : "Zizazorro"
},
"e3de536b-03a1-4fb4-b637-ebcebaae55c6" : {
"date" : 1459369285026,
"feed" : {
"-KGkxzQj0FM2CMAXo5Em" : true,
"-KGlCoD7kEa1k3DaAtlG" : true
},
"firstname" : "Anna",
"followers" : {
"87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d" : true
},
"following" : {
"87a8b6c8-7f72-45c2-a8c5-bb93fe9d320d" : true
},
"regUser" : "e3de536b-03a1-4fb4-b637-ebcebaae55c6",
"username" : "Sven8k"
}
}
}
The problem is: How can I display the actual content of the upload, when there is only a ID reference? I need a reference to the IDs for a user (userFeedRef) and a reference to the actual content of those specific IDs, not all uploads (uploadsRef).
How can I display this user ng-repeat in my html? So for example: ng-repeat {{feed.reasonUpload}} for user1 has to show:
Test
Test2
EDIT I've looked at this example, but I can't figure out how to render the content on the actual html feed in my case
var commentsRef =
new Firebase("https://awesome.firebaseio-demo.com/comments");
var linkRef =
new Firebase("https://awesome.firebaseio-demo.com/links");
var linkCommentsRef = linkRef.child(LINK_ID).child("comments");
linkCommentsRef.on("child_added", function(snap) {
commentsRef.child(snap.key()).once("value", function() {
// Render the comment on the link page.
));
});
You can do something like this:
$scope.finalData = (Object).values($scope.firebaseData.uploads);
//$scope.firebaseData is data what you retrieving from firebase.
Hope this plunker will help you.

Combining orderByChild and equalTo in Firebase queries

Given the following data structure:
{
"comments" : {
"-JcBbk64Gpm1SKoFHv8b" : {
"content" : "blah",
"createdAt" : 1417550954985,
"link" : "http%3A%2F%2Flocalhost%3A3000%2F",
"recommendedCount" : 0,
"replies" : {
"-JcBbk8gF_nQ_vjwag61" : true
},
"replyCount" : 1
},
"-JcBbk8gF_nQ_vjwag61" : {
"content" : "blah blah",
"createdAt" : 1417550955151,
"link" : "http%3A%2F%2Flocalhost%3A3000%2F",
"recommendedCount" : 0,
"replyCount" : 1,
"replyToComment" : "-JcBbk64Gpm1SKoFHv8b"
}
},
"links" : {
"http%3A%2F%2Flocalhost%3A3000%2F" : {
"author" : 5,
"commentCount" : 2,
"comments" : {
"-JcBbk64Gpm1SKoFHv8b" : true,
"-JcBbk8gF_nQ_vjwag61" : true
},
"createdAt" : 1417550954931,
"ratingCount" : 2,
"recommendedCount" : 32,
"score" : 91,
"title" : "A Christian vs. an Atheist: Round 2",
"url" : "http://localhost:3000/"
}
}
}
I would like to retrieve all the comments for a given link and sort them in reverse chronological order. I've gotten this far, but I can't figure out how to do the reverse sort because I've already used orderByChild to narrow down the results by link:
ref.orderByChild('link').equalTo(currentLink).on('value', function (snap) {
console.log(snapshot.val());
});
If I call orderByChild() a second time like this:
ref.orderByChild('link').equalTo(currentLink).orderByChild('createdAt').on('value', function (snap) {
console.log(snapshot.val());
});
it fails with this error message:
Uncaught Error: Query.orderByChild: You can't combine multiple orderBy calls.
I'm stumped. Any suggestions?
I encountered the same issues on my firebase project and I solved that like this.
Please take a look.
var formRef = firebase.child('links').child(currentLink);
formRef.orderByChild('createdAt').on('value', function (snap) {
console.log(snapshot.val());
});
It will works.
Best Regards.

Categories