Setting up data structure for querying in Firebase - javascript

I am currently creating an app where the administrator should be able to tag images. The visitors can search for a tag, and see the images that have that tag.
One image can have more than one tag. This represents how I currently have set up my data:
Images: {
PUSH_ID: {
url: "https://cdn.filestackcontent.com/o0ze07FlQjabT9nuteaE",
tags: {
PUSH_ID: {
tag: "1324"
},
PUSH_ID: {
tag: "4321"
}
}
}
}
When a visitor searches for a tag, I need to be able to query the tag, and find the URL of the images that have the given tag. I was thinking that something along the lines of this would work:
ref.orderByChild('tags/tag').equalTo(tag).once("value"...)
But after some reading I have come to the understanding that you can only query one level deep in Firebase.
If this is the case, I need to restructure my data, but I cannot figure out how it should be structured.
Can anyone help me?
Btw; I have been told that I should use something like Elasticsearch to query in Firebase, but this app is for an event with a limited ammount of traffic.
Thanks!

When modeling data in Firebase (and in most NoSQL databases), you need to model the data for how your app wants to use it.
For example:
Images: {
PUSH_ID_1: {
url: "https://cdn.filestackcontent.com/o0ze07FlQjabT9nuteaE",
tags: {
"tag1324": true,
"tag4321": true
}
},
PUSH_ID_2: {
url: "https://cdn.filestackcontent.com/shjkds7e1ydhiu",
tags: {
"tag1324": true,
"tag5678": true
}
}
},
TagsToImages: {
"tag1324": {
PUSH_ID_1: true,
PUSH_ID_2: true
},
"tag4321": {
PUSH_ID_1: true
}
"tag5678": {
PUSH_ID_2: true
}
}
I changed a few things from you model:
I still store the tags for each image, so that you can show them when a user is watching a single image
But now we store the tags in a "tag1234": true format, which prevents a image from being tagged with the same tag multiple times. Whenever you feel the need for an array that you want to do a contains() operation on, consider using this approach which is more akin to a set.
I prefix the tag numbers with a static string, which prevents Firebase from trying to interpret the tag numbers as array indices.
We now also store a map of tags-to-image-ids. In this map you can easily look up all image IDs for a specific tag and then load the images in a loop.
We've essentially duplicated some data (the tags are stored twice) to ensure that we can look the information up in two ways. If you have more ways you want to access the data, you may need to replicate even more. This is normal in NoSQL databases and is part of the reason they scale so well.
I highly recommend reading this article on NoSQL data modeling.

And there is a Tags node in this structure like this, isnt it??
Tags: {
"tag1324": {
tagName: pets
},
"tag4321": {
tagName: daffodil
}
"tag5678": {
tagName: tasmanian wolf
}

Related

Is it possible to reset specific item in Apollo Client Cache?

I have a query which uses fetchMore and the relayPagination which works fine for lazy loading when using the variables page and perPage, the issue now is I'm trying to refresh the query whenever i update the other variables for filtering like date and type which values can be either debited or credited. It fetches the data, but then appends the incoming data to apollo cache instead of replacing the old data with the new one.
for example in this sample typePolicy
PaginatedBooks: {
fields: {
allBooks: {
merge: relayPagination()
}
}
}
AllProducts: {
// Singleton types that have no identifying field can use an empty
// array for their keyFields.
keyFields: [],
},
I only want to reset PaginatedBooks
I've tried use fetchPolicy of no-cache but this stops pagination and fetchMore from working as i can't merging existing and incoming data. I opted to use client.resetStore(): https://www.apollographql.com/docs/react/api/core/ApolloClient/#ApolloClient.resetStore
but this also refetches other active queries and causes the ui to flicker. So far looking through the documentation and github repo I can't seem to find anything or anyone who has tried to do something similar, so I'm hoping I can get some insight and probably be offered a better solution. Thanks in advance

Query firebase realtime database where child has property

I'm trying to query my Firebase Realtime Database to find all games a user belongs to.
I have a list of games, each game has a property of players. Each child of players has a key of $uid, and within that {key: $uid, name: "joe"}
Is it possible to get all games this way? Or do I need to start keeping another index of players_games/$uid/$game?
I've tried firebase.database().ref('games').orderByChild('players').equalTo(token.uid), but this yields null
It looks like database.ref('games').orderByChild('players/${token.uid}') works, but then I'd need to give .read access to all of games, or do this server-side.
Your current data structure makes it easy to find all the users for a specific game. It does not however make it easy to find all the games for a specific user. To allow that, you'll want to add an addition data structure that inverts the information.
So that'd look something like this:
player_games: {
"XDYNyN8il6TDsM4LuttwDzNuytj1": {
"-M5vf...U5zK": true
},
"NxH14...mxY2": {
"-M5vf...U5zK": true
}
}
Also see:
Firebase query if child of child contains a value
Firebase Query Double Nested
I recommend you also study the Firebase documentation on structuring your database, specifically the section on avoiding nested data. By mixing entity types as you currently do, you'll likely run into problems with security, and scalability.
The most idiomatic way to model your many-to-many relationship in the Firebase database is with four top-level lists:
players: {
$playerId: { ... }
}
games: {
$gameId: { ... }
}
player_games: {
$playerId: {
$gameId: true
}
}
game_players: {
$gameId: {
$playerId: true
}
}
Also see:
Many to Many relationship in Firebase

Using Apollo's writeFragment to update nested list

I am working on a application in which a ship can be configured using rudders and other stuff. The database structure is sort of nested, and so far I have been keeping my GraphQL queries in correspondence with the database.
That means: I could fetch a ship using some query ship(projectId, shipId), but instead I am using a nested query:
query {
project(id:1) {
id
title
ship(id:1) {
id
name
rudders {
id
position
}
}
}
}
Such a structure of course leads to a lot of nested arrays. For example, if I have just added a new rudder, I would have to retrieve using cache.readQuery, which gives me the project object rather than the rudder list. To add the rudder to the cache, I'd get a long line with nested, destructured objects, making the code hard to read.
So I thought of using GraphQL fragments. On the internet, I see them being used a lot to prevent having to re-type several fields on extensive objects (which I personally find very useful as well!). However, there are not so many examples where a fragment is used for an array.
Fragments for arrays could save all the object destructuring when appending some data to an array that is nested in some cached query. Using Apollo's readFragment and writeFragment, I managed to get something working.
The fragment:
export const FRAGMENT_RUDDER_ARRAY = gql`
fragment rudderArray on ShipObject {
rudders {
id
position
}
}
`
Used in the main ship query:
query {
project(id: ...) {
id
title
ship(id: ...) {
id
name
...rudderArray
}
}
}
${RUDDER_FRAGMENT_ARRAY}
Using this, I can write a much clearer update() function to update Apollo's cache after a mutation. See below:
const [ createRudder ] = useMutation(CREATE_RUDDER_MUTATION, {
onError: (error) => { console.log(JSON.stringify(error))},
update(cache, {data: {createRudder}}) {
const {rudders} = cache.readFragment({
id: `ShipObject:${shipId}`,
fragment: FRAGMENT_RUDDER_ARRAY,
fragmentName: 'rudderArray'
});
cache.writeFragment({
id: `ShipObject:${shipId}`,
fragment: FRAGMENT_RUDDER_ARRAY,
fragmentName: 'rudderArray',
data: {rudders: rudders.concat(createRudder.rudder)}
});
}
});
Now what is my question? Well, since I almost never see fragments being used for this end, I find this working well, but I am wondering if there's any drawbacks to this.
On the other hand, I also decided to share this because I could not find any examples. So if this is a good idea, feel free to use the pattern!

Get user's events with Firebase (many to many relationship)

I'm a Front-End Developer and for the first-time, I'm using Firebase to build an application. I read some documentation/articles and watched this interesting video about foreign keys (many to many relationship): https://www.youtube.com/watch?v=ran_Ylug7AE&list=PLl-K7zZEsYLlP-k-RKFa7RyNPa9_wCH2s&index=2.
I'm trying to apply this approach to retrieve events for a specific user. Ideally, I want to get this information without requesting/paying for too much data. First, this is how I set-up the following database structure (see video):
{
users:
"abc": {
firstname: "Maxime",
...
},
"def": {
firstname: "John",
...
},
},
events: {
"eventAbc-": {
title: "Some great event",
...
},
"eventDef": {
title: "Another great event",
...
}
},
eventAttendees: {
"eventAbc": {
abc: true,
def: true,
},
"eventDef": {
abc: true,
}
}
}
To get user's events, I have the following which actually works:
getEvents(userId) {
const self = this;
const query = firebase.firestore().collection('eventAttendees');
var promises = [];
query.onSnapshot(function(snap) {
snap.docs.forEach(doc => {
const data = doc.data();
if (data[userId]) {
// user has been invited to this event
promises.push(self.getEvent(doc.id));
}
});
Promise.all(promises).then((results) => {
console.log("All events");
console.log(results);
});
});
}
getEvent(eventId) {
return new Promise((resolve) => {
const query = firebase.firestore()
.collection('events')
.doc(eventId);
query.onSnapshot(function(snap) {
resolve(snap.data());
});
});
}
getEvents('abc');
Questions
I don't think/know if I have the right and optimized approach? Any documentation/Github project I should look into as reference?
What does happen if 'abc' has been invited to 1 million events? I feel, I'm looping and need to handle pagination out of the box. Is there a better way?
Let's assume, each event has a lot of information (details). However, on the homepage, I just need to display main information (event title, event date). Where should I store that to avoid loading a lot of information.
Hopefully, there is someone who can find the time to reply to my questions.
Thanks.
You seem to be watching David's great video series Firebase for SQL developers, which was written for the Firebase Realtime Database. Since your code uses the Cloud Firestore API, I'd recommend switching over to Todd's Getting to know Cloud Firestore series, which contains similar information and much more, but then tailored towards Firestore.
What does happen if 'abc' has been invited to 1 million events?
No user is ever going to watch those million events. So don't load them. Think of how many items a user will realistically see, and then load those. Typically this will be one or a few screenfuls. If you think they may want to load more, allow them to load a new page. Search firebase and pagination to see many questions about that topic.
Let's assume, each event has a lot of information (details). However, on the homepage, I just need to display main information (event title, event date). Where should I store that to avoid loading a lot of information.
You should only load the data you need. If you need a subset of the data for each event in the initial list view, create a separate node or collection with just that information for each event. Yes, you're indeed duplicating data that way, but that is quite normal in NoSQL databases.
Thanks for taking the time to reply to my questions. Indeed, I like this video series and I'm looking forward to watch the one you've recommended which seems to be more recent.
I agree with you and definitely want to load only what's needed. At the moment, I don't have much datas so it's difficult for me to conceive/anticipate that. I should have probably taken advantage of .limit(10) or .startAt() in the function getEvents?
I'm not familiar with database in general and don't really know where I should have a subset of the data for each event? Is the following what you would call initial list view? For my example, is it the right place? If not, where would put that?
eventAttendees: {
"eventAbc": {
title: "Some great event",
eventDate: "",
moreKeyInformation: "",
abc: true,
def: true,
},
"eventDef": {
title: "Another great event",
eventDate: "",
moreKeyInformation: "",
abc: true,
}
}

Redux data structuring

I'm trying to build a complex fully-dynamic app with Redux. I mean my App has lots of dynamic-generated forms with generated fields-components on-the-fly. I want to store in my Redux-store visual data about my components too. But how should i do it without mixing real data with visual component data?
For example if i have structure like this
Store {
visual: {...deeply nested visual-data-tree...},
data: {...deeply-nested real-data-tree...}
}
It is hard to render component because i need to search visual data first, then react component "value" in two trees.
But if have a structure similar to this:
Store {
form {
visual: {...form visual data...},
data: {
//Ok here the form "data" - widgets. Or it must to be visual? :)
widget1 {
visual: {type:"ComboBox", opened: true},
data: 1
}
}
}
}
You see the problem, now i have visual data inside real data of Form widget.
(form - data - widget1 - visual)
Visual data inside the real data is out of the concept.
How do you guys solve same problems of mixing data?
Really sorry for my poor english. I hope i clearly explained the problem.
Isn't the distinction superficial? I think a more important rule is that the data in the state should be normalized. For example, if you have Combobox widget letting you choose users, your data shape better be
{
chosenUserId: 10, // Good!
users: {
10: { name: 'Alice' }
}
rather than
{
chosenUser: { name: 'Alice' }, // Bad!
users: {
10: { name: 'Alice' }
}
If the data is duplicated in the state tree, it's hard to update it correctly and avoid inconsistencies.
As long as you keep the data normalized, I see no real need to divide visual and data. You might want to have top-level entity cache that looks like a database (e.g. entities which includes users, posts, or whatever data objects your app uses), but other than that, go with whatever state shape feels most comfortable when retrieving the relevant state.

Categories