Pagination for Firebase Realtime database - javascript

I have been trying to search for a way to do pagination for Firebase Realtime Database. I see a lot of tutorial/articles on pagination for Cloud Firestore but nothing for Realtime Database. Below is my code and its working as expected. Can anyone point me in the right direction for adding pagination to this? If even possible? Any help would be appreciated.
const [ufoSightings, setUfoSightings] = useState([]);
const [userStateSelection, setUserStateSelection] = useState("");
useEffect(() => {
let allUfo = [];
//referencing firebase db
const ufoRef = firebase.database().ref("ufos");
//filter database searching for specific state user is looking for
const query = ufoRef
.orderByChild("state")
.equalTo(`${userStateSelection}`)
.limitToFirst(12);
query.once("value").then((snapshot) => {
//storing ufoSightings in state
snapshot.forEach((snap) => {
allUfo.push(snap.val());
});
setUfoSightings(allUfo);
});
}, [userStateSelection]);

when I tried Frank van Puffelen's answer to query for the next page, I got an error that says
Error: startAfter: Starting point was already set (by another call to startAt, startAfter, or equalTo).
After going through this firebase documentation here is a query that worked for me to get next page
const query = ufoRef.orderByChild("state")
.startAt(`${userStateSelection}`,lastKey)
.endAt(`${userStateSelection}`).limitToFirst(12)
.once("value");
The only drawback is lastKey's list value will be fetched every time you query for the next page which is redundant, so you have to take care of that.
The reason why it worked for me:
From this firebase documentation
startAt ( value : number | string | boolean | null , key ? : string ) : Query
The starting point is inclusive, so children with exactly the specified value will be included in the query. The optional key argument can be used to further limit the range of the query. If it is specified, then children that have exactly the specified value must also have a key name greater than or equal to the specified key.
endAt ( value : number | string | boolean | null , key ? : string ) : Query
The ending point is inclusive, so children with exactly the specified value will be included in the query. The optional key argument can be used to further limit the range of the query. If it is specified, then children that have exactly the specified value must also have a key name less than or equal to the specified key.
So the query will try to fetch
state >= userStateSelection && state <= userStateSelection (lexicographically)
which will equal to
state == userStateSelection

To get the next page, you pass in the state and key of the node to start at or after
So say you capture the values in your listener with this:
var lastState, lastKey;
query.once("value").then((snapshot) => {
//storing ufoSightings in state
snapshot.forEach((snap) => {
allUfo.push(snap.val());
lastState = snap.val().state; // 👈
lastKey = snap.key; // 👈
});
setUfoSightings(allUfo);
});
Now you can get the next page with this query:
const query = ufoRef
.orderByChild("state")
.equalTo(`${userStateSelection}`)
.startAfter(lastState, lastKey) // 👈
.limitToFirst(12);
The startAfter() method is relatively new to the Realtime Database, so if you can't find it or are having trouble with it try the (much older) startAt() method with the same arguments.
Also check out some of the many other questions on firebase-realtime-database pagination, as this has been covered here quite frequently already.

Related

Generate Firestore document's doc Id based on Users' uids

In my chat app, I have private chat between the two users. I intend to set the chat document's id using these two user's docId/uid in such a way that it doesn't depend on the order they're combined and I can determine the chat document's docId using the uid of users irrespective of the order of uid.
I know, I can use where clauses to get the chat doc as well. Is there any major flaw with my approach of generating the chat document's docId? Should I let it be generated automatically and use normal where clauses supported by firestore and limit(1) to get the chat?
basically, it seems I'm looking for is to encrypt uid1 in such a way that it returns a number only and then same with uid2 and then add them together to create the ChatId. This way it'll not depend on the order I use to add them and I can get the chatId and maybe convert that number back to a string using Base64 encode. This way, if I know the users participating in the chat, I can generate the same ChatId. Will that work or is there any flaw to it?
Converting each user ID to a number and then adding them together will likely lead to collisions. As a simple example, think of the many ways you can add up to the number 5: 0+5, 1+4, 2+3.
This answer builds upon #NimnaPerera's answer.
Method 1: <uid>_<uid>
If your app doesn't plan on using large groups, you can make use of the <uid>_<uid> format. To make sure the two user IDs are ordered in the same way, you can sort them first and then combine them together using some delimiter.
A short way to achieve this is to use:
const docId = [uid1, uid2].sort().join("_");
If you wanted to have a three-way group chat, you'd just add the new userID in the array:
const docId = [uid1, uid2, uid3].sort().join("_");
You could also turn this into a method for readability:
function getChatIdForMembers(userIds) {
return userIds.sort().join("_");
}
Here's an example of it in action:
const uid1 = "apple";
const uid2 = "banana";
const uid3 = "carrot";
[uid1, uid2].sort().join("_"); // returns "apple_banana"
[uid1, uid3].sort().join("_"); // returns "apple_carrot"
[uid2, uid1].sort().join("_"); // returns "apple_banana"
[uid2, uid3].sort().join("_"); // returns "banana_carrot"
[uid3, uid1].sort().join("_"); // returns "apple_carrot"
[uid3, uid2].sort().join("_"); // returns "banana_carrot"
// chats to yourself are permitted
[uid1, uid1].sort().join("_"); // returns "apple_apple"
[uid2, uid2].sort().join("_"); // returns "banana_banana"
[uid3, uid3].sort().join("_"); // returns "carrot_carrot"
// three way chat
[uid1, uid2, uid3].sort().join("_"); // returns "apple_banana_carrot"
[uid1, uid3, uid2].sort().join("_"); // returns "apple_banana_carrot"
[uid2, uid1, uid3].sort().join("_"); // returns "apple_banana_carrot"
[uid2, uid3, uid1].sort().join("_"); // returns "apple_banana_carrot"
[uid3, uid1, uid2].sort().join("_"); // returns "apple_banana_carrot"
[uid3, uid2, uid1].sort().join("_"); // returns "apple_banana_carrot"
Method 2: Member list properties
If you intend on supporting group chats, you should use automatic document IDs (see CollectionReference#add()) and store a list of chat members as one of it's fields as introduced in #NimnaPerera's answer for better use of queries.
I recommend two fields:
"members" - an array containing each chat member's ID. This allows you to query the /chats collection for chats that contain the given user.
"membersAsString" - a string, built from sorting "members" and joining them using "_". This allows you to query the /chats collection for chats that contain the exact list of members.
"chats/{chatId}": {
"members": string[], // list of users in this chat
"membersAsString": string, // sorted list of users in this chat, delimited using "_"
/* ... */
}
To find all chats that I am a part of:
const myUserId = firebase.auth().currentUser.uid;
const myChatsQuery = firebase.firestore()
.collection("chats")
.where("members", "array-contains", myUserId);
myChatsQuery.onSnapshot(querySnapshot => {
// do something with list of chat documents
});
To find all three-way chats between Apple, Banana and I:
const myUserId = firebase.auth().currentUser.uid;
const members = [myUserId, "banana", "apple"];
const membersAsString = members.sort().join("_");
const groupChatsQuery = firebase.firestore()
.collection("chats")
.where("membersAsString", "==", membersAsString);
groupChatsQuery.onSnapshot(querySnapshot => {
// do something with list of chat documents
// normally this would return 1 result, but you may get
// more than one result if a user gets added/removed a chat
});
A normal flow, would be to:
Get a list of the relevant chats
For each chat, get the most recent message
Based on the most recent message, sort the chats in your UI
You can very well use a combination of two users uids to define your Firestore document IDs, as soon as you respect the following constraints:
Must be valid UTF-8 characters
Must be no longer than 1,500 bytes
Cannot contain a forward slash (/)
Cannot solely consist of a single period (.) or double periods (..)
Cannot match the regular expression __.*__
What I'm not sure to understand in your question is "in such a way that it doesn't depend on the order they're combined". If you combine the uids of two users you need to combine them in a certain order. For example, uid1_uid2 is not equal to ui2_uid1.
As you are asking #lightsaber you can follow following methods to achieve your objective. But my personal preference is using an where clause, because firestore is supporting that compound queries which cannot be done in real time database.
Method 1
Create a support function to generate a chatId and check whether document is exist from that id. Then you can create chat document or retrieve the document using that id.
const getChatId = (currentUserId: string, guestUserId: string) => {
/* In this function whether you changed the order of the values when passing as parameters
it will always return only one id using localeCompare */
const comp = currentUserId.localeCompare(guestUserId);
if (comp === 0) {
return null;
}
if (comp === -1) {
return currentUserId + '_' + guestUserId;
} else {
return guestUserId + '_' + currentUserId;
}
}
Method 2
Use where clause with array-contains query for retrieving the chat document. And when creating add two user Ids to array and set the array with a relevant field name.
Firestore docs for querying arrays

How to get first and last document in collection?

I have google-cloud-firestore collection with many documents.
I want to get first and last document in collection (sorted by timestamp).
How can I query for this?
I'm not sure by which Timestamp you want to sort, so I'm gonna assume your doc looks something like this
/id/: {
timestamp: FirebaseFirestore.Timestamp
// ... whatever other properties
}
You can get a Query by querying on a CollectionReference<T> via the .orderBy method.
const collectionQuery = firestore
.collection('collectionName')
.orderBy('timestamp', 'asc')
Now we want to get a QuerySnapshot (which will allow us to access the documents), by using the .get() method.
const collectionSnapshot = await collectionQuery.get()
Now we just access the first and the last document.
const firstDocument = collectionSnapshot.isEmpty
? null
: collectionSnapshot.docs[0]
const lastDocument = collectionSnapshot.isEmpty
? null
: collectionSnapshot.docs[collectionSnapshot.docs.length - 1]
And voila, you have your first and last document! :-)

Firestore "startAt" keeps bringing the same data

I am doing a pagination with firestore, the problem is that even if I change the startAt it still brings the same results.
An example of my problem
const snaps = await db.collection('blogs').
.orderBy('createdAt')
.startAt(0)
.limit(5).get();
const snaps2 = await db.collection('blogs').
.orderBy('createdAt')
.startAt(5)
.limit(5).get();
let billList = []
let billList2 = []
snaps.forEach(x => billList.push(x.data()) )
snaps2.forEach(x => billList.push(x.data()) )
console.log(billList)
console.log(billList2)
The pagination API doesn't work the way you are expecting. It startAt() doesn't accept an integer offset. As you can see from the linked API documentation, it requires either:
A DocumentReference of the document to start at
A array of field values relative to the order of the query (themselves also typically taken from documents
The paging API doesn't work with offsets at all. You can't skip ahead by N documents at a time. What you have to do is read N documents, then read the next N documents, and so on. I suggest reading the documentation on pagination for specific examples. Note that the first example is not requesting an offset of 1000000 - it's actually asking to start with cities at or above a population field value of 1000000.

Returning a single child's value on Firebase query using orderByChild and equalTo

I am trying to pull a URL for an image in storage that is currently logged in the firebase real time database.
This is for a game of snap - there will be two cards on the screen (left image and right image) and when the two matches the user will click snap.
All of my image urls are stored in the following way:
Each one has a unique child called "index" - I also have another tree that is just a running count of each image record. So currently I am running a function that checks the total of the current count, then performs a random function to generate a random number, then performs a database query on the images tree using orderByChild and an equalTo that contains the random index number.
If I log the datasnap of this I can see a full node for one record (So index, score, url, user and their values) however if I try to just pull the URL I get returned a value of Null. I can, rather annoyingly, return the term "URL" seemingly at my leisure but I can't get the underlying value. I've wondered if this is due to it being a string and not a numeric but I can't find anything to suggest that is a problem.
Please bare in mind I've only been learning Javascript for about a week at max, so if I'm making obvious rookie errors that's probably why!
Below is a code snippet to show you what I mean:
var indRef = firebase.database().ref('index')
var imgRef = firebase.database().ref('images')
var leftImg = document.getElementById('leftImg')
var rightImg = document.getElementById('rightImg')
document.addEventListener('DOMContentLoaded', function(){
indRef.once('value')
.then(function(snapShot){
var indMax = snapShot.val()
return indMax;
})
.then(function(indMax){
var leftInd = Math.floor(Math.random()* indMax + 1)
imgRef.orderByChild('index').equalTo(leftInd).once('value', function(imageSnap){
var image = imageSnap.child('url').val();
leftImg.src=image;
})
})
})
When you execute a query against the Firebase Database, there will potentially be multiple results. So the snapshot contains a list of those results. Even if there is only a single result, the snapshot will contain a list of one result.
Your code needs to cater for that list, by looping over Snapshot.forEach():
imgRef.orderByChild('index').equalTo(leftInd).once('value', function(imageSnap){
imageSnap.forEach(function(child) {
var image = child.child('url').val();
leftImg.src=image;
})
})

How can I paginate in Mongoose without duplicates?

I'm building an API for my site (using Node.js and Mongoose) and I would like to incorporate pagination in it. My problem is the following: if the page size is for example 15 and I make a request for the first page so it sends me the first 15 items ordered by date of creation but then what if before I make the request for the second page, 15 new items are created in the database, the returned data will be the same as previously if I just use a skip on mongoose.
Is there a way to avoid doublons with mongoose? What I have at the moment is an "exclude" parameter in the query so it excludes all items already loaded but I'm thinking if there are lots of loaded items, the URL might be very long and I'm not sure that's a good thing...
Is there a better way to do this or do I have to just leave it with the risk of having doublons?
You can use mongoose-paginate-v2
And to prevent any duplicates from returned documents, you can pass (last document id & limit)
And your query should be like :
import { PaginateModel } from 'mongoose-paginate-v2';
constructor(
private readonly _paginateModel: PaginateModel<any>
) {}
const query = this._paginateModel
.find({ _id: { $lt: lastId } });
this._paginateModel.paginate(query, { limit: 10 });
You can use skip and limit function in which you can pass min and max value .
like in first page if you get 15 records values for min and max will be :
min = 0 and max = 15
for page 2
min = 15 and max = 30

Categories