How to paginate getting bulk data from firebase with axios? - javascript

Assuming I have 1000+ blog posts. What will be the best practice to get data from firebase using axios to store in nuxtServerInit?
Can I somehow get the first 10 blog posts first during the first load and get even more data later on?
Right now I have vuex action as following:
nuxtServerInit(vuexContext, context) {
return axios
.get('https://my-blog.firebaseio.com/posts.json')
.then(res => {
const postsArray = []
for (const key in res.data) {
postsArray.push({ ...res.data[key], uid: key })
}
vuexContext.commit('setPosts', postsArray)
})
.catch(e => context.error(e))
},

You're using the REST API to access the Firebase Database. To retrieve a limited number of items, use a limit query.
https://my-blog.firebaseio.com/posts.json?limitToFirst=10
Since a limit only makes sense when you know how the items are order, you'll want to also order the items, i.e. on their key:
https://my-blog.firebaseio.com/posts.json?orderBy="$key"&limitToFirst=10
Note that the results may not be ordered, since the order of properties in a JSON object is undefined. So the result will contains the first 10 posts, but it may not show them in the right order.
Next step is to get the next 10 items. Unlike on most traditional databases, Firebase doesn't support an offset clause on its queries. So you can't tell it to skip the first 10 items to get to the next 10.
Instead Firebase queries use anchors/ranges: you must know the last item of the previous page to build the query for the next page. Say that the 10th post had a key of keyOfPost10, then you can get the next page with:
https://my-blog.firebaseio.com/posts.json?orderBy="$key"&startAt="keyOfPost10"&limitToFirst=11
We need to retrieve 11 posts here, since we're also getting the 10th post. That also means you'll need to filter the overlapping post in your client code.

Related

Reducing the amount of reads in a where query

I have recently implemented firebase into my project and I have created a user collection, this collection has a document for each user and each document has about 8 fields, when my user launches the app, I am trying to pull the document that corresponds to his data, so im doing the following query:
async function getUserData() {
const _collection = collection(db, "users")
const _query = query(_collection, where("userid", "==", uniqueUserID))
const querySnapshot = await getDocs(_query)
querySnapshot.forEach((doc) => {
console.log(doc.data())
})
setLoadingStatus(false)
}
This query works and gives me the corresponding user data, but the problem is, if the user is too far down the collection, this will execute 8 reads per document until it gets to the corresponding user, I have tried to implement a cache system using a lastModified but I still need to read the document data for that field and it will end up using more or less the same amount of reads. My question is: How do I reduce the amount of read operations that get executed when im trying to compare values in the documents, I have also thought of adding an a like so a_uniqueUserID so it gets ordered alphabetically and takes the first spot of the document but it's hacky.
EDIT: Here is what my structure looks like:
I think you are misunderstanding the definition of a document and a field. When you read a document, you always get all fields out of it. The snapshot contains everything read, even if you don't use it. There is no additional cost per field, other than the storage required to hold it all. In your screenshot, you show 5 documents, and one of those documents have 8 fields.
You are probably misunderstanding the metrics in the console. When you read and write documents using the console, those are also billed as reads and writes - use of the console is not "free". What you are seeing is a combination of what your app is doing in combination with what you're doing in the console.

How to read last message node in firebase for js app?

I want to read last message node as shown in image below.
I am using below query to retrieve last message where suid and ids are known(as shown in img). but not working.
firebase.database().ref("Messages/"+suid).child(ids+"/chat").orderByKey().limitToLast(1);
How to retrieve last message from firebase realtime db and display it to my console? Thanks
Your existing code looks fine, so let's store that query in a variable first:
const query = firebase.database().ref("Messages/"+suid).child(ids+"/chat").orderByKey().limitToLast(1);
Now you can read the message(s) this query matches with either once() or get(). I'll use once(), since it's much older and hence you're more likely to find examples of it elsewhere:
query.once("value").then((snapshot) => {
snapshot.forEach((message) => {
console.log(message.val().message.msg);
});
})
Some things to note here:
Since a query may have multiple results, the snapshot contains a list of those results. Even if there's only a single result (like here), the snapshot is a list of one result. To loop over these results, we use snapshot.forEach().
Your msg is nested under message in each child, so we use message.val().message.msg to get at its value.

get more data after some given id in firebase

I'm new using firebase and I'd like to make an infinite scroll pagination to my posts.
Suppose I have this post IDS:
-M5HMGs3EnBv6O2NSxG
-M5HMGsA3YyW_NJ3fSV
-M5HMGsdfLOOEPPBR_s
-M5HMGsiH9HIZw9DNaz
-M5HMHFYoUX8kNolLrP
-M5HMHFZvDrV27S2hSt
-M5HMHFnrQ_l4mVQ0rX
-M5HMHFoaV8wbexwPCd
-M5HMJQaaFGxF450lJB
-M5HMJQe319R19Cvak6 // THE LAST ONE IN FIRST PAGINATION
-M5HMJQh2gWuah7GSht
-M5HMJQlrcfTTF3fbbI
-M5HMJQo6QT1HwWP2lz
-M5HMJQrIUiZyzhiqK3
-M5HMJQudSkFdvs42D1
-M5HMJRCmbFbxKp1NgA
-M5HMJRFwfo7yN8Is3-
-M5HMJRIsKT3YmoukQ0
-M5HMJRKDTq7XuqtshT
-M5HMJRNRabXbYQUMi6
-M5HMJRS1t9UBzH_3Jh
-M5HMJRXEv6BaPeQPMn
-M5HMJR_bRdxCBy-uma
-M5HMJRdOPBA-SMoMjB
So, first I get the first 10 posts till the id M5HMJQe319R19Cvak6 using this:
var starCountRef = firebase.database().ref('/posts/').limitToFirst(11);
it is ok. When I go to the last post my reactjs will call this function again sending the last loaded post ID (the M5HMJQe319R19Cvak6) in variable pageNumber.
What I'd like to do is get posts after this last id given to continue the pagination, like so:
-M5HMJQh2gWuah7GSht
-M5HMJQlrcfTTF3fbbI
-M5HMJQo6QT1HwWP2lz
-M5HMJQrIUiZyzhiqK3
-M5HMJQudSkFdvs42D1
...
I tried to use startAt(ID) but no result has shown:
var starCountRef = firebase.database().ref('/posts/').limitToFirst(11).startAt(pageNumber);
any ideas how can I get posts after an id in firebase?
Firebase Realtime Database queries consists of two steps:
You call one of the orderBy methods to order the child nodes on a specific value.
You then call one of the filtering methods (startAt, endAt, equalTo) to determine where to start and stop returning data.
Your code does that second step, but fails to do the first step. This means that you're using Firebase's default sort-order, which is by each node's priority. Priorities are a left-over from the days then this API didn't have any better sorting options. These days, you'll always want to call orderBy... before filtering.
With that knowledge, your query for getting the second page of results should look something like:
firebase.database()
.ref('/posts/')
.orderByKey()
.startAt("-M5HMJQe319R19Cvak6")
.limitToFirst(11)
I highly recommend calling the "-M5HMJQe319R19Cvak6" value something else than pagenumber as that variable name makes it seem as if Firebase queries are offset-based, which they aren't.

Unable to access element from array of JSON objects

I am trying to access the specific elements of an array of JSON objects. To test I simply have:
{console.log(this.state.stockCharts)}
This returns (in browser):
This is great, but now I want to access a specific element. Say the first element. I type:
{console.log(this.state.stockCharts[0])}
And the browser is like: nah mate
undefined
It's probably something really simple, but I have been banging my head against my keyboard for the past 45 minutes and nothing has worked. Thanks guys!
Edit 1 (For Akrion)
The query that I am using to access the API is:
https://www.alphavantage.co/query?function=TIME_SERIES_WEEKLY&symbol=MSFT&apikey=demo
This is what I get back from the call:
I call this API twice, and after I get a response back I push it to my stockCharts array:
this.state.stockCharts.push(result)
Edit 2 (For Beginner)
I initialize the state how you would normally do it:
this.state = {
stockCharts: []
}
I verified with the api given in my local and I am able to get the data.
First thing the way you push api response to stockCharts is not recommended. Which means direct mutation of the state is not recommended.
You can push api response in the following way
this.setState(prevState => ({
stockCharts: [...prevState.stockCharts, result]
}));
Now in render
render(){
this.state.stockCharts.map((data, i) => {
console.log("data", data); // this will give each object
console.log("Meta Data", data["Meta Data"]); //This will give meta data information
console.log("Weekly Time Series", data["Weekly Time Series"]);// this will print weekly time information
});
return(
)
}
Is this what your expectation?
It might be because you mutate the state which is not recommended.
try instead of calling this.state.stockCharts.push(result) do this.setState({stockCharts: [...this.state.stockCharts, result]})
https://reactjs.org/docs/state-and-lifecycle.html

Pagination in React with fetch()

I am fetching objects from an API simply with
getData() {
fetch('API_URL').then(res => res.json()).then(data => {
this.setState({ jobs: data.jobs });
}).catch(console.log);
}
componentDidMount() {
this.getData();
}
But I want to be able to click a button to load more objects.
I guess that I should create the API such that it only prints e.g. 10 objects at a time and keeps a variable "pageNumber" and if I click the "load more" button, it should fetch from next page and append the new objects.
Is this the right approach? Or can I simply load thousands of objects when mounting the component and use React to limit how many are seen? I guess this would be a very inefficient way to fetch the data, but I am not really sure how well React handles this.
Can I, for instance, in my API just keep print X number of objects and in my fetch request decide how many objects are loaded? So when I have pressed "load more" 2 times, the API endpoint will return 30 objects instead of only 10 - even though the first 20 have already been fetched before?
I have tried searching for pagination in React, but I only get a lot of pagination libraries. I just want to understand the very basic initial fetching and the fetching following clicking load more.
Edit
But if I have an API endpoint which returns something like
{
page: 1,
objectsPerPage: 10,
numPages: 30,
objects: [
...
]
}
and I am initially retrieving the objects on page 1, and every time I click "Load more", I increase the page number and append the objects on the next page (with this.setState({ jobs: this.state.jobs.concat(data.jobs) }); where data.jobs is the list of objects on the next page, then I would be afraid that new objects are created in the database, so which objects belong to which page is completely screwed up and not all or some duplicates are shown.
Yes, it is the right approach to have a pageNumber on the API, so you only look for the registers you don't have.
On the other size if your data is not too big you can make the fake pagination having all the objects in memory and only showing the ones that you are interested in.
I don't recommend to increase the number of objects you are looking for because you are not getting the advantage of the ones you have already fetched and everytime you increase the number, the request will last more and more.

Categories