TL;DR
I'm working on a Chat List functionality very much like any of the big social networks have, and i'm having issues with React Native state management because a very common problem with Firestore onSnapshot "in" conditions.
As workaround i'm working in batches generated from a state array.onSnapshot makes changes to the state array based on such batches, HOWEVER i'm having trouble refreshing the batches after each change.
Full Description
One of its complexities is that i must condition the realtime updates from Firestore in a way that it's not yet supported by Firebase:
const watchedGroups = db.collection('group').where('__name__', 'in', groupArray?.map(({ id }) => id));
unsubscribeListener = watchedGroups.onSnapshot((querySnapshot) => {
querySnapshot.forEach((doc) => {
//...
(Please note that group = chat)
The problem with this approach is that Firestore does not support a IN condition (groupArray) with more than 10 elements and this code block will crash if the case materializes.
To solve that, i approached groupArray in batches that do not violate such constrait:
const [recentChats, setRecentChats] = useState([]);
// ...
useFocusEffect(useCallback(() => {
const grupos = [...recentChats];
if (grupos && grupos.length > 0) {
handleRefreshSuscriptions();
const collectionPath = db.collection('group');
while (grupos.length) {
const batch = grupos.splice(0, 10);
console.log(">> QUERYING", batch?.length, batch.map(({ lastMsgForMe }) => lastMsgForMe))
const unsuscribe = collectionPath.where(
'__name__',
'in',
[...batch].map(({ id }) => id)
).onSnapshot((querySnapshot) => {
if (querySnapshot !== null) {
querySnapshot.forEach((doc) => {
const validGroup = batch.find(grupo => doc.id == grupo.id);
if (validGroup) {
lastMsg(doc.id).then((lastM) => {
console.log(batch.map(({ lastMsgForMe }) => lastMsgForMe))
if (validGroup.lastMsgForMe !== doc.data().recentMessage.messageText) {
mergeChat({
...validGroup,
messageText: doc.data().recentMessage.messageText,
lastMsgForMe: lastM.messageText,
dateMessageText: lastM.sentAt,
viewed: lastM.viewed
});
}
}).catch(error => console.log(error));
}
})
}
})
setRefreshSuscription(prevState => [...prevState].concat(unsuscribe))
}
}
return () => {
handleRefreshSuscriptions();
}
}, [recentChats.length]));
It works (almost) perfectly, every change reachs the view succesfully. However, there is an issue, here are the logs when i recieve the first update:
// Initialization (12 groups shown, 2 batches)
>> QUERYING 10 ["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"]
>> QUERYING 2 ["Veremoss", "Hjjj"]
// Reception of a message "C" that updates last message shown ("B") of first group in the list.
["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"] //several repetitions of this log, i've erased it for simplicity
update idx 0 - B -> C
At this point, there isn't any noticeable issue. However, if i keep interacting with other groups and then pay attention to the logs when i recieve a message to the above shown group, i will see this:
["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"]
update idx 1 - Bbnnm -> Bbnnm // unexpected
update idx 0 - 12 -> 12 // unexpected
update idx 2 - C -> D // expected
Notice how the batch still shows "B" when i've already recieved "C" and "D" messages on that group. The problem repeats on other two groups, and because of that, now i get a real change and another two false positives.
The problem is that, because of how batches are generated, inside of onSnapshot the batch content is always the same. This results on as many false "updates" as groups have been updated since batch generation, per recieved message.
How can i keep the batch up-to-date inside onSnapshot?
One possible solution that i came with is updating the batches on the go, by switching from find to findIndex and work the updates inside the batch
querySnapshot.forEach((doc) => {
const validGroupIdx = batch.findIndex(grupo => doc.id == grupo.id);
if (validGroupIdx !== -1) {
lastMsg(doc.id).then((lastM) => {
console.log(batch.map(({ lastMsgForMe }) => lastMsgForMe))
if (batch[validGroupIdx].lastMsgForMe !== doc.data().recentMessage.messageText) {
batch[validGroupIdx] = {
...batch[validGroupIdx],
messageText: doc.data().recentMessage.messageText,
lastMsgForMe: lastM.messageText,
dateMessageText: lastM.sentAt,
viewed: lastM.viewed
}
mergeChat(batch[validGroupIdx]);
}
}).catch(error => console.log(error));
}
})
However, to my understanding this is still suboptimal, because when i navigate to other components i will get the old batch and not the updated one, provoking the false positive at least once.
I'm wondering if i could directly handle the state in batches, instead of generating batches from it.
However, sorting and merging it would be a pain afaik.
Related
I'm not sure if this is related to React or just JavaScript.
I'm building a simple voting app. You can add some options and vote +1 for each option.
My App has options as state like below. storedOptions is from localStorage.
function App() {
const [options, setOptions] = useState(
storedOptions ? JSON.parse(storedOptions) : []
);
And handleVote increases count by 1 for the given option.
const handleVote = useCallback((option) => {
setOptions((options) => {
console.log("previous: ", options);
let updatedOptions = [...options];
console.log(updatedOptions); // THIS PART IS STRANGE
const index = updatedOptions.indexOf(option);
console.log(index);
updatedOptions[index] = { ...option, count: option.count + 1 }; // Change reference of the given option only
console.log("new: ", updatedOptions);
updatedOptions = sortByValue(updatedOptions, "count"); // I think this is not related to my problem though, this is why I declared updatedOptions with 'let'. sortByValue function returns new array.
localStorage.setItem(OPTIONS_KEY, JSON.stringify(updatedOptions)); // I'm working with localStorage too, you can ignore this
return updatedOptions;
}, []);
});
But when I voted for an option, it didn't work as it supposed to be. So I logged them out like above and found out that console.log(updatedOptions)(second log) already reflected future operation(increasing count).
Shouldn't count be 0 at that moment? why 1 already?
CONTEXT
First question here so thank you all in advance and please do let me know if you require anything else to help answer my question!
I'm creating a sorted list of HTML cards. The cards pull data from Firestore documents that each have metadata: numViews and summary.
I am loading paginated data of 5 cards at a time, and want a 'load more' button to reveal the next 5.
Im following this tutorial: https://youtu.be/vYBc7Le5G6s?t=797 (timestamped to part on orderBy, limit, and creating a Load More button).
Not sure if its relevant but I eventually intend to make this an infinite scroll, ideally using the same tutorial.
PROBLEM
I have a working solution (exactly the same as tutorial) for sorting by ascending (see below)). It creates cards for the documents with the lowest number of views, increasing by views as you scroll down the page. The load more button creates the next 5 cards.
When I change to orderBy() descending, no cards load.
When I change the pagination from startAfter() to endBefore(), and orderBy(descending) it sorts the first 5 by descending correctly (starting with the highest views, and descending as you scroll down the page), but the Load More button just reloads the same 5 cards, not the next 5.
Here is my code
// References an empty container where my cards go
const container = document.querySelector('.containerload');
// Store last document
let latestDoc = null;
const getNextReviews = async () => {
// WORKING ascending
var load = query(colRef, orderBy('numViews', 'asc'), startAfter(latestDoc || 0), limit(5))
// BROKEN descending - returns nothing
// var load = query(colRef, orderBy('numViews', 'desc'), startAfter( latestDoc || 0), limit(5))
// HALF-BROKEN descending - returns 5 cards with highest views, but load more returns the same 5
// var load = query(colRef, orderBy('numViews', 'desc'), endBefore(latestDoc || 0), limit(5))
const data = await getDocs(load);
// Output docs
let template = '';
data.docs.forEach(doc => {
const grabData = doc.data();
template += `
<div class="card">
<h2>${grabData.summary}</h2>
<p>Views ${grabData.numViews}</p>
</div>
`
});
container.innerHTML += template;
// Update latestDoc
latestDoc = data.docs[data.docs.length-1]
// Unattach event listeners if no more documents
if (data.empty) {
loadMore.removeEventListener('click',handleClick)
}
}
// Load more docs (button)
const loadMore = document.querySelector('.load-more button');
const handleClick = () => {
getNextReviews();
console.log(latestDoc);
}
loadMore.addEventListener('click', handleClick);
// wait for DOM content to load
window.addEventListener('DOMContentLoaded', () => getNextReviews());
I have looked through a number of similar questions and cannot get a solution working:
Firestore startAfter() returning the same data in infinite scrolling when ordered by descending timestamp - Cannot refactor solution to my circumstances
How to combine Firestore orderBy desc with startAfter cursor - Cannot refactor solution to my circumstances (this uses Firebase v8)
Firestore how to combine orderBy desc with startAfter(null) - This suggested using endBefore which is how I ran into the problem of Load More loading the same 5 datapoints
I am unsure why orderBy('numViews', 'desc') and startAfter() returns nothing.
I think orderBy('numViews', 'desc') and endBefore() returns the same 5 because it is just getting 5 docs that end before latestDoc, which doesn't change to a cursor 5 documents down. I have tried playing around with this line but cannot get it to load the next 5 documents:
latestDoc = data.docs[data.docs.length-1]
Thanks everyone :)
Upon re-checking the Firebase documentation on how to paginate a query, and following the given options from this related post:
Use a Firebase query to get the correct data, then re-order it client-side
Add a field that has a descending value to the data
It easier if you just add field like timestamp, for example:
Firestore collection:
Code:
import { initializeApp } from "firebase/app";
import { getFirestore, collection, query, getDocs, orderBy, startAfter, limit } from "firebase/firestore";
const firebaseConfig = {
projectId: "PROJECT-ID"
};
const app = initializeApp(firebaseConfig);
const db = getFirestore(app);
const q = query(collection(db, "users"));
const getNextReviews = async () => {
// For this example, there's only 6 data
// Query the first 5 data
var load = query(q,
orderBy("timestamp", "desc"),
limit(5))
// Retrieve the first 5 data
const data = await getDocs(load);
// Update latestDoc reference
const latestDoc = data.docs[data.docs.length-1]
// To output the retrieved first 5 data
data.forEach((doc) => {
// doc.data() is never undefined for query doc snapshots
console.log("First: ", doc.data().first_name , " => ", doc.data().timestamp.toDate());
});
// Query the next data or remaining data
var next = query(q,
orderBy("timestamp", "desc"),
startAfter(latestDoc),
limit(5));
// Automatically pulls the remaining data in the collection
const data_next = await getDocs(next);
// Outputs the remaining data in the collection
data_next.forEach((doc) => {
// doc.data() is never undefined for query doc snapshots
console.log("Second: ", doc.data().first_name , " => ", doc.data().timestamp.toDate());
});
}
// You can add the other conditions for 'load-more button' in here
getNextReviews();
Result:
First: bob => 2022-01-30T16:00:00.000Z
First: mike => 2022-01-29T16:00:00.000Z
First: teddy => 2022-01-26T16:00:00.000Z
First: grace => 2022-01-25T16:00:00.000Z
First: john => 2022-01-23T16:00:00.000Z
Second: lory => 2022-01-19T16:00:00.000Z
I want to retrieve a list of products in relation to the user's position, for this I use Geofirestore and update my Flatlist
When I have my first 10 closest collections, I loop to have each of the sub-collections.
I manage to update my state well, but every time my collection is modified somewhere else, instead of updating my list, it duplicates me the object that has been modified and adds it (updated) at the end of my list and keep the old object in that list too.
For example:
const listListeningEvents = {
A: {Albert, Ducon}
B: {Mickael}
}
Another user modified 'A' and delete 'Ducon', I will get:
const listListeningEvents = {
A: {Albert, Ducon},
B: {Mickael},
A: {Albert}
}
And not:
const listListeningEvents = {
A: {Albert},
B: {Mickael},
}
That's my useEffect:
useEffect(() => {
let geoSubscriber;
let productsSubscriber;
// 1. getting user's location
getUserLocation()
// 2. then calling geoSubscriber to get the 10 nearest collections
.then((location) => geoSubscriber(location.coords))
.catch((e) => {
throw new Error(e.message);
});
//Here
geoSubscriber = async (coords) => {
let nearbyGeocollections = await geocollection
.limit(10)
.near({
center: new firestore.GeoPoint(coords.latitude, coords.longitude),
radius: 50,
})
.get();
// Empty array for loop
let nearbyUsers = [];
// 3. Getting Subcollections by looping onto the 10 collections queried by Geofirestore
productsSubscriber = await nearbyGeocollections.forEach((geo) => {
if (geo.id !== user.uid) {
firestore()
.collection("PRODUCTS")
.doc(geo.id)
.collection("USER_PRODUCTS")
.orderBy("createdDate", "desc")
.onSnapshot((product) => {
// 4. Pushing each result (and I guess the issue is here!)
nearbyUsers.push({
id: product.docs[0].id.toString(),
products: product.docs,
});
});
}
});
setLoading(false);
// 4. Setting my state which will be used within my Flatlist
setListOfProducts(nearbyUsers);
};
return () => {
if (geoSubscriber && productsSubscriber) {
geoSubscriber.remove();
productsSubscriber.remove();
}
};
}, []);
I've been struggling since ages to make this works properly and I'm going crazy.
So I'm dreaming about 2 things :
Be able to update my state without duplicating modified objects.
(Bonus) Find a way to get the 10 next nearest points when I scroll down onto my Flatlist.
In my opinion the problem is with type of nearbyUsers. It is initialized as Array =[] and when you push other object to it just add new item to at the end (array reference).
In this situation Array is not very convenient as to achieve the goal there is a need to check every existing item in the Array and find if you find one with proper id update it.
I think in this situation most convenient will be Map (Map reference). The Map indexes by the key so it is possible to just get particular value without searching it.
I will try to adjust it to presented code (not all lines, just changes):
Change type of object used to map where key is id and value is products:
let nearbyUsersMap = new Map();
Use set method instead of push to update products with particular key:
nearbyUsersMap.set(product.docs[0].id.toString(), product.docs);
Finally covert Map to Array to achieve the same object to use in further code (taken from here):
let nearbyUsers = Array.from(nearbyUsersMap, ([id, products]) => ({ id, products }));
setListOfProducts(nearbyUsers);
This should work, but I do not have any playground to test it. If you get any errors just try to resolve them. I am not very familiar with the geofirestore so I cannot help you more. For sure there are tones of other ways to achieve the goal, however this should work in the presented code and there are just few changes.
Let say I have these 2 fields in my firebase database.
user: {
postCount: 2
posts: [
{title: hello, content: world},
{title: hello again, content: world}
]
}
I want the user to have permission to update his posts. but I don't want him to be able to update his post count. I want the post counts to always represent the number of posts and prevent the user from cheating it.
How can I do this in firebase? Is it possible with front end javascript only? If not what would be the option that requires the least server side code possible?
This is the code I'm using but it doesn't prevent users from cheating and just calling the increment function by themselves infinite times.
const push = (objectToInsert, firebasePath) => {
const key = firebase.database().ref().child(firebasePath).push().key
let updates = {}
updates[firebasePath + key] = objectToInsert
firebase.database().ref().update(updates)
}
const increment = (firebasePath) => {
const ref = firebase.database().ref(firebasePath)
ref.transaction( (value) => {
value++
return value
})
}
push(post, `/${user}/${posts}/`)
increment(`/${user}/${postCount}`)
Referring you to firebase rules:
https://firebase.google.com/docs/database/security/#section-authorization
Either you set the rules on each of the user properties based on your security setup and keep the structure as you mentioned, Or move the counts to another node and set the rules (ex. user_post_counts).
Recently Firebase introduce Cloud Functions.
In my case this feature is very usefull to count elements in my database.
Firebase posted a sample code to count elements but I ask myself some questions with big data.
In our example we consider that we need to count likes for a post.
In the sample code, at each new like, the function count all likes for the current post and update the count.
Do you think it's a good solution for big data ? (For example if we have 1M likes)
Thank you in advance !
Agreed that the code in the functions sample is not ideal for large sets of data.
For a long time I've used a two-stepped approach in my counters:
when a child is added/removed, increase/decrease the counter
when the counter gets deleted, recount all the children (as it does now)
So case #2 is memory-bound the same as the current code. But case #1 triggers on child writes, so is a lot less memory hungry.
The code:
// Keeps track of the length of the 'likes' child list in a separate property.
exports.countlikechange = functions.database.ref("/posts/{postid}/likes/{likeid}").onWrite((event) => {
var collectionRef = event.data.ref.parent;
var countRef = collectionRef.parent.child('likes_count');
return countRef.transaction(function(current) {
if (event.data.exists() && !event.data.previous.exists()) {
return (current || 0) + 1;
}
else if (!event.data.exists() && event.data.previous.exists()) {
return (current || 0) - 1;
}
});
});
// If the number of likes gets deleted, recount the number of likes
exports.recountlikes = functions.database.ref("/posts/{postid}/likes_count").onWrite((event) => {
if (!event.data.exists()) {
var counterRef = event.data.ref;
var collectionRef = counterRef.parent.child('likes');
return collectionRef.once('value', function(messagesData) {
return counterRef.set(messagesData.numChildren());
});
}
});
I also submitted this in a PR for the repo.
See the sample of this in functions-samples.
Given a data structure similar to this:
/functions-project-12345
/posts
/key-123456
likes_count: 32
/likes
user123456: true
user456789: true
user786245: true
...
This function would do the trick:
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
// Keeps track of the length of the 'likes' child list in a separate attribute.
exports.countlikes = functions.database.ref('/posts/{postid}/likes').onWrite(event => {
return event.data.ref.parent.child('likes_count').set(event.data.numChildren());
});
Note that this code is copyright Google and apache licensed. See the code for more details.