How to reduce / aggregate multiple fields in a table? This does not seem efficient:
r.object(
'favorite_count',
r.db('twitterdb').table('tweets').map(tweet => tweet('favorite_count')).avg().round(),
'retweet_count',
r.db('twitterdb').table('tweets').map(tweet => tweet('retweet_count')).avg().round()
)
(Expected) Result:
{
"favorite_count": 17 ,
"retweet_count": 156
}
I'm not sure if it's possible to make RethinkDB work like you want in one go using its built-ins, but you can implement the avg function yourself really easy:
r.db('twitterdb')
.table('tweets')
.fold(
{_: 0, favorite_count: 0, retweet_count: 0},
(a, tweet) => ({
_: a('_').add(1),
favorite_count: a('favorite_count').add(tweet('favorite_count')),
retweet_count: a('retweet_count').add(tweet('retweet_count'))
})
)
.do(a => ({
favorite_count: r.branch(a('_').gt(0), a('favorite_count').div(a('_')).round(), null),
retweet_count: r.branch(a('_').gt(0), a('retweet_count').div(a('_')).round(), null)
}))
I have quickly-tested the above over a small set of data, and enabling the query profiling showed at least /2 shard accesses and less time to execute.
However I'm not sure about the overall profiler output and I don't think I can interpret its details (I believe that native avg is more optimized, but it looks cheaper that accesing the data at least in two rounds).
Additionally, this custom avg function implementation is more 0-elements friendly not throwing an error.
If the length of the array is known (e.g. 7430), this is faster:
r.db('twitterdb').table('tweets')
.reduce((agg, item) => {
return {
favorite_count: agg('favorite_count').add(item('favorite_count')),
retweet_count: agg('retweet_count').add(item('retweet_count'))
}
})
.do(result => r.object('favorite_count', result('favorite_count').div(7430).round(), 'retweet_count', result('retweet_count').div(7430).round()))
Related
I have an array with nested objects that looks like the one below.
What I'd like to do is loop through it calculate the sum of each item per date.
For example pc + screen = ?
I cannot seem to figure out how to do it properly. I have found this solution, it works great in console.log() but I cannot figure out how to output the result in a div. Should I use a map function ?
const amountPerDate = data.forEach(function (i) {
const sum = i.item.reduce(function (sum, elem) {
return sum + elem.price;
}, 0);
console.log("the total sum is " + sum);
});
The array:
The code you have posted doesn't seem quite right since forEach won't return anything, and the inner variable sum is not actually available for React to render since it is not in scope (in JavaScript, variables can not escape their containing function, which is function (i) { -- nothing outside of that function can see it).
You were roughly on the right tracks with needing map since that will return an array that represents an accumulation of the return values in the nested callback.
const amountsPerDate = data.map((i) => {
return i.item.reduce(function (sum, elem) {
return sum + elem.price;
}, 0);
});
amountsPerDate will now be an array of the sums. However, in this process, youve lost the info about which sum correlates to which date. So we need more. We can modify to return both the sum alongside the date (an array of objects, each with a sum and date inside).
const amountsPerDate = data.map((i) => {
return {
sum: i.item.reduce(function (sum, elem) {
return sum + elem.price;
}, 0),
date: i.date
});
Now, you should have something in amountsPerDate that looks like this:
[
{ date: '01/01/2022', sum: 200 },
{ date: '02/01/2022', sum: 30},
]
To display in your react component, it's just a case of rendering it, which will require you to map over this new data and return an element for each entry. You haven't posted your full component, but it will be something like this in your JSX:
<div>
{amountsPerDate.map(sum =>
<div>Date: {sum.date}. Total: {sum.sum}</div>
)}
</div>
Of course you can play with this and move it around as you see fit so it fits however you want it laid out.
It's really worth your time understanding map and the differences with foreach since it's so ubiquitous in functional programming. Foreach and map both loop over each item. But map allows you to return a value within the loop callback, and that value goes on to be part of a new array returned from map that represents that item. You can think of it as a transformation from one array to another -- both with the same length -- but with each item replaced with something of your choosing, calculated from each items original contents.
TL;DR
I'm working on a Chat List functionality very much like any of the big social networks have, and i'm having issues with React Native state management because a very common problem with Firestore onSnapshot "in" conditions.
As workaround i'm working in batches generated from a state array.onSnapshot makes changes to the state array based on such batches, HOWEVER i'm having trouble refreshing the batches after each change.
Full Description
One of its complexities is that i must condition the realtime updates from Firestore in a way that it's not yet supported by Firebase:
const watchedGroups = db.collection('group').where('__name__', 'in', groupArray?.map(({ id }) => id));
unsubscribeListener = watchedGroups.onSnapshot((querySnapshot) => {
querySnapshot.forEach((doc) => {
//...
(Please note that group = chat)
The problem with this approach is that Firestore does not support a IN condition (groupArray) with more than 10 elements and this code block will crash if the case materializes.
To solve that, i approached groupArray in batches that do not violate such constrait:
const [recentChats, setRecentChats] = useState([]);
// ...
useFocusEffect(useCallback(() => {
const grupos = [...recentChats];
if (grupos && grupos.length > 0) {
handleRefreshSuscriptions();
const collectionPath = db.collection('group');
while (grupos.length) {
const batch = grupos.splice(0, 10);
console.log(">> QUERYING", batch?.length, batch.map(({ lastMsgForMe }) => lastMsgForMe))
const unsuscribe = collectionPath.where(
'__name__',
'in',
[...batch].map(({ id }) => id)
).onSnapshot((querySnapshot) => {
if (querySnapshot !== null) {
querySnapshot.forEach((doc) => {
const validGroup = batch.find(grupo => doc.id == grupo.id);
if (validGroup) {
lastMsg(doc.id).then((lastM) => {
console.log(batch.map(({ lastMsgForMe }) => lastMsgForMe))
if (validGroup.lastMsgForMe !== doc.data().recentMessage.messageText) {
mergeChat({
...validGroup,
messageText: doc.data().recentMessage.messageText,
lastMsgForMe: lastM.messageText,
dateMessageText: lastM.sentAt,
viewed: lastM.viewed
});
}
}).catch(error => console.log(error));
}
})
}
})
setRefreshSuscription(prevState => [...prevState].concat(unsuscribe))
}
}
return () => {
handleRefreshSuscriptions();
}
}, [recentChats.length]));
It works (almost) perfectly, every change reachs the view succesfully. However, there is an issue, here are the logs when i recieve the first update:
// Initialization (12 groups shown, 2 batches)
>> QUERYING 10 ["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"]
>> QUERYING 2 ["Veremoss", "Hjjj"]
// Reception of a message "C" that updates last message shown ("B") of first group in the list.
["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"] //several repetitions of this log, i've erased it for simplicity
update idx 0 - B -> C
At this point, there isn't any noticeable issue. However, if i keep interacting with other groups and then pay attention to the logs when i recieve a message to the above shown group, i will see this:
["B", "Dddffg", "Dfff", ".", null, "Hvjuvkbn", "Sdsdx", "Vuvifdfhñ", "Ibbijn", "asdasdasd"]
update idx 1 - Bbnnm -> Bbnnm // unexpected
update idx 0 - 12 -> 12 // unexpected
update idx 2 - C -> D // expected
Notice how the batch still shows "B" when i've already recieved "C" and "D" messages on that group. The problem repeats on other two groups, and because of that, now i get a real change and another two false positives.
The problem is that, because of how batches are generated, inside of onSnapshot the batch content is always the same. This results on as many false "updates" as groups have been updated since batch generation, per recieved message.
How can i keep the batch up-to-date inside onSnapshot?
One possible solution that i came with is updating the batches on the go, by switching from find to findIndex and work the updates inside the batch
querySnapshot.forEach((doc) => {
const validGroupIdx = batch.findIndex(grupo => doc.id == grupo.id);
if (validGroupIdx !== -1) {
lastMsg(doc.id).then((lastM) => {
console.log(batch.map(({ lastMsgForMe }) => lastMsgForMe))
if (batch[validGroupIdx].lastMsgForMe !== doc.data().recentMessage.messageText) {
batch[validGroupIdx] = {
...batch[validGroupIdx],
messageText: doc.data().recentMessage.messageText,
lastMsgForMe: lastM.messageText,
dateMessageText: lastM.sentAt,
viewed: lastM.viewed
}
mergeChat(batch[validGroupIdx]);
}
}).catch(error => console.log(error));
}
})
However, to my understanding this is still suboptimal, because when i navigate to other components i will get the old batch and not the updated one, provoking the false positive at least once.
I'm wondering if i could directly handle the state in batches, instead of generating batches from it.
However, sorting and merging it would be a pain afaik.
Currently learning RxJS, I'm not gonna lie, I had difficulties understanding it, especially in terms of "Why do we even want to use if we have promises"
But now I think I made small step further.
I know I should avoid nested subscriptions.
Trying to write as short as possible code to merge two streams result into single variable.
So I have two arrays simulating result of streams, which I want to join.
Fights array should became new obj array inside boxers objects
const boxers = [
{
user_id:1,
first_name:'Lennox ',
last_name:'Lewis',
nationality:"UK"
},
{
user_id:2,
first_name:'Mike',
last_name:'Tyson',
nationality:'USA'
},
{
user_id:3,
first_name:'Riddick',
last_name:'Bowe',
nationality:'USA'
},
];
const fights = [
{
fight_id:1,
user_id:1,
opponnent_id:2,
winner_id:1,
venue:'Memphis Pyramid, Tennessee'
}
]
And then I wrote code:
const boxersWithFights2 = boxersStream.pipe(
flatMap(boxers => {
return fightsStream.pipe(
flatMap(fights => {
boxers.map(boxer => boxer.fights = fights.filter(fight => fight.user_id === boxer.user_id ||fight.opponnent_id === boxer.user_id ))
return boxers;
})
)
}
));
Surprisingly this works as expected.
When I subscribe to boxersWithFights, it console.logs me with properly mapped objects.
So it probably also work when returned from external api, but then I would of course need another map() operator.
My question: Is this code written well? Can it be written to be more clean & elegant ?
I also know I could do that easier with e.g. forkJoin, but I really wanted to test flatMap(mergeMap) operator.
You shouldn't mutate data in the stream but boxer.fights = does it.
Also you can combine the streams together via forkJoin because they don't depend on each other.
Try to use map operator instead:
const boxersWithFights2 = forkJoin([boxersStream, fightsStream]).pipe(
map(([boxers, fights]) => {
return boxers.map(boxer => ({
...boxer,
fights: fights.filter(fight => fight.user_id === boxer.user_id ||fight.opponnent_id === boxer.user_id ),
})),
));
Here is some code from I project I am working in:
const profile = userdataDocs
.filter(isValidUserdataDocument)
.find((document: ICouchDBDocumentDoc) => document._id === profileId);
if (profile) {
return {
id: hashSensitive(profile._id, environment),
type: profile.type,
creationDate: profile.creationDate,
updatedDate: profile.updatedDate,
entityVersion: profile.entityVersion,
};
}
Here is how I would like to have my code look:
return userdataDocs
.filter(isValidUserdataDocument)
.filter((document: ICouchDBDocumentDoc) => document._id === profileId)
.map((profile: ICouchDBDocumentDoc) => ({
id: hashSensitive(profile._id, environment),
type: profile.type,
creationDate: profile.creationDate,
updatedDate: profile.updatedDate,
entityVersion: profile.entityVersion,
}))
.slice(0, 1);
But I get feedback from the rest of my team that I should not use filter because it will continue searching after having found an item. Premature optimization in mind, but still a pretty valid and popular opinion.
Is there some other array method (or altogether different solution) that I can use to write code the way I want, with 'pipes', without getting the performance penalty of moving from find to filter?
Also let me know if I am an idiot and should let go of the pipe dream (pun intended).
Let me start that I like the first solution. In my opinion, it looks good.
But if you are really desperate for a solution that fulfills your pipe dream
const array = [10, 20, 30];
function singleMapFind(args, fn) {
const currentArray = args[2];
const duplicate = [...currentArray];
currentArray.splice(1, currentArray.length - 1);
return duplicate.find(fn);
}
const modified = array.map((...args) => singleMapFind(args, (e) => e > 20));
I would never use it though. Wish you luck with the PR.
I am having a bit of trouble wrapping my head around getting this search function to work. I have it setup right now so it will get the right item when I search it but it's spelling has to be exact including capitalization and punctuation. I want it to be able to get the item regardless of the users search term's capitalization and if they just typed the letter 'b' it will include all items that have a 'b' in the items fields.
I know that I want to query the call to the database since it would be quite heavy to do it on the client side but what do you guys think or how would you go about achieving this?
setFilteredItems() {
this.employeeListRef = this.database.list('userProfile',
ref=> ref.orderByChild('lastName'));
this.employeeList = this.employeeListRef.snapshotChanges()
.map(
changes => {
return changes.map(c => ({
key: c.payload.key, ...c.payload.val()
}))
}
);
//if searchterm is null it returns so it can set back the list to all values
//searchterm is declared in constructor
if(!this.searchTerm) {
return;
}
//var term = this.searchTerm.toLowerCase();
this.employeeListRef = this.database.list('userProfile',
ref => ref.orderByChild('lastName').equalTo(term));
this.employeeList = this.employeeListRef.snapshotChanges()
.map(
changes => {
return changes.map(c => ({
key: c.payload.key, ...c.payload.val()
}))
}
);
}
If you look at the ASCII table you can get a good idea of how Firebase stores it's records and why orderByChild might not work as you expect.
b is 98 and B is 66. Their in different positions on the ASCII table.
There are two things you can try to help you access the data in the expression you want.
Try converting the searchable data to lowercase with the user of database methods
Use a cloud function and on-write of a record, save a lowercase version of that record in the object, then search by that record. An example would be;
{ lastName: 'Smith', lowercaseLastName: 'smith' }
You then orderByChild('lowercaseLastName').