We Keep Coding

Order of Async function calls

I'm making a React App for my Football team to keep track of Payments and Debts for each Member. I'm trying to search the database for Records for each Season and delete a Member if no records are found. Currently the Member gets deleted then the database searched.
This is the code block (it can also be seen with the rest of my App on GitHub here). I use Firebase to store all my data.
import database from '../firebase/firebase';
export const startRemoveMember = ( playerUuid, seasonList ) =>
{
return (dispatch, getState) =>
{
let canDelete = true;
const uid = getState().auth.uid;
// This should resolve first
seasonList.forEach( (season) =>
{
database.ref(`subs-tracker/users/${uid}/debts_and_payments/${season.seasonUuid}`)
.once('value')
.then((records) =>
{
records.forEach((childRecord) =>
{
if(childRecord.val().playerUuid === playerUuid)
{
canDelete = false;
return true; // breaks loop if record is found
};
});
});
});
// This resolves first before block above
if(canDelete)
{
alert('Deleted');
return database.ref(`subs-tracker/users/${uid}/members/${playerUuid}`)
.remove()
.then((ref) =>
{
dispatch(removeMember(playerUuid)); // Calls removeMember() function to remove the Member from the State of the App
})
}
else
{
alert('Cannot Delete. Member has records');
return false;
}
};
};
I'm probably missing something basic. I'm no expert with Databases and Async calls. Any help you would be appreciated :)

In your outer forEach callback you are creating a promise in each iteration. All those promises must resolve first.
You'll have to await all of them.
So instead of doing forEach, use map so you can collect those promises in an array. Then call Promise.all on those, so you have a promise that will resolve when all those have resolved. Then put the deletion code inside a then callback. It cannot obviously be executed synchronously.
This also means you have to think about the return value of your (dispatch, getState) => function. That function cannot give an indication of success in a synchronous way. So it should best return a promise as well.
const promises = seasonList.map( (season) =>
database.ref(`subs-tracker/users/${uid}/debts_and_payments/${season.seasonUuid}`)
.once('value')
.then((records) =>
records.forEach((childRecord) => // breaks on true
childRecord.val().playerUuid === playerUuid
);
);
);
return Promise.all(promises).then(findings =>
findings.includes(true)
).then(cannotDelete => {
if (cannotDelete) {
alert('Cannot Delete. Member has records');
return false;
} else {
alert('Deleted');
return database.ref(`subs-tracker/users/${uid}/members/${playerUuid}`)
.remove()
.then((ref) => {
dispatch(removeMember(playerUuid)); // Calls removeMember() function to remove the Member from the State of the App
return true; // To distinguish from false
});
}
});

How to define if #mentioned is subscriber? TMI.js

Trying to make little bot for my friend's twitch and want to make it define if "#mentioned" is a sub.
I decided to test if it define mods first but it always says "2"
client.on("chat", function (channel, user, message, self) {
if (message.toLowerCase().startsWith("!love")) {
var loved = message.split(' ')[1];
if (loved.mod || loved['user-type'] === 'mod'){
client.action(channel, "1 ");  //says sth
      }
else {
client.action(channel, "2 ");//says sth different
      }
}

user.mod
Is only available for user objects that are returned by the message handler. If you want to check if a mentioned user is a mod you have to request the chat list
function getAllChatters(channel, _attempts) {
return rp({
uri: `https://tmi.twitch.tv/group/user/${channel.replace('#','')}/chatters`,
json: true
})
.then(data => {
return Object.entries(data.chatters)
.reduce((p, [ type, list ]) => p.concat(list.map(name => {
return {name, type};
})), []);
})
.catch(err => {
if(_attempts < 3) {
return getChatters(channel, _attempts + 1);
}
throw err;
})
}
Then filter it for only mods
function getModsOnline(channel) {
return getAllChatters(channel)
.then(data => {
var mods = data.filter(chatter => chatter.type == 'moderators' || chatter.type == 'broadcaster').map(chatter => chatter.name);
return mods;
})
}
And then see if the user is in the list
function isModOnline(channel, username) {
// This will only return true if the mod is online in chat aka in the chat list
return getMods(channel)
.then(mods => {
return mods.includes(username);
})
}
Keep in mind that
This is for getting mods, not subscribers
It will only work for mods that are currently in chat, that list tends to update rather slowly
You need to work with promises here so
const rp = require('request-promise');
and to use the value
```
isModOnline(channel, username)
.then(value => {
// Code to work with the value here
console.log(`User is mod: ${value}`);
})

setState in nested async function - React Hooks

How can I build a function which gets some data asynchronously then uses that data to get more asynchronous data?
I am using Dexie.js (indexedDB wrapper) to store data about a direct message. One thing I store in the object is the user id which I'm going to be sending messages to. To build a better UI I'm also getting some information about that user such as the profile picture, username, and display name which is stored on a remote rdbms. To build a complete link component in need data from both databases (local indexedDB and remote rdbms).
My solution returns an empty array. It is being computed when logging it in Google Chrome and I do see my data. However because this is not being computed at render time the array is always empty and therefor I can't iterate over it to build a component.
const [conversations, setConversations] = useState<IConversation[]>()
const [receivers, setReceivers] = useState<Profile[]>()
useEffect(() => {
messagesDatabase.conversations.toArray().then(result => {
setConversations(result)
})
}, [])
useEffect(() => {
if (conversations) {
const getReceivers = async () => {
let receivers: Profile[] = []
await conversations.forEach(async (element) => {
const receiver = await getProfileById(element.conversationWith, token)
// the above await is a javascript fetch call to my backend that returns json about the user values I mentioned
receivers.push(receiver)
})
return receivers
}
getReceivers().then(receivers => {
setReceivers(receivers)
})
}
}, [conversations])
/*
The below log logs an array with a length of 0; receivers.length -> 0
but when clicking the log in Chrome I see:
[
0: {
avatarURL: "https://lh3.googleusercontent.com/..."
displayName: "Cool guy"
userId: "1234"
username: "cool_guy"
}
1: ...
]
*/
console.log(receivers)
My plan is to then iterate over this array using map
{
receivers && conversations
? receivers.map((element, index) => {
return <ChatLink
path={conversations[index].path}
lastMessage={conversations[index].last_message}
displayName={element.displayName}
username={element.username}
avatarURL={element.avatarURL}
key={index}
/>
})
: null
}
How can I write this to not return a empty array?
Here's a SO question related to what I'm experiencing here

I believe your issue is related to you second useEffect hook when you attempt to do the following:
const getReceivers = async () => {
let receivers: Profile[] = []
await conversations.forEach(async (element) => {
const receiver = await getProfileById(element.conversationWith, token)
receivers.push(receiver)
})
return receivers
}
getReceivers().then(receivers => {
setReceivers(receivers)
})
}
Unfortunately, this won't work because async/await doesn't work with forEach. You either need to use for...of or Promise.all() to properly iterate through all conversations, call your API, and then set the state once it's all done.
Here's is a solution using Promise.all():
function App() {
const [conversations, setConversations] = useState<IConversation[]>([]);
const [receivers, setReceivers] = useState<Profile[]>([]);
useEffect(() => {
messagesDatabase.conversations.toArray().then(result => {
setConversations(result);
});
}, []);
useEffect(() => {
if (conversations.length === 0) {
return;
}
async function getReceivers() {
const receivers: Profile[] = await Promise.all(
conversations.map(conversation =>
getProfileById(element.conversationWith, token)
)
);
setReceivers(receivers);
}
getReceivers()
}, [conversations]);
// NOTE: You don't have to do the `receivers && conversations`
// check, and since both are arrays, you should check whether
// `receivers.length !== 0` and `conversations.length !== 0`
// if you want to render something conditionally, but since your
// initial `receivers` state is an empty array, you could just
// render that instead and you won't be seeing anything until
// that array is populated with some data after all fetching is
// done, however, for a better UX, you should probably indicate
// that things are loading and show something rather than returning
// an empty array or null
return receivers.map((receiver, idx) => <ChatLink />)
// or, alternatively
return receivers.length !== 0 ? (
receivers.map((receiver, idx) => <ChatLink />)
) : (
<p>Loading...</p>
);
}
Alternatively, using for...of, you could do the following:
function App() {
const [conversations, setConversations] = useState<IConversation[]>([]);
const [receivers, setReceivers] = useState<Profile[]>([]);
useEffect(() => {
messagesDatabase.conversations.toArray().then(result => {
setConversations(result);
});
}, []);
useEffect(() => {
if (conversations.length === 0) {
return;
}
async function getReceivers() {
let receivers: Profile[] = [];
const profiles = conversations.map(conversation =>
getProfileById(conversation.conversationWith, token)
);
for (const profile of profiles) {
const receiver = await profile;
receivers.push(receiver);
}
return receivers;
}
getReceivers().then(receivers => {
setReceivers(receivers);
});
}, [conversations]);
return receivers.map((receiver, idx) => <ChatLink />);
}

i think it is happening because for getReceivers() function is asynchronous. it waits for the response, in that meantime your state renders with empty array.
you can display spinner untill the response received.
like
const[isLoading,setLoading]= useState(true)
useEffect(()=>{
getReceivers().then(()=>{setLoading(false)}).catch(..)
} )
return {isLoading ? <spinner/> : <yourdata/>}

Please set receivers initial value as array
const [receivers, setReceivers] = useState<Profile[]>([])
Also foreach will not wait as you expect use for loop instead of foreach
I am not sure it is solution for your question
but it could help you to solve your error

Recursion and Observable RxJs

I am performing pagination inside and Observable stream.
The pagination is implemented with a cursor and a total count using recursion.
I am able to emit the every page using the following code observer.next(searches);, by the way I would like to use just observable and no promises but I cannot express recursion using RxJs operators.
Any suggestions?
const search = id =>
new Observable(observer => { recursePages(id, observer) })
const recursePages = (id, observer, processed, searchAfter) => {
httpService.post(
"http://service.com/search",
{
size: 50,
...searchAfter ? { search_after: searchAfter } : null,
id,
})
.toPromise() // httpService.post returns an Observable<AxiosResponse>
.then(res => {
const body = res.data;
const searches = body.data.hits.map(search => ({ data: search.data, cursor: search.id }));
observer.next(searches);
const totalProcessed = processed + searches.length;
if (totalProcessed < body.data.total) {
return recursePages(id, observer, totalProcessed, searches[searches.length - 1].cursor);
}
observer.complete();
})
}
// General Observer
incomingMessages.pipe(
flatMap(msg => search(JSON.parse(msg.content.toString()))),
concatAll(),
).subscribe(console.log),

these methods will recursively gather all the pages and emit them in an array. the pages can then be streamed with from as shown:
// break this out to clean up functions
const performSearch = (id, searchAfter?) => {
return httpService.post(
"http://service.com/search",
{
size: 50,
...searchAfter ? { search_after: searchAfter } : null,
id,
});
}
// main recursion
const _search = (id, processed, searchAfter?) => {
return performSearch(id, searchAfter).pipe( // get page
switchMap(res => {
const body = res.data;
const searches = body.data.hits.map(search => ({ data: search.data, cursor: search.id }));
const totalProcessed = processed + searches.length;
if (totalProcessed < body.total) {
// if not done, recurse and get next page
return _search(id, totalProcessed, searches[searches.length - 1].cursor).pipe(
// attach recursed pages
map(nextPages => [searches].concat(nextPages)
);
}
// if we're done just return the page
return of([searches]);
})
)
}
// entry point
// switch into from to emit pages one by one
const search = id => _search(id, 0).pipe(switchMap(pages => from(pages))
if what you really need is all of the pages to emit one by one before they're all fetched, for instance so you can show page 1 as soon as it's available rather than wait on page 2+, then that can be done with some tweaking. let me know.
EDIT: this method will emit one by one
const _search = (id, processed, searchAfter?) => {
return performSearch(id, searchAfter).pipe( // get page
switchMap(res => {
const body = res.data;
const searches = body.data.hits.map(search => ({ data: search.data, cursor: search.id }));
const totalProcessed = processed + searches.length;
if (totalProcessed < body.total) {
// if not done, concat current page with recursive call for next page
return concat(
of(searches),
_search(id, totalProcessed, searches[searches.length - 1].cursor)
);
}
// if we're done just return the page
return of(searches);
})
)
}
const search = id => _search(id, 0)
you end up with an observable structure like:
concat(
post$(page1),
concat(
post$(page2),
concat(
post$(page3),
post$(page4)
)
)
)
and since nested concat() operations reduce to a flattened structure, this structure would reduce to:
concat(post$(page1), post$(page2), post$(page3), post$(page4))
which is what you're after and the requests run sequentially.
it also seems like expand might do the trick as per #NickL 's comment, soemthing like:
search = (id) => {
let totalProcessed = 0;
return performSearch(id).pipe(
expand(res => {
const body = res.data;
const searches = body.data.hits.map(search => ({ data: search.data, cursor: search.id }));
totalProcessed += searches.length;
if (totalProcessed < body.data.total) {
// not done, keep expanding
return performSearch(id, searches[searches.length - 1].cursor);
}
return EMPTY; // break with EMPTY
})
)
}
though I've never used expand before and this is based off some very limited testing of it, but I am pretty certain this works.
both of these methods could use the reduce (or scan) operator to gather results if you ever wanted:
search(id).pipe(reduce((all, page) => all.concat(page), []))

This is my used solution combining the expand and reduce operator
searchUsers(cursor?: string) {
return from(
this.slackService.app.client.users.list({
token: this.configService.get('SLACK_BOT_TOKEN'),
limit: 1,
...(cursor && { cursor }),
}),
);
}
Usage
.......
this.searchUsers()
.pipe(
expand((res) => {
if (!!res.response_metadata.next_cursor) {
return this.searchUsers(res.response_metadata.next_cursor);
}
return EMPTY;
}),
reduce((acc, val) => {
return [...acc, ...val.members];
}, []),
)
.subscribe((users) => {
console.log(JSON.stringify(users));
});
....

Is there a way to use a observable returning function for each element of another observable array?

I get an Observable<Group[]> from my Firebase collection.
In this Group class is an id which I wanna use to retrieve another dataset array from Firebase, which would be messages for each unique group Observable<Message[]>.(each group has its own chat: Message[])
And it want to return an observable which hold an array of a new Type:
return { ...group, messages: Message[] } as GroupWithMessages
the final goal should be Observable<GroupWithMessages[]>
getGroupWithChat(): Observable<GroupWithMessages[]> {
const groupColl = this.getGroups(); // Observable<Group[]>
const messages = groupColl.pipe(
map(groups => {
return groups.map(meet => {
const messages = this.getMessagesFor(group.uid);
return { messages:messages, ...group} as GroupWithMessages
});
})
);
return messages;
}
}
and here the Message function
getMessagesFor(id: string): Observable<Message[]> {
return this.afs.collection<Message>(`meets/${id} /messages`).valueChanges();
}
sadly that doesnt work because when i create the new Obj I cannot bind messages:messages because messages ist vom typ Observable<Message[]>
I hope that cleares things
UPDATE:
my main problem now comes down to this:
getGroupsWithMessages() {
this.getJoinedGroups()
.pipe(
mergeMap(groups =>
from(groups).pipe(
mergeMap(group => {
return this.getMessagesFor(group.uid).pipe(
map(messages => {
return { ...group, messages } as GroupIdMess;
})
);
}),
tap(x => console.log('reaching here: ', x)),
toArray(),
tap(x => console.log('not reaching here = completed: ', x))
)
),
tap(x => console.log('not reaching here: ', x))
)
.subscribe(x => console.log('not reaching here: ', x));
}
when i call that function my console.log is as follows:

Not sure if I follow what you're doing here but the logic look like you'd want:
getGroupWithChat() {
return this.getGroups.pipe(map(groups=> {
return groups.map(group => this.getMessagesFor(group.uid));
})).subscribe(); // trigger "hot" observable
}
Let me know if I can help further after you clarify.
UPDATE:
So it looks like you need to get the UID of the group before making the call to get the GroupMessages[]?
get Group: Observable
call getMessagesFor(Group.uid)
this example gets groups result$ then
concatMap uses groups result$ to make the messages query
this.getGroups().pipe(
concatMap((group: Group) => this.getMessagesFor(group.uid))
).subscribe((messages: GroupWithMessages[]) => {
console.log(messages);
});
You may still want to map them together but it seems like you know how to do that. concatMap waits for the first to finish, then makes the second call which you need.
Is this closer?

Use forkJoin to wait for messages to be received for all groups. Then map the result of forkJoin to an array of GroupWithMessages like this -
getGroupWithChat(): Observable<GroupWithMessages[]> {
return this.getGroups()
.pipe(
switchMap(groups => {
const messagesForAllGroups$ = groups.map(group => this.getMessagesFor(group.uid));
return forkJoin(messagesForAllGroups$)
.pipe(
map(joined => {
//joined has response like -
//[messagesArrayForGroup0, messagesArrayForGroup1, messagesArrayForGroup2....];
const messagesByGroup = Array<GroupWithMessages>();
groups.forEach((group, index) => {
//assuming that GroupWithMessages has group and messages properties.
const gm = new GroupWithMessages();
gm.group = group;
gm.messages = joined[index];
messagesByGroup.push(gm);
});
return messagesByGroup;
})
)
})
)
}

I usually do that by splitting Observable<any[]> to Observable<any> and then mergeMap the results to inner Observable.
Something like this should work:
getMessagesFor(id: string): Observable<number> {
return of(1);
}
getGroups(): Observable<string[]> {
return of(["1", "2"]);
}
getGroupWithChat() {
this.getGroups().pipe(
mergeMap(groups => from(groups)), // Split the stream into individual group elements instead of an array
mergeMap(group => {
return this.getMessagesFor(group).pipe(
map(messages => {
return Object.assign(group, messages);
})
);
})
);
}
Edit:
Consider BehaviorSubject. It doesn't complete at all:
const behSub: BehaviorSubject<number[]> = new BehaviorSubject([1, 2, 3]);
setTimeout(() => {
behSub.next([4, 5, 6]);
}, 5000);
behSub
.pipe(
mergeMap(arr =>
from(arr).pipe(
tap(), // Do something with individual items, like mergeMap to messages
toArray() // Go back to array
)
)
)
.subscribe(console.log, null, () => {
console.log('Complete');
});