Thanks for reading.
I have two Collections in Firestore and I'm using Angularfire2.
I have a collections of "Clients" and a collection of "Jobs". Clients can have a number of Jobs and each job has a linked client.
I've created a component to show a list of all the jobs, and I'm trying to pull in the associated client for each job using the Firestore key.
Here's my data:
I have been able to "hack" a solution but it is extremely buggy, and it looses all async - so I might as well write a MySQL backend and forget Firestore - the whole point of using Firestore is it's "live".
public jobs = {};
[...]
processData() {
const clients = [];
this.clientCollection.ref.get().then((results) => {
results.forEach((doc) => {
clients[doc.id] = doc.data();
});
const jobs = {};
this.jobsCollection.ref.get().then((docSnaps) => {
docSnaps.forEach((doc) => {
jobs[doc.id] = doc.data();
jobs[doc.id].id = doc.id;
jobs[doc.id].clientData = clients[doc.data().client];
});
this.jobs = jobs;
});
});
}
This works up to a point - but it strips out the async.
Key question: is there any way of doing a "join" as we would in an SQL database to pull these two data sets together? And is there a way of doing it in a way that will keep the async nature of the data?
You can try to use Promise.all() to fetch multiple promises into one object. Here is a revised version of your method that should give the desired result if I am understanding your question correctly:
async processData() {
const clientCollection = await firebase.firestore().collection(`clients`).get();
const jobsCollection = await firebase.firestore().collection(`jobs`).get();
const clients = [];
clients.push(clientCollection);
clients.push(jobsCollection);
const res = await Promise.all(clients);
// res should equal a combination of both querySnapshots
}
I just recreated the collection variables to show what to add to the array, but Promise.all() takes in an array of promises and resolves all of them into one array as the get() method in firestore is a Promise. This is also using async/await. Hope this can be of help!
EDIT:
Since you are using AngularFire2, you should use their method for this.
In you component you are going to want to import the angularfire2/firestore module and use the Observable methods they provide:
first import the module: import { AngularFireStore } from 'angularfire2/firestore';
then provide it to your constructor:
constructor(
private firestore: AngularFireStore
) { }
Then you could use Observable.combineLatest() to receive all the data at one time:
clients$: Observable<any[]>;
clients: any[];
jobs$: Observable<any[]>;
jobs: any;
joined: any;
ngOnInit() {
this.clients$ = this.getCollection('clients');
this.jobs$ = this.getCollection('jobs');
this.joined = Observable.combineLatest(this.clients$, this.jobs$);
this.joined.subscribe(([clients, jobs]) => {
this.clients = clients;
this.jobs = jobs;
});
}
getCollection(collectionName) {
return this.firestore.collection(`${collectionName}`).valueChanges();
}
In your markup you would just loop the data using *ngFor:
<div *ngFor="let client of clients">{{ client.name }}</div>
This way, your component will be listening for new data once firestore has the data and it will all come in at one time so you don't have nested subscriptions that are vulnerable to creating multiple subscriptions. Hope this can be of help.
late for solutions but after 4 hours searching I found this solution library its very useful for join collection and get data.
main resource : https://github.com/AngularFirebase/133-firestore-joins-custom-rx-operators
first create one type script file
file name : collectionjoin.ts
import { combineLatest, pipe, of, defer } from 'rxjs';
import { map, switchMap, tap } from 'rxjs/operators';
export const leftJoinDocument = (afs: AngularFirestore, field, collection) => {
return source =>
defer(() => {
// Operator state
let collectionData;
const cache = new Map();
return source.pipe(
switchMap(data => {
// Clear mapping on each emitted val ;
cache.clear();
// Save the parent data state
collectionData = data as any[];
const reads$ = [];
let i = 0;
for (const doc of collectionData) {
// Skip if doc field does not exist or is already in cache
if (!doc[field] || cache.get(doc[field])) {
continue;
}
// Push doc read to Array
reads$.push(
afs
.collection(collection)
.doc(doc[field])
.valueChanges()
);
cache.set(doc[field], i);
i++;
}
return reads$.length ? combineLatest(reads$) : of([]);
}),
map(joins => {
return collectionData.map((v, i) => {
const joinIdx = cache.get(v[field]);
return { ...v, [field]: joins[joinIdx] || null };
});
}),
tap(final =>
console.log(
`Queried ${(final as any).length}, Joined ${cache.size} docs`
)
)
);
});
};
after that in your homepage.ts or any page typescript file.
import {
AngularFirestore,
AngularFirestoreCollection,
AngularFirestoreDocument,
} from "#angular/fire/firestore";
jobs: any
constructor( public afStore: AngularFirestore ) { }
this.afStore.collection('Jobs').valueChanges().pipe(
leftJoinDocument(this.afStore, 'client', 'Clients'),
shareReplay(1)
).subscribe((response) => {
this.products = response;
})
in this step we are passing three arguments
this.afStore = this is object of lib.
'client' = this is key/id of jobs collections
'Clients' = this is collection name which we are join.
now last step Display result
<ion-list lines="full" *ngFor="let job of Jobs">
<ion-item button detail>
<ion-label class="ion-text-wrap">{{ job.Name }}</ion-label>
<p>{{ job.Clients.companyName}}</p>
</ion-item>
</ion-list>
finally this code given two collection record.
thank you.
After trying multiple solution I get it done with RXJS combineLatest, take operator. Using map function we can combine result.
Might not be an optimum solution but here its solve your problem.
combineLatest(
this.firestore.collection('Collection1').snapshotChanges(),
this.firestore.collection('Collection2').snapshotChanges(),
//In collection 2 we have document with reference id of collection 1
)
.pipe(
take(1),
).subscribe(
([dataFromCollection1, dataFromCollection2]) => {
this.dataofCollection1 = dataFromCollection1.map((data) => {
return {
id: data.payload.doc.id,
...data.payload.doc.data() as {},
}
as IdataFromCollection1;
});
this.dataofCollection2 = dataFromCollection2.map((data2) => {
return {
id: data2.payload.doc.id,
...data2.payload.doc.data() as {},
}
as IdataFromCollection2;
});
console.log(this.dataofCollection2, 'all feeess');
const mergeDataFromCollection =
this.dataofCollection1.map(itm => ({
payment: [this.dataofCollection2.find((item) => (item.RefId === itm.id))],
...itm
}))
console.log(mergeDataFromCollection, 'all data');
},
Related
I am using this request to get my documents from Firestore asynchronously.
During an operation (delete for example) the list of my documents is not updated automatically. How can I transform my asynchronous function into an observable in order to take advantage of the real-time functionalities of Firestore and get the id of the document ?
import { Injectable } from '#angular/core';
import { SentencePair } from '../models/sentence-pair.model';
import { Firestore, collectionData, deleteDoc,limit, limitToLast,orderBy, increment,
addDoc, collection, doc, updateDoc, setDoc,query, where, getDocs } from
'#angular/fire/firestore';
import { Observable,combineLatest,map, defer } from 'rxjs';
#Injectable({ providedIn: 'root'})
export class SentencesPairsService {
constructor(private firestore: Firestore) { }
async FilterPairs(field: string, boolean: boolean, display:any) {
const sentencepairsRef = collection(this.firestore, 'SentencesPairs');
const q = query(sentencepairsRef,
where('validation', '==', boolean),
where('origin_trad', 'in', display),
orderBy(field),limitToLast(10));
const truc = await getDocs(q)
return truc.docs.map(docData=>({...docData.data(), id:docData.id}));
}
I use AngularFire 7.2. Thanks for your help.
Get an observable for data from Firestore query
If you want an observable from Firestore, you need to return the .valueChanges() on a AngularFirestoreCollection. Read this doc for reference: https://github.com/angular/angularfire2/blob/master/docs/firestore/querying-collections.md.
For example :
getUserByEmail(email: string): Observable<User> {
const collection = this.firestore.collection<User>('users', ref => ref.where('email', '==', email))
const user$ = collection
.valueChanges()
.pipe( map(users => {
const user = users[0];
console.log(user);
return user;
}));
return user$;
}
Get ID of the document
If you want the ID( which is not returned in valueChanges())on your document you need to use snapshotChanges(). Sometimes it's easier to maintain the id on the user data when saving to Firestore to avoid using snapshotChanges. Also SwitchMap is helpful in this case as when a new value comes from the source (userId in this case) it will cancel its previous firestore subscription and switch into a new one with the new user id. If for some reason you want to maintain firestore subscriptions for all userIds that come through at once, then use mergeMap instead. But typically you only want to be subscribed to one user's data at a time.
// Query the users by a specific email and return the first User with ID added using snapshotChanges()
return this.firestore.collection<User>('users', ref => ref.where('email', '==', email))
.snapshotChanges()
.pipe(map(users => {
const user = users[0];
if (user) {
const data = user.payload.doc.data() as User;
const id = user.payload.doc.id;
return { id, ...data };
}
else { return null;
}
}));
Note to subscribe to the changes :
The user will be null until you call subscribe on this.user$ :
const queryObservable = size$.pipe(
switchMap(size =>
afs.collection('items', ref => ref.where('size', '==', size)).snapshotChanges()
)
);
queryObservable.subscribe(queriedItems => {
console.log(queriedItems);
});
OR
In your html use the async pipe which handles subscribe and unsubscribe on an observable like below:
<div *ngIf="(user$ | async) as user">{{ user.email }}</div>
Try to avoid using the async pipe on the same observable multiple times in your html, and instead set it as a local html variable which i do above (as user) to avoid unnecessary data trips to the db.
So I am running into an issue with my collections object in the Discord.js library. I am working on a command and event handler to populate a collection of commands and events based on the files written in each directory. Each collection appears to populate properly as long as I check it within the map function. Immediately after the second map is complete, whichever collection I populate second becomes empty, yet the first collection remains set.
If I reverse the order they are set, the problem changes to whichever collection is set second. And because they both set fine when debugged within the map or if they are set first, I am confident it is not related to anything directory wise or with the files they are importing. I suspect it is somehow related to how collections work on an object that I am unaware of.
Any insight to this would be great!
import { Command, Event, Config } from "../Interfaces/index"
import { Client, Collection, Intents } from "discord.js"
import glob from "glob";
import { promisify } from "util";
const globPromise = promisify(glob)
class Bot extends Client {
public events: Collection<string, Event> = new Collection()
public commands: Collection<string, Command> = new Collection()
public aliases: Collection<string, Command> = new Collection()
public config: Config
public constructor() {
super({ ws: { intents: Intents.ALL } })
}
public async init(config: Config): Promise<void> {
this.config = config
this.login(this.config.token)
const commandFiles: string[] = await globPromise(`${__dirname}/../Commands/**/*.ts`)
commandFiles.map(async (filePath: string) => {
const { command }: { command: Command } = await import(filePath)
this.commands.set(command.name, command)
if (command.aliases?.length !== 0) {
command.aliases?.forEach((alias) => {
this.aliases.set(alias, command)
})
}
})
const eventfiles: string[] = await globPromise(`${__dirname}/../Events/**/*.ts`)
eventfiles.map(async (filePath: string) => {
const { event }: { event: Event } = await import(filePath)
this.events.set(event.name, event)
console.log(this) // Events and commands collection are populated
})
console.log(this) // Events collection is empty and commands collection is populated
}
}
You may not know, but you mapped each of the commandFiles and eventFiles items to a Promise. To make sure that the async function was actually completed before you called console.log(this) you need to await the Promises returned by the map function.
To await every item that is returned from the map, wrap the call in Promise.all:
const commandFiles: string[] = await globPromise(`${__dirname}/../Commands/**/*.ts`)
await Promise.all(
commandFiles.map(async (filePath: string) => {
...
})
);
const eventfiles: string[] = await globPromise(`${__dirname}/../Events/**/*.ts`)
await Promise.all(
eventfiles.map(async (filePath: string) => {
...
})
);
console.log(this) // collections should be populated at this point
// because you awaited the `map` results
Leaving 'dangling' Promises can very often lead to some unexpected errors.
I am trying to check the validity of a function I have written in Typescript, in congruence with RxJS observables, that fetches some bookings from one service and then for each booking fetches its corresponding location and activity from another service.
I am simply writing this post to verify the validity of what I have written and to ask if there is anything I could have done more efficiently.
let params = new HttpParams();
params = params.append('status', 'C');
params = params.append('offset', offset.toString());
params = params.append('limit', limit.toString());
return this.http.get(`${this.environment.booking.url}/my/bookings`, { params }).pipe(
mergeMap((bookings: Booking[]) => {
if(bookings.length > 0) {
return forkJoin(
bookings.map((booking: Booking) =>
forkJoin(
of(booking),
this.activityService.getActivity(booking.activity),
this.locationService.getLocation(booking.finalLocation),
).pipe(
map((data: [ Booking, Activity, Location ]) => {
let booking = data[0];
booking.activityData = data[1];
booking.finalLocationData = data[2];
return booking;
})
)
)
)
}
return of([]);
}),
catchError((err: HttpErrorResponse) => throwError(err))
);
I am expecting for this function to return a list of bookings alongside their corresponding location and activity. However more importantly I want to verify that what I am doing is correct and sensible. Is there anything I could have done differently to make it cleaner/ more human-readable (not nit-picking, please 😁 )?
On a different note, that of performance, I also have a follow-up question with regards to performance. Given that a list of bookings has common activities and locations. Is there a way to only fetch activities and locations without any duplicate HTTP requests? Is this already handled under the hood by RxJS? Is there anything I could have done to make this function more efficient?
I'm not sure about the efficiency, but, at least for me, it was a little hard to read
Here's how I'd do it:
I used a dummy API, but I think it correlates with your situation
const usersUrl = 'https://jsonplaceholder.typicode.com/users';
const todosUrl = 'https://jsonplaceholder.typicode.com/todos';
const userIds$ = of([1, 2, 3]); // Bookings' equivalent
userIds$
.pipe(
filter(ids => ids.length !== 0),
// Flatten the array so we can avoid another nesting level
mergeMap(ids => from(ids)),
// `concatMap` - the order matters!
concatMap(
id => forkJoin(ajax(`${usersUrl}/${id}`), ajax(`${todosUrl}/${id}`))
.pipe(
map(([user, todo]) => ({ id, user: user.response, todo: todo.response }))
)
),
toArray()
)
.subscribe(console.log)
Here is a StackBlitz demo.
With this in mind, here is how I'd adapt it to your problem:
this.http.get(`${this.environment.booking.url}/my/bookings`, { params }).pipe(
filter(bookings => bookings.length !== 0),
// Get each booking individually
mergeMap(bookings => from(bookings)),
concatMap(
b => forkJoin(
this.activityService.getActivity(b.activity),
this.locationService.getLocation(b.finalLocation),
)
.pipe(
map(([activity, location]) => ({ ...b, activity, location }))
)
),
// Getting the bookings array again
toArray()
catchError((err: HttpErrorResponse) => throwError(err))
);
This is how I would tackle this using RxJS:
Fetch all the Bookings
For Each Booking fetch Location and Activities cuncurrently
const { from, of, forkJoin, identity } = rxjs;
const { mergeMap, tap, catchError } = rxjs.operators;
const api = 'https://jsonplaceholder.typicode.com';
const endpoints = {
bookings: () => `${api}/posts`,
locations: (id) => `${api}/posts/${id}/comments`,
activities: (id) => `${api}/users/${id}`
};
const fetch$ = link => from(fetch(link)).pipe(
mergeMap(res => res.json()),
catchError(() => from([])),
);
fetch$(endpoints.bookings()).pipe(
mergeMap(identity),
mergeMap(booking => forkJoin({
booking: of(booking),
locations: fetch$(endpoints.locations(booking.id)),
activities: fetch$(endpoints.activities(booking.userId)),
})),
).subscribe(console.log);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.3/rxjs.umd.js" integrity="sha256-Nihli32xEO2dsnrW29M+krVxoeDblkRBTkk5ZLQJ6O8=" crossorigin="anonymous"></script>
note:
Reactive Programming, and more generically declarative approaches, focus on avoiding imperative control flows... You should try to write your pipes without conditions (or any other control flow). To discard empty bookings you can use the filter operator.
Avoid nesting streams because this comes at the cost of readability.
forkJoin also takes a spec object which is very useful (part of the overloads)
So i have 2 collections
1 collection is 'users'. There i have documents (objects) with property 'profile', that contains string. It's id of profile, that is stored in other collection 'roles' as document.
So i'm trying to get this data, but without success. Is there exist join method or something like that? Or i must use promise for getting data from collection roles, and then assign it with agent?
async componentDidMount() {
firebase
.firestore()
.collection('users')
.orderBy('lastName')
.onSnapshot(async snapshot => {
let changes = snapshot.docChanges()
const agents = this.state.agents
for (const change of changes) {
if (change.type === 'added') {
const agent = {
id: change.doc.id,
...change.doc.data()
}
await firebase
.firestore()
.collection('roles')
.doc(change.doc.data().profile).get().then( response => {
//when i get response i want to set for my object from above this val
agent['profile'] = response.data().profile
//after that i want to push my 'agent' object to array of 'agents'
agents.push(agent)
console.log(agent)
}
)
}
}
this.setState({
isLoading: false,
agents: agents
})
})
}
To do async operation on array of objects you can use promise.all, i have given a example below that is similar to your use case where multiple async operation has to be done
const all_past_deals = await Promise.all(past_deals.map(async (item, index) => {
const user = await Users.get_user_info(item.uid);
const dealDetails = await Deals.get_deal_by_ids(user.accepted_requests || []);
const test = await Users.get_user_info(dealDetails[0].uid);
return test
}))
}
This way you can get data from once api call and make other api call with the obtained data
*SOLVED
The problem was in how I was creating and responding to the observable created by the firebase callback.
I also had way too much stuff going on inside my firebase callbacks.
I ended up splitting it up a bit more, using the firebase promise structure: https://firebase.googleblog.com/2016/01/keeping-our-promises-and-callbacks_76.html
and creating an Observable.fromPromise for the firebase callback within what is now called firebaseAPI.checkForUser.
*
Working epic:
export const contactFormFirebaseSubmitEpic = (action$) =>
action$.ofType(START_CONTACT_FORM_FIREBASE_SUBMIT)
.flatMap((firebaseSubmitAction) => {
const values = firebaseSubmitAction.values;
const formattedEmail = firebaseAPI.getFormattedEmail(values);
const contactsRef = firebaseAPI.getContactsRef(formattedEmail);
return firebaseAPI.checkForUser(values, formattedEmail, contactsRef);
})
.flatMap((data) => concat(
of(firebaseAPI.recordUserAndUpdateDetails(data))
))
.flatMap((data) => concat(
of(firebaseAPI.setQuoteData(data))
))
.switchMap((x) => merge(
of(stopLoading()),
of(contactFormFirebaseSuccess())
));
// original question
Ok so, what I'm trying to achieve is to perform the first action (firebaseAPI.checkUserAndUpdate), then the next, then when both of them are done essentially discard what's there and send out two actions (contactFormFirebaseSuccess and stopLoading).
This all works fine except for one weird thing, the setQuoteData function always runs before the checkUser function. Does anyone know why this might be?
Also if there's a better way to lay this out I'd be very open to suggestions! Cheers. Also I've taken out quite a few variables and things that would make it even more complicated. Basically I just wanted to show that in each case I'm returning an observable from 'doing something with firebase'. But I don't think that's the problem as I have console logs in each of the firebase functions and the setQuoteData one just fires first and then executes the firebase stuff then when it's done the checkUserAndUpdate one runs.
export const contactFormFirebaseSubmitEpic = action$ =>
action$.ofType(START_CONTACT_FORM_FIREBASE_SUBMIT)
.flatMap((firebaseSubmitAction) => {
const values = firebaseSubmitAction.values;
return merge(
firebaseAPI.checkUserAndUpdate(values),
firebaseAPI.setQuoteData(values),
)
.takeLast(1)
.mergeMap((x) => {
return merge(
of(contactFormFirebaseSuccess()),
of(stopLoading()),
);
});
});
const firebaseAPI = {
checkUserAndUpdate: (values) => {
const checkUserAndUpdateDetails = firebaseRef.once('value', snapshot => {
const databaseValue = snapshot.val();
checkUserExistsAndUpdateDetails(databaseValue, values);
});
return Observable.from(checkUserAndUpdateDetails);
},
setQuoteData: (value) => {
const setQuote = setQuoteData(values);
return Observable.from(setQuote);
},
};
const stopLoading = () => ({ type: STOP_BUTTON_LOADING });
const contactFormFirebaseSuccess = () => ({ type: SUCCESS });
checkUserAndUpdate: (values, contactsRef) => {
const checkUser$ = Observable.from(contactsRef.once('value').then(
snapshot => {
const databaseValue = snapshot.val();
checkUserExistsAndUpdateDetails(
values,
databaseValue,
contactsRef,);
})
);
return checkUser$;
},
const checkUserExistsAndUpdateDetails = (
values,
databaseValue,
contactsRef,
) => {
if (databaseValue) { console.log('user exists'); }
else {
console.log('new user, writing to database');
contactsRef.set({
name: values.name,
email: values.email,
phone: values.phone,
});
}
};
The problem is that merge does not maintain the order of the streams that you subscribe to, it simply emits events from any of the source streams regardless of what order they emit.
If you need to maintain order you should use concat instead of merge
i.e.
const values = firebaseSubmitAction.values;
return concat(
firebaseAPI.checkUserAndUpdate(values),
firebaseAPI.setQuoteData(values),
)
Side note, I don't know why you are using the of operator there, you already have Observables returned from your API so you can just pass those to merge or concat in this case.