Multiple RxJS AJAX Requests - javascript

I'm using rxjs to make several http requests and I want to end up with an object that looks something like:
{
100: {
...response from api call...
},
205: {
...response from api call...
},
...etc...
}
Here's what I have so far:
const projectIds = [100, 205, 208, 300]
const source = Rx.Observable
.from(projectIds)
.flatMap(id => get(`projects/${id}/builds`))
.map(response => response['data'])
.zip(projectIds)
.toArray()
source.subscribe(pipelines => {
console.log(pipelines)
})
This gives me back an array of arrays where the first element is the response from the call and the second element is the id of the project.
The problem is that the response doesn't match the project id as the responses come back in different orders depending on which request completes first.
How can I preserve the order (or at least know which projectId goes with each response) while also ending up with an object at the end (currently is an array)?

Option 1
Instread of flatMap you could use concatMap, that should preserve the order.
Note: This won't make any concurrent requests though, if that is what you are looking for.
Option 2
If you want to make concurrent requests (at least from the RxJS side, depending on the browser this could still be limited) you could use some construct using forkJoin like the following:
const projectIds = [100, 205, 208, 300]
const source = Rx.Observable
.from(projectIds)
.map(id => get(`projects/${id}/builds`).pluck('data'))
.toArray()
.switchMap(requestArray => Rx.Observable.forkJoin(requestArray))
.zip(projectIds)
source.subscribe(pipelines => {
console.log(pipelines)
})

Just use the flatMap with elementSelector overload:
.flatMap(
projectId => getProjectDetails(projectId),
(projectId, details) => ({ id: projectId, details })
)
function getProjectDetails(id){
return get(`projects/${id}/builds`)
.map(response => response['data']);
}
This will let you combine the input argument and every output value from flatMap as you require, effectively preserving context. If you require the output order to stay the same you can use .concatMap but then all emissions are done after each other instead of concurrently.
Then finally use a .reduce to combine all objects back to one big emission:
.reduce((acc, curr) => acc[curr.id] = curr.details, {})

Related

How to combine requests in to an array in angular?

How to combine two responses in to an array in angular?
Following is making an http post requests to two endpoints. How can I combine both of them an provide a return value as an array?
Like this:
postCombined() {
return combineLatest([this.link1$, this.link2$])
.pipe(
mergeMap(([link1, link2]: [string, string]) => {
return [
this.http.post(link1, values1, { headers }),
this.http.post(link2, values2, { headers }),
];
})
)
.subscribe(console.log);
}
Is my implementation is correct? or do I need to use forkJoin?
forkJoin seems the better option as it allows us to group multiple observables and execute them in parallel, then return only one observable.
mergeMap maintains multiple active inner subscriptions at once, so it’s possible to create a memory leak through long-lived inner subscriptions.
postCombined() {
return combineLatest([this.link1$, this.link2$])
.pipe(
let http1$ = this.http.post(link1, values1, {
headers
}), http2$ = this.http.post(link2, values2, {
headers
})
forkJoin([https1$, http2$])
)
.subscribe(console.log);
}
Taken from this article

Using setImmediate to improve JSON.stringify performance of large amounts of data

I have some code that JSON.stringify's an array of objects like so:
const postsCommentsReadyForDB = postsComments.map(postComments => ({
id: getPostId(postComments),
comments: JSON.stringify(getComments(postComments)),
}))
However on very large amounts of data (e.g. an array of 6000+ objects), it can take up to 3+ seconds for this to complete because of all the JSON.stringify processing.
I'm not so worried about the time it takes to do this, but I would like to avoid blocking the event loop or any I/O, so I thought I could perhaps use setImmediate to solve this.
Here is my approach to doing this with setImmediate:
const postsCommentsReadyForDB = []
postsComments.forEach(postComments => {
setImmediate(_ => {
postsCommentsReadyForDB.push({
id: getPostId(postComments),
comments: JSON.stringify(getComments(postComments)),
})
})
})
However when I run this it doesn't seem to work and postsCommentsReadyForDB is empty.
Is it possible to use setImmediate in this scenario?
Based on Bergi's suggestion, this approach seems to work:
const util = require('util')
const setImmediatePromise = util.promisify(setImmediate)
async function foo(){
const postsCommentsReadyForDB = []
for (const postComments of postsComments) {
await setImmediatePromise()
postsCommentsReadyForDB.push({
id: getPostId(postComments),
comments: JSON.stringify(getComments(postComments)),
})
}
}
foo().catch(err => console.error(err))

RXJS convert single observable to array of observables

I have an API (getNewStories) that returns the data as an array of numbers(ids) such as [1,2,3,4...].
There is another API (getItem) that uses the number(id) and give its details.
How can I accomplish this with rxjs operators, so that I should only subscribe to it once and it gives me an array of the records with those ids?
I am able to accomplish this using 2 subscriptions, but I want it with one. Is it possible? and if it's, then, how?
this.hnService.getNewStories().subscribe(data => {
// data is [1,2,3,4,5]
// create an array of observables for all the ids and get the record for that id
const observables = data.map(item => this.hnService.getItem(item));
// use forkJoin to combine the array to single results variable
forkJoin(...observables).subscribe(results => {
this.stories = results;
});
});
with this I have to subscribe to both the APIs.
You were going the right direction with using forkJoin:
this.hnService.getNewStories()
.pipe(
concatMap(data => {
const items$ = data.map(item => this.hnService.getItem(item));
return forkJoin(...items$);
}),
)
.subscribe(allItems => ...);
forkJoin will wait until all source Observables complete and only then emit all results as a single array.
I think you can achieve this using a flattening operator like this.
this.hnService.getNewStories().pipe(
.mergeMap(data => this.hnService.getItem(data))
.subscribe(res => this.stories = res);
Other option can be to create two observable streams and use a combineLatest.
You can simply have implemented as in the the following snippet: (Yes, mergeAll flattens an observable containing an array, for further explanation refer to #Martin's post about the Best way to “flatten” an array inside an RxJS Observable
)
getNewStories().pipe(mergeAll(), concatMap(this.getItem), toArray()).subscribe()
You can try running the following snippet:
const { of } = rxjs;
const { concatMap, toArray, mergeAll } = rxjs.operators;
function getItem(x) {
return of({ item : x })
}
of([1, 2, 3, 4])
.pipe(
mergeAll(),
concatMap(getItem),
toArray()
)
.subscribe(console.log)
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.6.2/rxjs.umd.min.js"></script>

Angular - Observer map not working as intended

I'm trying to fetch data from my backend, creating a class object for each item I get
getRankingList(type: RankingListType, page: number) {
let params = new HttpParams().set("pid", String(page)).set("limit", String(5));
return this.http.get(`http://127.0.0.1:3333/ranking/player/all`, { params })
.pipe(
map(item => new RankingGuild(item['guild'], item['name'], item['country'], item['honor'], item['RawKey']))
);
}
The data I'm receiving from the backend looks like this:
[
{
"RawKey": "1",
"honor": 0,
"guild": "Test",
"name": "test",
"country": 1
},
{
"RawKey": "2",
"honor": 0,
"guild": "Test2",
"name": "test2",
"country": 1
}
]
But instead of iterating through the object, "item" is the object itself, meaning there is only one iteration returning the object that I had in the first place, rather than its entries. I've been searching for hours to find a solution, but it seems like this is the correct way to do it, not sure why it doesn't work.
This is because the RxJS map operator and JavaScript's Array.map() are 2 different things altogether. You should read up on their differences.
In short, the RxJS map operator allows you to apply a given project function to each value emitted by the source Observable, and emit the resulting values as an Observable. On the other hand, the Array.map() method merely creates a new array with the results of calling a provided function on every element in the calling array.
If you want to map the value returned by the response from the HTTP request, I believe this is what you should be doing instead.
getRankingList(type: RankingListType, page: number) {
let params = new HttpParams().set("pid", String(page)).set("limit", String(5));
return this.http.get(`http://127.0.0.1:3333/ranking/player/all`, { params })
.pipe(
map(response => response.map(item => new RankingGuild(item['guild'], item['name'], item['country'], item['honor'], item['RawKey'])))
);
}
Then, on your component itself, you may subscribe to the method to return the actual values itself.
getRankingList.subscribe(res => {
// do the rest
})
The rx map operator is not the array map operator. The array map transforms items in an array, the rx map transforms items in a stream, and the item in the stream in this case is an array. Do this:
return this.http.get(`http://127.0.0.1:3333/ranking/player/all`, { params })
.pipe(
map(items => items.map(item => new RankingGuild(item['guild'], item['name'], item['country'], item['honor'], item['RawKey'])))
);
Use the array map inside your rx map.

Angular make multiple http request but wait for each one to finish before making a new one

In angular app I have an array of literal objects containing a url property.
I need to make a http request for each of these object, but one after another.
example:
let arr = [
{url: 'http://example.com/url1'},
{url: 'http://example.com/url2'},
{url: 'http://example.com/url3'}
]
(that is an example, they may be more of objects, and I don't know how many)
Now, I want to make a request to first url and when we have a response from it (or error, doesn't matter) THEN I want to make a request to second etc. Any idea how to efficiently implement that?
I don't want to make these request at single time - each one should be made only after previous was successful or failure.
A possible solution would be to use concatMap, toArray and switchMapTo.
So first you have a list of urls:
let arr = [{url: 'http://example.com/url1'},{url: 'http://example.com/url2'}]
Then you transform them to an Observable:
of(arr)
.pipe(
concatMap(r=> http.get(r.url)), //MAKE EACH REQUEST AND WAIT FOR COMPLETION
toArray(), // COMBINE THEM TO ONE ARRAY
switchMapTo(http.get("FINALURL") // MAKE REQUEST AFTER EVERY THING IS FINISHED
)).subscribe()
We can use tricky method for this. You have to create method in the service like below.
// TestService.ts
callUrlAsync(url): any[] {
return this._http.get<any>(url);
}
In the component you have to call this method as follows.
//component.ts
let arr = [{url: 'http://example.com/url1'},{url: 'http://example.com/url2'},
{url:'http://example.com/url3'}]
public i = 0;
//trigger method
testMethod(){
this.callUrl(this.arr[0]);
}
callUrl(url){
this.testService.callUrlAsync(url)
.subscribe(data => {
console.log(data);
if(this.arr.length > this.i){
this.i++;
this.callUrl(this.arr[this.i]);
}
}
}, error => {
this.Error(error);
if(this.arr.length > this.i){
this.i++;
this.callUrl(this.arr[this.i]);
}
}
);
}
You can combine your observables with flatMap. Since you have a list of observables (or a list of urls that you want to transform into observables), you can do this with reduce.
Example:
// just some options to make a simple GET without parsing JSON
// and reading the response status
const httpOptions = {
headers: new HttpHeaders({
'Accept': 'text/html, application/xhtml+xml, */*',
'Content-Type': 'application/x-www-form-urlencoded'
}),
responseType: 'text',
observe: 'response'
};
const urls = [
{ url: 'www.google.com' },
{ url: 'www.stackoverflow.com' },
{ url: 'www.imgur.com' },
{ url: 'www.reddit.com' },
];
const reducer = (cur, acc) => acc.pipe(
flatMap(r => cur)
);
const all$ = urls.map(({url}) =>
// create the get requests
http.get(url, httpOptions).pipe(
// do something with the result
tap(r => console.log(r.url + ': ' + r.status))
))
.reverse()
.reduce(reducer);
all$.subscribe(x => {});
User forkJoin
Fork Join
let arr = [{url: 'http://example.com/url1'},{url: 'http://example.com/url2'},{url:
'http://example.com/url3'}]
forkJoin(arr);
Use concatAll() method from rxJs which collect observables and subscribe to next when previous completes. (Or you can use forkJoin for multiple request at single time.)
forkJoin - This will group all your request and execute one by one.
forkJoin waits for each http request to complete and group’s all the observables returned by each http call into a single observable array and finally return that observable array.
It accepts array as parameter. for example -
let response1 = this.http.get(requestUrl1);
let response2 = this.http.get(requestUrl2);
let response3 = this.http.get(requestUrl3);
return Observable.forkJoin([response1, response2, response3]);
For ref. - https://medium.com/#swarnakishore/performing-multiple-http-requests-in-angular-4-5-with-forkjoin-74f3ac166d61
This operator is best used when you have a group of observables and only care about the final emitted value of each.
In another way you can call each request in previous request's error or success block, which is lengthy process.
Use concatMap, make sure to catchError because you don't want the entire chain to fail if one link produced an error.
StackBlitz
let results$ = from(arr).pipe(
concatMap(({ url }) =>
requestData(url).pipe(catchError(err => of(`error from ${url}`)))
)
);

Categories