I have to code a tree component that displays multiple data, which worked fine with mocked data for me. The problem here is when I try to get data from servers, let me explain:
I have three main objects : Districts, buildings and doors. As you may guess, doors refers to buildingId and buildingID to districts. So to retrieve data and create my tree nodes, I have to do some http calls in forEach loops which is not asynchronous.
I won't share with you everything but just a minimized problem so I can get help easily:
This method retrieves a district array from the server and puts it in a local array:
async getDistricts(){
this.districtDataService.getDistrictData().toPromise().then(async districts => {
this.districts = await districts.results as District[];
});
}
On my ngOnInit :
ngOnInit() {
this.getDistricts().then(async () => {
console.log(this.districts);
for (const district of this.districts){
console.log(district);
}
})
The first console.log (in NgOnInit) returns an empty array, which is quite surprising because the first method puts the data in "this.districts". and logging data in the first method just after I put it in returns an array with my data. I guess it have something to do with the async/await I've used. Can anyone help?
EDIT 1: Tried to use this.getDistricts().finally() instead of this.getDistricts().then(), but didn't work.
EDIT 2: console.log in getDistrict get executed after the one before my loop. The expected behavior would be the opposite.
SOLVED: putting the for loop in a finally block after my HTTP call solves this. So as the answer says, I think I'm over engineering the async/await calls. I have to rethink my work based on this. Thank you everyone!
Well, you should return your Promise from getDistricts. Also you are very much over engineering and complicating the async/await concept. I understand you don't want to use Observables, but I would advise you to use them anyways.
With promises and async/await so you kinda see how to use them:
async getDistricts(): Promise<District[]> {
const { results } = await this.districtDataService.getDistrictData();
return results;
}
async ngOnInit(): Promise<void> {
this.districts = await this.getDistricts();
for (const district of this.districts){
console.log(district);
}
}
With Observable it would look like this:
getDistricts(): Observable<District[]> {
return this.districtDataService.getDistrictData().pipe(
map(({ results }) => results as District[])
);
}
ngOnInit(): void {
this.getDistricts().subscribe((districts) => {
this.districts = districts;
for (const district of this.districts){
console.log(district);
}
});
}
Just to provide whoever needs to make multiple http calls in a desired order.
as mentionned by others, i overcomplicated the concept of async await.
The trick was to use observables, convert them to Promises using .toPromise(), using .then() to get data into my variables, then making other async calls in finally block using .finally(async () => { ... }).
here's what my final code looks like :
async ngOnInit(): Promise<void> {
await this.districtDataService.getDistrictData().toPromise().then(response => {
this.districts = response.results as District[];
console.log(this.districts);
}).finally(async () => {
for (const district of this.districts){
await this.districtDataService.getBuildingsOfDistrict(district.id).toPromise().then(response => {
this.buildings = response.results as Building[];
console.log(this.buildings);
}).finally(async ()=> {
for(const building of this.buildings){
await this.districtDataService.getDoorsOfBuilding(building.id).toPromise().then(response => {
this.doors = response.results as Door[];
console.log(this.doors);
}).finally(async () => {
for(const door of this.doors){
await this.doorNodes.push(new districtNodeImpl(false,null,null,door,null));
}
})
await this.buildingNodes.push(new districtNodeImpl(false,null,building,null,this.doorNodes));
}
})
await this.dataSource.push(new districtNodeImpl(false,district,null,null,this.buildingNodes));
console.log(this.dataSource);
this.buildingNodes = new Array();
this.doorNodes = new Array();
}
})
Hope this will help ! have a nice day.
Related
I have below code in javascript in which some asynchronous task is being performed:
async function fetchData(id){
for(let i=1;;++i){
res = await fetch(`https://some-api/v1/products/${id}/data?page=${i}`,{//headers here});
res = await res.json();
if(res.length==0) break;
else{ //do some work here and continue for next iteration}
}
}
async function callApi(){
var arr = [//list of id's here to pass to api one by one, almost 100 id's here];
await Promise.all(arr.map(async(e)=>{
await fetchData(e);
}));
}
callApi();
The above code looks fine to me, except that it doesn't work as expected. Ideally, what should happen is that unless one id's call is not completed( unless break condition not satisfies for one id), the for loop should not proceed to next iteration. Rather, I am getting totally different results. The api calls are happening in random order because the loop is not waiting the iteration to complete. My hard requirement is that unless one iteration is not complete, it should not move to next one.
await seems to have no effect here. Please guide me how can I achieve this. I am running out of ideas.
Thank You!
Your arr.map(...) is not awaiting the different fetchData calls before the next map call, so I'd turn this into a specific for loop to be sure it waits:
async function callApi(){
const arr = [...];
for(let i = 0; i < arr.length; i++){
await fetchData(arr[i]);
}
}
or alternatively use a for of
async function callApi(){
const arr = [...];
for(let a of arr){
await fetchData(a);
}
}
The fetchData function also looks like it could use some improvements with error handling, but since you shortened your code quite a bit, I'm assuming there is something like that going on there, too, and your issue is actually with the callApi() code instead, as the fetch and await looks good to me there.
You should decide either to use promises or async await. Don't mix them.
With promises you can always use funky abstractions but with a simple recursive approach you can do like
function fetchData(hds, id, page = 1, pages = []){
return fetch(`https://some-api/v1/products/${id}/data?page=${page}`,hds)
.then(r => r.ok ? r.json() : Promise.reject({status:r.status,pages})
.then(j => fetchData(hds, id, ++page, pages.push(doSomethingWith(j))))
.catch(e => (console.log(e.status), e.pages));
}
So we use recursion to fetch indefinitelly until the API says enough and r.ok is false.
At the callApi side you can use reduce since we have an ids array.
const ids = [/* ids array */],
hds = { /* headers object */ };
function callApi(ids){
return ids.reduce( (p,id) => p.then(_ => fetchData(hds,id))
.then(pages => process(pages))
, Promise.resolve(null)
)
.catch(e => console.log(e));
}
So now both accesses to the id and page data are working asynchronously but only fired once the previous one finishes. Such as
(id=1,page=1) then (id=1,page=2) then (id=1,page=3) then (process 3 pages of id=1) then
(id=2,page=1) then (id=2,page=2) then (process 2 pages of id=2) etc...
While I love the promises, you can also implement the same functionality with the asyc await abstraction. I believe the idea behind the invention of the async await is to mimic sync imperative code. But keep in mind that it's an abstraction over an abstraction and I urge you to learn promises by heart before even attemting to use async await. The general rule is to never mix both in the same code.
Accordingly the above code could have been written as follows by using async await.
async function fetchData(hds, id){
let page = 1,
pages = [],
res;
while(true){
res = await fetch(`https://some-api/v1/products/${id}/data?page=${page++}`,hds);
if (res.ok) pages.push(await res.json())
else return pages;
}
}
Then the callApi function can be implemented in a similar fashion
const ids = [/* ids array */],
hds = { /* headers object */ };
async function callApi(ids){
let pages;
for(let i = 0; i < ids.length; i++){
try {
pages = await fetchData(hds,ids[i]);
await process(pages); // no need for await if the process function is sync
}
catch(e){
console.log(e);
}
}
}
I'm playing with the Rick and Morty API and I want to get all of the universe's characters
into an array so I don't have to make more API calls to work the rest of my code.
The endpoint https://rickandmortyapi.com/api/character/ returns the results in pages, so
I have to use recursion to get all the data in one API call.
I can get it to spit out results into HTML but I can't seem to get a complete array of JSON objects.
I'm using some ideas from
Axios recursion for paginating an api with a cursor
I translated the concept for my problem, and I have it posted on my Codepen
This is the code:
async function populatePeople(info, universePeople){ // Retrieve the data from the API
let allPeople = []
let check = ''
try {
return await axios.get(info)
.then((res)=>{
// here the current page results is in res.data.results
for (let i=0; i < res.data.results.length; i++){
item.textContent = JSON.stringify(res.data.results[i])
allPeople.push(res.data.results[i])
}
if (res.data.info.next){
check = res.data.info.next
return allPeople.push(populatePeople(res.data.info.next, allPeople))
}
})
} catch (error) {
console.log(`Error: ${error}`)
} finally {
return allPeople
}
}
populatePeople(allCharacters)
.then(data => console.log(`Final data length: ${data.length}`))
Some sharp eyes and brains would be helpful.
It's probably something really simple and I'm just missing it.
The following line has problems:
return allPeople.push(populatePeople(res.data.info.next, allPeople))
Here you push a promise object into allPeople, and as .push() returns a number, you are returning a number, not allPeople.
Using a for loop to push individual items from one array to another is really a verbose way of copying an array. The loop is only needed for the HTML part.
Also, you are mixing .then() with await, which is making things complex. Just use await only. When using await, there is no need for recursion any more. Just replace the if with a loop:
while (info) {
....
info = res.data.info.next;
}
You never assign anything to universePeople. You can drop this parameter.
Instead of the plain for loop, you can use the for...of syntax.
As from res you only use the data property, use a variable for that property only.
So taking all that together, you get this:
async function populatePeople(info) {
let allPeople = [];
try {
while (info) {
let {data} = await axios.get(info);
for (let content of data.results) {
const item = document.createElement('li');
item.textContent = JSON.stringify(content);
denizens.append(item);
}
allPeople.push(...data.results);
info = data.info.next;
}
} catch (error) {
console.log(`Error: ${error}`)
} finally {
section.append(denizens);
return allPeople;
}
}
Here is working example for recursive function
async function getAllCharectersRecursively(URL,results){
try{
const {data} = await axios.get(URL);
// concat current page results
results =results.concat(data.results)
if(data.info.next){
// if there is next page call recursively
return await getAllCharectersRecursively(data.info.next,results)
}
else{
// at last page there is no next page so return collected results
return results
}
}
catch(e){
console.log(e)
}
}
async function main(){
let results = await getAllCharectersRecursively("https://rickandmortyapi.com/api/character/",[])
console.log(results.length)
}
main()
I hesitate to offer another answer because Trincot's analysis and answer is spot-on.
But I think a recursive answer here can be quite elegant. And as the question was tagged with "recursion", it seems worth presenting.
const populatePeople = async (url) => {
const {info: {next}, results} = await axios .get (url)
return [...results, ...(next ? await populatePeople (next) : [])]
}
populatePeople ('https://rickandmortyapi.com/api/character/')
// or wrap in an `async` main, or wait for global async...
.then (people => console .log (people .map (p => p .name)))
.catch (console .warn)
.as-console-wrapper {max-height: 100% !important; top: 0}
<script>/* dummy */ const axios = {get: (url) => fetch (url) .then (r => r .json ())} </script>
This is only concerned with fetching the data. Adding it to your DOM should be a separate step, and it shouldn't be difficult.
Update: Explanation
A comment indicated that this is hard to parse. There are two things that I imagine might be tricky here:
First is the object destructuring in {info: {next}, results} = <...>. This is just a nice way to avoid using intermediate variables to calculate the ones we actually want to use.
The second is the spread syntax in return [...results, ...<more>]. This is a simpler way to build an array than using .concat or .push. (There's a similar feature for objects.)
Here's another version doing the same thing, but with some intermediate variables and an array concatenation instead. It does the same thing:
const populatePeople = async (url) => {
const response = await axios .get (url)
const next = response .info && response .info .next
const results = response .results || []
const subsequents = next ? await populatePeople (next) : []
return results .concat (subsequents)
}
I prefer the original version. But perhaps you would find this one more clear.
First of all, there are some issues with console.log in Google Chrome not functioning as expected. This is not the case as I am working in VSCode.
We begin with two async calls to the server.
promise_a = fetch(url)
promise_b = fetch(url)
Since fetch results are also promises, .json() will needed to be called on each item. The helper function process will be used, as suggested by a Stackoverflow user -- sorry lost the link.
let promiseResults = []
let process = prom => {
prom.then(data => {
promiseResults.push(data);
});
};
Promise.all is called. The resulting array is passed to .then where forEach calls process on item.json() each iteration and fulfilled promises are pushed to promiseResults.
Promise.all([promise_a, promise_b])
.then(responseArr => {
responseArr.forEach(item => {
process(item.json());
});
})
No argument is given to the final .then block because promiseResults are in the outer scope. console.log show confusing results.
.then(() => {
console.log(promiseResults); // correct results
console.log(promiseResults[0]); // undefined ?!?
})
Any help will be greatly appreciated.
If you are familiar with async/await syntax, I would suggest you not to use an external variable promiseResults, but return the results on the fly with this function:
async function getJsonResults(promisesArr) {
// Get fetch promises response
const results = await Promise.all(promisesArr);
// Get JSON from each response promise
const jsonResults = await Promise.all(results.map(r => r.json()));
return jsonResults
}
This is usage example:
promise_a = fetch(url1)
promise_b = fetch(url2)
getJsonResults([promise_a, promise_b])
.then(theResults => console.log('All results:', theResults))
Use theResults variable to extract necessary results.
You can try this, it looks the array loop is not going properly in the promise env.
Specifically: the promiseResults is filled after you are logging.
var resultAll = Promise.all([promise_a, promise_b])
.then(responseArr => {
return Promise.all(responseArr.map(item => return item.json()));
});
resultAll.then(promiseResults => {
console.log(promiseResults);
});
sup guys, i'm working on this firebase project and i need to iterate trought a subcollection of all sales of all stores in the root collection and sum their values... the problem that i'm getting is that i'm getting the sum printed before the iteration. I'm new to TS and Firebase... this is what i got so far:
export const newBilling = functions.firestore.document('billings/{billId}').onCreate(event =>
{
const valueArray = []
const feeArray = []
const storesCollection = afs.collection('stores').where('active', '==', true).get().then(stores => {
stores.forEach(store => {
const salesCollection = afs.collection('stores').doc(store.id).collection('sales').get().then(sales => {
sales.forEach(sale => {
return valueArray.push(sale.data().value) + feeArray.push(sale.data().fee)
// other aproach
// valueArray.push(sale.data().value)
// feeArray.push(sale.data().fee)
})
})
})
}).catch(error => {console.log(error)})
let cashbackSum, feeSum : number
cashbackArray.forEach(value => {
cashbackSum += value
})
feeArray.forEach(value => {
feeSum += value
})
console.log(cashbackSum, feeSum)
return 0
})
TKS =)
You're not using promises correctly. You've got a lot of get() method call, each of which are asynchronous and return a promise, but you're never using them to case the entire function to wait for all the work to complete. Calling then() doesn't actually make your code wait - it just runs the next bit of code and returns another promise. Your final console.log is executing first because none of the work you kicked off ahead of it is complete yet.
Your code actually needs to be substantially different in order to work correctly, and you need to return a promise from the entire function that resolves only after all the work is complete.
You can learn better how to use promises in Cloud Functions by going through the video tutorials.
I am working on a project where I am building a simple front end in Angular (typescript) / Node to make call to a back end server for executing different tasks. These tasks take time to execute and thus need to be queued on the back end server. I solved this issue by following the following tutorial: https://github.com/realpython/flask-by-example and everything seems to work just fine.
Now I am finishing things up on the front end, where most of the code has been already written in Typescript using Angular and Rxjs. I am trying to replicate the following code in Typescript:
https://github.com/dimoreira/word-frequency/blob/master/static/main.js
This code consists of two functions, where first function "getModelSummary"(getResults in the example) calls a post method via:
public getModelSummary(modelSummaryParameters: ModelSummaryParameters): Observable<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters)
.map(res => res.json())
;
}
to put the job in queue and assign a jobID to that function on the back end server. The second function "listenModelSummary", ideally should get executed right after the first function with the jobId as it's input and loops in a short interval checking if the job has been completed or not:
public listenModelSummary(jobID: string) {
return this.http.get(`${SERVER_URL}results/` + jobID).map(
(res) => res.json()
);
}
Once the job is done, it needs to return the results, which would update the front end.
I am new to Typescript, Observables and rxjs and wanted to ask for the right way of doing this. I do not want to use javascript, but want to stick to Typescript as much as possible in my front end code stack. How can I use the first function to call the second function with it's output "jobID" and have the second function run via interval until the output comes back?
Observables are great, and are the type of object returned by Angular's HttpClient class, but sometimes, in my opinion, dealing with them is a lot more complicated than using promises.
Yes, there is a slight performance hit for the extra operation to convert the Observable to a Promise, but you get a simpler programming model.
If you need to wait for the first function to complete, and then hand the returned value to another function, you can do:
async getModelSummary(modelSummaryParameters: ModelSummaryParameters): Promise<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters).toPromise();
}
async doStuff(): Promise<void> {
const modelSummary = await this.getModelSummary(params);
// not sure if you need to assign this to your viewmodel,
// what's returned, etc
this.listenModelSummary(modelSummary)
}
If you're dead-set on using Observables, I would suggest using the concatMap pattern, which would go something like this:
doStuff(modelSummaryParameters: ModelSummaryParameters): Observable<ModelSummary> {
return this.http
.post(`${SERVER_URL}start`, modelSummaryParameters)
.pipe(
concatMap(modelSummary => <Observable<ModelSummary>> this.listenModelSummary(modelSummary))
);
}
Here's an article on different mapping solutions for Observables: https://blog.angularindepth.com/practical-rxjs-in-the-wild-requests-with-concatmap-vs-mergemap-vs-forkjoin-11e5b2efe293 that might help you out.
You can try the following:
getModelSummary(modelSummaryParameters: ModelSummaryParameters): Promise<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters).toPromise();
}
async someMethodInYourComponent() {
const modelSummary = await this.get(modelSummary(params);
this.listenModelSummary(modelSummary)
}
// OR
someMethodInYourComponent() {
this.get(modelSummary(params).then(() => {
this.listenModelSummary(modelSummary);
});
}
After doing more reading/researching into rxjs, I was able to make my code work and I wanted to thank you guys for the feedback and to post my code below.
In my services I created two observables:
First one is to fetch a jobId returned by queue server:
// API: GET / FUNCTION /:jobID
public getModelSummaryQueueId(modelSummaryParameters: ModelSummaryParameters): Observable<JobId>{
return this.http.post(${SERVER_URL}start, modelSummaryParameters).map(
(jobId) => jobId.json()
)
}
Use the jobId from first segment to fetch data:
// API: GET / FUNCTION /:results
public listenModelSummary(jobId: JobId): Observable <ModelSummary>{
return this.http.get(${SERVER_URL}results/+ jobId).map(
(res) => res.json()
)
}
Below is the component that works with the 2 services above:
`
this.subscription = this.developmentService.getModelSummaryQueueId(this.modelSummaryParameters)
.subscribe((jobId) => {
return this.developmentService.listenModelSummary(jobId)
// use switchMap to pull value from observable and check if it completes
.switchMap((modelSummary) =>
// if value has not changed then invoke observable again else return
modelSummary.toString() === 'Nay!'
? Observable.throw(console.log('...Processing Request...'))
// ? Observable.throw(this.modelSummary = modelSummary)
: Observable.of(modelSummary)
)
.retryWhen((attempts) => {
return Observable
// specify number of attempts
.range(1,20)
.zip(attempts, function(i) {
return(i);
})
.flatMap((res:any) => {
// res is a counter of how many attempts
console.log("number of attempts: ", res);
res = 'heartbeat - ' + res
this.getProgressBar(res);
// this.res = res;
// delay request
return Observable.of(res).delay(100)
})
})
// .subscribe(this.displayData);
// .subscribe(modelSummary => console.log(modelSummary));
.subscribe((modelSummary) => {
console.log("FINAL RESULT: ", modelSummary)
this.modelSummary = modelSummary;
this.getProgressBar('Done');
});
});
`