We Keep Coding

Javscript Promises: How Combine .fetch() .then() and all()

I am trying to combine .fetch(), .then(). and .all(). But I can't get it right. My case looks like this:
fetch() a record from an API. (My Item)
then() Search in this record for more API links. (Here I find the links to the media that belong to the item. Each item can have any number of media).
all() Execute an API call for each of these links with .fetch()
then() if all previous promises are fulfilled, execute the render
function that depends on this data.
// Prepare Request Header
this.url_params = new URLSearchParams(window.location.search);
let httpHeader = {};
let request = new Request(api_itemURL + this.url_params.toString(), {
method: "GET",
headers: new Headers(httpHeader),
});
// The first fetch I have wrapped a seperate class. I still need it from other places.
omekaAPI
.getItemOnly(request)
.then(
function (result) {
this.itemObject = result;
let media_array = Array.from(result["o:media"], (x) => x["#id"]);
// Up to here it works wonderfully. I have list with all links. But after that it doesn't even jump into the all.
console.log(media_array);
return media_array;
}.bind(this)
)
.all(
media_array.map((media_array) =>
fetch(url).then(function () {
console.log("Its works!?!");
})
)
);
I don't know what I'm doing wrong and could use a push in the right direction.
Follow-Up Question
Today I have now tried to process the input. Unfortunately, I cheered too early. I only got back the underfilled Promise. With which my loop of course does not work. I then tried to append .then(). Both inside Promise.all() and after the .then() in which I assumed it. Unfortunately this has the same result. I thought then should go back a step and read about Promises in general. Is it possible that I have to do this with await/async? But then I have to write the complete function differently or? Also, my babel/uglify completely freaks out when I have one of these commands inside. It would be great if someone could give me a hint if I can get anywhere with my code here?
getItemWithAllMedia(request) {
return fetch(request)
.then((response) => {
return response.json();
})
.then((data) => {
console.log(data);
return data;
})
.then((result) => {
let media_links = Array.from(result["o:media"], (x) => x["#id"]);
return media_links;
})
.then((media_links) => {
return Promise.all(media_links.map((url) => fetch(url)));
})
.then((media_response) => {
/// debugging results from promise.all
console.log(media_response); // (2) [Response, Response]
console.log(media_reponse[0]); // ReferenceError: media_reponse is not defined
console.log(media_reponse[0].json()); // ReferenceError: media_reponse is not defined
/// desired function: all responses to json and push to an array
media_response.forEach((element) => {
media_data.push(element.json());
});
/// show created array for debugging
console.log(media_data);
return media_data;
})

React Hooks, setState not working with an async .map call

So, I'm trying to change a state in my component by getting a list of users to call an api.get to get data from these users and add on an new array with the following code:
function MembersList(props) {
const [membersList, setMembersList] = useState(props.members);
const [devs, setDevs] = useState([]);
useEffect(() => {
let arr = membersList.map((dev) => {
return dev.login;
});
handleDevs(arr);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [membersList]);
function handleDevs(membersArr) {
membersArr.map(async (dev) => {
let { data } = await api.get(`/users/${dev}`);
/** If i console.log data here i actualy get the data from the user
but i think theres something wrong with that setDevs([...devs, data]) call
**/
setDevs([...devs, data]);
});
}
but the devs state always return an empty array, what can I do to get it to have the actual users data on it?

The issue you were having is because you were setting devs based on the data from the original render every time due to the closure around handleDevs function. I believe this should help take care of the issues you were having by using the callback method of using setDevs.
This also takes care of some issues with the dependency arrays and staleness in the useEffect hook. Typically using // eslint-disable-next-line react-hooks/exhaustive-deps should be your last resort.
function MembersList(props) {
// this isn't needed unless you are using it separately
const [membersList, setMembersList] = useState(props.members);
const [devs, setDevs] = useState([]);
useEffect(() => {
let arr = membersList.map((dev) => dev.login);
arr.forEach(async (dev) => {
let { data } = await api.get(`/users/${dev}`);
setDevs((devs) => [...devs, data]);
})
}, [membersList]);
}

You need to understand how React works behind scenes.
In short, it saves all the "sets" until it finishes the cycle and just after that actually update each state.
I think that why you do not see the current state updated.
For better understanding read this post: Medium article

The issue is that setDevs uses devs which is the version of devs when handleDevs is defined. Therefore setDevs will really only incorporate the data from the last time setDevs is called.
To fix this you can use the callback version of setDevs, like so:
setDevs(prevDevs => [...prevDevs, data])
Also since you are not trying to create a new array, using map is not semantically the best loop choice. Consider using a regular for loop or a forEach loop instead.

you call setDevs in the async execution loop. Here is the updated handleDevs function.
function handleDevs(membersArr) {
const arr = membersArr.map(async (dev) => {
let { data } = await api.get(`/users/${dev}`);
return data;
/** If i console.log data here i actualy get the data from the user
but i think theres something wrong with that setDevs([...devs, data]) call
**/
});
setDevs([...devs, ...arr]);
}

Typescript Observables http.post and http.get Handing Question

I am working on a project where I am building a simple front end in Angular (typescript) / Node to make call to a back end server for executing different tasks. These tasks take time to execute and thus need to be queued on the back end server. I solved this issue by following the following tutorial: https://github.com/realpython/flask-by-example and everything seems to work just fine.
Now I am finishing things up on the front end, where most of the code has been already written in Typescript using Angular and Rxjs. I am trying to replicate the following code in Typescript:
https://github.com/dimoreira/word-frequency/blob/master/static/main.js
This code consists of two functions, where first function "getModelSummary"(getResults in the example) calls a post method via:
public getModelSummary(modelSummaryParameters: ModelSummaryParameters): Observable<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters)
.map(res => res.json())
;
}
to put the job in queue and assign a jobID to that function on the back end server. The second function "listenModelSummary", ideally should get executed right after the first function with the jobId as it's input and loops in a short interval checking if the job has been completed or not:
public listenModelSummary(jobID: string) {
return this.http.get(`${SERVER_URL}results/` + jobID).map(
(res) => res.json()
);
}
Once the job is done, it needs to return the results, which would update the front end.
I am new to Typescript, Observables and rxjs and wanted to ask for the right way of doing this. I do not want to use javascript, but want to stick to Typescript as much as possible in my front end code stack. How can I use the first function to call the second function with it's output "jobID" and have the second function run via interval until the output comes back?

Observables are great, and are the type of object returned by Angular's HttpClient class, but sometimes, in my opinion, dealing with them is a lot more complicated than using promises.
Yes, there is a slight performance hit for the extra operation to convert the Observable to a Promise, but you get a simpler programming model.
If you need to wait for the first function to complete, and then hand the returned value to another function, you can do:
async getModelSummary(modelSummaryParameters: ModelSummaryParameters): Promise<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters).toPromise();
}
async doStuff(): Promise<void> {
const modelSummary = await this.getModelSummary(params);
// not sure if you need to assign this to your viewmodel,
// what's returned, etc
this.listenModelSummary(modelSummary)
}
If you're dead-set on using Observables, I would suggest using the concatMap pattern, which would go something like this:
doStuff(modelSummaryParameters: ModelSummaryParameters): Observable<ModelSummary> {
return this.http
.post(`${SERVER_URL}start`, modelSummaryParameters)
.pipe(
concatMap(modelSummary => <Observable<ModelSummary>> this.listenModelSummary(modelSummary))
);
}
Here's an article on different mapping solutions for Observables: https://blog.angularindepth.com/practical-rxjs-in-the-wild-requests-with-concatmap-vs-mergemap-vs-forkjoin-11e5b2efe293 that might help you out.

You can try the following:
getModelSummary(modelSummaryParameters: ModelSummaryParameters): Promise<ModelSummary> {
return this.http.post(`${SERVER_URL}start`, modelSummaryParameters).toPromise();
}
async someMethodInYourComponent() {
const modelSummary = await this.get(modelSummary(params);
this.listenModelSummary(modelSummary)
}
// OR
someMethodInYourComponent() {
this.get(modelSummary(params).then(() => {
this.listenModelSummary(modelSummary);
});
}

After doing more reading/researching into rxjs, I was able to make my code work and I wanted to thank you guys for the feedback and to post my code below.
In my services I created two observables:
First one is to fetch a jobId returned by queue server:
// API: GET / FUNCTION /:jobID
public getModelSummaryQueueId(modelSummaryParameters: ModelSummaryParameters): Observable<JobId>{
return this.http.post(${SERVER_URL}start, modelSummaryParameters).map(
(jobId) => jobId.json()
)
}
Use the jobId from first segment to fetch data:
// API: GET / FUNCTION /:results
public listenModelSummary(jobId: JobId): Observable <ModelSummary>{
return this.http.get(${SERVER_URL}results/+ jobId).map(
(res) => res.json()
)
}
Below is the component that works with the 2 services above:
`
this.subscription = this.developmentService.getModelSummaryQueueId(this.modelSummaryParameters)
.subscribe((jobId) => {
return this.developmentService.listenModelSummary(jobId)
// use switchMap to pull value from observable and check if it completes
.switchMap((modelSummary) =>
// if value has not changed then invoke observable again else return
modelSummary.toString() === 'Nay!'
? Observable.throw(console.log('...Processing Request...'))
// ? Observable.throw(this.modelSummary = modelSummary)
: Observable.of(modelSummary)
)
.retryWhen((attempts) => {
return Observable
// specify number of attempts
.range(1,20)
.zip(attempts, function(i) {
return(i);
})
.flatMap((res:any) => {
// res is a counter of how many attempts
console.log("number of attempts: ", res);
res = 'heartbeat - ' + res
this.getProgressBar(res);
// this.res = res;
// delay request
return Observable.of(res).delay(100)
})
})
// .subscribe(this.displayData);
// .subscribe(modelSummary => console.log(modelSummary));
.subscribe((modelSummary) => {
console.log("FINAL RESULT: ", modelSummary)
this.modelSummary = modelSummary;
this.getProgressBar('Done');
});
});
`

Why the nodejs with amqplib consume function is closure?

I use the nodejs amqplib module to connect rabbitmq.
I found the consume function is become a closure function, but I couldn't understand why. I didn't use closure.
My code is below. I found the corr in the returnOK still get the first time value. When I fire this function second times. The corr still the value at first time.
I think that is odd. Someone could explain this?
const corr = new Date().getTime();
try {
const params = JSON.stringify(req.body);
console.log('corr first =', corr);
await ch.sendToQueue(q, Buffer.from(params), {
deliveryMode: true,
correlationId: corr.toString(),
replyTo: queue.queue,
});
const returnOK = (msg) => {
if (msg.properties.correlationId === corr.toString()) {
console.info('******* Proxy send message done *******');
res.status(HTTPStatus.OK).json('Done');
}
};
await ch.consume(queue.queue, returnOK, { noAck: true });
} catch (error) {
res.status(HTTPStatus.INTERNAL_SERVER_ERROR).json(error);
}

It appears you're calling ch.consume on every request, in effect creating a new consumer every time. You should only do that once.
What is happening is that the first consumer is picking up the messages.
To fix this, you probably want to move ch.consume outside the request handler.

Axios API call returning Promise object instead of result?

Before I start, let me say that I'm new to Javascript and very new to axios API calls, so I'm probably making a rookie mistake...
I have this function getObjects() that's meant to map over an array and return the data from an Axios API call. The API call and map function are both working, but I'm getting a Promise object instead of the data I'm trying to get.
I figure this is because the data is returned before there's enough time to actually get it, but not sure how to fix? I tried a .setTimeout(), but that didn't seem to work.
getObjects() {
let newsItems = this.state.arrayofids.map((index) => {
let resultOfIndex = axios.get(`https:\/\/hacker-news.firebaseio.com/v0/item/${index}.json`).then((res) => {
let data = res.data;
//console.log(data.by); // this prints the correct byline, but
// all the bylines are printed after
// the console.log below...
if (!data.hasOwnProperty('text')) return data;
}); /// end of axios request
return resultOfIndex;
}); /// end of map
/// ideally this would end in an array of response objects but instead i'm getting an array of promises...
console.log(newsItems);
}
(The extra escape characters are for my text editor's benefit.)
Here's a link to a codepen with the issue - open up the console to see the problem. It's a React project but I don't think any of the React stuff is the issue. EDIT: Codepen is link to working solution using axios.all as suggested below
Thanks!
EDIT: Here is my working solution.
getObjects() {
let axiosArr = [];
axios.all(this.state.arrayofids.map((id) => {
return axios.get(`https:\/\/hacker-news.firebaseio.com/v0/item/${id}.json`)
})).then((res) => {
for (let i = 0; i < this.state.arrayofids.length; i++) {
axiosArr.push(<li key={i} data-url={res[i].data.url} onClick={(e) => this.displayTheNews(e)}>{res[i].data.title}</li>);
}
if (axiosArr.length == this.state.arrayofids.length) {
this.setState({arrayofdata: axiosArr});
console.log('state is set!');
}
})
}

axios.all function should be more appropriate to your current scenario.

Your console.log is executing immediately, rather than waiting for the requests to finish, because they are not synchronous. You have to wait for all the responses before you console.log.
OPTION 1 (the hard way):
replace your console.log with
newsItems.forEach((promise, index) => {
promise.then((object)=>{
newsItems[index] = object
if (index+1 == newsItems.length) {
console.log(newsItems)
}
})
})
OPTION 2 (the better way):
using axios.all
getObjects() {
axios.all(this.state.arrayofids.map((id) => {
return axios.get(`https:\/\/hacker-news.firebaseio.com/v0/item/${id}.json`)
})).then((res) => {
console.log(res)
})
}
by the way, I would definitely reccommend changing
this.state.arrayofids.map((index) => {
let resultOfIndex = axios.get(`https:\/\/hacker-news.firebaseio.com/v0/item/${index}.json`)...
to be called id instead of index