We Keep Coding

javascript: Update the DOM only when the result is ready

I have some api endpoint.
one returns all server details (https://dnscheck.io/api/serverDetails/)
others are server specific endpoint. (https://dnscheck.io/api/query/?id=2&type=A&hostname=test.com) for each server_Id(which I got from serverDetails endpoint), I have to call each api endpoint.
what I have done is.
I loop over the results array (which I got from serverDetails endpoint)
and for each iteration of loop, I call each endpoint for getting the ip.
loop:
for (const [index, item] of data.entries()) {
const res = await fetch(
`https://dnscheck.io/api/query/?id=${item.id}&type=${query.type}&hostname=${query.host}`
);
const result = await res.json();
renderResult(result, item, index);
}
render-function:
const renderResult = (result, data, index) => {
const ip = document.querySelector(`.ip-address${index + 1}`);
ip.innerHTML = result.answers[0].address;
};
In this way, results are displayed in the DOM in a sync way. (one after another)
But, what I want is, update the dom with the result, as soon as the result is ready.
what can I do?

Don't use await, as that blocks the for loop and orders the results. Use .then() instead.
for (const [index, item] of data.entries()) {
fetch(
`https://dnscheck.io/api/query/?id=${item.id}&type=${query.type}&hostname=${query.host}`
).then(res => res.json())
.then(result => renderResult(result, item, index));
}

You can do them in parallel by using map on the array and using fetch within. You can know when they've all finished by using Promise.all to observe the overall result:
await Promise.all(
data.entries().map(async (index, item) => {
const res = await fetch(
`https://dnscheck.io/api/query/?id=${item.id}&type=${query.type}&hostname=${query.host}`
);
// You need to check `res.ok` here
const result = await res.json();
renderResult(result, item, index);
)
);
Note that Promise.all will reject its promise immediately if any of the input promises rejects. If you want to know what succeeded and what failed, use allSettled instead:
const results = await Promise.allSettled(
data.entries().map(async (index, item) => {
const res = await fetch(
`https://dnscheck.io/api/query/?id=${item.id}&type=${query.type}&hostname=${query.host}`
);
// You need to check `res.ok` here
const result = await res.json();
renderResult(result, item, index);
)
);
// Use `results` here, it's an array of objects, each of which is either:
// {status: "fulfilled", value: <the fulfillment value>}
// or
// {status: "rejected", reason: <the rejection reason>}
About my "You need to check res.ok here" note: this is unfortunately a footgun in the fetch API. It only rejects its promise on network failure, not HTTP errors. So a 404 results in a fulfilled promise. I write about it here. Typically the best thing is to have wrapper functions you call, for instance:
function fetchJSON(...args) {
return fetch(...args)
.then(response => {
if (!response.ok) {
throw new Error(`HTTP error ${response.status}`); // Or an error subclass
}
return response.json();
});
}

Problems with the fetch api on nodejs [duplicate]

This question already has answers here:
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 2 years ago.
I am creating an array of objects from an endpoint with the fetch api. Each object in the response contains a new url and I scan this array for new objects. The purpose is to access the endpoint, obtain the urls and access these urls to store your objects and attributes. But when accessing the array with objects and from an index it returns undefined.
let url = [];
let pokemon = [];
function getPokemons(){
fetch("https://pokeapi.co/api/v2/type/11/")
.then(async response =>{
await response.json().then(data =>
data.pokemon.forEach(item =>
url.push(item.pokemon.url)
)
)
}).then( response => {
url.map(item =>
fetch(item).then(response =>{
response.json().then(data =>
pokemon.push(data)
)
})
)
})
}
getPokemons()
console.log(pokemon[1])

Solution for web browser :
The node-fetch module is not required, I just clean your code using async/await syntax.
// getPokemon is now an async function for cleaner syntax
async function getPokemon() {
const pokemon = [];
// Then we can use await
const response = await fetch("https://pokeapi.co/api/v2/type/11/"),
json = await response.json(),
urls = json.pokemon.map(item => item.pokemon.url);
for (const url of urls) {
const response = await fetch(url),
json = await response.json();
pokemon.push(json)
}
// We return the pokemon array instead of polluting global scope
return pokemon;
};
getPokemon().then(pokemon => {
console.log(pokemon)
});
Hope it help !
NodeJS solution
You must install node-fetch (see npmjs.org :
npm install -D node-fetch
Then you can use a fetch:
// Importing the node module.
// You can delete this line if your not using a node environment.
const fetch = require('node-fetch');
// getPokemon is now an async function for cleaner syntax
async function getPokemon() {
const pokemon = [];
// Then we can use await
const response = await fetch("https://pokeapi.co/api/v2/type/11/"),
json = await response.json(),
urls = json.pokemon.map(item => item.pokemon.url);
for (const url of urls) {
const response = await fetch(url),
json = await response.json();
pokemon.push(json)
}
// We return the pokemon array instead of polluting global scope
return pokemon;
};
getPokemon().then(pokemon => {
console.log(pokemon)
});
Explanation
fetch is not part of the Javascript language specification but is a Web API. Each browser may or not choose to implement it. Each Implementations may work differently under the hood but the Javascript API provided must match the standard (MDN Web Docs).
This is why you need a module to fetch the data.
EDIT : Adding solution for web browser environment

The problem is that the function getPokemons should be async.
You should await it before accessing:
getPokemons().then(()=> console.log(pokemon[1]))
// or if it inside an async function:
await getPokemons();
console.log(pokemon[1]);
But there's another reason also. You have internal promises outside the parent promise chain. I mean:
.then((response) => {
// this is array of promises
// you should return it to have ability await it on parent promise
url.map((item) =>
fetch(item).then((response) => {
response.json().then((data) => pokemon.push(data));
})
);
});
Your code might look like:
// if you really need global variable
let pokemon = [];
async function getPokemons() {
const result = await fetch("https://pokeapi.co/api/v2/type/11/")
.then((response) => response.json())
.then((data) =>
Promise.all(
data.pokemon.map((item) =>
fetch(item.pokemon.url).then((response) => response.json())
)
)
);
pokemon.push(...result);
}
getPokemons().then(() => {
console.log(pokemon[1]);
});
Or, the same result without global variables:
function getPokemons() {
return fetch("https://pokeapi.co/api/v2/type/11/")
.then(...)
.then(...);
}
getPokemons().then((pokemons) => {
console.log(pokemons[1]);
});

Returning array of results from Firebase query

I'm using the #react-native-firebase wrapper to interact with Firebase's Firestore. I have a function which performs some querying to one of my collections and the intended response should be an Array object containing each found document.
My function currently looks like:
export const backendCall = async => {
let response = []
firestore()
.collection('collection')
.where('id', '==', 1)
.onSnapshot(documentSnapshot => {
documentSnapshot.forEach(x => response.push(x.data()))
})
return response
On my UI, I use a react-native Button component which calls this function onPress:
<Button
title="get"
onPress={() => {
backendCall().then(response => console.log(response))
}}
/>
Using console.log I am able to observe the expected Array object (but with a "value below was evaluated just now" icon. If however, I change the onPress to console.log(JSON.stringify(response)), the Array object is empty.
I'm assuming this has something to do with the async calls but I can't quite figure out how to fix it. Would appreciate some pointers.

You're returning the response without waiting for the result from your firebase query. To wait for the response to arrive before returning, you can use Promise.
export const backendCall = () => {
return new Promise(resolve => {
firestore()
.collection('collection')
.where('id', '==', 1)
.onSnapshot(documentSnapshot => {
const response = []
documentSnapshot.forEach(x => response.push(x.data()))
resolve(response)
})
})
}
You can use Array.map to make the for loop looks nicer.
const response = documentSnapshot.map(x => x.data())
resolve(response)

You can also read docs from a QuerySnapshot:
export const backendCall = async () => {
const qs = firestore().collection('collection').where('id', '==', 1).get()
return qs.docs.map(x => x.data())
}

Swapi API pagination using (data.next) vanilla JS

I have this async function to get three separate requests from the swapi API to retrieve data. However, I'm only getting back the first page of data as it's paginated. I know I have to create a loop for data.next to make new requests but I'm unsure the best way to run it through my function.
(async function getData() {
//Utility Functions for fetch
const urls = ["https://swapi.co/api/planets/", "https://swapi.co/api/films/", "https://swapi.co/api/people/"];
const checkStatus = res => res.ok ? Promise.resolve(res) : Promise.reject(new Error(res.statusText));
const parseJSON = response => response.json();
//Get Data
await Promise.all(urls.map(url => fetch(url)
.then(checkStatus)
.then(parseJSON)
.catch(error => console.log("There was a problem!", error))))
.then(data => {
let planets = data[0].results,
films = data[1].results,
people = data[2].results;
buildData(films, planets, people);
});
})();

You are trying to access all the data.results keys in the loop, which misses the point of using Promise.all. Promise.all collects all the results from promises and stores it in a single array when all the promises are resolved.
So wait for the promises to resolve and use the array returned from Promise.all to build your data.
To get all the pages you need to have a recursive function. Which means that this function will keep calling itself until a condition is met. Sort of like a loop but with callbacks.
Every time you fetch a page check if the there is a next page by checking the next property in the response object. If there is call the getAllPages again until there are no more pages left. At the same time all the results are concatenated in a single array. That array is passed on to the next call which concatenates it again with the result. And at the end the collection variable, which contains all the concatenated arrays, is returned.
Let me know if you have any questions regarding the code.
(async function getData() {
//Utility Functions for fetch
const urls = ["https://swapi.co/api/planets/", "https://swapi.co/api/films/", "https://swapi.co/api/people/"];
const checkStatus = res => res.ok ? Promise.resolve(res) : Promise.reject(new Error(res.statusText));
const parseJSON = response => response.json();
// Get a single endpoint.
const getPage = url => fetch(url)
.then(checkStatus)
.then(parseJSON)
.catch(error => console.log("There was a problem!", error));
// Keep getting the pages until the next key is null.
const getAllPages = async (url, collection = []) => {
const { results, next } = await getPage(url);
collection = [...collection, ...results];
if (next !== null) {
return getAllPages(next, collection);
}
return collection;
}
// Select data out of all the pages gotten.
const [ planets, films, people ] = await Promise.all(urls.map(url => getAllPages(url)));
buildData(films, planets, people);
})();

Bluebird Promise Order issue

I am watching videos to learn MongoDB Express.js VueJS Node.js (MEVN) stack.
And I want to create a seed directory and also use promise functions
// const delay = require('delay')
const Promise = require('bluebird')
const songs = require('./songs.json')
const users = require('./users.json')
const bookmarks = require('./bookmarks.json')
const historys = require('./history.json')
sequelize.sync({ force: true })
.then( async function () {
await Promise.all(
users.map( user => {
User.create(user)
})
)
await Promise.all(
songs.map( song => {
Song.create(song)
})
)
//I have to add this line
// ---> await delay(1000)
await Promise.all(
bookmarks.map( bookmark => {
Bookmark.create(bookmark)
})
)
await Promise.all(
historys.map( history => {
History.create(history)
})
)
})
I have four tables with seeds to create, and the last two tables data must be created after the former two tables data. (They are foreign keys)
But every time I run this file, the last two tables data will be created first
The only way I can prevent this is to add delay(1000) between them.
I am wondering if there exists any efficient way to solve this issue~
Thank you.

Race conditions like this one is always caused by that promises weren't properly chained.
A promise should be returned from map callback:
await Promise.all(
users.map( user => User.create(user))
);
etc.
Not returning a value from map is virtually always a mistake. It can be prevented by using array-callback-return ESLint rule.
If User.create(user), etc. were Bluebird promises with default configuration, not chaining them would also result in this warning.

My assumption why your code might fail:
You're not returning the Promises that I guess /(User|Song|Bookmark|History).create/g return to the Promise.all() function, since your map callback is not returning anything.
If you're using Arrow functions with brackets, then you need to explicitly specify the return value (using the familar return keyword).
Otherwise you can just omit the curly brackets.
My suggestion is, that you refactor you're code by utilizing Promise .then()-Chaining.
For you're example, I would suggest something like this:
const Promise = require('bluebird')
const songs = require('./songs.json')
const users = require('./users.json')
const bookmarks = require('./bookmarks.json')
const histories = require('./history.json')
sequelize.sync({
force: true
}).then(() =>
Promise.all(
users.map(user =>
User.create(user)
)
).then(() =>
Promise.all(
songs.map(song =>
Song.create(song)
)
)
).then(() =>
Promise.all(
bookmarks.map(bookmark =>
Bookmark.create(bookmark)
)
)
).then(() =>
Promise.all(
histories.map(history =>
History.create(history)
)
)
)
);