We Keep Coding

How do I update my list after POST if im using a different endpoint?

I need help with updating my list after POST. I can't see any answers online. Usually what I will just do is push object into array but I think in my case is different.
This function uses 2 api endpoint. The first api will get the list data. The weather api will base from the first api endpoint data, iterate through the list and get the data of the city that matches the name.
async getPreviousWeather() {
let endpoint = "/api/";
let promises = [];
try {
const response1 = await axios.get(endpoint);
this.cities = response1.data;
for (let i = 0; i < this.cities.length; i++) {
const response2 = await axios.get(
`https://api.openweathermap.org/data/2.5/weather?q=${this.cities[i].city_name}&units=metric&appid={API_KEY}`
);
this.infos.push(response2.data);
}
} catch (error) {
console.log(error);
}
},
Now this endpoint post data from the first endpoint. The only problem that I have here is how to update the list on post. I don't know how to push it or i tried calling this.getPreviousWeather(); what happens is it adds the new data but also adds the previous ones.
async onSubmit() {
let endpoint = "/api/";
try {
const response = await axios.post(endpoint, {
city_name: this.city_query,
});
this.city_query = null;
this.getPreviousWeather();
} catch (error) {
console.log(error);
}
},
created() {
this.getPreviousWeather();
},

I created the answer I'm not sure if it is effective but it works.
methods: {
async onSubmit() {
let endpoint1 = `https://api.openweathermap.org/data/2.5/weather?q=${this.city_query}&units=metric&appid={API_KEY}`;
let endpoint2 = "/api/";
try {
const response1 = await axios.get(endpoint1);
const response2 = await axios.post(endpoint2, {
city_name: this.city_query,
});
this.city_query = null;
if (this.error) {
this.error = null;
}
this.infos.push(response1.data);
} catch (error) {
console.log(error);
}
},

How to recursively fetch data from paginated API then combine into one array

Below I tried to write a conditional that would fetch a paginated api and then map it to another api that is being fetched. One issue that is coming up is that it's not continue to loop after it pulls one paginated page or one next page. The second issue is that that the data being fetched from the pages aren't being combined into one array. What am I doing wrong or missing?
const fetchURL = `${baseURL}?owner=${accounts[0]}`;
fetch(fetchURL, {
method: 'GET',
redirect: 'follow',
})
.then(resp => resp.json())
.then(data => {
console.log(data);
const pageKey = data.pageKey
if (pageKey !== 0) {
fetch(`${baseURL}?owner=${accounts[0]}&pageKey=${pageKey}`, {
method: 'GET',
redirect: 'follow',
})
.then(resp => resp.json())
.then(data => {
console.log(data)
})
return data.ownedNfts.concat(data.ownedNfts)
} else {
return data
}
const responses = data.ownedNfts.map((ownedNfts) =>
fetch(`${baseURL1}stats?address=${ownedNfts.contract.address}`)
.then((res) => res.json()),
);

To manage pagination from api, you could try a recursive like this.
You should have a script with a request loop with increment params, and a threshold to break the loop. You have to manage the request delay from your api with a time sleep or something like this.
This example bellow work in a node env with axios, you can try it and adapt it with your environnement.
const { default: axios } = require('axios');
// Init a bigData array to push new data on each iteration
const bigData = [];
async function fetchAllPaginateData(
pageKey = 0 /** init by default page index 0 */,
) {
try {
const fetchURL = `https://api.instantwebtools.net/v1/passenger?page=${pageKey}&size=1`;
const response = await axios.get(fetchURL);
const { data } = response;
const { totalPages } = data; // Your api should give you a total page count, result or something to setup your iteration
bigData.push(data); // push on big data response data
// if current page isn't the last, call the fetch feature again, with page + 1
if (
pageKey < totalPages &&
pageKey < 10 // (this is a test dev condition to limit for 10 result) */
) {
pageKey++;
await new Promise((resolve) => setTimeout(resolve, 200)); // setup a sleep depend your api request/second requirement.
console.debug(pageKey, '/', totalPages);
return await fetchAllPaginateData(pageKey);
}
console.clear();
return console.info('Data complete.');
} catch (err) {
console.error(err);
}
}
fetchAllPaginateData().then(() => console.table(bigData));

I modified the previous answer slightly to be self contained in one method instead of modifying a payload in the larger scope.
const axios = require('axios');
// These outer brackets are "simulating" global variables
//// `API_URL` would probably be configured globally on your request module
//// `LIMIT` might be set based on the current environment
((API_URL, LIMIT = 10) => {
// This method could take any structure of parameters
// Placing them in an object may actually be better at this point
// Defaults to an empty `combineData` array
const fetchAllPaginateData = async (accountId, reqSize = 20, currentPage = 0, combineData = []) => {
try {
const fetchURL = `${API_URL}/passenger?owner=${accountId}&size=${reqSize}&page=${currentPage}`;
const response = await axios.get(fetchURL);
const { totalPages, totalPassengers, data } = response;
console.debug(`${totalPassengers} Passengers`, '/', `Needing ${totalPages} pages`, '/', `${reqSize} passengers per page`);
// Here's the secret sauce,
// Combine data on each recursion and return it at the end
combineData = combineData.concat(data);
if (currentPage < totalPages && currentPage < LIMIT) {
currentPage++;
console.debug(currentPage, 'of', totalPages, 'pages');
// Slow down requests so as not to DOS the API
await new Promise((resolve) => setTimeout(resolve, 200));
return await fetchAllPaginateData(reqSize, currentPage, combineData);
}
// Send big data back
return combineData;
} catch (err) {
console.error(err);
}
}
// Presumably this would be called from other methods or files
// providing the owners account number for the request query
fetchAllPaginateData(1007).then(data => console.table(data));
})('https://api.instantwebtools.net/v1', 2);

How to store API response data for analysis in javascript

I am attempting to build a bot that will periodically poll an API using axios for the price of multiple cryptocurrencies across multiple exchanges. I need to be able to then look at this stored data to analyse price differences of a given token between multiple exchanges to calculate if there is profit to be made if I were to purchase it on one and then sell on another.
So far I am able to get access to the price data and console.log it as the request is returned, however I need to add this data to an array so that it can be analysed at a later point. I understand that I will need to use promises to handle this but I find it slightly confusing.
The following code immediately outputs an empty array and then outputs all the individual prices. How can I gain access to those values at a later point in the code?
const axios = require('axios');
const { exchanges } = require('./resources/exchanges.json');
const { currencies } = require('./resources/currencies.json');
const buyCurrency = currencies.buy[0];
const poll = async () => {
const data = new Array();
await currencies.sell.forEach(async sellCurrency => {
await exchanges.forEach(async exchange => {
try {
const allOtherExchanges = exchanges.filter(x => x !== exchange).map(x => `,${x}`).join();
const response = await axios.get(`https://api.0x.org/swap/v1/quote?buyToken=${buyCurrency}&sellToken=${sellCurrency}&sellAmount=1000000000000000000&excludedSources=0x${allOtherExchanges}`)
if (response && response.data.price) {
console.log(exchange, sellCurrency, response.data.price)
data.push({
exchange,
price: response.data.price
});
}
} catch {}
});
});
console.log(data);
};
poll()

One of the solutions would be as follows:
const axios = require('axios');
const { exchanges } = require('./resources/exchanges.json');
const { currencies } = require('./resources/currencies.json');
const buyCurrency = currencies.buy[0];
const poll = async () => {
const data = new Array();
const promises = [];
currencies.sell.forEach(sellCurrency => {
exchanges.forEach(exchange => {
const allOtherExchanges = exchanges.filter(x => x !== exchange).map(x => `,${x}`).join();
promises.push(
axios.get(`https://api.0x.org/swap/v1/quote?buyToken=${buyCurrency}&sellToken=${sellCurrency}&sellAmount=1000000000000000000&excludedSources=0x${allOtherExchanges}`)
.then(response => {
if (response && response.data &&response.data.price) {
console.log(exchange, sellCurrency, response.data.price)
data.push({
exchange,
price: response.data.price
});
}
}).catch(err => console.error(err))
);
});
});
await Promise.all(promises);
console.log(data);
};
poll();

Save fetched JSON data to sessionStorage

I just figured out how to write an async/await function to fetch data from an API, and it's working, but it's hitting the API like crazy. So now I'm trying to save the fetched data to sessionStorage and only fetch from the API if the data isn't in the sessionStorage.
Here's my code:
const fetchMeetingData = async () => {
console.log('api hit')
try {
const response = await fetch(https://sheet.best...)
const data = await response.json()
validate(data) // Clean data to remove null key values
return data
} catch (e) {
console.log('Fetch error with getMeetingData()')
}
}
const filterMeetings = async (filters) => {
meetings = await fetchMeetingData()
meetings.forEach((meeting) => {
meeting.time2 = moment(meeting.time, ["h:mm A"]).format("HHmm")
})
let today = moment().format("dddd").toString()
let hour = moment().format('HHmm').toString()
let filteredMeetings = meetings.filter(function (matches) {
if (document.querySelector('#select-day').selectedIndex === 0 && filters.searchText === '') {
return matches.day === today &&
moment(matches.time, ["h:mm A"]).format("HHmm") > hour
} else {
return true
}
})
Here's what I've tried:
const fetchMeetingData = async () => {
console.log('api hit')
try {
const response = await fetch(https://sheet.best...)
const data = await response.json()
validate(data) // Clean data to remove null key values
sessionStorage.setItem('meetingData', JSON.stringify(data)) // added this line
return data
} catch (e) {
console.log('Whoa! Fetch error with getMeetingData()')
}
}
I'm not really sure where to go from here, or if this is even the correct approach. My noob instinct was to do something like this, which didn't work.
savedMeetingData = sessionStorage.getItem('meetingData')
const getSavedMeetingData = async () => {
if (savedMeetingData) {
meetings = savedMeetingData
return meetings
} else {
fetchMeetingData()
meetings = await data
return meetings
}
const filterMeetings = async (filters) => {
meetings = await getSavedMeetingData() // replaces call to fetchMeetingData
meetings.forEach((meeting) => {
meeting.time2 = moment(meeting.time, ["h:mm A"]).format("HHmm")
})
I'm not sure if that's exactly the code I was trying but it's close. The problem was the API was still getting hit, even though the data was stored successfully to sessionStorage.
I'd really appreciate some help and/or suggestions on how to clarify this question.
SOLUTION:
Based on answer from #Christian
// StackOverflow Q/A
async function getMeetingData() {
const preLoadedData = sessionStorage.getItem('meetingData')
if(!preLoadedData) {
try {
const response = await fetch('https://sheet.best...')
const data = await response.json()
validate(data)
sessionStorage.setItem('meetingData', JSON.stringify(data))
console.log('api hit')
return data
} catch (e) {
console.log('Whoa! Fetch error with getMeetingData()')
}
} else {
console.log('no api hit!!!')
return JSON.parse(preLoadedData)
}
}
async function getSavedMeetingData() {
const meetings = await getMeetingData()
return meetings
}
const filterMeetings = async (filters) => {
meetings = await getSavedMeetingData()
meetings.forEach((meeting) => {
meeting.time2 = moment(meeting.time, ["h:mm A"]).format("HHmm")
})

If you could be more explicit on what exactly did not work it would be great :) (did not save data in sessionStorage?, could not retrieve it?, etc...). Anyway, maybe you could try something like this and see if it helps:
async function getSavedMeetingData() {
const meetingData = await getMeetingData();
}
async function getMeetingData() {
const preloadedData = sessionStorage.getItem('meetingData');
if (!preloadedData) {
try {
const response = await fetch('https://myapiurl.com/');
const data = validate(response.json());
sessionStorage.setItem('meetingData', JSON.stringify(data));
return data;
} catch (e) {
console.log('Whoa! Fetch error with getMeetingData()');
}
} else {
return JSON.parse(preloadedData);
}
}
One more reminder (just in case), keep in mind you are saving this to sessionStorage, so if you close the tab do not expect to have the information saved, in that case you should use localStorage.

"Exit promise" with multiples fetch requests

I need to merge data from API. I do a first call to an endpoint that gives me a list of ids, then I do a request for each id. My goal is to return a list with the responses of all requests but I lost myself in promises ...
My code runs on NodeJS. Here is the code :
const fetch = require('node-fetch')
const main = (req, res) => {
fetch('ENDPOINT_THAT_GIVES_LIST_OF_IDS')
.then(response => response.json())
.then(response => {
parseIds(response)
.then(data => {
console.log(data)
res.json(data)
// I want data contains the list of responses
})
})
.catch(error => console.error(error))
}
const getAdditionalInformations = async function(id) {
let response = await fetch('CUSTOM_URL&q='+id, {
method: 'GET',
});
response = await response.json();
return response
}
const parseIds = (async raw_ids=> {
let ids= []
raw_ids.forEach(function(raw_id) {
let informations = {
// Object with data from the first request
}
let additionalInformations = await
getAdditionalInformations(raw_id['id'])
let merged = {...informations, ...additionalInformations}
ids.push(merged)
})
return ids
})
main()
I get this error : "await is only valid in async function" for this line :
let additionalInformations = await getAdditionalInformations(raw_id['id'])
Help me with promise and async/await please.

You're almost there, just a slight bit of error here with your parentheses:
// notice the parentheses'
const parseIds = async (raw_ids) => {
let ids= []
raw_ids.forEach(function(raw_id) {
let informations = {
// Object with data from the first request
}
let additionalInformations = await getAdditionalInformations(raw_id['id'])
let merged = {...informations, ...additionalInformations}
ids.push(merged)
})
return ids
}

You are missing an async after forEach
const parseIds = (async raw_ids=> {
let ids= []
raw_ids.forEach(async function(raw_id) {
let informations = {
// Object with data from the first request
}
let additionalInformations = await
getAdditionalInformations(raw_id['id'])
let merged = {...informations, ...additionalInformations}
ids.push(merged)
})
return ids
})
One suggestion: you are mixing promises (.then()) with async/await. Prefer async/await is more readable.
Note that getAdditionalInformations inside forEach doesn't wait for it to be done before going to the next entry of the array.
You can use plain old for(var i=0; .... instead