We Keep Coding

axios problem with sending all requests in react

I have an array of items in the local storage and i want to send them all at once to my API
problem is when I send them all the API tells me it got all the requests but this loop doesnt remove them from the local storage its like it didn't get a response or something
let local_items = localStorage.getItem("items")
local_items = JSON.parse(local_items)
loadingToast("Syncing items")
for (let i = 0; i < local_items.length; i++) {
axios.post('/items', {
item: local_items[i],
shop: '1',
employee: '1'
}, config).then((response) => {
if (response.status === 200) {
local_items.splice(i, 1)
//it counts down 2 or 3 times then gets stuck on a number
alert("Item synced" + local_items.length)
localStorage.setItem("items", JSON.stringify(local_items))
} else {
dismissToast()
errorToast("Error syncing items")
localStorage.setItem("items", JSON.stringify(local_items))
return
}
}).catch((error) => {
dismissToast()
errorToast("Error syncing items")
localStorage.setItem("items", JSON.stringify(local_items))
return
})
}
}
the alert shows up for all items but the countdown gets stuck.

local_items.splice(i, 1) is not a good way to remove the elements, it will only work for the first half elements and not work for the rest.
Try local_items.shift() instead.

Javascript API fetch loop using offset

I'm fetching data from an API that has a limit of 50 records per call. There are 10,000 records in total that I need to retrieve so looking at using pagination via the offset parameter to achieve this.
I'm not too familiar with the syntax I need to use but I would like to keep looping through the fetch until the number of records returned for a single call is < 50 to signal the final page.
I need my JSON responses from each call to be appended together, in addition to a final 'count' that gives me 10,000
let offset = 0
fetch(`https://api.opensea.io/api/v1/assets?collection=womenandweapons&format=json&offset=${offset}&limit=50&order_direction=desc`,
{
method: 'GET',
headers: {
'Accept': 'application/json'
}
})
.then(response => response.json())
.then(data => {
let len = Object.keys(data.assets).length
console.log(len)
console.log(data.assets)
})
offset += 50

I've had some luck doing something like this:
const main = async () => {
const url = new URL(AIRTABLE_URL);
let data = await fetcher(url, AIRTABLE_API_KEY);
storiesTotal = mapData(data.records);
while (data.offset) {
data = await fetcher(url, AIRTABLE_API_KEY, data.offset);
storiesTotal = [...storiesTotal, ...mapData(data.records)];
}
}
main();

Firebase realtime database halt after reaching 29k+ data

Our firebase real-time database reaches 29k+, after that our app keeps on loading and loading and not showing any more data. i used this script to populate data return from firebase:
firebase.database().ref('/registrant/').orderByChild('id').on("value", function(snapshot) {
console.log(snapshot.val());
});
any help will be much appreciated thanks a lot.

Along with Franks answer, you could also do a staggered approach where you use a shallow query to get the top-level elements and load them asynchronously.
To my knowledge it is only available to the rest API, but it gives you the root nodes to iterate through directly
import request from 'request';
request({ url: "https://[YOUR-APP-ID].firebaseio.com/path/to/data.json?shallow=true" }, (error, response, body) => {
const shallowData = JSON.parse(body);
console.log(shallowData);
});
Source:
https://firebase.google.com/docs/database/rest/retrieve-data#shallow
https://firebase.google.com/docs/reference/rest/database#section-param-shallow

If you're loading 29K nodes, is it possible that you're running low on memory in where it is that you're running this code?
If so, consider loading less data by for example by add a limit... clause to your query:
firebase.database().ref('/registrant/').orderByChild('id').limitToFirst(100).on("value", function(snapshot) {
...

Alright got it to work this is derived from Frank and DIGI's answer kudos to them what i did is this:
let start = 20;
let end = 30;
$scope.execute = function() {
start = start + 10;
end = end + 10;
var settings = {
"url": "https://lcqrregistry-default-rtdb.firebaseio.com/registrant.json?orderBy=\"age\"&startAt=" + start + "&endAt=" + end + "&print=pretty",
"method": "GET",
"timeout": 0,
};
$.ajax(settings).done(function(response) {
console.log(start,end)
$timeout(function() {
$scope.$apply(function() {
let returnArr = [];
angular.forEach(response, function(value, key) {
returnArr.push(value);
});
$scope.registereds = returnArr;
console.log($scope.registereds)
});
})
});
}
var settings = {
"url": "https://my-url/registrant.json?orderBy=\"age\"&startAt=" + 18 + "&endAt=" + 30 + "&print=pretty",
"method": "GET",
"timeout": 0,
};
$.ajax(settings).done(function(response) {
$timeout(function() {
$scope.$apply(function() {
let returnArr = [];
angular.forEach(response, function(value, key) {
returnArr.push(value);
});
$scope.registereds = returnArr;
console.log($scope.registereds)
});
})
});
I filter by age and just add the next button to populate the filtered data

GIthub Api get all commits per repo of user

I am trying to use Github API to get all commits of user. Github API doesn't explicitly provide this, so the only way is to loop through all the repos and find number of commits authored by the user for each repo.
The idea for getting this is
for each repo in repoList:
response = https://api.github.com/repos/{user}/{repo_name}/commits?per_page=100&page={page}
for each item in response:
if(item.commit.author===user)
totalCommmitsPerRepo += 1
update state as {repo : totalCommitsPerRepo}
(Max results per page = 100)
I am having problems when updating the page number. My code is given below
getCommitsInfo = async = () => {
let flag = 0
const repos = [...this.state.repos]
let totalCommits = 0
const requests = repos.map((repo) => {
do {
flag = 0;
axios.get("https://api.github.com/repos/" + this.state.user + '/' + repo + "/commits?per_page=100&page=" + page)
.then(res => {
totalCommits = 0
//CHECK IF RESULTS PER PAGE = MAXIMUM
if (res.data.length === 100) {
flag = 1
}
res.data.map(commit => {
let committer = commit.author.login.toLowerCase()
let user = this.state.user.toLowerCase()
if (user === committer) {
totalCommits += 1
}
})
let obj = {
name: repo,
commits: totalCommits
}
this.setState({
reposAndCommits: [...this.state.reposAndCommits, obj]
})
})
//UPDATE PAGE NUMBER
if (flag) {
page += 1
}
} while (flag);
})
await Promise.all(requests)
}
The do while loop doesn't seem to work as expected. I am not getting the page number updated. This is possibly because the do-while loop is asynchronous and I don't know how to make the loop wait until each response is fully complete.
Can someone please help regarding this, Thanks

How can I return different values from a function depending on code inside an Axios promise? NodeJS - a

I have a block of code that calls an Api and saves results if there are differences or not. I would like to return different values for DATA as layed out on the code. But this is obviously not working since Its returning undefined.
let compare = (term) => {
let DATA;
//declare empty array where we will push every thinkpad computer for sale.
let arrayToStore = [];
//declare page variable, that will be the amount of pages based on the primary results
let pages;
//this is the Initial get request to calculate amount of iterations depending on result quantities.
axios.get('https://api.mercadolibre.com/sites/MLA/search?q='+ term +'&condition=used&category=MLA1652&offset=' + 0)
.then(function (response) {
//begin calculation of pages
let amount = response.data.paging.primary_results;
//since we only care about the primary results, this is fine. Since there are 50 items per page, we divide
//amount by 50, and round it up, since the last page can contain less than 50 items
pages = Math.ceil(amount / 50);
//here we begin the for loop.
for(i = 0; i < pages; i++) {
// So for each page we will do an axios request in order to get results
//Since each page is 50 as offset, then i should be multiplied by 50.
axios.get('https://api.mercadolibre.com/sites/MLA/search?q='+ term +'&condition=used&category=MLA1652&offset=' + i * 50)
.then((response) => {
const cleanUp = response.data.results.map((result) => {
let image = result.thumbnail.replace("I.jpg", "O.jpg");
return importante = {
id: result.id,
title: result.title,
price: result.price,
link: result.permalink,
image: image,
state: result.address.state_name,
city: result.address.city_name
}
});
arrayToStore.push(cleanUp);
console.log(pages, i)
if (i === pages) {
let path = ('./compare/yesterday-' + term +'.json');
if (fs.existsSync(path)) {
console.log("Loop Finished. Reading data from Yesterday")
fs.readFile('./compare/yesterday-' + term +'.json', (err, data) => {
if (err) throw err;
let rawDataFromYesterday = JSON.parse(data);
// test
//first convert both items to check to JSON strings in order to check them.
if(JSON.stringify(rawDataFromYesterday) !== JSON.stringify(arrayToStore)) {
//Then Check difference using id, otherwise it did not work. Using lodash to help.
let difference = _.differenceBy(arrayToStore[0], rawDataFromYesterday[0],'id');
fs.writeFileSync('./compare/New'+ term + '.json', JSON.stringify(difference));
//if they are different save the new file.
//Then send it via mail
console.log("different entries, wrote difference to JSON");
let newMail = mail(difference, term);
fs.writeFileSync('./compare/yesterday-' + term +'.json', JSON.stringify(arrayToStore));
DATA = {
content: difference,
message: "These were the differences, items could be new or deleted.",
info: "an email was sent, details are the following:"
}
return DATA;
} else {
console.log("no new entries, cleaning up JSON");
fs.writeFileSync('./compare/New'+ term + '.json', []);
DATA = {
content: null,
message: "There were no difference from last consultation",
info: "The file" + './compare/New'+ term + '.json' + ' was cleaned'
}
return DATA;
}
});
} else {
console.error("error");
console.log("file did not exist, writing new file");
fs.writeFileSync('./compare/yesterday-' + term +'.json', JSON.stringify(arrayToStore));
DATA = {
content: arrayToStore,
message: "There were no registries of the consultation",
info: "Writing new file to ' " + path + "'"
}
return DATA;
}
}
})
}
}).catch(err => console.log(err));
}
module.exports = compare
So I export this compare function, which I call on my app.js.
What I want is to make this compare function return the DATA object, so I can display the actual messages on the front end,
My hopes would be, putting this compare(term) function inside a route in app.js like so:
app.get("/api/compare/:term", (req, res) => {
let {term} = req.params
let data = compare(term);
res.send(data);
})
But as I said, Its returning undefined. I tried with async await, or returning the whole axios first axios call, but Im always returning undefined.
Thank you