I'm fetching data from a PS4 Games API but it's split into 400+ pages. I wanted to get the data from all pages, but the solution I came up with did not work very well. It gives me an error 'JSON Value of type NSNull cannot be converted to a valid URL'. Also, I don't think the for loop works well either, it shows me it going through all the pages when it displays the results in my list.
Additionally, this API is dynamic because new games keep getting released. So how could I get data up to latest page without manually changing my last page number everytime? I looked at some questions here but I couldn't fit it into my code
My code is rather long so I'm just going to post the part that matter:
componentDidMount() {
var i;
for (i = 0; i < 400; i++) {
fetch(`https://api.rawg.io/api/games?page=${i+1}&platforms=18`, {
"method": "GET",
"headers": {
"x-rapidapi-host": "rawg-video-games-database.p.rapidapi.com",
"x-rapidapi-key": "495a18eab9msh50938d62f12fc40p1a3b83jsnac8ffeb4469f"
}
})
.then(res => res.json())
.then(json => {
const { results: games } = json;
this.setState({ games });
//setting the data in the games state
});
}
}
The API also has an item that gives me the link of the next page, I think there is a way to use 'next' and fetch data from that URL
If anyone could help, that would be AWESOME. Thank you in advance
its my first answer on this forum, so... I hope be helpfull.
For me you have 2 options:
Each request send 2 variables in the answer count and next, or you make a for loop with count/20 as limit (20 is the number of items gaves in the answer), or you make a while loop until the next variable give null as answer (currently at page 249).
What is currently happening is you are making 400 requests and when each comes in it is overwriting the component with the response it received. It does not care about what is already there or any of the other requests.
An approach you could try instead is as the responses come in append the results to the ongoing list and update the state with the running list.
Going forward and for your other question about handling new releases. Instead of running 400 queries every time the application is used, try looking into caching the results. When the app loads you can see if cache exists and load or query if it does not. The rawg.io /games endpoint has a parameter for ordering by release. When the application loads in future you can conditionally loop until you reach a game that is already in cache at which point terminate.
Related
I have this fetch statement that returns 19 building names, but I only want 10; the following is what I attempted, but I still get 19 building names.
fetchBuildings(energyProgramId) {
fetch(`http://localhost:1001/api/energyprograms/${energyProgramId}/buildings/?results=10`)
.then(res => res.json())
.then(json => {
this.setState({
isLoaded: true,
buildings: json,
})
});
}
Is there something extra I need to add?
Here is an example:
1.fetch('http://jsonplaceholder.typicode.com/posts/')
The above URL gives array of objects with 100 elements because it originally is an array of 100 elements.
2.fetch('http://jsonplaceholder.typicode.com/posts/?_limit=10')
This URL gives array of objects with 10 elements.
Notice the difference?
I only did this : ?_limit=10 ----> Add any number in place of 10 and hopefully you will get desired results.
As the other answer already points out the best/most normal solution would be to change on the backend how the API returns data. Typically REST API's support query parameters such as limit and start or page and resultsPerPage.
If this is not available - e.g. when you're fetching an external resource - an alternative which is often supported by static file servers and sometimes by API's is the Range header which allows you to retrieve only a specific byte range of the resource (do note, in the case that an API supports this it will still load the entire resource on the server, but it will not transmit the entire resource). An example with fetch would look like
fetch('', { headers: { range: 'bytes=0-1000'} })
When doing this on XML or JSON resources it can be somewhat difficult to work with, but with for example CSV files it's ideal.
No different from fetch to XHR or axios or anything else. actually, no different from react or angular or vue or anything else.
This is an API that backend developers wrote it and it is based on REST API, so when you call it as GET or POST and anything else you just fetch the JSON that the backend developers designed it. BUT
There is a new technology that name is GraphQL. you can call API and then you just fetch the JSON what you want. Also, it must be implemented in backend but it is possible.
It's not closely bound up with React. If you need less data you must reduce data before set the state.
const reducedBuildings = [];
fetch(`http://localhost:1001/api/energyprograms/${energyProgramId}/buildings/?results=10`)
.then(res => res.json())
.then(json => {
json.forEach(building => {
if (reducedBuildings.length < 10) {
reducedBuildings.push(building);
}
});
this.setState({
isLoaded: true,
buildings: reducedBuildings,
})
});
I'm trying to send multiple webhooks based on the number of items in a JSON array. I'm using a example from:
how to trigger webhook from zapier code
I tried it and it almost works as planned. I have a problem that when I have a JSON array of 4 items, it sends 16 webhooks instead of 4. If I have A JSON array of 3 items, it sends out 9 webhooks instead of 3.
I use inputData.items for insertion of the JSON array. Does anyone know why the items in the JSON array are multiplied?
I used:
const elements = JSON.parse(inputData.items)
var body = elements;
var options = {
"url": "URL.COM",
"method": "POST",
"headers": {'Content-Type': 'application/json'},
"body": JSON.stringify(body)
},
requests = elements.map(mapDataToSettings);
function mapDataToSettings(elem) {
var settings = Object.assign({}, options);
settings.data = JSON.stringify(elem);
return settings;
};
Promise.all(requests.map(grabContent))
.then(function(data){ callback(null, {requestsMade: data});});
function grabContent(options) {
return fetch(options.url, options)
.then(function(res) {return res.json();});
};
Does anyone see why my webhooks are triggering to often?
Thanks
David here, from the Zapier Platform team.
It's hard to say for sure without seeing your zap, but my guess is that you're feeding .items in from a previous Code step where it's returned as an array? If so, you're running into an undocumented code feature where subsequent steps are run for each item in the array. This is usually desirable, but since you're doing a loop in the code, you don't need it.
Assuming that's the case, you've got two options:
Return a sjon string from the previous step (instead of an array) so this code step only runs once.
Change this code to only receive one item and perform a request with it.
If something else is going on, update your question with the zap id (it's not sensitive info) and I can take a look!
I'm creating a mock application with JSON server as the backend and I'm wondering if it is possible to get the total number of records contained at an end point without loading all the records themselves? Assuming the db.json file looks like the JSON snippet below, how would I find out that the end point only has one record without fetching the record itself, provided it's possible?
{
"books": [{
"title": "The Da Vinci Code",
"rating": "0"}]
}
You can simply retrieve the X-Total-Count header
This is a screen-shot of a response headers returned by JSON Server when enabling pagination i.e using the _page parameter (e.g. localhost:3000/contacts?_page=1)
Whenever you fetch the data, json-server actually returns the total count by default (it has an x-total-count property:
Example:
axios
.get("http://localhost:3001/users", {
params: {
_page: 1,
_limit: 10
}
})
.then(res => {
console.log(res.data); // access your data which is limited to "10" per page
console.log(res.headers["x-total-count"]); // length of your data without page limit
});
You've three options. I'd recommend the 3rd one to you:
Return all the records and count them. This could be slow and send a lot of data over the wire but probably is the smallest code change for you. It also opens you up to attacks where people can hammer your server by requesting many records repeatedly.
Add a new endpoint. You could add a new endpoint that simply returns the count. It's simple but slightly annoying having a 2nd endpointime to document and maintain.
Modify the existing endpoint. Return something like
{
count: 157,
rows: [...data]
}
The benefit of 3 is its all in one endpoint. It also nears you toward a point where you can add a skip and take parameter in future to allow pagination of the resultant data.
You will write another end point that returns number of records. Usually also you may want end point for limit and offset to be used with pagination.
let response = await fetch("http://localhost:3001/books?_page=1");
let total = response.headers.get('X-Total-Count');
I'm using Big Cartel's Dugway development environment, and trying to build a live search with Product.search (available with their JS API). It appears to return a set of 20 products no matter what search term, or limit I place on the call, and I'm wondering if anybody else has experienced this? Is it just an issue with the Dugway environment?
More specifically, this is the call I'm making:
Product.search('mugs', { limit: 5 }, function(data) {
console.log(data);
});
This is just the way that products.js works with Dugway, I can confirm that the limit works properly in a live Big Cartel shop. For now maybe you just want to create a for loop and stop at 5 results, or only retrieve 5 from the response another way.
What I'm trying to achieve is an effective way of passing each item found in a specific attribute through a JSON request in jQuery, so that I can gather information about it from an API. The API is Twitch and each data attribute is a streamers name.
Each streamer would be defined like so -
<div streamer='streamer_name'></div>
and from the API I'd like to gather their status and information. However, the Twitch API does not showcase information about offline streamers in the original API url, and so I have to use two API URL's to get all the required data.
I have created something that works, and as I'm rather new to this, I've most likely missed out on important features required to make this as effective as possible.
$('[streamer]').each(function() {
var data = $(this);
var streamer = $(data).attr('streamer');
$.getJSON('https://api.twitch.tv/kraken/streams/' + streamer, function(x) {
if (x.stream != null) {
$(data).append(
//online data
);
} else {
$.getJSON('https://api.twitch.tv/kraken/channels/' + streamer, function(x) {
$(data).append(
//offline data
);
});
}
})
})
http://jsfiddle.net/W4Km8/8801/ for an example of what this code actually does.
EDIT: There is also a way to search for multiple streamers at once, although again, it does not show any data for those that are offline.
https://api.twitch.tv/kraken/streams?channel=riotgames,scrags
EDIT2: To be more specific about my doubts, occasionally it will append the data incorrectly and will break the page layout, or not show up at all. This happens quite frequently upon page refreshes.
Thanks