How to limit the amount of data returned from a JSON file using fetch? - javascript

I have this fetch statement that returns 19 building names, but I only want 10; the following is what I attempted, but I still get 19 building names.
fetchBuildings(energyProgramId) {
fetch(`http://localhost:1001/api/energyprograms/${energyProgramId}/buildings/?results=10`)
.then(res => res.json())
.then(json => {
this.setState({
isLoaded: true,
buildings: json,
})
});
}
Is there something extra I need to add?

Here is an example:
1.fetch('http://jsonplaceholder.typicode.com/posts/')
The above URL gives array of objects with 100 elements because it originally is an array of 100 elements.
2.fetch('http://jsonplaceholder.typicode.com/posts/?_limit=10')
This URL gives array of objects with 10 elements.
Notice the difference?
I only did this : ?_limit=10 ----> Add any number in place of 10 and hopefully you will get desired results.

As the other answer already points out the best/most normal solution would be to change on the backend how the API returns data. Typically REST API's support query parameters such as limit and start or page and resultsPerPage.
If this is not available - e.g. when you're fetching an external resource - an alternative which is often supported by static file servers and sometimes by API's is the Range header which allows you to retrieve only a specific byte range of the resource (do note, in the case that an API supports this it will still load the entire resource on the server, but it will not transmit the entire resource). An example with fetch would look like
fetch('', { headers: { range: 'bytes=0-1000'} })
When doing this on XML or JSON resources it can be somewhat difficult to work with, but with for example CSV files it's ideal.

No different from fetch to XHR or axios or anything else. actually, no different from react or angular or vue or anything else.
This is an API that backend developers wrote it and it is based on REST API, so when you call it as GET or POST and anything else you just fetch the JSON that the backend developers designed it. BUT
There is a new technology that name is GraphQL. you can call API and then you just fetch the JSON what you want. Also, it must be implemented in backend but it is possible.

It's not closely bound up with React. If you need less data you must reduce data before set the state.
const reducedBuildings = [];
fetch(`http://localhost:1001/api/energyprograms/${energyProgramId}/buildings/?results=10`)
.then(res => res.json())
.then(json => {
json.forEach(building => {
if (reducedBuildings.length < 10) {
reducedBuildings.push(building);
}
});
this.setState({
isLoaded: true,
buildings: reducedBuildings,
})
});

Related

Sending an array with axios.get as params is undefined

I am making a get request with additional params options, since I am using that request on a filter, so the params are filters for what to get back:
const res = await axios.get("http://localhots:3000/getsomedata", {
params: {
firstFilter: someObject,
secondFilter: [someOtherObject, someOtherObject]
}
});
The request goes through just fine, on the other end, when I console.log(req.query); I see the following:
{
firstFilter: 'someObject',
'secondFilter[]': ['{someOtherObject}', '{someOtherObject}'],
}
If I do req.query.firstFilter that works just fine, but req.query.secondFilter does not work and in order for me to get the data, I have to do it with req.query["secondFilter[]"], is there a way to avoid this and be able to get my array of data with req.query.secondFilter?
My workaround for now is to do:
const filter = {
firstFilter: req.query.firstFilter,
secondFilter: req.query["secondFilter[]"]
}
And it works of course, but I don't like it, I am for sure missing something.
Some tools for parsing query strings expect arrays of data to be encoded as array_name=1&array_name=2.
This could be a problem if you have one or more items because it might be an array or might be a string.
To avoid that problem PHP required arrays of data to be encoded as array_name[]=1&array_name[]=2 and would discard all but the last item if you left the [] out (so you'd always get a string).
A lot of client libraries that generated data for submission over HTTP decided to do so in a way that was compatible with PHP (largely because PHP was and is very common).
So you need to either:
Change the backend to be able to parse PHP style
Change your call to axios so it doesn't generate PHP style
Backend
The specifics depend what backend you are using, but it looks like you might be using Express.js.
See the settings.
You can turn on Extended (PHP-style) query parsing by setting it to "extended" (although that is the default)
const app = express()
app.set("query parser", "extended");
Frontend
The axios documentation says:
// `paramsSerializer` is an optional function in charge of serializing `params`
// (e.g. https://www.npmjs.com/package/qs, http://api.jquery.com/jquery.param/)
paramsSerializer: function (params) {
return Qs.stringify(params, {arrayFormat: 'brackets'})
},
So you can override that
const res = await axios.get("http://localhots:3000/getsomedata", {
params: {
firstFilter: someObject,
secondFilter: [someOtherObject, someOtherObject]
},
paramsSerializer: (params) => Qs.stringify(params, {arrayFormat: 'repeat'})
});
My example requires the qs module
This has to do with params not being serialized correctly for HTTP GET method. Remember that GET has no "body" params similar to POST, it is a text URL.
For more information I refer to this answer, which provides more detailed info with code snippets.

Javascript - Is it possible to use fetch or axios.get with a specific property path?

I am a student, in my 3rd week of learning JavaScript, and today I've been completely roadblocked by something I can't find my way around.
Suppose I have the following:
axios.get("www.examplewebsite.com/api/......")
.then((response)=>{
console.log(response)
})
The json in question contains a tremendous amount of data, and all I need is one url from it. I found the property path of:
data.data.examples[0].example.week3.uri
Is there any way to use axios, or even fetch, to grab that single piece of data?
Unless the API provides that option, the client cannot specify which bits of information are wanted. With a regular REST API you get what the server sends you.
But, you can chain .then() calls if you want to narrow your data down and disregard everything you don't care about:
axios.get("www.examplewebsite.com/api/......")
.then(response => response.data.data.examples[0].example.week3.uri)
.then(uri => {
console.log(uri)
})
Or, if you want to grab the URI from the same path for all your examples, you could narrow it down to an array of URIs:
axios.get("www.examplewebsite.com/api/......")
.then(response => response.data.data.examples)
.then(examples => examples.map(item => item.example.week3.uri))
.then(uris => {
console.log(uris)
})
It depends on the server and the backend. It might be possible to get the value, but you have to provide the required information in your request (URL).
For example, this URL (http://site.come/moives) can give you a list of movies, but this (http://site.come/moives/movieid) can give you information about a certain movie. Again it depends on the server ...

Fetching data from multiples pages of API

I'm fetching data from a PS4 Games API but it's split into 400+ pages. I wanted to get the data from all pages, but the solution I came up with did not work very well. It gives me an error 'JSON Value of type NSNull cannot be converted to a valid URL'. Also, I don't think the for loop works well either, it shows me it going through all the pages when it displays the results in my list.
Additionally, this API is dynamic because new games keep getting released. So how could I get data up to latest page without manually changing my last page number everytime? I looked at some questions here but I couldn't fit it into my code
My code is rather long so I'm just going to post the part that matter:
componentDidMount() {
var i;
for (i = 0; i < 400; i++) {
fetch(`https://api.rawg.io/api/games?page=${i+1}&platforms=18`, {
"method": "GET",
"headers": {
"x-rapidapi-host": "rawg-video-games-database.p.rapidapi.com",
"x-rapidapi-key": "495a18eab9msh50938d62f12fc40p1a3b83jsnac8ffeb4469f"
}
})
.then(res => res.json())
.then(json => {
const { results: games } = json;
this.setState({ games });
//setting the data in the games state
});
}
}
The API also has an item that gives me the link of the next page, I think there is a way to use 'next' and fetch data from that URL
If anyone could help, that would be AWESOME. Thank you in advance
its my first answer on this forum, so... I hope be helpfull.
For me you have 2 options:
Each request send 2 variables in the answer count and next, or you make a for loop with count/20 as limit (20 is the number of items gaves in the answer), or you make a while loop until the next variable give null as answer (currently at page 249).
What is currently happening is you are making 400 requests and when each comes in it is overwriting the component with the response it received. It does not care about what is already there or any of the other requests.
An approach you could try instead is as the responses come in append the results to the ongoing list and update the state with the running list.
Going forward and for your other question about handling new releases. Instead of running 400 queries every time the application is used, try looking into caching the results. When the app loads you can see if cache exists and load or query if it does not. The rawg.io /games endpoint has a parameter for ordering by release. When the application loads in future you can conditionally loop until you reach a game that is already in cache at which point terminate.

Retrieving the client request ip address

This post isn't really a question anymore; I just want to post this to help other people out in the future to avoid lost time.
Goal: Retrieve the client IP address and set some specific values based on certain octet in IP.
I was developing a react web-app for my company and needed to support three facilities. The three locations of-course existed in different geographical regions and had slightly different IP schema's.
I needed to set some session identifier based on an octet value from the client IP. To do so, I did the following steps.
Setup express route for user to hit on initial visit of app.
Get client IP and store in const/var.
Explode IP string by ".".
Perform If/Then or Switch to determine value of desired octet.
Set some session/logic within matching condition.
Thanks to express, the req object contains an ip key with the value of the requests IP address. We can utilize this or some other third party library to get the needed info. Of course there are better/more secure ways to do this, but this is a simple method I've researched and setup. Definitely thanks to community for helping me resolve my issue with this.
apiRouter.route('/test')
.get((req, res) => {
const request_ip = req.ip; // Returns string like ::ffff:192.168.0.1
const ip_array = request_ip.split('.') // Returns array of the string above separated by ".". ["::ffff:192","168","0","1"]
// The switch statement checks the value of the array above for the index of 2. This would be "0"
switch(ip_array[2]) {
case('0'):
res.json({'request-ip':ip_array, 'location':'Location A'});
break;
case('1'):
res.json({'request-ip':ip_array, 'location':'Location B'});
break;
case('2'):
res.json({'request-ip':ip_array, 'location':'Location C'});
break;
default:
res.json({'request-ip':ip_array, 'location':'Default Location'});
}
})
One of my main issues was that I was developing on my local laptop. My node server was running express here. I was also trying to get my request ip from my local machine. This didn't make sense because I was constantly getting back "::1" as my request IP. Baffled, I did much research and finally found it to be an obvious PEBKAC issue. Thanks to nikoss in this post, it made all the sense in the world.
You can get this information by fetching it from an open IP
https://api.ipdata.co/
fetch("https://api.ipdata.co")
.then(response => {
return response.json();
}, "jsonp")
.then(res => {
console.log(res.ip)
})
.catch(err => console.log(err))
This works!
async componentDidMount() {
const response = await fetch('https://geolocation-db.com/json/');
const data = await response.json();
this.setState({ ip: data.IPv4 })
alert(this.state.ip)
}
use it in jsx as
{this.state.ip}
It seems like https://api.ipdata.co doesn't work anymore, even when specifying a key. I ended up using Ipify (typescript):
private getMyIp() {
fetch('https://api.ipify.org?format=json').then(response => {
return response.json();
}).then((res: any) => {
this.myIp = _.get(res, 'ip');
}).catch((err: any) => console.error('Problem fetching my IP', err))
}
This is a good reference for alternative IP retrieval services: https://ourcodeworld.com/articles/read/257/how-to-get-the-client-ip-address-with-javascript-only
If https://api.ipdata.co doesn't work you can use geolocation-db.com/json. Advantage of geolocation it also gives you other important values like latitude, longitude, country, state, zip
fetch(`https://geolocation-db.com/json/`)
.then(res => res.json())
You can console.log(...) the res.json() to view the JSON values.
You can use this one as well.
fetch('https://get-ip-only.herokuapp.com/') .then(r => r.json()) .then(resp => console.log(resp.ip))
https://get-ip-only.herokuapp.com/
This API provides you the IP only.

meteor.js - temporary server-side app state

I need to use some data from a 3rd party API in my app, poll for the needed data with certain frequency from the server, and make it available to the client. The easiest way would be to create a collection and update it, and make the data available to the client via pub/sub. But, in this particular case I don't need to store that data or keep track of it, and it updates very frequently, so storing it to db would actually be just additional unneeded work. I would prefer to store it somehow in the RAM, and make it available to the client in some other way except collections (perhaps, return from a method call). But I'm not sure, how to do that. Could someone suggest some nice approach?
You could use this package meteor-publish-join to fetch data from external API and publish to client periodically (disclaimer: I am the author):
Server:
import { JoinServer } from 'meteor-publish-join';
Meteor.publish('test', function() {
// Publish a random value from an external API, plays well with promise, re-run every 10 seconds
JoinServer.publish({
context: this,
name: 'withPromise',
interval: 10000,
doJoin() {
const id = parseInt(Math.random() * 100, 10);
return fetch(`https://jsonplaceholder.typicode.com/posts/${id}`)
.then(res => res.json())
.then(data => data.title)
.catch(err => console.error(err));
},
});
});
Client:
import { JoinClient } from 'meteor-publish-join';
Meteor.subscribe('test');
// Get the values published within `test` publication. All these values are reactive
JoinClient.get('withPromise')

Categories