Intercept a certain request and get its response (puppeteer) - javascript

Once that puppeteer goes to a certain url, I want that it listens to all the requests that are made, then find a specific request and return its response. The response should be a json object.
I managed in listening to all the requests and intercepting the desired one, but I don't know how to get its response. Here's my attempt: I get the error TypeError: Cannot read property 'then' of null.
Any suggestion?
page.on('request',async(request)=>{
console.log(request.url())
if (request.url().includes('desiredrequest.json')){
console.log('Request Intercepted')
request.response().then(response => {
return response.text();
}).then(function(data) {
console.log(data); // this will be a string
alert(data)
});
}
request.continue()
})

Since the response may have not arrived yet, the better method would be listening on the response event and get the request object from it.
page.on('response', async(response) => {
const request = response.request();
if (request.url().includes('desiredrequest.json')){
const text = await response.text();
console.log(text);
}
})

You might wanna use the "requestfinished" event instead of the "request" one.
page.on('requestfinished', async (request) => {
const response = await request.response();
const responseHeaders = response.headers();
let responseBody;
if (request.redirectChain().length === 0) {
// Because body can only be accessed for non-redirect responses.
if (request.url().includes('desiredrequest.json')){
responseBody = await response.buffer();
}
}
// You now have a buffer of your response, you can then convert it to string :
console.log(responseBody.toString());
request.continue()
});

Related

testing fastapi response to next.js with fetch, promise stuck in pending?

my fetch is stuck in pending when I query a fastapi endpoint in local dev.
followed this blog and a few others - https://damaris-goebel.medium.com/promise-pending-60482132574d
Using this fetch code (having simplified it drastically just to get a simple solution working)
function fastapiRequest(path) {
return fetch(`${path}`)
.then((response) => {
return response;
}
);
into a constant variable i.e.,
const xxxx = fastapiRequest(
`http://0.0.0.0:8008/xcascasc/Dexaa/Emo.json?Magic=Dexxaa&emotion=expressions`
);
Ideally I want to use UseSWR to do this as I'm using next.js, but first of all, just need it to work :)
A postman query like this works fine to return a value
curl --location --request GET 'http://0.0.0.0:8008/xcaxc/dexxa/emo.json?analysis=statistical&substance=dexxa&emo=powa' \
--header 'x_token: 13wdxaxacxasdc1'
the value is left like this in console.log
data show here? Promise {<pending>}
With the initial response being
Response {type: 'cors', url: 'url', redirected: false, status: 200, ok: true, …}
Update based on answers.
Using each of the proposed answers, I am still not getting the data returned appropriately. i.e.,
function fastApiRequest(path) {
console.log("really begins here");
return fetch(`${path}`, { mode: 'cors' })
.then((response) => {
console.log('response', response);
return response.json();
})
.catch((err) => {
throw err;
});
}
async function test() {
console.log('begins');
return await fastApiRequest(
`http://0.0.0.0:8008/xxxx/dex/adea.json?deaed=adedea&adee=deaed&adeada=adeeda`
);
}
const ansa = test();
Is giving a response of pending at the moment.
The backend is built with fastapi, with these CORS, I'm wondering if I need to give it more time to get the data? (postman works fine :/)
def get_db():
try:
db = SessionLocal()
yield db
finally:
db.close()
origins = [
"http://moodmap.app",
"http://localhost:3000/dashboard/MoodMap",
"http://localhost:3000",
"http://localhost",
"http://localhost:8080",
]
app.add_middleware(
CORSMiddleware,
allow_origins=origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
max_age=3600,
)
I am running the fastapi code in a docker container as well btw
As per Documentation
The Response object, in turn, does not directly contain the actual JSON response body but is instead a representation of the entire HTTP response. So, to extract the JSON body content from the Response object, we use the json() method, which returns a second promise that resolves with the result of parsing the response body text as JSON.
.json() is an async method (it returns a Promise itself), so you have to assign the parsed value in the next .then(). So your code can be changed like this.
function fastApiRequest(path) {
let res;
fetch(`${path}`)
.then((response) => response.json())
.then((data) => (res = data))
.then(() => console.log(res));
return res;
}
response = fastApiRequest('https://proton.api.atomicassets.io/atomicassets/v1/accounts?limit=10');
console.log('response')
If you want to use async/await approach, below is the code.
async function fastApiRequest(path) {
try {
const response = await fetch(path);
const data = await response.json();
return data;
} catch (error) {
console.error(error);
}
}
async function test() {
console.log(await fastApiRequest('https://proton.api.atomicassets.io/atomicassets/v1/accounts?limit=10'))
}
test()
first you need to parse the response into json if it's a json API.
function fastapiRequest(path) {
return fetch(`${path}`)
.then((response) => {
return response.json();
});
}
you need to 'await' for the rsponse
you need to write the below code in an async function
const xxxx = await fastapiRequest(
`http://0.0.0.0:8008/xcascasc/Dexaa/Emo.json?Magic=Dexxaa&emotion=expressions`
);
When you make an http request using fetch in javascript it will return a Promise, it's not stuck it's just need to be resloved, you can resolve it just like the above code with async await, or you can use the .then(() => { /* code... */ }) function, you can also use .catch(() => { /* handle error... */ }) function to handle errors.
In Your curl you use x_token as header variable, if it's required you need to pass a header with your path too. All other answers are valid too.

Javascript fetch api use custom error message

I'm looking for a way of handling errors with the native javascript fetch api. Used to use jQuery, but I'm trying to use more native javascript functions.
I found this blog and like the approach: https://learnwithparam.com/blog/how-to-handle-fetch-errors/
fetch(url)
.then((response) => {
if (response.status >= 200 && response.status <= 299) {
return response.json();
}
throw Error(response.statusText);
})
.then((jsonResponse) => {
// do whatever you want with the JSON response
}).catch((error) => {
// Handle the error
console.log(error);
});
However, in the catch I'm getting the statusText that belongs to the HTTP code. For 400 for example Bad request. But that is not wat I want, my call to the server will respond with exactly what is wrong. So I want to use the response body text as a the error. I tried different ways, but I can't get the response body incase the HTTP code is 400. With jQuery I used response.responseJSON.html. But this is not available with the fetch api.
So how can I can use the response body as error code.
The fetch API was designed to work best with async functions. If you can make your outer function async, your code would become:
try {
const response = await fetch(url);
if (!response.ok) {
const text = await response.text();
throw Error(text);
}
const jsonResponse = await response.json();
// do whatever you want with the JSON response
} catch (error) {
console.log(error);
}
Otherwise, it gets a bit more complicated:
fetch(url)
.then((response) => {
if (response.ok) {
return response.json();
}
return response.text().then((text) => throw Error(text));
})
.then((jsonResponse) => {
// do whatever you want with the JSON response
}).catch((error) => {
// Handle the error
console.log(error);
});

Https request from react sending incorrect(null) values

I have been trying to make a http PUT request for sending data to the server. Each time, when I try to send the request the data in the server becomes null. I have tested the server side code by sending correct format of data seems the server side is working fine. Going back to the client side code, I have noticed that during fetch operation the data in the response becomes in 'ReadableStream'. Don't know actually what does that mean. Can anybody help me finding out the mistakes if it is possible from the below information.
I am trying to send the data from the react component:
updateData(data)
the format of the data I am trying to send is an Object:
{first:"first", second:"second"}
And then in my api handler:
const updateData = async (data) => {
const resource = await getResource('PUT', {
data,
});
const url = `${process.env.updateAPI}/Update`;
return handleFetch(url, resource);
};
And then in my handleFetch function:
const handleFetch = async (url, resource) => {
const res = await fetch(url, resource);
if (!res.ok) {
throw new Error(`Request failed, status ${res.status}`);
}
const resData = await res.json();
return resData;
};
getResource function is accepting the parameters and returns the required methods, data and the header where 'Content-Type': 'application/json', is provided and handling the data like: body = JSON.stringify(data);
During debugging I have noticed that the res from fetchHandlers becomes like this:
I am actually confused, why in the body the data becomes 'ReadableStream' instead of the actual data I am trying to send or is this the problem I am having null values. Can anybody help me with that?, it will be highly appreciated. Thanks in Advance
If your data return type is not JSON or you don't want JSON then use text()
As an example:
fetch('https://jsonplaceholder.typicode.com/posts/1')
.then(function(response) {
return response.text();
}).then(function(data) {
console.log(data); // this will be a string
});

How to get string response of fetch api and assign it to a variable?

I am sending a request to backend with fetchApi and the response is coming as a just only string (not in a json body)
Here is my frontend code:
async decryptText() {
let response = await this.getDecryptedText(this.entry.text, this.user.userId)
if(response){
console.log(response)
}
}
async getDecryptedText(inputText: string, userId: number) {
let url = "http://localhost:8080/getDecryptedText?textToDecrypt=" + inputText + "&userId=" + userId;
return fetch(url);
}
But I cannot parse the response like other json responses because it is just a string.
When I print it to console the output is this:
I just want to print on to console the response string
How can I get the string response and assign it to a variable ?
I have edited the code as on answers.
await means wait until promise is resolved. so you should not use .then
callback function. It should be,
async decryptText() {
let response = await this.getDecryptedText(this.entry.text, this.user.userId)
// promise is resolved here with success or failure
if(response){
console.log(response) // you should get result here
}
}
hope it help's you try this :
decryptText() {
this.getDecryptedText(this.entry.text, this.user.userId)
.then(response => response.json())
.then(response => {
console.log(response)
})
}
getDecryptedText(inputText, userId) {
let url = "http://localhost:8080/getDecryptedText?textToDecrypt=" + inputText + "&userId=" + userId;
return fetch(url);
}
You need to use Response.text() method
let response = await this.getDecryptedText(this.entry.text, this.user.userId)
.then(res=>{
if(res.ok){
console.log(res.text())
}
This might work. res.ok there is a boolean which says was request successful or not. You must use this because construction if(res) is useless — you always get response even if request failed.
try to print like
console.log(response.body)
Because, you are getting body in your response.

Angular 4.0 http put request

I've written a function to send a http put request to update some data but it says, that it is not recieving any data:
updateHuman(human: Human) {
const url = `${this.url}/${human.id}`;
const data = JSON.stringify(human);
return this.http.put(url, data).map(
response => response.json().data as Human,
error => console.log(error)
);
}
After I've changed my function to the following, it is working:
updateHuman(human: Human) {
const url = `${this.url}/${human.id}`;
const data = JSON.stringify(human);
return this.http.put(url, data).map(() => human);
}
Could someone explain me, why the first function is not working but second is working?
Observables are lazy, you need to be subscribed to them for them to work and retrieve anything. Did you subscribe to your method? Example:
methodToUpdateHuman(human): void{
...
this.updateHuman(human).subscribe((response) => {
//do something with the response
console.log.("Response is: ", response);
},
(error) => {
//catch the error
console.error("An error occurred, ", error);
});
}
I suggest you read through the Angular Tour Of Heroses, it's based in angular 2 and most of the functionality is functional in angular 4, there is a section dedicated to http requests: https://angular.io/tutorial/toh-pt6
In the second example you are not returning the response within the map, you are returning the human that was originally passed in.
So, basically you are creating an illusion that it is working, when it isn't.
Probably best to test your API with something like PostMan, to see if you can get it working with that first.
You use map method incorrectly, read more about this method in documentation: http://xgrommx.github.io/rx-book/content/observable/observable_instance_methods/map.html
If you want receive response from server your code should look like that:
updateHuman(human: Human) {
const url = `${this.url}/${human.id}`;
const data = JSON.stringify(human);
return this.http.put(url, data).subscribe(
response => response.json().data as Human,
error => console.log(error)
);
}
You can use map method if you want to modify server response(map some objects to other structures etc.):
updateHuman(human: Human) {
const url = `${this.url}/${human.id}`;
const data = JSON.stringify(human);
return this.http.put(url, data)
.map(response => { return response.json() }) // you can get json response here
.subscribe(
response => response.data as Human, // -- change here --
error => console.log(error)
);
}
map method returns Observable object, so you can subscribe that and wait for response, error or simple complete method(third parameter of subscribe()):
http://xgrommx.github.io/rx-book/content/observable/observable_instance_methods/subscribe.html

Categories