I am trying to intercept a request, change the method to post and pass Data.
...
page.on('request', request => {
payload = {...}
var data = {'method': 'POST', 'postData': payload
request.continue(data)
});
await page.goto(url_post_product)
The problem is the URL url_post_product doesn't return any body, the status_code is a 302 so I always have a TimeoutError: Navigation Timeout Exceeded.
I think I need to know the location reponse and redirected it bur nothing happens when I try to incertecept the reponse.
page.on('response', response => {console.log(response.status())});
I know the request and posted data is well formed.
EDIT
it's seems to a chromium bug:
Puppeteer GitHub bug
Related
I have the following simple PHP page:
<?php
echo 'done';
When I send a Javascript fetch request to that URL, I can inspect the response in the browser's dev tools and see that it returns the string 'done'.
Here is the request:
const response = await fetch(url, {
credentials: 'include'
});
On the other hand, if I start a session, the response is blank:
<?php
session_start();
echo 'done';
In both cases, the request headers are exactly the same, the response headers are exactly the same and the HTTP code is 200. It also works correctly if I manually go to that URL in the browser. It only fails specifically with fetch requests.
Note, I have display_errors and display_startup_errors set to On and nothing is outputted to the browser and nothing is logged in the log file either.
This behavior is because of a bug with Chromium that the devs have decided they "WontFix" and have stopped answering comments.
In order to get it to work, you need to manually read response.text() or response.json():
const response = await fetch(url);
const text = await response.text();
Once you do that, the response body will show up in your dev tools. If not, it will appear as if the response was empty, even if it wasn't.
You can use async with Await
Asyn function getData(url){
const response = await fetch(url);
const data = await response.json();
Console.log(data).
}
You can change the .json() to text()
I wish to give the client feedback before a redirect occurs, so they can store it in session storage, then when the cached page arrive from the service worker, they check session storage while the page is being rendered (not after!), and can handle the cached response accordingly.
I tried:
Adding a custom header to the response, but the client JavaScript can't read it for security reasons.
I have tried to edit the response directly. This only works for GET requests. Unfortunately when I sync a POST request, because it returns a redirect, so then it looks like a normal GET. So I need some additional way of saying, this is a GET after a sync POST, tell the user the POST was saved, its not just a normal "get the page"
Post Message, but slow as.
LocalStorage and SessionStorage is forbidden for the service worker
I could write to IndexedDB in the service worker, and then read from the client. But IndexedDB is such a confusing beast I really don't want to go down this route.
URL search parameters, redirect and url cleaning strategy became spaghetti code very quickly. The server would have to clean up URLs, and so would the service worker for the injected query args.
Is there any recommended machanism for relaying information to a client that would suite this purpose?
Side note about the post message being slow:
I currently use post message, but the problem is its really slow, and the reason I think is this:
Client attempts offline POST
Service worker serializes and stores it for when online again. In the fetch interrupt it responds with the cached response. It also calls an async postmessage to tell the client it was saved. Unfortunately if I await the postmessage, it errors out the fetch. So then one has to leave it to be async. Which means the post message happens only after the redirect
Client receive redirect response
Client redirects
Client paints the page
The cahed paged is showed
Only after about two seconds later it shows the 'was saved banner'
Heres some code if applicable:
Note: Orginally the code would set a value in the session storage when receiving the message (assumed it would receive the message before the redirect), and then pop it after the redirect at page render. However because the post message was coming so much later, I changed to performing the change on the page directly.
async function msgClientSyncSaved(event) {
const data = {
type: 'MSG_SYNC_SAVED',
};
const client = await getClient(event);
client.postMessage(data);
}
// Applicable parts of runFetch:
async function runFetch(event) {
const urlObj = new URL(event.request.url);
if (utils.getIsMethodTx(event.request.method)) {
// If a Sync URL
const clonedRequest = event.request.clone();
const response = await new strategies.NetworkOnlyStratey(log, event, cacheMutator).run();
if (!response.isDefaultResponse && !response.isCachedResponse) {
event.waitUntil(syncAllRequests());
return response;
} else {
const [syncKey, syncValue] = settings.PWA_SYNC_POST_URL_PARAM.split("=");
if (urlObj.searchParams.get(syncKey) === syncValue) {
// A failed POST, that requires SYNCING
console.log(`SW: Sync later: ${event.request.method} to ${event.request.url}`);
event.waitUntil(storeRequest(clonedRequest)); // no need to wait for this to finish before returning response
event.waitUntil(msgClientSyncSaved(event)); <--- HERE message client
// After a post, return a redirect
urlObj.searchParams.delete(syncKey);
const redirectUrl = String(urlObj);
// 302 means GET the redirect, 307 means POST to the redirect
console.log('REDIRECT TO', redirectUrl)
return Response.redirect(redirectUrl, 302);
}
}
}
}
function handleFetch(event) {
event.respondWith(runFetch(event));
}
self.addEventListener("fetch", handleFetch);
Reciever on client side:
async function handleMessage(event) {
switch (event.data.type) {
case 'MSG_SYNC_SAVED':
document.body.setAttribute('data-pwa-cached-page', 'true data-tx')
break;
}
}
navigator.serviceWorker.addEventListener("message", handleMessage);
I've been trying to send a GET request to an api to fetch data using Axios but always get a response object with status, headers, config, agents etc and response.data is always empty.
For example, the following code returns me an Axios response object with the hasBody set to true and data being empty.
axios.get(`https://fantasy.premierleague.com/api/leagues-classic/12000/standings/`).then(response => {console.log(response);
console.log(response.data);});
However, when I switched over to using Request library which has been deprecated, I am able to get the response body. For example, the following code works:
request(`https://fantasy.premierleague.com/api/leagues-classic/12000/standings/`, { json: true }, (err, res, body) => {
if (err) { return console.log(err); }
console.log(body);
});
Can someone tell me what am I doing wrong and how can I get the response body using axios? I'm a beginner and have spent hours trying to figure out so I would really appreciate any form of help.
It's not an axios library issue. From what I can tell, the server does't like the user-agents starting with "axios/". Specifying some user agent gives you the expected result:
const axios = require("axios");
axios.get(`https://fantasy.premierleague.com/api/leagues-classic/12000/standings`, {
headers: {
'user-agent': 'not axios',
}
}).then(response => {
console.log(response.data);
});
As for why the requests library works but axios does not: axios is setting the user-agent header to something like axios/0.21.1 or whatever version you have. requests on the other side, leaves the user-agent header unset. It's the server right to handle the request as he pleases.
I have verified the response from this URL https://fantasy.premierleague.com/api/leagues-classic/12000/standings/ - there is no data property in the response
Try like below to read the values:
It seem like your URL at https://fantasy.premierleague.com/api/leagues-classic/12000/standings/ had invalid response body.
I am trying to send POST requests through fetch API to JSON-server. Function is called on a simple button click (type 'button', not 'submit'). When I replace POST request with GET request everything works like it supposed to, but with POST I have a problem. Request passes, on the JSON-server entity gets created but keeps refreshing the page after each request. Also, I don't have a response from JSON-server, google chrome says 'Failed to load response data'.
Where I'm making a mistake?
const comment = {
text: "test comment",
article_id: 3
};
console.log(JSON.stringify(comment));
const options = {
method: 'post',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(comment)
}
fetch(`${URL_COMMENTS}`, options)
.then(response => { return response.json() })
.then(data => {
console.log(data)
});
If you use Live Server extension, try disabling that and try again.
Check out for Json sever port number running on your machine
attach the html form code
So we can try it on oru local machine to reproduce the issue.... Which help us to resolve the issue easy
I use the following code to POST the users position to my own backend-service via the Fetch API:
window.onload = () => {
let getPosition = (options) => {
return new Promise((resolve, reject) => {
navigator.geolocation.getCurrentPosition(resolve, reject, options);
});
};
getPosition().then(pos => {
const data = new FormData();
data.append('latitude', String(pos.coords.latitude));
data.append('longitude', String(pos.coords.longitude));
fetch('/', { // <--- error is thrown in this line
method: 'POST',
body: data
}).then(response => {
if (!response.ok) {
throw Error('Data sent - Network response NOT OK');
} else {
console.log('Data sent - Network response OK')
}
});
});
};
This works flawlessly and the backend-service sends back a positive empty response. Data sent - Network response OK is logged in the browser's console and everything is fine except that immediately after, this is logged:
How come I get a Fetch failed loading: POST even though the POST succeeded, the response is OK and the status code in the Chrome network tab is 200 OK? Is there something wrong with my code?
I had the same behaviour that you were seeing.
My server responded to the POST request with a 204 (Empty Response) and an empty (Content-Length=0) response.
I tried changing that to just respond with a "ok" message instead (edit: actually just returning the created object now) and the fetch error log disappeared.
It seems to me that fetch (at least in Chrome) erroneously logs "Fetch Failed" when the response body is empty, even if the request was successful.
This happens because you don't read the empty response body. I noticed this, because Django triggers a broken pipe error on the server side. To fix this, just read the empty body out.
async function fetchSomething() {
const response = await fetch(url, {
method: 'POST'
})
await response.text()
return response.ok
}