For my little Javascript app I wrote serverside API function with CGI.
I made it very simple, and full example script looks like that:
#!/usr/bin/env perl
use strict; use warnings; use 5.014;
use CGI;
use JSON;
use Data::Dumper;
my $q = new CGI;
my %p = $q->Vars;
_api_response();
sub _api_response {
my ( $error ) = #_;
my $res;
my $status = 200;
my $type = 'application/json';
my $charset = 'utf-8';
if ( $error ) {
$status = 500;
$res->{data} = {
status => 500,
};
$res->{error} = {
error => 'failure',
message => $error,
detail => Dumper \%p,
};
} else {
$res->{data} = {
status => 200,
};
}
print $q->header(
-status => $status,
-type => $type,
-charset => $charset,
);
my $body = encode_json( $res );
print $body;
}
When I call it from JS script with fetch, it gets no response body. If I checked from Developers Tools/Network, it has also no response body. If I enter the same URL into browser, it shows JSON body. If I use curl as
curl -v 'https://example.com/my_api?api=1;test=2;id=32'
response seems have also correct body:
< HTTP/2 200
< date: Mon, 13 Sep 2021 14:04:42 GMT
< server: Apache/2.4.25 (Debian)
< set-cookie: example=80b7b276.5cbe0f250c6c7; path=/; expires=Thu, 08-Sep-22 14:04:42 GMT
< cache-control: max-age=0, no-store
< content-type: application/json; charset=utf-8
<
* Connection #0 to host example.com left intact
{"data":{"status":200}}
Why fetch does not see it as a body?
For sake of completeness, I include JS part also:
async function saveData(url = '', data = {}) {
const response = await fetch(url, {
method: 'GET',
mode: 'no-cors',
cache: 'no-cache',
credentials: 'omit',
headers: {
'Content-Type': 'application/json'
},
redirect: 'follow',
referrerPolicy: 'no-referrer',
});
console.log(response); // body is null
return response.json();
}
Using the function as:
saveData('https://example.com/my_api?api=1;test=2;id=32', { answer: 42 })
.then(data => {
console.log(data);
})
.catch( error => {
console.error( error );
});
On console I see error:
SyntaxError: Unexpected end of input
One possible reason for this error is empty JSON string.
I was able to reproduce your problem, and then I was able to fix it.
It was a CORS issue. You need to enable CORS on both the front and the back end.
On the front end you need to set the content security policy with a meta tag in your page's <head>:
<meta http-equiv="Content-Security-Policy" content="default-src *; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval' http://localhost">
(Don't forget to change localhost to whatever your real domain is.)
On the back you need to add the CORs header:
print $q->header(
-status => $status,
-type => $type,
-charset => $charset,
-access_control_allow_origin => '*', # <-- add this line
);
As a side note, none of the settings you're passing into fetch are necessary. And since you're awaiting the response and then returning another promise anyway, there is really no reason for that to even be a an async function.
Until you're ready to do something with the unused data argument, the following code will suffice:
function saveData(url = '', data = {}) {
return fetch(url).then(response=>response.json());
}
You have to await for response.json() too.
Try return await response.json();, instead of return response.json();
Related
This Netlify function should run as an endpoint on example.com/.netlify/functions/github and is supposed to proxy a fetch request from my website, reach out to the GitHub API and send data back to the website.
As far as I have understood, I can use to GET data from the GitHub API without authentication. Hitting their API directly in the browser works: https://api.github.com/orgs/github/repos?per_page=2 (also works from Postman).
The data is an array of objects where each object is a repository.
There has been multiple issues the past couple of years where Netlify functions (running on AWS lambdas) have had hickups that resulted in error messages similar to mine, so I'm confused whether this is an error in my code or something weird on their side.
First, the proxy function which – according to the Netlify admin console – runs without error. In a support article Netlify requires the result returned as JSON.stringify(), so I follow that convention here:
const fetch = require('node-fetch')
const url = 'https://api.github.com/orgs/github/repos?per_page=2'
const optionsHeaders = {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': 'Content-Type'
}
const fetchHeaders = {
'Content-Type': 'application/json',
'Host': 'api.github.com',
'Accept': 'application/vnd.github.v3+json',
'Accept-Encoding': 'gzip, deflate, br'
}
exports.handler = async (event, context) => {
if (event.httpMethod === 'OPTIONS') {
return {
'statusCode': '200',
'headers': optionsHeaders,
}
} else {
try {
const response = await fetch(url, {
method: 'GET',
headers: fetchHeaders
})
const data = await response.json()
console.log(JSON.stringify({ data }))
return {
statusCode: 200,
body: JSON.stringify({ data })
}
} catch (err) {
console.log(err)
}
}
}
Client fetch that hits https://example.com/.netlify/functions/github. URL is correct, the function is executed (verified that in the Netlify admin panel):
const repos = document.querySelectorAll('.repo')
if (repos && repos.length >= 1) {
const getRepos = async (url) => {
try {
const response = await fetch(url, {
method: "GET",
mode: "no-cors"
})
const res = await response.text()
// assuming res is now _text_ as per `JSON.stringify` but neither
// that nor `.json()` work
console.log(res[0].name)
return res[0].name
} catch(err) {
console.log(err)
}
}
const repoName = getRepo('https://example.com/.netlify/functions/github')
repos.forEach((el) => {
el.innerText = repoName
})
}
Not 100% sure where this error message originates from, it is probably not the console.log(err) although it displays in the browser console, because the error code is 502 and the error also shows up directly in the response body in Postman.
error decoding lambda response: error decoding lambda response: json: cannot unmarshal
string into Go value of type struct { StatusCode int "json:\"statusCode\""; Headers
map[string]interface {} "json:\"headers\""; MultiValueHeaders map[string][]interface {}
"json:\"multiValueHeaders\""; Body string "json:\"body\""; IsBase64Encoded bool
"json:\"isBase64Encoded,omitempty\""; Metadata *functions.Metadata
"json:\"metadata,omitempty\"" }
Haven't found any clear information on this issue, could any of you enlighten me?
The only response that don't comply with the schema is the preflight request. From the error message, I assume you need to change:
'statusCode': '200',
to
'statusCode': 200, // StatusCode int
Even better, because there's no content, you may want to use 204 instead.
If that's still not enough, I may still want to include the body there as well, as it doesn't seem optional:
return {
'statusCode': 204,
'headers': optionsHeaders,
'body': ''
}
I'm working against an HTTP server that sometimes returns a body that is longer than the response's Content-length header. For example, curl shows me this:
curl -v -k -H "Cookie: VerySecretCookie" https://fqdn/path
[...]
< HTTP/1.1 200 200
< Date: Thu, 01 Jul 2021 10:45:32 GMT
[...]
< X-Frame-Options: SAMEORIGIN
<
* Excess found in a non pipelined read: excess = 83, size = 2021, maxdownload = 2021, bytecount = 0
I'm trying to issue a similar request via axios and read the entire body, even beyond the Content-length. However, it seems to stop after Content-length bytes have been read.
This is the config I'm using:
const config: AxiosRequestConfig = {
method: 'GET',
responseType: 'stream',
maxRedirects: 0,
decompress: false,
timeout: 60000,
};
And I then retrieve the response via:
private async streamToString(stream: Stream): Promise<string> {
return new Promise((res, rej) => {
let data = '';
stream.on('end', () => {
res(data);
});
stream.on('error', (err) => {
rej(err);
});
stream.on('data', (chunk) => {
data += chunk;
});
});
}
const data = await this.streamToString(response.data);
From a quick glance at axios' code, I couldn't find a place where it inspects the Content-length header, so perhaps this is done by Node's http? Is there a way to tell it to ignore Content-length and just read the entire stream until it closes (assuming HTTP keepalive is not used)?
I have an existing PHP code that is doing a curl request to a 3rd-party PHP server.
The 3rd-party server returns a GZIP string.
In PHP, I can use gzdecode to decode the gzip string.
How can I do it in NodeJS/Javascript? I tried using decompress-response with no avail.
Also tried using got instead of request, enabled auto-decompress, also doesn't work.
Edit: Also tried zlib and pako, also doesn't work.
Sample Code [ PHP ]
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => $params,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 3000000,
CURLOPT_ENCODING => '',
CURLOPT_CUSTOMREQUEST => "GET",
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo false;
} else {
$response = gzdecode($response);
echo $response;
}
This is the solution that works for me.
I used got instead of axios because I can't get it to work there.
I set my request options:
const requestOptions = {
encoding: null, // this is important
headers: {
"Accept-Encoding": "gzip",
}
...
};
Don't forget that encoding: null line, because without that, the response will be automatically converted to a string. ( We need a buffer to make this work )
Then I created a function like this to handle my request:
const zlib = require("zlib");
async function performRequest(url, options) {
try {
const response = await got(url, options);
if (response.headers["content-encoding"] === "gzip") {
const body = response.body;
try {
const dezziped = zlib.gunzipSync(response.body);
response.body = JSON.parse(dezziped.toString());
} catch (error) {
response.body = body;
}
}
return response.body;
} catch (error) {
return error;
}
}
Note: This is a synchronous operation, you can use gunzip instead if you want to do the async way.
The guy responsible for API requests is gone for the week, so nothing can be done on server side.
fetch("https://url.com/api/login/", {
method: "post",
headers: {
// 'Accept': 'application/json',
'Content-Type': 'application/x-www-form-urlencoded'
},
body: JSON.stringify({
username: "test#mail.com",
password: "123"
})
}).then(function (response) {
return response.json();
}).then(function (myJson) {
console.log(myJson);
});
It works on Postman, but as I heard, Postman doesn't follow the same security as browsers, therefore this isn't an issue for Postman. But I doubt this is the case, as the authors php-solution works fine.
This is an example of php-solution that works (He wrote it):
function login($username, $password) {
$curl = curl_init(); curl_setopt_array($curl, array(
CURLOPT_URL => "https://url.com/api/login/",
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => false,
CURLOPT_TIMEOUT => 30,
CURLOPT_MAXREDIRS => 10,
CURLOPT_POSTFIELDS => "username=".$username."&password=".$password,
CURLOPT_HTTPHEADER => array(
"cache-control: no-cache",
"content-type: application/x-www-form-urlencoded"),
));
$response = curl_exec($curl);
curl_close($curl);
$authdata = json_decode($response);
if ($authdata -> success) {
//success
return true;
} else {
//fail
return false;
}
}
What's missing in my code? How can I make it work like his php solution. (Have no experience in php).
Any help is much appreciated.
EDIT:
What worked on Postman:
Raw json format in Body.
Adding values as Key and Value in x-www-form-urlencoded
To solve this error you can do 3 things:
Add your origin server side.
Run your javascript on the same domain.
Check this answer for disabling same origin policy in chrome. This will allow you to test your code until the guy responsible for de API returns.
I don't seem to find any missed end input. This is the error that I recieve:
Uncaught (in promise) SyntaxError: Unexpected end of input
at fetch.then.response (InventoryOnHand.js:33)
Below is my code: I have a value for the url.
fetch(url + "GetItemMasterList", { 'mode': 'no-cors' })
.then(response => response.json())
.then(function (data) {
console.log(data)
});
This is a well known problem with CORS policy. To overcome this problem you need access rights to the server side API. In particular, you have to add a line in the header of php or another server endpoint:
<?php
header('Access-Control-Allow-Origin: *');
//or
header('Access-Control-Allow-Origin: http://example.com');
// Reading JSON POST using PHP
$json = file_get_contents('php://input');
$jsonObj = json_decode($json);
// Use $jsonObj
print_r($jsonObj->message);
...
// End php
?>
Also, make sure NOT to have in the header of your server endpoint:
header("Access-Control-Allow-Credentials" : true);
Model of working fetch code with POST request is:
const data = {
optPost: 'myAPI',
message: 'We make a research of fetch'
};
const endpoint = 'http://example.com/php/phpGetPost.php';
fetch(endpoint, {
method: 'POST',
body: JSON.stringify(data)
})
.then((resp) => resp.json())
.then(function(response) {
console.info('fetch()', response);
return response;
});