We Keep Coding

Fetch raw gzip-encoded web page into Uint8Array

I'm using fetch to retrieve a URL. This is in code that is acting as a proxy and if it gets a response with content-encoding: gzip, I want to get the raw data so I can send it back to the consumer without decoding it.
But I see no way to do this. Response.blob, Response.arrayBuffer, Response.body.getReader(), .... these all seem to decode the content.
So, for example, in my test case, I end up with a 15K array of bytes instead of the actual response body which was only 4k.
Is there a way to get the raw contents of the response without it being decoded?

The browser automatically decodes compressed, gzip-encoded HTTP responses in its low-level networking layer before surfacing response data to the programatic Javascript layer. Given that the compressed bytes are already transmitted over the network and available in users' browsers, that data has already been "sent to the customer".
The Compression Streams API can performantly create a Gzip-encoded, ArrayBuffer, Blob, etc:
const [ab, abGzip] = await Promise.all((await fetch(url)).body.tee().map(
(stream, i) => new Response(i === 0
? stream
: stream.pipeThrough(new CompressionStream('gzip'))
).arrayBuffer()
))
Full example
https://batman.dev/static/74634392/
A gzip-encoded SVG image is provided to users as a downloadable file link. The original SVG image is retrived using fetch() from a gzip-encoded response.

Sending pdf as binary in POST request (from Javascript)

I want:
To send a file (I have my eye on .docx and text files, but let's use a .pdf as an example) as binary in the body of a POST request from a browser (Javascript).
Main problem
I can do this just fine in POSTMAN. You can select "binary" as the body type, and viola! your file is configured to be the body. But I don't know how to mimic that behavior in Javascript.
My question is: On the client-side, in Javascript, how can I get a file into my POST request as binary?
Specifically, how can I get the file into the same format that POSTMAN uses when you select Body -> Binary in a POST request?
For context:
I have been using this guide to get everything configured how I want in AWS. It ends with making requests in POSTMAN. But adding a file as binary in POSTMAN is one thing - doing it from a browser in Javascript is another, and the main question that I have.
I am sending this through API Gateway to a Lambda function. I have API Gateway configured to handle application/pdf as binary, and the Lambda function to decode it once it arrives.
So I think I want to hand it in as a binary blob, not base64. But not sure exactly how.
JavaScript
postBinary() {
var settings = {
"url": "https:<my-aws-api>.amazonaws.com/v1/upload",
"method": "POST",
"timeout": 0,
"headers": {
"Content-Type": "application/pdf"
},
"data": <my pdf file here as binary>
};
$.ajax(settings).done(function (response) {
console.log(response);
});
},
API Gateway:
Integration has 'When there are no templates defined (recommended)' set to 'Content-Type':'application/pdf'. The API's Binary Media Types have 'application/pdf' set. I know I have CORS set correctly - I can pass strings through the POST request and get success messages back, but I would like to handle files here, not just a simple string. I also want to avoid requiring the client side to parse out data on their end.
My Lambda function will take the file and then parse information out of it, then send it back.
Lambda function:
import json
import base64
import boto3
BUCKET_NAME = 'my-bucket'
def lambda_handler(event, context):
file_content = base64.b64decode(event['content'])
parsed_data = some_function(file_content) # parse information from file
return {
'statusCode': 200,
'body': {
'file_path': file_path
}
}
In the end, we want a user experience of: choose a file, send to API, get back parsed data. Simple.
Note: I know there are lots of good reasons to put files in s3 instead of going through Lambda first. But our files are small, and we are not concerned about taking considerable compute time/power in Lambda. Further, we want to avoid sending to s3 right away because we would like the user to only have to make one call to the API: send file in POST request, get results. If we send to s3 first, the user has to send multiple requests: request pre-signed URL, send file to s3, request results from parsing.
I am mostly concerned with the fact that this is possible in POSTMAN and it must be possible via browser/Javascript as well.
Thanks, everyone!

Fetch API - Download Headers Only [duplicate]

Here I have this sample fiddle, where I try to get only the content-type header from a large image, using both XMLHttpRequest() and fetch() in Javascript. If you run the fiddle with Chrome's Developer Tools / Network tab (or similar tools in other browsers) open, you'll see that while the XMLHttpRequest() method gets only the 600 B where the headers are located, the fetch() method gets the whole 7.8 MB image, despite my code only needing the content-type headers.
How can I get just those few bytes needed for the headers (instead of the whole content) using the Fetch APi's fetch()?
Note: I am interested in this, because apparently, if I try to get the headers of all the links of a regular Google page, the XMLHttpRequest() method logs me out and sometimes changes my language as well, while the fetch() method does not. I assume this is because the former sends the credentials / cookies / etc., while the latter is more 'restrictive' by default...

You need to specify the method type of HEAD:
fetch(url, {method: 'HEAD'})
See updated fiddle

Nodejs read image from server with promise

I already found some posts regarding on how to send an image to a client with nodejs. However the in the backend I get the image from another url, where I need to pass an authentification header.
From the client, I want to access the response and the image what so ever, but this doesn't seem to work, as the response only shows the image binary encoded.
Any solutions to this? Or how would you access an image that is user:password protected on the server (so a basic auth header is required for that)

AngularJS - $http get returns error with status 0?

I've an AngularJS application in which I'm trying to get an XML data with $http get from a server say http://example.com/a/b/c/d/getDetails?fname=abc&lname=def (this when accessed manually by entering the link in a browser shows a tree structure of the XML file).
When I run the application the data from that link is not fetched. Instead its showing an error with status 0.
//url = http://example.com/a/b/c/d/getDetails?fname=abc&lname=def
$http.get(url).success(function (data){
alert("Success");
deferred.resolve(data);
}).error(function (data, status){
console.log("Error status : " + status);
});
I'm not sure why the $http.get fails and goes to the error function returning status 0.
But if I pass an URL to the local server which has a JSON object, it works.
Is the problem because of me trying to access the XML file from a different domain(CORS issue?) or something else?
Please help me out!

You have HTTP access control (CORS) issues .
The server answering your REST requests MUST include the headers specified by CORS in order to allow Angular to consume properly the response. Essentially this means including the Access-Control-Allow-Origin header in your response, specifying the servers from where the request comes from, that are allowed. (ref)
There is directive Angular allows one to get/set the whitelists and blacklists used to ensure that the URLs used for sourcing Angular templates are safe.

yes you got it right, its because default content-type for $http is “application/json” if you want to change the request data or response data you can change it with the help of tranformRequest and tranformResponse. you can find more information on it here
also i find an article on implementation of tranformRequest change post data from json to form post in angularjs