I'm trying to send a simple image file to a lambda function. Once it gets to the function I need to turn it into a buffer and then manipulate it. Right now when data is received there are a bunch of characters prepended to the data:
"body": "--X-INSOMNIA-BOUNDARY\r\nContent-Disposition: form-data; name=\"image\"; filename=\"americanflag.png\"\r\nContent-Type: image/png\r\n\r\n�PNG\r\n\n\rIHDR0�\b�;BIDATx��]u|����{g��.H\b^h�F)PJ�������WwwP���\"E��$!nk3���3�l���{��=�L�����=��=�|)����ٿ)��\"�$��q�����\r���'s��4����֦M��\"C�y��*U�YbUEc����|�ƼJ���#�=�/ �6���OD�p�����[�Q�D��\b�<hheB��&2���}�F�*�1M�u������BR�%\b�1RD�Q�������Q��}��R )%ĉ�Idv�髝�S��_W�Z�xSaZ��p�5k�{�|�\\�?
I have no idea how to handle that. My plan has just been to create a buffer as you normally would in Node:Buffer.from(data, 'utf8'). But it's throwing an error
Things I've tried:
I've been testing the function with Insomniac and Postman, both with the same result.
I've gone with both a multipart/form and binary file for the body
of the request.
I've tried multiple image files.
I've set the header of content-type to image/png and other file
types.
I've removed the headers.
I know that I could upload the files to S3 and that would be much easier but it negates the point of what I'm writing. I don't want to store the images I just want to manipulate them and then discard them.
This is what the response looks like when I send it back to myself.
Edit: The full code is uploaded. Again, I'm not sending via node at this very moment. It's simply through Postman/Insomniac. If the answer is simply "write your own encoder" then please put that as an answer.
Because you did not upload full code so based on my best prediction I post an answer here. There are probably any of the solutions may help to you.
Encoding Base64 Strings:
'use strict';
let data = '`stackoverflow.com`';
let buff = new Buffer(data);
let base64data = buff.toString('base64');
console.log('"' + data + '" converted to Base64 is "' + base64data + '"');
Decoding Base64 Strings:
'use strict';
let data = 'YHN0YWNrb3ZlcmZsb3cuY29tYA==';
let buff = new Buffer(data, 'base64');
let text = buff.toString('ascii');
console.log('"' + data + '" converted from Base64 to ASCII is "' + text + '"');
Encoding binary data to base64 string:
'use strict';
const fs = require('fs');
let buff = fs.readFileSync('image-log.png');
let base64data = buff.toString('base64');
console.log('Image converted to base 64 is:\n\n' + base64data);
Decoding Base64 Strings to Binary Data:
'use strict';
const fs = require('fs');
let data = 'encoded binary string';
let buff = new Buffer(data, 'base64');
fs.writeFileSync('image-log.png', buff);
console.log('Base64 image data converted to file: image-log.png');
Base64 encoding is the way to converting binary data into plain ASCII text. It is a very useful format for communicating between one or more systems that cannot easily handle binary data, like images in HTML markup or web requests.
In Node.js the Buffer object can be used to encode and decode Base64 strings to and from many other formats, allowing you to easily convert data back and forth as needed.
I am trying to build a PDF file out of a binary stream which I receive as a response from an Ajax request.
Via XmlHttpRequest I receive the following data:
%PDF-1.4....
.....
....hole data representing the file
....
%% EOF
What I tried so far was to embed my data via data:uri. Now, there's nothing wrong with it and it works fine. Unfortunately, it does not work in IE9 and Firefox. A possible reason may be that FF and IE9 have their problems with this usage of the data-uri.
Now, I'm looking for any solution that works for all browsers. Here's my code:
// responseText encoding
pdfText = $.base64.decode($.trim(pdfText));
// Now pdfText contains %PDF-1.4 ...... data...... %%EOF
var winlogicalname = "detailPDF";
var winparams = 'dependent=yes,locationbar=no,scrollbars=yes,menubar=yes,'+
'resizable,screenX=50,screenY=50,width=850,height=1050';
var htmlText = '<embed width=100% height=100%'
+ ' type="application/pdf"'
+ ' src="data:application/pdf,'
+ escape(pdfText)
+ '"></embed>';
// Open PDF in new browser window
var detailWindow = window.open ("", winlogicalname, winparams);
detailWindow.document.write(htmlText);
detailWindow.document.close();
As I have said, it works fine with Opera and Chrome (Safari hasn't been tested). Using IE or FF will bring up a blank new window.
Is there any solution like building a PDF file on a file system
in order to let the user download it? I need the solution that works in all browsers, at least in IE, FF, Opera, Chrome and Safari.
I have no permission to edit the web-service implementation. So it had to be a solution at client-side. Any ideas?
Is there any solution like building a pdf file on file system in order
to let the user download it?
Try setting responseType of XMLHttpRequest to blob , substituting download attribute at a element for window.open to allow download of response from XMLHttpRequest as .pdf file
var request = new XMLHttpRequest();
request.open("GET", "/path/to/pdf", true);
request.responseType = "blob";
request.onload = function (e) {
if (this.status === 200) {
// `blob` response
console.log(this.response);
// create `objectURL` of `this.response` : `.pdf` as `Blob`
var file = window.URL.createObjectURL(this.response);
var a = document.createElement("a");
a.href = file;
a.download = this.response.name || "detailPDF";
document.body.appendChild(a);
a.click();
// remove `a` following `Save As` dialog,
// `window` regains `focus`
window.onfocus = function () {
document.body.removeChild(a)
}
};
};
request.send();
I realize this is a rather old question, but here's the solution I came up with today:
doSomethingToRequestData().then(function(downloadedFile) {
// create a download anchor tag
var downloadLink = document.createElement('a');
downloadLink.target = '_blank';
downloadLink.download = 'name_to_give_saved_file.pdf';
// convert downloaded data to a Blob
var blob = new Blob([downloadedFile.data], { type: 'application/pdf' });
// create an object URL from the Blob
var URL = window.URL || window.webkitURL;
var downloadUrl = URL.createObjectURL(blob);
// set object URL as the anchor's href
downloadLink.href = downloadUrl;
// append the anchor to document body
document.body.append(downloadLink);
// fire a click event on the anchor
downloadLink.click();
// cleanup: remove element and revoke object URL
document.body.removeChild(downloadLink);
URL.revokeObjectURL(downloadUrl);
}
I changed this:
var htmlText = '<embed width=100% height=100%'
+ ' type="application/pdf"'
+ ' src="data:application/pdf,'
+ escape(pdfText)
+ '"></embed>';
to
var htmlText = '<embed width=100% height=100%'
+ ' type="application/pdf"'
+ ' src="data:application/pdf;base64,'
+ escape(pdfText)
+ '"></embed>';
and it worked for me.
The answer of #alexandre with base64 does the trick.
The explanation why that works for IE is here
https://en.m.wikipedia.org/wiki/Data_URI_scheme
Under header 'format' where it says
Some browsers (Chrome, Opera, Safari, Firefox) accept a non-standard
ordering if both ;base64 and ;charset are supplied, while Internet
Explorer requires that the charset's specification must precede the
base64 token.
I work in PHP and use a function to decode the binary data sent back from the server. I extract the information an input to a simple file.php and view the file through my server and all browser display the pdf artefact.
<?php
$data = 'dfjhdfjhdfjhdfjhjhdfjhdfjhdfjhdfdfjhdf==blah...blah...blah..'
$data = base64_decode($data);
header("Content-type: application/pdf");
header("Content-Length:" . strlen($data ));
header("Content-Disposition: inline; filename=label.pdf");
print $data;
exit(1);
?>
Detect the browser and use Data-URI for Chrome and use PDF.js as below for other browsers.
PDFJS.getDocument(url_of_pdf)
.then(function(pdf) {
return pdf.getPage(1);
})
.then(function(page) {
// get a viewport
var scale = 1.5;
var viewport = page.getViewport(scale);
// get or create a canvas
var canvas = ...;
canvas.width = viewport.width;
canvas.height = viewport.height;
// render a page
page.render({
canvasContext: canvas.getContext('2d'),
viewport: viewport
});
})
.catch(function(err) {
// deal with errors here!
});
I saw another question on just this topic recently (streaming pdf into iframe using dataurl only works in chrome).
I've constructed pdfs in the ast and streamed them to the browser. I was creating them first with fdf, then with a pdf class I wrote myself - in each case the pdf was created from data retrieved from a COM object based on a couple of of GET params passed in via the url.
From looking at your data sent recieved in the ajax call, it looks like you're nearly there. I haven't played with the code for a couple of years now and didn't document it as well as I'd have cared to, but - I think all you need to do is set the target of an iframe to be the url you get the pdf from. Though this may not work - the file that oututs the pdf may also have to outut a html response header first.
In a nutshell, this is the output code I used:
//We send to a browser
header('Content-Type: application/pdf');
if(headers_sent())
$this->Error('Some data has already been output, can\'t send PDF file');
header('Content-Length: '.strlen($this->buffer));
header('Content-Disposition: inline; filename="'.$name.'"');
header('Cache-Control: private, max-age=0, must-revalidate');
header('Pragma: public');
ini_set('zlib.output_compression','0');
echo $this->buffer;
So, without seeing the full response text fro the ajax call I can't really be certain what it is, though I'm inclined to think that the code that outputs the pdf you're requesting may only be doig the equivalent of the last line in the above code. If it's code you have control over, I'd try setting the headers - then this way the browser can just deal with the response text - you don't have to bother doing a thing to it.
I simply constructed a url for the pdf I wanted (a timetable) then created a string that represented the html for an iframe of the desired sie, id etc that used the constructed url as it's src. As soon as I set the inner html of a div to the constructed html string, the browser asked for the pdf and then displayed it when it was received.
function showPdfTt(studentId)
{
var url, tgt;
title = byId("popupTitle");
title.innerHTML = "Timetable for " + studentId;
tgt = byId("popupContent");
url = "pdftimetable.php?";
url += "id="+studentId;
url += "&type=Student";
tgt.innerHTML = "<iframe onload=\"centerElem(byId('box'))\" src='"+url+"' width=\"700px\" height=\"500px\"></iframe>";
}
EDIT: forgot to mention - you can send binary pdf's in this manner. The streams they contain don't need to be ascii85 or hex encoded. I used flate on all the streams in the pdf and it worked fine.
You can use PDF.js to create PDF files from javascript... it's easy to code... hope this solve your doubt!!!
Regards!
Background Information
I'm trying to write some javascript / HTML logic that will export some data to a csv format, and open up a new document in Excel or whatever is the default csv app on the workstation.
Problem
The code successfully creates a new csv file and prompts to save the file, but when it's displayed in Excel, everything is in one column instead of being split up into multiple columns.
Code:
function exportTableToCSV($tableName, fileName) {
var csv = GetCellValues($tableName);
console.log(csv);
console.log("filename is:" + fileName);
if (navigator.userAgent.search("Trident") >= 0) {
//this is the path that is execute in IE10 browser...
window.CsvExpFrame.document.open("text/html", "replace");
window.CsvExpFrame.document.write(csv);
window.CsvExpFrame.document.close();
window.CsvExpFrame.focus();
window.CsvExpFrame.document.execCommand('SaveAs', true, fileName + ".csv");
} else {
var uri = "data:text/csv;charset=utf-8," + escape(csv);
var downloadLink = document.createElement("a");
downloadLink.href = uri;
downloadLink.download = fileName + ".csv";
document.body.appendChild(downloadLink);
downloadLink.click();
document.body.removeChild(downloadLink);
}
};
function GetCellValues($table) {
var $rows = $table.find('tr:has(td),tr:has(th)'),
// Temporary delimiter characters unlikely to be typed by keyboard
// This is to avoid accidentally splitting the actual contents
tmpColDelim = String.fromCharCode(11), // vertical tab character
tmpRowDelim = String.fromCharCode(0), // null character
// actual delimiter characters for CSV format
colDelim = ',',
rowDelim = '"\r\n"',
// Grab text from table into CSV formatted string
csv = '"' + $rows.map(function(i, row) {
var $row = $(row),
$cols = $row.find('td, th');
return $cols.map(function(j, col) {
var $col = $(col),
text = $col.text();
return text.replace('"', '""'); // escape double quotes
}).get().join(tmpColDelim);
}).get().join(tmpRowDelim)
.split(tmpRowDelim).join(rowDelim)
.split(tmpColDelim).join(colDelim) + '"',
// Data URI
//csvData = 'data:application/csv;charset=utf-8,' + encodeURIComponent(csv);
csvData = csv;
return csvData;
};
CSV File In Excel
Instead of having 7 columns of data, i have one column of data with all 7 values delimited by comma in it.
Console Output From Above Code:
HTML1300: Navigation occurred.
group_membership_list.html
"Group Name","PD Number","Dest","Department","First Name","Last Name","ttp"
"Test Group","45721","test1#hotmail.com","undefined","Daniel","Pamplemouse","smtp"
"Test Group","45721","test1#hotmail.com","undefined","Daniel","Pamplemouse","smtp"
"Test Group","46481","test1#hotmail.com","undefined","Benjamin","Grouper","smtp"
"Test Group","48381","test1#hotmail.com","undefined","Gregory","Erock","smtp"
"Test Group","48381","test1#hotmail.com","undefined","Gregory","Erock","smtp"
"Test Group","48382","test1#hotmail.com","undefined","Jonathan","David","smtp"
"Test Group","48382","test1#hotmail.com","undefined","Jonathan","David","smtp"
"Test Group","45303","test1#hotmail.com","undefined","Harold","Cause","smtp"
"Test Group","45303","test1#hotmail.com","undefined","Harold","Cause","smtp"
"Test Group","45306","test1#hotmail.com","undefined","Micah","Latin","smtp"
filename is:Group Membership List.csv
What I've Tried So far:
I've tried to change this:
colDelim = ',',
to
colDelim = '\,',
But that didn't make a difference.
I'm sure it's something simple that I'm missing but I can't seem to see my error. Any help would be appreciated.
EDIT 1
I also tried changing:
window.CsvExpFrame.document.write(csv);
to this:
window.CsvExpFrame.document.write('sep=,\r\n' + csv);
based on the following stackoverflow question:
Javascript/ jQuery : Exporting data in CSV not working in IE
But it's still a no go.
EDIT 2
As a test, I took a copy of the csv file that the web app created, and tried to import it via Excel. I chose comma as the delimiter. The end result was that it failed - aka, all the data ends up in one column.
I guess this means the way I'm creating the delimiter in my code is wrong.
I changed my code to use this:
colDelim = '","',
And retried. It still fails when the file opens via the web app, but if I take that file and manually import it again in Excel, this time it does work.
Edit 3
It seems to be related to quotes. As you can see from the console output, each field there seems to have been properly enclosed in quotes and separated by commas. But when I examine the data in Excel, the first field (Test Grp) is actually missing the quotes. It looks like this:
Group Name,"PD Number","Dest","Department","First Name","Last Name","ttp"
Test Group,"45721","test1#hotmail.com","undefined","Daniel","Pamplemouse","smtp"
EDIT 4
So I changed my code to look like this:
// actual delimiter characters for CSV format
colDelim = '"\,"',
rowDelim = '"\r\n"',
And now while IE still fails, other browsers like Chrome consistenly work with this combo. Also, although IE fails to open the file up properly, when I manually import the csv file that the web page generates into Excel, it parses correctly.
Another interesting artifact. The name of the file that IE generates is
Group Membership List_csv.csv
whereas Chrome creates:
Group Membership List.csv
Not sure this is the solution, but it is worth trying:
If it only happens using the SaveAs option (Trident condition in the question's code), it could be the BOM prefix, missing when the file is saved and possibly required by Excel. You can add it to your CSV text:
window.CsvExpFrame.document.write("\uFEFF"+csv);
However it could as well be the opposite case, being the BOM prefix added when the file is saved and then not expected by Excel. In this case you might need to remove it with a text editor like Notepad++, encoding and saving the file without BOM.
I can't comment so I'll post this as an answer:
In your GetCellValues function you're setting your rowDelimeter to be "\r\n".
The first thing I would try is changing it to be just \r\n. I can't say for sure, but the double-quotes may be messing with Excel's import CSV functions.
rowDelim = '\r\n';
In addition, when I created a similar script to import to excel, I only needed the \n return, didn't need to include \r. You might want to try that if changing the rowDelim doesn't work.
Edit:
I found the code I wrote before. Here's what I did:
// Create the first row, title, for the CSV
thecsv = "Full Title,CRN,Section,Class,Max,Actual,Open,Start,End,Day,Room\n";
// Split an array of info into comma-delimited values where each row is delimited by \n
for(course in courses) {
for (detail in courses[course]) {
thecsv += courses[course][detail] + ","
}
thecsv += "\n";
}
// Encode the CSV file.
var encodedCSVURI = encodeURIComponent(thecsv);
// Create a link to open the file (in my case, download)
var link = document.createElement('a');
link.setAttribute("href", "data:attachment/csv," + encodedCSVURI)
link.setAttribute("target", "_blank");
var filename = prompt("Please enter a name for the file.", "EXTRACTED_DATA");
link.setAttribute("download", filename + ".csv");
document.body.append(link);
link.click();
Edit 2:
Here I am encoding the CSV string (which would be your dumped table), creating a link that downloads a file of the encoded CSV string:
var encodedCSVURI = encodeURIComponent('some,comma,delimited,string\n,etc,etc,etc,etc'); // Encode the CSV
link.setAttribute("href", "data:attachment/csv," + encodedCSVURI); // Create the download file link with the encoded CSV data
link.setAttribute("download", "filename.csv"); // Here I was prompting the user for a filename, you don't have to
link.setAttribute("target","_blank"); // This is to prevent leaving the current page. You don't have to include this
document.body.append(link); // Add it to the document
link.click(); // Click the link (download the file)
I am using AngularJs with a REST API. I don't have the hand on the REST API.
I can store digital object with the API by sending a REST request.
I can get it also with a GET request.
The requests needs to have some specific headers.
My goal is to give the user a "download and save as" link.
For now on the click event i make the request :
this.file = function (file) {
var url = config.domain + 'file/' + file;
var methods = resource(url, null, {
'get': {
method:'GET',
headers:{ 'Authorization' : user.auth,
'secret-key' : user.secretkey}
}
transformResponse : function(data, headersGetter){
return {content:data}; //transform octet stream into text, angular returns an array containing 1 character per element.
},
});
return methods;
};
in the return body I have the file content (see below). I would like to download it. How is it possible ? Notice that I can't store the file as a URL.
Would it be possible to open a window wich make the rest call with the good headers and save the file ?
EDIT
I need the solution to be able to work well with a 50Mo File.
example of a PDF file content I have :
%PDF-1.7
£´ÅÖçø
2 0 obj
[/ICCBased 3 0 R]
endobj
3 0 obj
<<
/Filter /FlateDecode
/Length 2596
/N 3
>>
stream
xwTSÙϽ7½PÐkhRH
½H.*1 JÀ"6DTpDQ¦2(à£C±"Q±ëDÔqpIdß¼yïÍß÷~k½ÏÝgï}ÖºüÂLX ¡XáçÅg`ðlàp³³BøF|Ølø½º ùû*Ó?Áÿ¹Y"1PçòøÙ\É8=W%·Oɶ4MÎ0JÎ"Y2Vsò,[|öe9ó2<ËsÎâeðäÜ'ã9¾`çø¹2¾&ctI#Æoä±|N6(Ü.æsSdl-c(2- ãyàHÉ_ðÒ/XÌÏËÅÎÌZ.$§&\SáÏÏMçÅÌ07#â1ØYárfÏüYym²";Ø8980m-m¾(Ô]ü÷v^îDøÃöW~
°¦eµÙúmi]ëP»ýÍ`/²¾u}qº|^RÄâ,g+«ÜÜ\Kk)/èïúC_|ÏR¾Ýïåaxó8t1C^7nfz¦DÄÈÎâpùæøþuü$¾/ED˦L Lµ[ÈB#øøÃþ¤Ù¹ÚøÐX¥!#~(* {d+Ðï}ÆGùÍÑûÏþ}W¸LþÈ$cGD2¸QÎìüZ4 E#ê#èÀ¶À¸àA(q`1àD µ `'¨u 46ptcà48.Ë`ÜR0)ð
Ì#ÈRt CȲXäCP%CBH#ë R¨ªê¡fè[è(tº
C· Qhúz#0 ¦ÁZ°l³`O8ÁÉð28.·Àp|îOÃàX
?§:¢0ÂFBx$ !«¤i#Ú¤¹H§È[EE1PLÊ⢡V¡6£ªQP¨>ÔUÔ(j
õMFk¢ÍÑÎèt,:.FW Ðè³èô8ú¡c1L&³³³Ó9Æa¦±X¬:Öë
År°bl1¶
{{{;}#âtp¶8_\<N+ÄU
[.....]
I think you could using blob, something like
var content=...the content of your request;
var mypdf = new Blob(content, {type : 'application/pdf'});
and check answer from "panzi" in this other question Using HTML5/Javascript to generate and save a file
(One character per element in the array seem pretty nice binary. Probably you don't need to transform it. https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Sending_and_Receiving_Binary_Data )
Maybe you could do something like this?
var a = document.createElement('a');
a.href = 'data:attachment/pdf,' + encodeURI(data);
a.target = '_blank';
a.download = 'filename.pdf';
a.click();
You'd just have to make sure that data was in the correct format, which IIRC is base64.
you can use this instead of above code :
var url = config.domain + 'file/' + file;
var parameters = "Authorization=" + user.auth +
"&secret-key=" + user.secretkey;
var reportParameters = url + encodeURIComponent(parameters);
window.location.assign(reportParameters);
Thanks everybody for helping find a solution.
I am not able to find a satisfying solution in javascript and client side application.
I am going to make a proxy class communicating with the API.
This will send the REST request with security headers and will give as response the file content.
This class will be called by a HTTP GET request, so the 'save as' process will be managed easily thanks to right response headers.
I know this question is asked several times, but unfortunately nothing seems to work for me.
I post the src of an img to my node/express. It looks like this:
data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD//gA+Q1JF ... UUUUAFFFFAH/2Q==
The data is saved in picture. I cut of the data:image-stuff and got the raw base64 and the filetype.
var result = {
"type":"",
"data":""
}
var matches = picture.match(/^data:image\/([A-Za-z-+\/]+);base64,(.+)$/),response = {};
result.type = matches[1];
result.data = new Buffer(matches[2], 'base64');
require('fs').writeFile(mediaFolder+'/test.'+result.type, result.data, "binary", function(err){
res.status(500).send("error");
});
res.status(200).send("success");
When I try to open the saved image it says: Damaged or too big. I also tried to set the "binary" parameter in the writeFile methode. The client always gets the 200 http status.
I don't know what's wrong with this. I checked the raw base64 String with an online decoder. It worked perfectly.
I logged every string/match and everything looked okay to me.
Any help would be nice to solve this Problem.
Thanks in advance!
EDIT:
This is how I send the picture:
var base64Image = $('#show-picture').attr('src');
xmlhttp.open("POST","/webapp-ajax",true);
xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
xmlhttp.send("picture="+base64Image);
I believe you need to use encodeUriComponent(base64) before sending to server.
try sending a JSON object, and parsing the image on the client side.
For example:
var mimeType = image.split(';')[0];
var base64 = encodeUriComponent(image.split(',')[1]);
var imageData = {
"mimeType" : mimeType,
"src" : base64
}
...
xmlhttp.setRequestHeader("Content-type","application/json");
xmlhttp.send(imageData);