I want to upload a file as JSON from the client to a Python webserver (Tornado) and save it on the server. This is my simplified setup:
Client HTML:
<input type="file" id="myFile" onchange="fileChange" />
Client JS:
function fileChange(event) {
const file = event.target.files[0];
const fileReader = new FileReader();
fileReader.onload = (e) => uploadFile(e.target.result, file.name);
fileReader.readAsText(file);
}
function uploadFile(fileContent, fileName) {
const data = {fileContent, fileName};
axios.post('http://localhost:8080/api/uploadFile', JSON.srtingify(data));
}
Python Webserver:
class UploadFileHandler(tornado.web.RequestHandler):
def post(self):
requestBody = tornado.escape.json_decode(self.request.body)
file = open(requestBody["fileName"], "w+")
file.write(requestBody["fileContent"].encode("UTF-8"))
file.close()
All uploaded files are empty (blank pages in a PDF, file type of JPG is 'not supported', Word file cannot be opened) and are nearly twice as big as the original file. How can I fix this?
Is there a way to improve this setup?
You are trying to upload binary files (word, jpg), serialised as JSON, and store them on the server.
To handle binary data in JSON, encode the binary data as base64 first, then call JSON.stringify.
Like this (untested):
function uploadFile(fileContent, fileName) {
// Encode the binary data to as base64.
const data = {
fileContent: btoa(fileContent),
fileName: fileName
};
axios.post('http://localhost:8080/api/uploadFile', JSON.stringify(data));
}
On the server side, you need to deserialise from JSON, decode the base64 and the open a file in binary mode to ensure that what you are writing to disk is the uploaded binary data. Opening the file in text mode requires that the data be encoded before writing to disk, and this encoding step will corrupt binary data.
Something like this ought to work:
class UploadFileHandler(tornado.web.RequestHandler):
def post(self):
requestBody = tornado.escape.json_decode(self.request.body)
# Decode binary content from base64
binary_data = base64.b64decode(requestBody[fileContent])
# Open file in binary mode
with open(requestBody["fileName"], "wb") as f:
f.write(binary_data)
Related
I have a MSSQL Server Database with files stored as a column Content of type VARBINARY(MAX) in a SQL table.
I developed a backend Node.js server with Express.js to access the database. In this backend I have a route called /api/file/:id_file which takes the content of the file with id_file
async getFile(req,res){
const {id_file} = req.query
const file= await db.Files.findOne({
where:{
'Id': id_file,
}
})
res.contentType('application/pdf')
res.send(file.Content)
}
Then in frontend using Javascript, I have an iframe
<iframe id="view" src="" style='height: 500px; width: 1000px;' />
And I use an ajax request to get the file. When I have the response, I just convert it to blob and set the src attribute of iframe element with the base64 encoding of the blob
const reader = new FileReader();
reader.onload = () => {
const b64 = reader.result.replace(/^data:.+;base64,/, "");
$('#view').attr('src','data:application/pdf;base64,'+ b64);
};
reader.readAsDataURL(new Blob([resp.data], { type: "application/pdf" }));
When I try to request a file and show, I see a blank PDF, nothing is shown.
Am I missing any step to convert the content of the file to base64?
You don't need a filereader since you're not reading a file from the filesystem or user input; you already have the file pulled down from the API. If your binary pdf data is in resp.data then you just need to use btoa
creates a Base64-encoded ASCII string from a binary string
$('#view').attr('src','data:application/pdf;base64,'+ btoa(resp.data));
I want to upload any file (Word, PDF, Image ...) with an HTML file input to a python server.
With the help of snakecharmerb (Upload file as JSON to Python webserver) I can send the file by encoding it in the following way in JavaScript:
function fileChange(event) {
const file = event.target.files[0];
const fileReader = new FileReader();
fileReader.onload = function(e) {
const encodedFile = btoa(unescape(encodeURIComponent(e.target.result)));
uploadFile(JSON.stringify(encodedFile), file.name);
}
fileReader.readAsText(file);
}
In Python, I can read and save the file in the following way:
import json
from base64 import b64decode
binary_data = b64decode(json.loads(file));
with open(file_name, "wb") as f:
f.write(binary_data)
But the resulting file is not readable or the content is not correct encoded. I think i need to encode the binary_data in utf-8. But binary_data.encode("utf-8") results in an error.
test_file.txt
Hello World!
Special chars: äöüß
Python error
UnicodeDecodeError('ascii', 'Hello World!\r\nSpecial chars: \xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd', 29, 30, 'ordinal not in range(128)')
I also tried combining it with .decode('string_escape')
I converted a excel file as base64 String as web backend side, and could recover it as excel file as expected with Java code below:
String destinationPath = "e:\out1.xls";
//decode Base64 String to excel file
FileOutputStream fos = new FileOutputStream(destinationPath);
bytes = new sun.misc.BASE64Decoder().decodeBuffer(content);
fos.write(bytes);
fos.flush();
fos.close();
I can response frontend with base64 string, but I always got messy code when I tried to export it as excel (.xls) file using javascript. Below is the code:
onClick: function(){
// below is base64 String snippet.
var base64Str = "0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAOwADAP7/CQAGAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAA\r\nEAAA/v///wAAAAD+////AAAAAAEAAAD/////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n////////////////////////////////////////////////////////////////////////////\r\n//////////////////////////////////////////////////////////////////////////9S\r\nAG8AbwB0ACAARQBuAHQAcgB5AAAAAAAAAAAAAA .........";
base64Str = base64Str.replace(/\\r\\n/g,"");
var decoder = Ext.util.Base64.decode(fileStr);
testApp.view.main.FileExport.saveAs(decoder,'SHANGTOUDI_JF_20180320_0800.xls','UTF-8');
}
Need to say, I can use this method to export txt file successfully. Any suggestion are appreciated.
Backend converts the xls file into base64 String and returns in the http response.On the browser, any way to convert it back to xls for downloading?
It is not necessary to convert the base64 string any further. Add the necessary portions to base64 string to form a valid data URI, then proceed to offer file for download.
const mime = "data:application/vnd.ms-excel,";
const data = mime + base64string;
var oReq = new XMLHttpRequest();
var reader = new FileReader();
reader.onload = function(e) {
var rawData = reader.result;
oReq.open("POST", '/upload', true);
oReq.send(rawData);
console.log(rawData);
}
reader.readAsBinaryString(postObj);
// postObj is an image who's src is set to a data uri, taken via a webcam.
console.log(rawData) yields ==>
"PNG
IHDRÈÜÇ IDATx^d½idizw"##rϬ}ëîªÞfÃ!%2
ø£üølpµ0ù Q&É©¡,Ú2l²!æSeÃ$Y-ÉÎLïKuí{îáëºófe³§¦*3#Nó¾Ïr?÷³¼úOþÓùÉ tóne¸Øm.»ÙhÐtÃÅQ7<w''³nοg³Y·Èkóãn<Xì§G]·²Ø
ç]·8wÇ\å¨;á5vÁ°Ì»E¾?æ÷ywtÒuó¯ù{>Ïám]ÇëF£Qwt|\ßó5ä³NæÜÇ°^{r¿ùÜÁ|¡;>:êón0âß¼gûã6rm_ëgçõü4[vG¼gaa!?_à5#~³0\èxcß}Â}ú|ÜÓóóÅ!Ï=?â{®ã½ó^í½v'ü×úþÞ/×ñÏ}P®}Â÷ݸ;9fX¨?[àUÇ|?à^|þáb=çòɸ[æ·¬#7q]>giÜqÓ~#oq´èMdY4í3-.ùÈzÿü̺Íy
7Ľֺؿ\Ü6¯õ:Ü ß·5r-|ºv½½ôú¬w¶ìõáþ<{/íuí{ÞÎÚy¯Gì%kÎ}NfÓÚÞã5ݶÏÞÔr×
ø«Õｯk¯õï</ëïçz¬ïso§ÙsƳü5öß»{ÛÝádÌñö ïgÍÿù¿üWÝƹÝøüX7øK?ùó ê¶/!Ë«ÝÂxÔ"
G.²Éxbæ]ñg#úýgé£Þ;¿ThƳ/ùòû(D¿Ã>2Þ-ð³7¯18"P<=ç>GSr±;r\à;³°<6ÎòZNAázndûûtù¼1ÇVGÁ¼^vܼ\3ÚÆu¸9×g]ÉñA V°0óîÍ£TÃY½ïdaM">ÆhÎF9ú¿](3¯¢xeÜõt
¢DÞN¾E¾oÈEy×q®?ÂÀ»z=Bó|,¯JáæzíýÞ¯ÿöï&äM!rϼWóQÛ^ÏÛÖ5÷ܤûåϸÛüûÃkôF©]ß¿óÜ1F¤üNëÍsM¯Qü²úYX:f¯DSßÌõ4D1eTÛûüvÿþlrpÐM'SÇuòµÈïù?øî½>è~â'~¢ü$
How can I interpret this in Python? On the server, this data shows up as:
'...\x7f\xc2\xb4r\xc2\x87\x1c\xc2\xaa\n-\xc2\x9c\xc2\xa6QF\xc2\xac\xc2\xb0tS\xc3\xa4\xc2\xb0;\xc3\x8cisL#\xc3\x98\xc2\x94E\xc3\x94\xc2\xb8Bz\xc3\xb0\xc3\xa9\xc2\xaa#8\xc2\x90\xc3\xbb\xc3\xa5>\xc3\xbaO\xc2\xa8\xc3\x81H\xc3\x91\xc2\xaf:i\xc2\x8a\xc2\x926\xc2\x8b\xc2\x81\xc3\xbc\xc3\xa1Y\xc3\x93\xc2\x9b\xc2\xbat\xc2\x8f\xc3\x9e~\xc2\xa3PH4\x02_\x04\xc2\xbf\xc2\x92\xc2\xb7\xc3\xad\xc2\x8f\xc3\x9e\xc3\xbf\xc2\xb8<\xc2\x91V\xc3\xa0\xc3\x8b\x1f\xc3\x88\xc3\x9f\xc2\xa2>)\x1d\xc3\x94eY=\xc3\x8ct\xc2\xa9+L^7\xc2\xa2I\xc3\x84\xc2\xba\x03\xc3\xb5!1f\xc3\x97\xc3\x81\xc3\xbfD\xc3\x87\xc3\xb7\x06\xc2\xaa\xc3\xafcz\xc3\xad(\xc3\xb5\xc2\xab\xc3\x96\xc3\xb5<\xc3\x8e\xc2\xab\x08\xc3\x81\xc2\x88\x0b\xc3\x8a;\xc3\x8e!v\xc3\x84\xc2\xb1?\xc2\x8bVn\x19t\xc3\x80\xc2\x8bT`:\x1c\xc3\x8b\xc2\x99\xc3\xb2\xc3\x9c\xc3\xbf\x0fCsXi\xc3\xa6z\xc3\xb3l\x00\x00\x00\x00IEND\xc2\xaeB`\xc2\x82'
Writing this to file as a PNG yields an invalid PNG. Any guidance on saving the image would be helpful.
You will need to convert your image's binary representation into Base64 before uploading - this makes it safe to work with when transferring data over HTTP.
When you receive the Base64 encoded image server-side, you can convert it back to binary, and write this to a file.
Client side code:
To convert to Base64, you need to use
fileReader.readAsDataURL( fileObject )
this automatically represents your data in a safe to upload via HTTP format.
Server side code:
import base64
coded_string = '''Q5YACgA...'''
binary = base64.b64decode(coded_string)
# now write binary to file...
open('/path/to/new_file.png', 'wb').write(rawData)
On the browser:
// Client Side:
var xhr = new XMLHttpRequest();
xhr.open('POST', uploadUrl);
xhr.send(base64EncodedItem);
On the server (I am using Flask/Python 2.x)
# Server Side:
import re
import uuid
# Obtain the base64 string
image_string = request.data
image_string = re.search(r'base64,(.*)', image_string).group(1)
# Generate a file name.
# We can assume PNG format in my specific implementation, but you could search
# the image_string variable to detect filetype.
output = open(str(uuid.uuid4()) + '.png', 'wb')
output.write(image_string.decode('base64'))
output.close()
This worked for me. Hope it helps others.