I have been developing a web app where the user can upload a PDF file, and then later retrieve it and view it. What I have been doing to achieve this is having the PDF uploaded to a PostgreSQL database as bytea datatype (the column is called "attachment"), and then having nodeJS offer up this data to be fetched back and turned back into a PDF to view.
However I have been struggling to convert the data back into a valid PDF. This is the method I am using so far.
var file = new Blob([res[i].attachment], { type: 'application/pdf' });
var fileURL = URL.createObjectURL(file);
document.getElementById("pdf_box").data = fileURL;
The #pdf_box identifier refers to an object element in a HTML file which is used to display the PDF (this has been shown to work when I provide the file location of a dummy PDF file to the data attribute of this element).
The res[i].attachment is also shown to provide valid buffer data in JSON format, with an example provided below:
"attachment":{"type":"Buffer","data":[91,111,98,106,101,99,116,32,70,105,108,101,93]}
When I load the created fileURL into the data attribute of #pdf_box however, I get an error indicating along the lines of an invalid PDF file. My research so far appears to indicate this may be because the data is coming in as buffer whereas it needs to be in byte form, but I haven't found much that helps show me how this may be achieved or if there is a way to convert between the forms with the data I have access to? I have also seen occasional reference to a method called pdf.create(), but I cannot find documentation on this and I assume it must belong to a third-party library for JS.
If you know of any information that can help me with understanding what my problem is and what to search to figure out a solution, it would all be greatly appreciated, thank you.
To all reading this question, this method does not seem possible and would likely be incredibly inefficient to implement. If you send a string to nodeJS larger than the MTU of your link to this server (anywhere between 68 bytes and >1500 bytes depending on every component of your network) the packet will be silently dropped with no attempts to resolve or throw an error. What you must do is take the approach of using "multipart/form-data" encoding to send any files to the server (more on this found here).
It should also be mentioned that uploading a file to the database is not recommended in any capacity (due to databases being quite inefficient storage for large amounts of data) and what should be done is to upload a reference to the file path or URL to find the file at. Then on the client-side this file could be rendered as such when you have retrieved the file location...
<object id="pdf" data="../files/test.pdf"></object>
To change the data attribute, the following can be done...
document.getElementById("pdf").data = "../files/test2.pdf";
Related
I am new to React.
And I've done a file upload functionality.
It works well for a small file size, but some files are working not good with it.
So I want to have some advice about the right way. The project works via websockets and with another back-end software (php based), so we need to be in this architecture/style.
How I do it:
After a file selection I do this:
let reader = new FileReader();
reader.onloadend = () => {
this.setState({
avatarPreview: reader.result,
enableSave: true,
});
}
As a next step I send this base64 raw data (reader.result) via websockets.
On the back-end I split this data, do some validation, save the raw data (after base64_decode) to the file and write it to the database.
And all works perfectly, so no issues mostly, but some files - don't work well.
And I want to understand why. As I've analyzed - the websocket's frame can be huge and file can't be split because of it. So the problem is not in the file size. Also, not all the files have a big size too. Probably, the issue is in the main logic.
Is it ok to send a base64 raw data to the backend and after write it manually to a file and to the database?
Or do I need to create another one new endpoint, kind of a RestFull API access for a POST queries? In the official docs nothing about upload, only about file selection. Google has different results, so what is the best way?
I know that blob is a data type for binary data as integer is a datatype for int. As they say, It's used to store files directly in database (we move our audio file into blob, and save that blob in database).
Question 1) why to store blob for audio if I can just put the audio in storage for example path /var/www/audio.mp3 and in database I store path_name /var/www/audio.mp3?
Question 2) which is better ? how netflix stores movies? just blobs or what?
Question 3) Curious if there're any cons or prons if you could just give me ideas so that I know when to use them .
Putting the blob in the database, rather than a file, allows you to grow to multiple servers with load balancing. If you put the data in files, you would have to replicate the files between the server. Most databases have built-in replication features, this isn't as easy for regular files.
Better to use external storage/cdn for serving such kind of large content.
How Netflix and our works? They upload content on external bucket i. e. S3 and write file name in db for identification. According to user file access frequency that file cache on CDN/edge location. User will get awesome experience while content server from their nearest edge location
With blob you can store all kinds of stuff.
Do you communicate with an API via SOAP or JSON and want to store it in the database? Use a blob. Want to log what a user filled into a form when it threw an exception? Store the entire post as a blob. You can save everything as is. It's handy for logging if you have different data formats. I know an API which expects some data via SOAP and some as JSON. To log the communication I use blob because the response may be in XML, JSON, a number (http code 203 for empty but accepted) or an exception as array.
My application has to generate reports which should be available for download in XLS format. I have built a REST API using Django Rest Framework and there is an endpoint for report generation. It accepts POST requests with JSON body (report parameters, like from, to, etc., but there is also some data that represented with JSON objects) and returns JSON result. I successfully use it from Javascript, render the report as an HTML table and it works just fine.
My problem is that I need to allow users to save the report as an .xls file with a decent filename (like myawesomereport.04.12-10.12.xls. I tried JS data url approach, but as far as I understand, there is no way to set a filename if you go with that option (except setting a download attribute on an a tag, but its support is limited, so it's not the way to go). I thought that maybe I should open a new window with my API endpoint's url appropriately formed, so it outputs an XLS file, but the problem is that I do not understand if there is a way to send JSON with that request.
How should I approach this problem?
You can set the filename in the backend, by using the header Content-Disposition, so that in the frontend you can use a standard <a> tag.
In DRF, it would look like this:
response['Content-Disposition'] = 'attachment; filename={}'.format(
file_name
)
I want to upload a binary file using json.
I choose Json because with the file I would also like to send additional information.
I am going to do this by -
Select a file in the file input tag.
Use the HTML5 File Reader Api to read a file first.
Convert the file content into base64.
Add the base64 content to a JS object in a data uri format.
Convert the JS object to json and post it to the server.
I wonder if this is the only legitimate way to achieve my goal? Also, if there is a plugin already available somewhere which give me this ability?
No, this is not the only way - one of the other ways is just to submit a form with a file in it. Such form uses multipart/form-data content type.
See W3C documentation on the subject:
The content type "application/x-www-form-urlencoded" is inefficient for sending large quantities of binary data or text containing non-ASCII characters.
The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.
So, there is no need to reinvent the wheel - browsers already support sending the files along with additional information, in a simple way. You just create a form where the user can enter data and select files, then all of them are sent to the server with multipart/form-data content type, and your web framework should be able to understand that it deals with both files and textual data.
So i have a canvas on which the user signs, now instead of converting it to a base 64 string i simply want to save it as an image itslef. whats the easiest way to do it html5??
You can easily do that this way (specifying the format as png in this case):
var img = canvas.toDataURL("image/png");
You can specify different image formats.
Take a look at this answer.
I've answered a similar question here:
Simulating user event
Assuming you are saving locally
You can go the route of creating an image from a Data URL, but then saving it is the trickier part that currently isn't very nice using HTML5. It's hopefully going to get better soon, if browsers incorporate the download attribute of the a tag.
Obviously if you have higher permissions than a standard webpage... i.e. you are designing a browser plugin - then there are other options...
If I were to implement something like this myself, at the moment, I would conceed to using a flash plugin to handle the save to the local computer.
Assuming you are saving remotely
By the sounds of it you aren't saving to a server, but if so this is quite easy by just POSTing the base64 information to a script written in a server-side scripting language (i.e. like PHP) and getting that to write the data directly as binary to a file. Obviously you have to make certain you do this securely however, you don't want just any binary data written to your server's filesystem.
Best of both worlds
If you've got the development time, the best method to get a canvas image saved locally - without Flash - is to create a server-side script that instead of saving the data to your server actually writes the Base64 information you send it directly back as a realised binary image file. That way you can create a form that posts your Base64 data to a new tab, this request is evaluated by the server-side, and the binary image is returned... at which point the browser asks the user where they wish to save their image.
You'll need to define the correct headers to force an image to download (rather than display in-browser). A simple change to force this is to set the server-side script's Content-type header to 'image/octect-stream'... there are other header options to set which would be best researched (i.e. headers that control the filename and so forth).
reflect.php
<?php
/// a simple example..
if ( array_key_exists('data', $_POST) && ($data = $_POST['data']) ) {
header('Content-type: image/octet-stream');
echo base64_decode( $data );
exit;
}
and the form...
<form action="reflect.php" method="post" target="_blank">
<input name="data" type="hidden" value=" ... Base64 put here with JS ... ">
</form>
(The whole form should be created dynamically and submitted automatically with JavaScript)
Improving the user experience
There are ways to avoid a new tab being created, but you'd have to research to make sure these other methods don't cause cross-browser problems... for example you could post your form data as part of an iframe (which would keep the process hidden), or just post the data directly on the current window (..and hope that all the browsers receive the correct request and open a download rather than replace your page content - most modern browsers should handle this).
Improving security
With regards to a PHP script that automatically returns binary data, you should keep the access to this script secured by one time use key / authentication token or something similar, and keep a limit on how much Base64 data you are willing to accept. It might not seem like it poses a secutiry risk - as you are not modifying your server in any way with what the user sends - but the dodgy people of this world could take your script and use it to send download request to other users... which if downloaded (and turned out to be unwanted trojans or viruses) would make your server implicit in providing the dodgy file.
All in all
Due to the effort required to get a simple thing like an image saved to the desktop, I wouldn't blame you for doing the following:
Embed the image in the page (after taking your snapshot from canvas) and ask the user to right click and Save as...
Hopefully future things will make this situation better...