We Keep Coding

Saving file with JavaScript File API results wrong encoding

I have a problem (or may be two) with saving files using HTML5 File API.
A files comes from the server as a byte array and I need to save it. I tried several ways described on SO:
creating blob and opening it in a new tab
creating a hidden anchor tag with "data:" in href attribute
using FileSaver.js
All approaches allow to save the file but with breaking it by changing the encoding to UTF-8, while the file (in current test case) is in ANSI. And it seems that I have to problems: at the server side and at the client side.
Server side:
Server side is ASP.NET Web API 2 app, which controller sends the file using HttpResponseMessage with StreamContent. The ContentType is correct and corresponds with actual file type.
But as can be seen on the screenshot below server's answer (data.length) is less then actual file size calculated at upload (file.size). Also here could be seen that HTML5 File object has yet another size (f.size).
If I add CharSet with value "ANSI" to server's response message's ContentType property, file data will be the same as it was uploaded, but on saving result file still has wrong size and become broken:
Client side:
I tried to set charset using the JS File options, but it didn't help. As could be found here and here Eli Grey, the author of FileUplaod.js says that
The encoding/charset in the type is just metadata for the browser, not an encoding directive.
which means, if I understood it right, that it is impossible to change the encoding of the file.
Issue result: at the end I can successfully download broken files which are unable to open.
So I have two questions:
How can I save file "as is" using File API. At present time I cannot use simple way with direct link and 'download' attribute because of serverside check for access_token in request header. May be this is the "bottle neck" of the problem?
How can I avoid setting CharSet at server side and also send byte array "as is"? While this problem could be hacked in some way I guess it's more critical. For example, while "ANSI" charset solves the problem with the current file, WinMerge shows that it's encoding is Cyrillic 'Windows-1251' and also can any other.
P.S. the issue is related to all file types (extensions) except *.txt.
Update
Server side code:
public HttpResponseMessage DownloadAttachment(Guid fileId)
{
var stream = GetFileStream(fileId);
var message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = file.Size;
message.Content.Headers.ContentType = new MediaTypeHeaderValue(file.ContentType)
{
// without this charset files sent with bigger size
// than they are as shown on image 1
CharSet = "ANSI"
};
message.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = file.FileName + file.Extension,
Size = file.Size
};
return message;
}
Client side code (TypeScript):
/*
* Handler for click event on download <a> tag
*/
private downloadFile(file: Models.File) {
var self = this;
this.$service.downloadAttachment(this.entityId, file.fileId).then(
// on success
function (data, status, headers, config) {
var fileName = file.fileName + file.extension;
var clientFile = new File([data], fileName);
// here's the issue ---^
saveAs(clientFile, fileName);
},
// on fail
function (error) {
self.alertError(error);
});
}
My code is almost the same as in answers on related questions on SO: instead of setting direct link in 'a' tag, I handle click on it and download file content via XHR (in my case using Angularjs $http service). Getting the file content I create a Blob object (in my case I use File class that derives from Blob) and then try to save it using FileSaver.js. I also tried approach with encoded URL to Blob in href attribute, but it only opens a new tab with a file broken the same way. I found that the problem is in Blob class - calling it's constructor with 'normal' file data I get an instance with 'wrong' size as could be seen on first two screenshots. So, as I understand, my problem not in the way I try to save my file, but in the way I create it - File API

check uploaded file format on client side

I am creating a web portal where end user will upload a csv file and I will do some manipulation on that file on the server side (python). There is some latency and lag on the server side so I dont want to send the message from server to client regarding the bad format of uploaded file. Is there any way to do heavy lifting on client side may be using js or jquery to check if the uploaded file is "comma" separated or not etc etc?
I know we can do "accept=.csv" in the html so that file extension has csv format but how to do with contents to be sure.

Accessing local files from Javascript is only possible by using the File API (https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications) - by using this you might be able to check the content whether it matches your expectations or not.

Here's some bits of code I used to display a preview image clientside when a file is selected. You should be able to use this as a starting point to do something else with the file data. Determining whether its csv is up to you.
Obvious caveat:
You still have to check server side. Anyone can modify your clientside javascript to pretend a bad file is good.
Another caveat:
I'm pretty sure that you can have escaped comma characters in a valid csv file. I think the escape character might be different across some implementations too...
// Fired when the user chooses a file in the OS dialog box
// They will have clicked <input id="fileId" type="file">
document.getElementById('fileId').onchange = function (evt) {
if(!evt.target.files || evt.target.files.length === 0){
console.log('No files selected');
return;
}
var uploadTitle = evt2.target.files[0].name;
var uploadSize = evt2.target.files[0].size;
var uploadType = evt2.target.files[0].type;
// To manipulate the file you set a callback for the whole contents:
var FR = new FileReader();
// I've only used this readAsDataURL which will encode the file like data:image/gif;base64,R0lGODl...
// I'm sure there's a similar call for plaintext
FR.readAsDataURL($('#file')[0].files[0]);
FR.onload = function(evt2){
var evtData = {
filesEvent: evt,
}
var uploadData = evt2.result
console.log(uploadTitle, uploadSize, uploadType, uploadData);
}
}

How to serve static files in node.js after renaming them?

I am creating a node.js app in which user can upload files and can download later.
I am storing file information (original file name that user uploaded, ...) in mongodb document and named that file same as mongodb document id. Now i want my user to be able to download that file with the original file name.
What i want to know is when a user sends a GET request on http://myapp.com/mongoDocument_Id
user gets a file named myOriginalfile.ext
I know about node-static and other modules but i can't rename them before sending file.
i am using koa.js framework.

Here's a simple example using koa-file-server:
var app = require('koa')();
var route = require('koa-route');
var send = require('koa-file-server')({ root : './static' }).send;
app.use(route.get('/:id', function *(id) {
// TODO: perform lookup from id to filename here.
// We'll use a hardcoded filename as an example.
var filename = 'test.txt';
// Set the looked-up filename as the download name.
this.attachment(filename);
// Send the file.
yield send(this, id);
}));
app.listen(3012);
In short:
the files are stored in ./static using the MongoDB id as their filename
a user requests http://myapp.com/123456
you look up that ID in MongoDB to find out the original filename (in the example above, the filename is just hardcoded to test.txt)
the file ./static/123456 is offered as a download using the original filename set in the Content-Disposition header (by using this.attachment(filename)), which will make the browser store it locally as test.txt instead of 123456.

File Download at front end using angular js

On click of download button beside name of a file I am getting the file content from backend . This content is in the bytes . The file can be of any type . How to convert the data received from backend to file and download it automatically on receiving response(file Content) .New bee in html and angular js .Any pointers or suggestions needed :)

You will need to have your backend tell your front-end the location of the file, and then the front end can place a link to the file. The backend should probably generate a unique hash name for this file.
The actual file can be returned as part of a Rest GET request as long as the backend webserver has the correct mime types configured.
So in your controller you would call a service to get the file path:
SomeController.$inject = ['$http'];
var SomeController = function($http) {
var self = this;
$http.get('/download-file-path').then(function(path) {
self.path = path;
}
}
Then in your view
<div ng-controller='SomeController as vm'>
<a ng-href="{{vm.path}}">Download</a>
</div>
When angular calls GET: /download-file-path the backend should return the name and path of the file, say something like /download/file-7dje79ae.xml. Then angular puts this path in the a link. When the user clicks the download button, the users browser will then make a request to /download/file-7dje79ae.xml and download the file.

Upload file to Windows Azure with only the link of the file is provided

How can I upload a file in azure if I only have the URL of the file to upload. In this case, i 'm using Dropbox file chooser which selects file from dropbox and returns its url path.
eq
https://www.dropbox.com/s/o9myet72y19iaan/Getting%20Started.pdf
Now we need the file to be stored in Windows Azure blob. What is the easiest way to do this without downloading the file first.
I'm planning to use a asp.net web api for the uploading of file to azure blob.

At first, I thought it should be quite straight forward as Azure Blob Storage support copying blobs from external URL however I don't think this would work in case of Dropbox files. I just tried it and got an error even though.
The link you mentioned above is not the direct link to the file. It's a link to a page on Dropbox's website from where you can download a file. This is obviously you don't want. Here's an alternate solution which you can try:
Replace www.dropbox.com in your URL with dl.dropboxusercontent.com (based on #smarx's comments below) and use that URL in the following code:
First you would need to append dl=1 to your request URL as query string. So your Dropbox URL would be https://www.dropbox.com/s/o9myet72y19iaan/Getting%20Started.pdf?dl=1. dl query string parameter indicates the file needs to be downloaded.
Next, using HTTPWebRequest try accessing this URL. Dropbox will respond back with another link and 302 status code. This link would be something like https://dl.dropboxusercontent.com/s/o9myet72y19iaan/Getting%20Started.pdf?token_hash=<tokenhash>.
Use this link in the code below to copy file. This would work.
CloudStorageAccount acc = new CloudStorageAccount(new StorageCredentials("account", "key"), false);
var client = acc.CreateCloudBlobClient();
var container = client.GetContainerReference("container-name");
container.CreateIfNotExists();
var blob = container.GetBlockBlobReference("dropbox-file-name");
blob.StartCopyFromBlob(new Uri("dropbox URL with dl.dropboxusercontent.com"));
Console.WriteLine("Copy request accepted");
Console.WriteLine("Now checking for copy state");
bool continueLoop = true;
do
{
blob.FetchAttributes();
var copyState = blob.CopyState;
switch (copyState.Status)
{
case CopyStatus.Pending:
Console.WriteLine("Copy is still pending. Will check status again after 1 second.");
System.Threading.Thread.Sleep(1000);//Copy is still pending...check after 1 second
break;
default:
Console.WriteLine("Terminating process with copy state = " + copyState.Status);
continueLoop = false;
break;
}
}
while (continueLoop);
Console.WriteLine("Press any key to continue.");