We Keep Coding

.NET Framework MVC Large file upload

I am trying to upload large binary files from a web client to a .NET 4.6.1 Framework MVC API. These files could range anywhere from 5GB to 20GB.
I have tried splitting the file into chunks to upload each chunk and merge the results at the end, but the merged file is always corrupted. If I work with small files and don't split, the binary will work correctly. However, when I split and merge the file is "corrupted". It won't load or behave as expected.
I have looked all over and haven't seen a proper solution to this so i'm hoping someone can help me here.
I followed this https://forums.asp.net/t/1742612.aspx?How+to+upload+a+big+file+in+Mvc, but I can't get it to work and the corrected solution was never posted. I am keeping track of the order of files before merging on the server.
Javascript (Call to uploadData is made to initiate)
function uploadComplete(file) {
var formData = new FormData();
formData.append('fileName', file.name);
formData.append('completed', true);
var xhr3 = new XMLHttpRequest();
xhr3.open("POST", "api/CompleteUpload", true); //combine the chunks together
xhr3.send(formData);
return;
}
function uploadData(item) {
var blob = item.zipFile;
var BYTES_PER_CHUNK = 750000000; // sample chunk sizes.
var SIZE = blob.size;
//upload content
var start = 0;
var end = BYTES_PER_CHUNK;
var completed = 0;
var count = SIZE % BYTES_PER_CHUNK == 0 ? SIZE / BYTES_PER_CHUNK : Math.floor(SIZE / BYTES_PER_CHUNK) + 1;
while (start < SIZE) {
var chunk = blob.slice(start, end);
var xhr = new XMLHttpRequest();
xhr.onload = function () {
completed = completed + 1;
if (completed === count) {
uploadComplete(item.zipFile);
}
};
xhr.open("POST", "/api/MultiUpload", true);
xhr.setRequestHeader("contentType", false);
xhr.setRequestHeader("processData", false);
xhr.send(chunk);
start = end;
end = start + BYTES_PER_CHUNK;
}
}
Server Controller
//global vars
public static List<string> myList = new List<string>();
[HttpPost]
[Route("CompleteUpload")]
public string CompleteUpload()
{
var request = HttpContext.Current.Request;
//verify all parameters were defined
var form = request.Form;
string fileName;
bool completed;
if (!string.IsNullOrEmpty(request.Form["fileName"]) &&
!string.IsNullOrEmpty(request.Form["completed"]))
{
fileName = request.Form["fileName"];
completed = bool.Parse(request.Form["completed"]);
}
else
{
return "Invalid upload request";
}
if (completed)
{
string path = HttpContext.Current.Server.MapPath("~/Data/uploads/Tamp");
string newpath = Path.Combine(path, fileName);
string[] filePaths = Directory.GetFiles(path);
foreach (string item in myList)
{
MergeFiles(newpath, item);
}
}
//Remove all items from list after request is done
myList.Clear();
return "success";
}
private static void MergeFiles(string file1, string file2)
{
FileStream fs1 = null;
FileStream fs2 = null;
try
{
fs1 = System.IO.File.Open(file1, FileMode.Append);
fs2 = System.IO.File.Open(file2, FileMode.Open);
byte[] fs2Content = new byte[fs2.Length];
fs2.Read(fs2Content, 0, (int)fs2.Length);
fs1.Write(fs2Content, 0, (int)fs2.Length);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message + " : " + ex.StackTrace + " " + file2);
}
finally
{
if(fs1 != null) fs1.Close();
if (fs2 != null)
{
fs2.Close();
System.IO.File.Delete(file2);
}
}
}
[HttpPost]
[Route("MultiUpload")]
public string MultiUpload()
{
try
{
var request = HttpContext.Current.Request;
var chunks = request.InputStream;
string path = HttpContext.Current.Server.MapPath("~/Data/uploads/Tamp");
string fileName = Path.GetTempFileName();
string newpath = Path.Combine(path, fileName);
myList.Add(newpath);
using (System.IO.FileStream fs = System.IO.File.Create(newpath))
{
byte[] bytes = new byte[77570];
int bytesRead;
while ((bytesRead = request.InputStream.Read(bytes, 0, bytes.Length)) > 0)
{
fs.Write(bytes, 0, bytesRead);
}
}
return "test";
}
catch (Exception exception)
{
return exception.Message;
}
}

How to add an attachement to an email (not stored in file disk)

I want to open outlook with an attachement (zip) using Javascript.
It's working fine if the file in in some directory in the computer client.
this.sendEmail = function (zip) {
try {
var theApp = new ActiveXObject("Outlook.Application");
var objNS = theApp.GetNameSpace('MAPI');
var theMailItem = theApp.CreateItem(0); // value 0 = MailItem
theMailItem.to = ('test#gmail.com');
theMailItem.Subject = ('test');
theMailItem.Body = ('test');
theMailItem.Attachments.Add(zip);
theMailItem.display();
}
catch (err) {
alert(err.message);
}
};
But What I want to do is to add the file that I have alredy in memory
this.SenddZip = function (docDescription, fileName) {
var zip = new JSZip();
for (var i = 0; i < docDescription.length; i++) {
var doc = docDescription[i];
zip.file(doc.core_documenttitle + doc.core_fileextension, doc.fileBase64, { base64: true });
}
Utils.sendEmail(zip);
// Generate the zip file asynchronously
zip.generateAsync({ type: "blob" })
.then(function (content) {
// Force down of the Zip file
var name = "documents";
if (fileName) {
name = fileName;
}
// location.href = "data:application/zip;base64," + content;
saveAs(content, name + ".zip");
Utils.sendEmail(xxxxxxxx);
});
};
If their is no solution for this :
Is their a way to download the document without the confirmation dialog ?
saveAs ask always for confirmation before to store the file.
Thank you

Hi, is it possible to convert Blob/Base64 to fileURI format using javascript/angularjs and ionic1.

I am trying to capture images using cordovaCamera and imagePicker then remove the EXIF data and create a copy of the image that I get after the metadata is removed.
In the following code I get image file in Blob format which I pass to resolveLocalFileSystemURL but unable to proceed as resolveLocalFileSystemURL does not accept blob or base64 data so is it possible to convert Blob/Base64 to fileURI format. Or is there any alternative to solve this.
In html:
<input id="erd" type="file"/>
In controllers.js :
var input = document.querySelector('#erd');
input.addEventListener('change', load);
function load(){
var fr = new FileReader();
fr.onload = process;
fr.readAsArrayBuffer(this.files[0]);
console.log(" onload "+URL.createObjectURL(this.files[0]));
}
function process(){
var dv = new DataView(this.result);
var offset = 0, recess = 0;
var pieces = [];
var i = 0;
if (dv.getUint16(offset) == 0xffd8){
offset += 2;
var app1 = dv.getUint16(offset);
offset += 2;
while (offset < dv.byteLength){
if (app1 == 0xffe1){
pieces[i] = {recess:recess,offset:offset-2};
recess = offset + dv.getUint16(offset);
i++;
}
else if (app1 == 0xffda){
break;
}
offset += dv.getUint16(offset);
var app1 = dv.getUint16(offset);
offset += 2;
}
if (pieces.length > 0){
var newPieces = [];
pieces.forEach(function(v){
newPieces.push(this.result.slice(v.recess, v.offset));
}, this);
newPieces.push(this.result.slice(recess));
var br = new Blob(newPieces, {type: 'image/jpeg'});
console.log(" pieces.length "+URL.createObjectURL(br));
$scope.strimg=URL.createObjectURL(br).toString();
//var file = new File(URL.createObjectURL(br), "uploaded_file.jpg", { type: "image/jpeg", lastModified: Date.now() });
var myFile = blobToFile(br, "my-image.jpg");
console.log('myFile'+JSON.stringify(myFile));
$scope.strimg = myFile;
createFileEntry($scope.strimg);
}
}
}
function blobToFile(theBlob,fileName){
console.log('theBlob'+theBlob+fileName);
var b = theBlob;
b.lastModifiedDate = new Date();
b.name = fileName; //Cast to a File() type
return theBlob;
}
function createFileEntry(fileURI) {
window.resolveLocalFileSystemURL(fileURI, copyFile, fail);
}
// 5
function copyFile(fileEntry) {
console.log(" copyFile "+fileEntry);
var name = fileEntry.fullPath.substr(fileEntry.fullPath.lastIndexOf('/') + 1);
var newName = name;
$scope.filepathnew="file:///storage/emulated/0/Android/data/com.offermanagement.system/files/";
window.resolveLocalFileSystemURL($scope.filepathnew, function(fileSystem2) {
fileEntry.copyTo(
fileSystem2,
newName,
onCopySuccess,
fail
);
},
fail);
}
function onCopySuccess(entry) {
$scope.$apply(function () {
$rootScope.profilepic=entry.nativeURL;
$rootScope.images.push(entry.nativeURL);
$rootScope.items.push({src:entry.nativeURL,sub:'' });
});
}
function fail(error) {
console.log("fail: " + error.code);
}
I get the following error for the above code
"Uncaught TypeError: Wrong type for parameter "uri" of resolveLocalFileSystemURI: Expected String, but got Blob."

Cordova: Not able to copy file on Android using Cordova

Somehow I keep getting an "error code 5" when trying to set the following right.
What I want to do, is copy an existing file from the assets in android to an accessible spot on the android device to be able to share it across other apps (like mail).
Here is my code example:
window.requestFileSystem = window.requestFileSystem || window.webkitRequestFileSystem;
var storagefolder = cordova.file.dataDirectory;
var storagefolderpointer;
console.log("storage folder: " + storagefolder);
// Check for support.
if (window.requestFileSystem) {
console.log("filesystem beschikbaar");
var getFSfail = function () {
console.log('Could not open filesystem');
};
var getFSsuccess = function(fs) {
var getDIRsuccess = function (dir) {
console.debug('Got dirhandle');
cachedir = dir;
fileurl = fs.root.fullPath + '/' + storagefolder;
storagefolderpointer = dir;
};
var getDIRfail = function () {
console.log('Could not open directory');
};
console.debug('Got fshandle');
FS = fs;
FS.root.getDirectory(storagefolder, {create:true,exclusive:false}, getDIRsuccess, getDIRfail);
};
window.requestFileSystem(LocalFileSystem.PERSISTENT, 0, getFSsuccess, getFSfail);
setTimeout(function() {
console.log("directory beschikbaar");
var suc = function(entry){
var goe = function(){
console.log("copy success");
};
var fou = function(){
console.log("copy NOT NOT success");
};
entry.copyTo(storagefolder, "vcard.vcf", goe, fou);
};
var fai = function(e){
console.log("fail getFile: " + e.code);
};
window.resolveLocalFileSystemURL(storagefolderpointer + "www/visitekaart/vcard.vcf", suc, fai);
}, 1000);
} else {
console.log("filesystem NOT NOT NOT available");
}

have you use cordovaFile plugin instead?,you can use blob to read the content of your files than write a new one on android sdcard using cordovaFile plugin
$cordovaFile.writeFile('appdata/file.txt', blob, 0).then(function(fileEntry) {
//success
}, function(err) {
//err
}

How To Upload Multiple Files To Server Using Javascript

I am using PhoneGap, and uploading a file (using a HTTP POST) like this,
function uploadSingleFile()
{
var ft = new FileTransfer();
// set up parameters etc
ft.upload(imageName, "http://serviceaddress/UploadFile.ashx", win, fail, options);
}
function win(r)
{
// success callback
}
I am wanting to upload muliple files, so in the success callback I want to call the uploadSingleFile to move onto the next file.
How can I store which file I am up to? I am using the localStorage to store the file names. So I would want to do this,
upload file localStorage.file0
upload file localStorage.file1
upload file localStorage.file2
So all I would need to do would be to store the number on the end, 0, 1, etc of where we are up to. Do I need to use a global variable? Seems messy.
If only I could pass through to the success callback a number as a additional parameter?

Hmmm. Is the problem worth doubting? Just store an array of file names and use JSON.stringify / JSON.parse for conversion between array and string.

function uploadSingleFile(fileName) {
var ft = new FileTransfer();
ft.upload("fileUrl",
"server",
function (result , fileName) {
console.log(fileName + ' has been uploaded successfully to server');
},
function (error) {
console.log(error);
},
{fileName: fileName, fileKey: "file"});
}
function uploadFiles() {
var files = JSON.parse(localStorage.files);
for(var i=0; i < files.length; i++) {
uploadSingleFile(files[i]);
}
}
You can send the index of file as parameter to uploadSingleFile() then using it in console.log()

First add all your images to array :
var TemplstImg = [];
function UploadImages()
{
var lstImages = [localStorage.file0,localStorage.file1,localStorage.file2];
TemplstImg=lstImages ;
if (TemplstImg.length > 0) {
var img = TemplstImg.pop();
uploadPhoto(img);
}
}
function uploadPhoto(imageURI) {
imageURI = imageURI.ImageFile;
var options = new FileUploadOptions();
options.fileKey = "file";
options.fileName = imageURI.substr(imageURI.lastIndexOf('/') + 1);
options.mimeType = "image/jpeg";
var params = {};
params.value1 = "test";
params.value2 = "param";
options.params = params;
var ft = new FileTransfer();
ft.upload(imageURI, yourServerPath, winImg, failImg,options);
}
function winImg(r) {
if (TemplstImg.length == 0) {
alert ('Done , all files was uploaded');
} else {
var img = TemplstImg.pop();
uploadPhoto(img);
}
}
function failImg(error) {
alert("failImg An error has occurred: Code = " + error.code);
console.log("upload error source " + error.source);
console.log("upload error target " + error.target);
}