We Keep Coding

Chunked File Uploads have different file size/corrupt

I'm using jquery-file-upload and Python-Flask on the server side. Whenever I upload a large 100mb+ file, the uploaded version is slightly larger than the original and does not open (is corrupt). I have chunking enabled for large files at 10mb chunks, I've tried to set "disableImageResize" to "true" as well as tried single and multiple files and the result has been the same. Am I missing something in my code?
main.js
$(function () {
'use strict';
// Initialize the jQuery File Upload widget:
$('#fileupload').fileupload({
// Uncomment the following to send cross-domain cookies:
//xhrFields: {withCredentials: true},
url: 'rs_upload',
disableImageResize: true,
sequentialUploads: true,
// redirect: 'home',
done: function (e, data) {
console.log("uploaded: " + data.files[0].name)
}
, maxChunkSize: 10000000, // 10 MB,
}).bind('fileuploadstop', function (e, data) {
if (data.loaded == data.total){window.location.replace("rs_create")}
});
views.py
#app.route("/rs_upload", methods=["GET", "POST"])
def rs_upload():
if request.method == 'POST':
files = request.files['file']
fs = files
handle_file(fs)
fullpath = session.get('finalpath')
if 'Content-Range' in request.headers:
# extract starting byte from Content-Range header string
range_str = request.headers['Content-Range']
start_bytes = int(range_str.split(' ')[1].split('-')[0])
# append chunk to the file on disk, or create new
with open(fullpath, 'a') as f:
f.seek(start_bytes)
f.write(fs.stream.read())
else:
# this is not a chunked request, so just save the whole file
fs.save(fullpath)
return jsonify({"name": fs.filename,
"size": os.path.getsize(fullpath),
"url": 'uploads/' + fs.filename,
"thumbnail_url": None,
"delete_url": None,
"delete_type": None,})
return render_template('remote_sensing/upload.html')

not sure if this is the issue, but would try
with open(fullpath, 'ab') as f:
to open & append to the file in binary mode.

Unable to view image on thumbnail in fine-uploader

Hi I am working on upload feature which has been done successfully using fine uploader, but for new functionality for edit i searched for same plugin and found that session can handle this functionality.
but i am not getting view of image in fine uploader section as below is the view i am getting.
I am passing name,uuid and thumbnailUrl as response.
Edited:
At Server Side:
List<PropertyImageEntity> propertyImageEntity=propertyService.getImagesUrlNames(Integer.parseInt(request.getParameter("id")),Constant.PROP_VAL);
for(PropertyImageEntity propertyImagesDetails: propertyImageEntity)
{
ImageDataResponse imageResponseData=new ImageDataResponse();
imageResponseData.setName(propertyImagesDetails.getFilename());
String test=String.valueOf(UUID.randomUUID());
imageResponseData.setUuid(this.uuid);
imageResponseData.setId(String.valueOf(propertyImagesDetails.getImageid()));
imageResponseData.setSize(propertyImagesDetails.getSize());
imageResponseData.setStatus("upload successful");
imageResponseData.setThumbnailUrl(propertyImagesDetails.getUrl());
imageResponse.add(imageResponseData);
}
at client side:
var manualUploader1 = new qq.FineUploader(
{
element : document
.getElementById('fine-uploader-manual-trigger1'),
template : 'qq-template-manual-trigger1',
request : {
endpoint : '/server/uploads?${_csrf.parameterName}=${_csrf.token}&id=${id}'
},
thumbnails : {
placeholders : {
waitingPath : '../assets/js/property/fileupload/placeholders/waiting-generic.png',
notAvailablePath : '../assets/js/property/fileupload/placeholders/not_available-generic.png'
}
},
validation : {
allowedExtensions : [ 'png', 'jpeg', 'jpg' , 'gif'],
itemLimit : 6,
sizeLimit : 100000000
},
autoUpload : false,
debug : true,
callbacks: {
onError: function(id, name, errorReason, xhrOrXdr) {
$("#errorMsg4").html(errorReason);
}
},
session: {
endpoint: '/server/get?id=${id}',
params: {},
customHeaders: {},
refreshOnReset: true
},
messages: {
typeError: jQuery.i18n.prop("invalid.extention.error"),
sizeError: jQuery.i18n.prop("upload.filesize.error"),
noFilesError: jQuery.i18n.prop("nofiles.toupload.error"),
tooManyItemsError: jQuery.i18n.prop("toomany.items.error"),
retryFailTooManyItems: jQuery.i18n.prop("retry.fail.error")
}
});
qq(document.getElementById("trigger-upload1")).attach("click",
function() {
$("#errorMsg4").html("");
manualUploader1.uploadStoredFiles();
});
but response for image url in console showing 200 ok.
Response:
[{"name":"b.png","uuid":"e3a5581e-aee9-4b8d-813f-63e0d400c9bc","thumbnailUrl":"http://192.168.1.68/html/1465290007617b.png","id":"84","size":26507,"status"
:null}]
Console Log:

The above problem was solved by adding cors headers in apache2.conf.
Header set Access-Control-Allow-Origin "*"
Thanks to #Ray for his answer on this post.

Dropzone.js v4+ - Display existing files on server with work limiting the number of files and other functions

How to add existing files on server to dropzone with right work all functions and right styling?

I wrote a function to add files: addCustomFile(file, thumbnail_url , responce)
Powered by Version: 4.0.1 stable
Correct working: maxFiles limit, event maxfilesexceeded, event success and others
$("#dropzone-images").dropzone({
url: "...",
paramName: 'image_temp',
maxFiles: 1,
init: function () {
this.addCustomFile = function(file, thumbnail_url , responce){
// Push file to collection
this.files.push(file);
// Emulate event to create interface
this.emit("addedfile", file);
// Add thumbnail url
this.emit("thumbnail", file, thumbnail_url);
// Add status processing to file
this.emit("processing", file);
// Add status success to file AND RUN EVENT success from responce
this.emit("success", file, responce , false);
// Add status complete to file
this.emit("complete", file);
}
this.addCustomFile(
// File options
{
// flag: processing is complete
processing: true,
// flag: file is accepted (for limiting maxFiles)
accepted: true,
// name of file on page
name: "The name",
// image size
size: 12345,
// image type
type: 'image/jpeg',
// flag: status upload
status: Dropzone.SUCCESS
},
// Thumbnail url
"http://.../img.jpg",
// Custom responce for event success
{
status: "success"
}
);
}
});

Plupload: perform custom check before starting upload

I have an MVC 5 view with a form and a plupload file uploader section. Upload is triggered by a button on the form. I have no problem uploading file chunks to the server and setting the parameters to the query string and all, but what I do have a problem with is starting the upload only after a custom sanity check has been performed.
Here's what I have tried:
var uploader = new plupload.Uploader({
runtimes: 'html5',
drop_element: 'upload',
browse_button: 'browse',
url: "../UploadFile",
chunk_size: "1024kb",
multipart_params: { "uid": "uid", "chunk": "chunk", "chunks": "chunks", "name": "name" },
init: {
PostInit: function(file) {
document.getElementById("filelist").innerHTML = "";
document.getElementById('submit-all').onclick = function () {
document.getElementById("infoPopup").style.visibility = "visible";
document.getElementById('submit-all').enabled = false;
var uuid = Math.uuidFast();
document.getElementById("uid").value = uuid;
uploader.settings.multipart_params = { uid: uuid, chunk: file.chunk, chunks: file.chunks, name: file.name };
if (checkReq) {
uploader.start();
}
return false;
};
},
The crucial part here is this:
if(checkReq){
uploader.start();
}
"checkReq" is my custom sanity check script that verifies that form values are not nonsensical (e.g. single form entries might be perfectly valid while in combination they are simply wrong, etc.).
So the above does not prevent the upload, the check script is not even fired, Firebug console output shows no error.
Since googling tells me that there is also a "BeforeUpload" event, I tried this:
BeforeUpload: function(up, file) {
if (checkReq) {
up.stop();
return false;
}
return true;
},
Which also does not seem to fire at all.
Edit: Next attempt, I put the call to my checkReq fuction into BeforeUpload in "preinit", which should fire before any chunking etc is done, so before the upload is prepared. This also failed although I have no idea why it does not fire:
var uploader = new plupload.Uploader({
runtimes: 'html5',
drop_element: 'upload',
browse_button: 'browse',
url: "../UploadFile",
chunk_size: "1024kb",
multipart_params: { "uid": "uid", "chunk": "chunk", "chunks": "chunks", "name": "name" },
preinit: {
BeforeUpload: function (up) {
if (checkReq) {
uploader.stop();
uploader.splice(0, uploader.files.length);
return false;
}
return true;
}
},
init: {
PostInit: function(file) {
...
I had used "dropzone.js" before, and my script worked fine with that but I found that I needed chunked uploads so I had to move to plupload and now my script is being ignored.
Could someone please tell me where I am being stupid here? Thanks!

Got it solved.
It's a nasty, ugly hack, but it works:
Made the "actual" submit/upload button hidden
Made a second button that acts as pre-submit button with onclick function
onclick function calls checkReq and if that returns true, the function calls the click() function of the "actual" submit/upload button
Like I said: nasty but it works.

Extjs 4 upload success function does not react

i have a grid with an toolbar and on that toolbar an upload option is added, so the upload is alright and it works , but after the file was uploaded to the server the success function does not react.
here my upload code:
upload: function () {
Ext.create('Ext.window.Window', {
title: 'Upload',
width: 300,
layout: 'fit',
draggable: false,
resizable: false,
modal: true,
bodyPadding: 5,
items: [{
xtype: 'form',
bodyPadding: 10,
frame: true,
items: [{
xtype:'filefield',
name:'file',
fieldLabel: 'File',
buttonText: 'Select File',
labelWidth: 30,
anchor: '100%'
}, {
xtype: 'button',
text: 'Upload',
handler: function(){
var form = this.up('form').getForm();
if(form.isValid()){
form.submit({
method: 'POST',
url: 'http://localhost:3000/upload',
success: function (form, action) {
Ext.Msg.alert('Success', 'Your File has been uploaded.');
console.log(action);
},
failure : function (form,action) {
Ext.Msg.alert('Error', 'Failed to upload file.');
}
})
}
}
}]
}],
}).show();
},
});
and the server response :
app.post('/upload', function(req, res) {
res.header('Access-Control-Allow-Origin', '*');
res.header('Content-Type','application/json; charset=UTF8');
var tmp_path = req.files.file.path;
var newPath = __dirname + '/files/' + req.files.file.name;
fs.rename(tmp_path, newPath, function (err){
if (err) throw err;
});
var path = newPath;
var name = req.files.file.name;
connection.query('SELECT name FROM audio WHERE name = ?', [name] , function(err,result) {
if (result[0]) {
console.log('File already exist');
res.status(400).send(JSON.stringify());
} else {
connection.query('INSERT INTO audio (name, path) VALUES (?,?)', [name,path], function (err,result) {
if (err) throw err;
var test = {
success: true
};
res.send({success:true});
console.log('success');
});
}
});
});
i can provide more code if necessary, thanks in advance

The error message is explicit: your response is lost due to cross-domain iframe issue.
See the doc explanation of how file upload form are handled: a hidden iframe is created to receive the response from the server (because, before HTML5 it was not possible to upload a file using XHR). When the iframe is loaded, Ext parses its content to read the response.
But, it is only allowed for a page to manipulate its iframes content if both are on the same domain, including the port number.
Most probably you're accessing your page at http://localhost/, while you're posting your form to http://localhost:3000. So forbidden: error, and no response for you!

This is a Ext js bug identified by Uberdude in the Sencha Forum.
Description of the problem :
When you make an Ext.Ajax.request with a form containing a file input to be uploaded, or manually set the isUpload option to true, rather than doing a proper Ajax request Ext submits the form in the standard HTML way to a dynamically generated hidden . The json response body is then read out of the iframe to create a faked-up Ajax response. A problem arises if the page making the upload Ajax request has changed its document.domain property, e.g. a page at home.example.com includes resources from static.example.com which it wishes to manipulate with javascript without violating the browser's same-origin-policy, so both set their document.domain to "example.com". If home.example.com then makes an upload Ajax request to a url on the home.example.com server, the iframe into which the response is written will have its document.domain as "home.example.com". Thus when the ExtJS code within Ajax.request on the home.example.com page tries to extract the document body from the iframe, it will be blocked by the same-origin-policy and the response passed to the callback functions will incorrectly have empty responseText.
Work Around :
1. Pass the document.domain to the server when making the upload request.
2. In your server response, add the document.domain in your response text/html.
response.setHeader('Content-Type', 'text/html');
response.write('document.domain = "' + params.__domain + '";');
response.write(JSON.stringify({msg: 'Welcome ' + params.name}));
response.end('');
Detail :
Please refer to :
http://www.sencha.com/forum/showthread.php?136092-Response-lost-from-upload-Ajax-request-to-iframe-if-document.domain-changed