I have an interesting problem - I have my own IIS 2016 server, that I use to host a website which allows the users to upload a variety of files - some in text format, the others zip'ed up together. Initially the website would return error 500 from the server when trying to upload something bigger, like ~50MB. I Googled up that IIS requires configuration of maxAllowedContentLength (changed default to 209715200, ~300MB) and FastCGI's parameters for IDLE, ACTIVITY and REQUEST (changed to 600) in order to allow bigger files upload without hitting the file size limit. However, now that the files are getting uploaded, the upload speed for these bigger files slowed down to a crawl. Previously I could upload ~20MB files in 10sec on a local network, while now 50MB takes like ~160sec. Not a linear increase I would expect.
My website runs on Django, and my POST method file transfer is carried out by Ajax call in JS:
$('#sn').on('submit', function(event) {
event.preventDefault();
var post_data = new FormData($("#sn")[0]);
$.ajax({
xhr: function() {
var xhr = new window.XMLHttpRequest();
xhr.upload.addEventListener("progress", function(evt) {
var percent = Math.round(evt.loaded/evt.total * 100)
console.log(percent)
$('#query_button').attr('disabled', true)
$('#query_button').get(0).innerText = "Upload status: " + percent + '%'
}, false);
xhr.upload.addEventListener("load", function(evt) {
$('#query_button').get(0).innerText = "PLEASE WAIT..."
}, false);
return xhr;
},
url: '#',
type: "POST",
data: post_data,
processData: false,
contentType: false,
dataType: "json",
statusCode: {
200: function() {
// alert("Server received request and posted a response!");
},
404: function() {
alert("Error code 404: Page not found!");
},
408: function() {
alert("Error code 408: Request Timeout!");
},
500: function() {
alert("Error code 500: Internal server error!");
}
},
success: function(response) {
console.log(response)
}
});
})
Can anybody please tell me, if the Ajax call could be the culprit of this slow-down in upload speed, or if it is something else that needs to be adjusted on IIS?
Related
Problem:
I have a situation where I'd like to upload a file (pdf, image, etc.) to an API Endpoint that accepts one of these types of files. However, the file is located on another web service somewhere. I'm trying to devise a clever solution that will allow me to (a) download the remote file (and store it as bytes in memory or something) then (b) upload that file through the API.
I have jQuery code that demonstrates how to upload a local file using jQuery with no backend code, but I'd like to extend it to allow me to upload something that is stored remotely.
Constraints:
I don't want to use any backend infrastructure on my image uploading page (ie. no php, python, ruby, etc.)
I don't want the end user of my form to need to download the file to their machine and upload the file as a two-step process.
What I've got so far:
I've seen some solutions on SO that kind-of connect the dots here in terms of downloading a file as a bytearray, but nothing that demonstrates how you might upload that.
Download File from Bytes in JavaScript
jQuery-only File Upload to Stripe API*
Keep in mind, Stripe is the example I have, but I'd like to try and replicate this on say Imgur or another API (if I can get this working). Hopefully someone else has some ideas!
$('#fileinfo').submit(function(event) {
event.preventDefault();
var data = new FormData();
var publishableKey = 'pk_test_***';
data.append('file', $('#file-box')[0].files[0]);
data.append('purpose', 'identity_document');
$.ajax({
url: 'https://uploads.stripe.com/v1/files',
data: data,
headers: {
'Authorization': 'Bearer ' + publishableKey,
// 'Stripe-Account': 'acct_STRIPE-ACCOUNT-ID'
},
cache: false,
contentType: false,
processData: false,
type: 'POST',
}).done(function(data) {
$('#label-results').text('Success!');
$('#upload-results').text(JSON.stringify(data, null, 3));
}).fail(function(response, type, message) {
$('#label-results').text('Failure: ' + type + ', ' + message);
$('#upload-results').text(JSON.stringify(response.responseJSON, null, 3));
});
return false;
});
I actually got this working for Stripe by doing this:
https://jsfiddle.net/andrewnelder/up59zght/
var publishableKey = "pk_test_xxx"; // Platform Publishable Key
var stripeAccount = "acct_xxx"; // Connected Account ID
$(document).ready(function () {
$('#file-upload').on('submit', function (e) {
e.preventDefault();
console.log('Clicked!');
var route = $('#file-route').val(); // URL OF FILE
var fname = route.split("/").slice(-1)[0].split("?")[0];
var blob = fetchBlob(route, fname, uploadBlob);
});
});
function fetchBlob(route, fname, uploadBlob) {
console.log('Fetching...')
var oReq = new XMLHttpRequest();
oReq.open("GET", route, true);
oReq.responseType = "blob";
oReq.onload = function(e) {
var blob = oReq.response;
console.log('Fetched!')
uploadBlob(fname, blob);
};
oReq.send();
}
function uploadBlob(fname, blob) {
var fd = new FormData();
fd.append('file', blob);
fd.append('purpose', 'identity_document');
console.log('Uploading...');
$.ajax({
url: 'https://uploads.stripe.com/v1/files',
data: fd,
headers: {
'Authorization': 'Bearer ' + publishableKey,
'Stripe-Account': stripeAccount
},
cache: false,
contentType: false,
processData: false,
type: 'POST',
}).done(function(data) {
console.log('Uploaded!')
}).fail(function(response, type, message) {
console.log(message);
});
}
I am using Visual Studio 2013 with IIS 8.0 for development (local host) and am attempting to upload a file via jQuery Ajax, but it keeps throwing a 405 error stating that POST methods aren't allowed. I have tried adding multiple aspects to the Web.config, but none work. Here is my basic upload code (uploading an image converted from a Canvas) to a directory named myfiles:
function imageMe() {
var canvasImage = document.getElementById("c");
var img = canvasImage.toDataURL("image/png");
var filename = 'Test.png';
$.ajax({
url: 'myfiles/',
type: "POST",
data: img,
processData: false,
headers: {
},
success: function (result) {
console.log("Upload complete!");
},
error: function (error) {
console.log("Something went wrong!");
}
});
}
I have a couple of jQuery Ajax requests, which have to be synchronous, but they keep locking/freezing the browser, until the response is received. My main problem is, that until the response is received I have to display a spinning icon, but due to the freezing the spinner is not displayed and even if it miraculously is it doesn't animate.
This is the event displaying the spinner and sending the request:
$(document).on('click', '#open-button', function () {
var input = "some text";
var wrapper = $('#wrapperWindow');
wrapper.children().animate({
opacity: 0
}, 500);
wrapper.children().remove();
wrapper.append('<div id="loading-spinner" style="display:none;"></div>');
var spinner = $('#loading-spinner');
spinner.css({
backgroundImage: 'url("img/loading.gif")',
opacity: 0
});
spinner.show();
spinner.animate({
opacity: 1
}, 500);
var dataForTab = requestData(input); //<-- the request
if (dataForTab.length > 0) {
//do stuff
}
});
The request:
function requestData(input) {
var result = null;
$.ajax({
async: false,
type: "POST",
url: "/some/url?input=" + input,
dataType: "json",
retryLimit: 3,
success: function (json) {
result = json;
},
error: function (xhr, err) {
console.log(xhr);
console.log(err);
}
});
return result;
}
Until the request returns the received JSON data, everything stops moving. How can I fix this please?
That's the essence of synchronous requests, they are locking. You may want to try to move the requests to a web worker. Here's an example (not using XHR, but it can give you an implementation idea)
A web worker is implemented in a separate file, the scripting can look like:
onmessage = function (e) {
var result = null;
$.ajax({
async: false,
type: "POST",
url: "/some/url?input=" + input,
dataType: "json",
retryLimit: 3,
success: function (json) {
result = json;
postMessage({result: result});
},
error: function (xhr, err) {
postMessage({error: err});
}
});
}
Depending on your use case you can use something like
task.js Simplified interface for getting CPU intensive code to run on all cores (node.js, and web)
A example would be
// turn blocking pure function into a worker task
const syncWorkerRequest = task.wrap(function (url) {
// sync request logic
});
// run task on a autoscaling worker pool
syncWorkerRequest('./bla').then(result => {
// do something with result
});
You should not be doing this though, unless you need to do some heavy data processing, please use async requests.
I'm using jQuery to upload a photo, and have attached a listener for onprogress events. However even when uploading photos that are a few megabytes, the only onprogress event that gets fired is when its at 100%. I've seen other sites like dropbox and facebook show a much more fluid progress bar. How can I get more frequent updates on the upload progress?
Sample upload Code:
var file = $photoFile.get(0).files[0];
var fileBlob = file && file.slice();
var formData = new FormData();
var title = $photoTitle.val();
formData.append('file', fileBlob);
formData.append('title', title);
$.ajax({
url: '/api/v1/photo/submit',
data: formData,
cache: false,
contentType: false,
processData: false,
type: 'POST',
xhrFields: {
onprogress: function(ev) {
console.info('upload progress', ev);
if (ev.lengthComputable) {
var percentUploaded = Math.floor(ev.loaded * 100 / ev.total);
console.info('Uploaded '+percentUploaded+'%');
// update UI to reflect percentUploaded
} else {
console.info('Uploaded '+ev.loaded+' bytes');
// update UI to reflect bytes uploaded
}
}
}
}).done(function(response) {
// do stuff
}).fail(function() {
// handle error
});
I was able to do this borrowing code from this stackoverflow question.
The key mistake I was making was relying on the onprogress within the xhrFields property of the call to $.ajax. Instead, to get more frequent progress updates I passed a custom XMLHttpRequest using the xhr property:
$.ajax({
url: '/api/v1/photo/submit',
data: formData,
cache: false,
contentType: false,
processData: false,
type: 'POST',
xhr: function() {
var myXhr = $.ajaxSettings.xhr();
if (myXhr.upload) {
myXhr.upload.addEventListener('progress',function(ev) {
if (ev.lengthComputable) {
var percentUploaded = Math.floor(ev.loaded * 100 / ev.total);
console.info('Uploaded '+percentUploaded+'%');
// update UI to reflect percentUploaded
} else {
console.info('Uploaded '+ev.loaded+' bytes');
// update UI to reflect bytes uploaded
}
}, false);
}
return myXhr;
}
}).done(function(response) {
// do stuff
}).fail(function() {
// handle error
});
Warning: This leverages newer web technologies and doesn't work with older browsers, especially IE.
This is driving me bonkers!
Some of the users of my Chrome browser extension are having trouble with an ajax request (it works fine for me and most). The problem happens whenever they try to access the API provided by my webserver (HTTPS), but not when they try to access other ajax commands. I have confirmed that they can load the URL in a browser so its not a DNS issue.
The ajax error handler is being triggered but I can't figure out why. They can load the page fine in a browser. The 3 parameters given to the error handler are not helpful. Here's a scrrenshot of their log; the 3 paramaters given to the error handler can be seen in 3 lines after "Facebook session errored out"
Error log image screenshot
As you can see, I have put in all the error catching I can think of, still nothing! Source:
$.ajax({'url': url, type:'POST', data: data, timeout: SERVER_TIMEOUT, success: function(d) {
dlog('got login session data');
dlog(d);
try {
d = $.parseJSON(d);
callback(d);
} catch(e) {
errh('The JSON could not be parsed: '+d);
}
}, error: function(x,t,e){
dlog(x);
dlog(t);
dlog(e);
callback({});
},statusCode: {
404: function() { errh('page not found'); },
200: function() { errh('200'); },
201: function() { errh('201'); },
202: function() { errh('202'); },
500: function() { errh('internal server error'); }
}});