Jquery ajax, trouble receiving huge amount of data - javascript

I need to receive big amount of data via ajax response and seems like some kind of data size limit on live server(nginx) is causing trouble. On local test server( wamp ) and live shared hosting apache I have no limits or issues. The data is currently 2.7mb and it could get bigger. Problem is that the script is going to be used by multiple users and I cant force anyone to increase the post_max_size.
The script is fairly simple
$.ajax({
type: "post",
dataType: "json",
url: ajaxurl,
data: {
'action': 'get_data',
},
success: function(response, status, xhr) {
if (response.data.html !== null) {
//process html 2.7MB
}
}
});
the reason why data is so big is json validation and html data must have special chars , so the response is almost 3 times of the size than what the actual normal processed HTML would be. This is just a small example of it
<\/div>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\t
right now I have no idea how to approach this. To brake the response in chunks or ? If you have any suggestion have at it please.
Update , we figured out that is some kind of data size limit but we cant nail it
if we do
wp_send_json_success(array(
'html' => substr($html, 0, 1830000),
));
data is passing trough
but with 1840000
it is failing ,
our test server is nginx and in litespeed config client_max_body_size is set at 500M so we are not sure what this 1.8M limit is.

Related

Insert Python's mpld3 images to HTML without saving them

I have a Python script, which does some computations and makes a figure. Then, I am using mpld3 to generate an HTML version of this figure and save it (e.g., figure.html). The Python computations are launched via the AJAX request to the HTTP server. In my JS file, I use the jQuery.load() method to insert the plot into HTML:
$("#includedContent").load('/figure.html');
In this way, everything seems to work fine. The question is can I do the same thing without saving the figures? I tried to return a figure as a response, here is my modified AJAX request:
$.ajax({
type: "POST",
contentType: "application/json; charset=utf-8",
dataType: "json",
url: "/main.py",
data: JSON.stringify({ 'sim_dur': input }),
success: function (response) {
figure = response.figure;
var child = document.createElement('div');
child.innerHTML = figure;
child = child.firstChild;
document.getElementById('includedContent').appendChild(child);
}
});
If I assign an HTML-like string to child.innerHTML, the code works correctly. But when I try to assign it a figure (which has a string type as well), it is not showing on the HTML page. I'm new to JS, so I apologize if it is something basic. I did some research and it seems that people usually save the results in other answers, so maybe it is a common way to do it and what I am trying to do is not right.
In my experience, mlpd3 will only work when the client (browser) is loading the page. Once the page is loaded, updating or creating an mpld3 figure using an ajax call won't create the figure for you. I personally struggled with that for some time and then gave up. I am sure one can patch things up but then you'll be stuck to your own mpld3 patched version

How to choose the right dataType and contentType value for AJAX request?

I have written some jQuery + PHP code that takes the HTML from an element on a webpage and saves it on server. Here is the code that I am using:
var page = {
'html': document.querySelector("article").innerHTML,
'url': 'path/current/webpage.php' or (<?php echo "'$current_page'"; ?>)
// Both give same 'url' value. This is not an issue.
};
$.ajax({
url:'https://example.com/update.php',
type:'post',
data: page,
success:function(data){
window.location.reload();
}
});
Here is my code for update.php:
$content = $_REQUEST['html'];
$page = $_REQUEST['url'];
file_put_contents($page, $content, LOCK_EX);
I am not very comfortable with dataType and contentType so I skipped them initially. However the request succeeded sometimes but gave 403() error other times.I did a little research and found that this might be due to lack of dataType and contentType. So, I used the following values:
contentType: 'text/plain; charset=utf-8',
dataType: 'html'
I no longer get any errors but the pages are not actually updating. I also tried setting the values to:
contentType:'application/json',
dataType: 'html'
This time too, I did not get any 403() errors but the page would not actually update.
Does the post data needs to be accessed differently based on the value of contentType like 'application/json' or 'text/plain; charset=utf-8'? Because the updates don't seem to show up on the webpage even with a 200 response code.
Using application/x-www-form-urlencoded; charset=UTF-8 updates some pages but gives 403() error for others.
As Rory said (as did I, in an answer I wrote then deleted when I saw his comment; he was right to comment instead), a 403 response code probably doesn't mean there's a problem with either dataType or contentType. You should look for other reasons the server would refuse to satisfy the request. For instance, as you're posting HTML, perhaps you (or your web host) has some kind of anti-script-injection protection going on. You'll have to track that down, perhaps with your hosting company.
But two things: Some info for completeness, and a potential workaround:
dataType is the type you expect back from the server. contentType is the type of data you're sending to the server.
For the request you're sending, leaving off contentType is correct, because the default jQuery will use is what PHP will expect to see.
You shouldn't have to specify dataType at all; instead, you should ensure the response carries the correct Content-Type header. That means ensuring that your server is configured correctly (for static content) and that your PHP code sets the correct header if necessary via header("Content-Type: data/type-here") The only reason for specifying dataType is if you don't control the server and you know it sends back the wrong type.
If you need to try to work around it, first ask: What if someone sends me malicious HTML directly, not through my web page? The answer is: You need to be careful with what you do with the HTML. For example: If you are going to store this HTML and then display it (as HTML) to a user, that's a Cross-Site Scripting vulnerability and you have to rigorously sanitize that HTML before doing that.
Do not proceed with any workaround until you've answered that question for yourself.
Okay, so in terms of working around it (once you have robust safeguards in place): You might send JSON rather than a standard form, in hopes that whatever is rejecting the forms won't look at it. To do that, you'd change your ajax call:
var page = {
html: document.querySelector("article").innerHTML,
url: <?php echo "'$current_page'"; ?>
};
$.ajax({
url:'https://example.com/update.php',
type:'post',
data: JSON.stringify(page),
contentType: 'application/json; charset=UTF8',
success:function(data){
window.location.reload();
}
});
Then on the PHP side, you'd read that JSON and parse it (reading code below taken from this answer):
$entityBody = json_decode(stream_get_contents(STDIN));
$content = $entityBody['html'];
$page = $entityBody['url'];
file_put_contents($page, $content, LOCK_EX);
Again: Please do not use this unless you have robust anti-XSS safeguards in place. And again, if you do haev robust anti-XSS safeguards in place, you might be able to just use a normal form by changing your server config.

How to do an ajax request with large parameters?

I am getting 'Request-URI Too Large' error when I tried to call a rails controller function from javascript with a large json parameter. I'm using Webrick http server. Is there any way to resolve this without changing the server?
I have something like:
$.ajax({
url: 'application/get_list',
data: { options : options_json, selected_option : selected_option_string},
success: function(data) {
// Insert the data to a div (returned data is a select tag with options)
},
type: 'get'
});
The simplest fix would be to change it to a POST request, and set up the action to handle that, and then you won't run into this error.
If you need to it to be a GET request, you can add a file called webrick.rb to the config\initializers directory with this content:
if defined?(WEBrick::HTTPRequest)
WEBrick::HTTPRequest.const_set("MAX_URI_LENGTH", 10240)
end
and if you keep getting the error, keep increasing the number 10240 until it works.
Since your comment says it needs to be a GET request, you really have no option but to set the MAX_URI_LENGTH. From the WEBrick source:
if #request_line.bytesize >= MAX_URI_LENGTH and #request_line[-1, 1] != LF
raise HTTPStatus::RequestURITooLarge
end
If you need really long URI's, then set it to something absurd, like 9223372036854775807

Can’t POST huge json data to PHP server with Jquery ajax function

I am trying to post some variables to PHP server via json. One of this variable contains huge string. When I post data all other variable get posted successfully but one that contains huge string remains undefined.
If I make content of that variable sort, then its running fine.
$.ajax({
type: 'POST',
url: url,
data:{name:”aaa”,age:”aaaaa”,detail:”huge string”},
async:false
}).done(function( data ) {
alert(data);
}).fail(function() {
$.notify("Cannot connect to server", "error");
});
please check your php.ini setting for directive
post_max_size
and try to increase like as below
ini_set('post_max_size' '20m')

Sending image data over AJAX from jquery

I need to send image data (data:image/png;base64) from the clientside using AJAX to my PHP server. My AJAX call looks like this:(form_data contains the image)
$.ajax({
url: global_siteurl+'/save_image',
data: form_data,
dataType: 'json',
type: 'post',
contentType: "application/x-www-form-urlencoded; charset=UTF-8",
success: function (retval) {
process_save_image(retval);
}
});
Then I store the encoded image data as a blob in the database (yes - long story behind that!). When I retrieve the image data it seems to be corrupted and does not display correctly. Almost as if there are line breaks and spaces introduced into the image data. Am I missing any parameters in my ajax call? Any ideas as to what may be going wrong? Is there a size limit for the image data I can send across?
It's been a long 4 days of chasing this one.
Mmiz
The problem turned out to be the same described (and solved) in this posting:
Blob data replace '+' with space
Turns out I needed to make the blob data safe for URLs when I GET/POST it. On the PHP server side I used the function described in the above posting. On the Javascript side, I used the functions from:
http://notepad2.blogspot.com/2012/08/javascript-make-base64-encoded-string.html
Took a lot of staring at the encoded image data to notice that the +/= were replaced.
Try this when showing image.
echo '<img src="data:image/png;base64,' . base64_encode($blob_data) . '"/>

Categories