i am passing a huge javascript array to php via the $.ajax method of jquery. The Content-Type is JSON and i sent the data as RAW.
In PHP i get the data with file_get_contents('php://input').
Everything is working very well, but on huge javascript arrays, for example 2,5MB, php receives only the small vars and the huge vars are stripped out. Before i send the data with jquery i do a JSON.stringify to send the data as JSON. PHP still will strip out all huge vars in this JSON string and if i get the contents of "php://input" only the small vars are left over....
In the php.ini i have raised up the max_post_size and other stuff, but nothing will work...
Does anybody know how i can handle this right?
thx!
Probably not the best idea but you could split your data, somthing like (not tested, might need a few corrections):
var json = HUGEJSON;
while (json.length != 0) {
var toSend = json.length >= 4096 ? json.substr(0, 4096) : json;
$.ajax({
url: 'server',
method:'post',
type:'json',
data: toSend
});
json = json.length >= 4096? json.substr(4096, json.length-4096) : json;
}
Then find a way to rebuild the data server side until all the data have been send.
Related
I am practicing ajax and want to send big amount of data to server (for instance I want to make tags for a post which can be up to 20 tags). Currently all I do is concatenate each tag with specific symbol between them and then in server I filter it and convert it to many tags again but I don't think that's the natural way. So what is the best way to send say, 30 - 40 entries to server with ajax optimally.
UPDATE (As some of the people suggested I am showing js code example):
$(document).ready(function(){
var tagsToSend = "tag1%tag2%tag3%tag4%tag5%tag6%tag7%tag8%tag9%tag10%tag11%tag12%tag13";
$.ajax({
url: "test.php",
method: "POST",
data: {
tags: tagsToSend
},
success: function(result){
alert(result)
}
});
})
So basically in server I'll just iterate over the given tags string and filter each tag. And I want more natural way.
I think the better way is to sending tags as a json array and not GET parameter. Something like this:
var postData = {};
postData['tagsToSend'] = ["tag1", "tag2", ...];
And inside your ajax config:
data: JSON.stringify(data)
Now, you can get a json in your php file and parse it into php array.
This can help you to have more readable and cleaner request to the server.
I have a JSON that is getJSONed to a PHP script in order to update the database for a simple CMS:
{
"pid": "3",
"post": "<p><span style='text-decoration: underline;'><strong>Test</strong></span></p><br><p>blah</p><br><p> </p><br><p><span style='text-decoration: underline;'><strong>Test</strong></span></p><br><p>blah</p>",
"tagline": "",
"title": "About"
}
In PHP it is decoded with json_decode and then sent off to the database.
It works perfectly and JSONLint reports that the JSON is valid, however PHP's json_decode fails with error 4, which is 'syntax error'. I'm unsure which is correct (I copied the JSON above from the GET request I send, so should be valid on JS' side).
I use JSON.stringify to create the JSON from my array on JS' side. The array is as follows:
var arr = {
pid : "<?php echo $pId; ?>",
post : $("#edit_target_preview").html(),
tagline : document.getElementById('page_tagline').value,
title : document.getElementById('page_title').value,
};
This array is forwarded to the PHP script via getJSON:
$.getJSON("savepost.php?json=" + JSON.stringify(arr), function(data){
*stuff happens here*
});
What am I doing wrong here? Am I overthinking this, or using completely the wrong approach?
Hexdump:
7b22706964223a2233222c22706f7374223a223c703e3c7370616e207374796c653d5c22746578742d6465636f726174696f6e3a20756e6465726c696e653b5c223e3c7374726f6e673e546573743c2f7374726f6e673e3c2f7370616e3e3c2f703e3c62723e3c703e626c61683c2f703e3c62723e3c703e
You're missing a call to encodeURIComponent (or the use of a decent serialiser) when composing you're URL. As your JSON contains a &, decoding on the server will truncate your data. It would have been obvious if you had logged the data before json_decode.
Note that it's most probably NOT a good idea to pass your JSON in the URL in a GET request (due to length restrictions). I'd strongly recommend using a POST request, which would also make a lot more sense semantically.
Since you're using getJSON which :
Load JSON-encoded data from the server using a GET HTTP request.
This is causing error, the JSON ends just before it at <p>, since you're doing a GET request and & marks the end of the previous parameter and the beginning of another parameter...
Just use post instead and it should work:
$.post("savepost.php", {json : JSON.stringify(arr)}, function(data) {
// Do something here
}, 'json');
I am sending an AJAX POST request using jQuery on a chrome extension but the data doesn't arrive as expected, accented characters turn out malformed.
The text "HÄGERSTEN" becomes "HÄGERSTEN".
The text appears fine in the console etc, only via AJAX to this other page it appears as mentioned. My AJAX call is basic, I send a data-object via jQuery $.ajax. I've tried both with and without contentType, UTF-8 and ISO-8859-1. No difference.
This is how I make my AJAX call:
var newValues = {name: 'HÄGERSTEN'}
$.ajax({
url: POST_URL,
type: 'POST',
data: newValues,
success: function() ...
});
The newValues object has more values but I retrieve them from a form. However, I have tried to specify these values manually as newValues['name'] = 'ÄÄÄÄ'; and still would cause the same problem.
The original form element of the page that I am sending the AJAX to contains attribute accept-charset="iso-8859-1". Maybe this matters.
The target website is using Servlet/2.5 JSP/2.1. Just incase it might make a difference.
I assume this is an encoding issue and as I've understood it should be because Chrome extensions require the script files to be UTF-8 encoded which probably conflicts with the website the plugin is running on and the target AJAX page (same website) which is using an ISO-8859-1 encoding, however I have no idea how to deal with it. I have tried several methods of decoding/encoding it to and from UTF-8 to ISO-8859-1 and other tricks with no success.
I have tried using encodeURIComponent on my values which only makes them show that way exactly on the form that displays the values I have sent via POST, as e.g. H%C3%84GERSTEN.
I have no access to the websites server and cannot tell you whether this is a problem from their side, however I would not suppose so.
UPDATE
Now I have understood POST data must be sent as UTF-8! So a conversion is not the issue?
Seems like the data is UTF-8 encoded when it is sent and not properly decoded on the server side. It has to be decoded on the server side. Test it out with the following encode and decode functions:
function encode_utf8(s) {
return unescape(encodeURIComponent(s));
}
function decode_utf8(s) {
return decodeURIComponent(escape(s));
}
var encoded_string = encode_utf8("HÄGERSTEN");
var decoded_string = decode_utf8(encoded_string);
document.getElementById("encoded").innerText = encoded_string;
document.getElementById("decoded").innerText = decoded_string;
<div>
Encoded string: <span id="encoded"></span>
</div>
<div>
Decoded string: <span id="decoded"></span>
</div>
We too faced the same situation but in our case we always sent the parameters using JSON.stringify.
For this you have to make changes, 1) While making call to the page via AJAX you have to add content-type tag defining in which encoding data is sent
$.ajax
({
type: "POST",
url: POST_URL,
dataType: 'json',//In our case the datatype is JSON
contentType: "application/json; charset=utf-8",
data: JSON.stringify(newValues),//I always use parameters to be sent in JSON format
EDIT
After reading your question more clearly I came to know that your server side JSP uses ISO-8859-1 encoding and reading some posts, I came to know that all POST method data will be transmitted using UTF-8 as mentioned.
POST data will always be transmitted to the server using UTF-8 charset, per the W3C XMLHTTPRequest standard
But while reading post jquery-ignores-encoding-iso-8859-1 there was a workaround posted by iappwebdev which might be useful and help you,
$.ajax({
url: '...',
contentType: 'Content-type: text/plain; charset=iso-8859-1',
// This is the imporant part!!!
beforeSend: function(jqXHR) {
jqXHR.overrideMimeType('text/html;charset=iso-8859-1');
}
});
Above code is taken from Code Posted by iappwebdev
I don't know if it could have been solved using POST-data and AJAX. Perhaps if I made a pure JavaScript XHR AJAX call, I might be able to send POST-data encoded the way I like. I have no idea.
However, in my desperation I tried my final option (or what seemed like it); send the request as GET-data. I was lucky and the target page accepted GET-data.
Obviously the problem still persisted as I was sending data the same way, being UTF-8 encoded. So instead of sending the data as an object I parsed the data into a URL friendly string with my own function using escape, making sure they are ISO-8859-1 friendly (as encodeURIComponent encodes the URI as UTF-8 while escape encodes strings making them compatible with ISO-8859-1).
The simple function that cured my headaches:
function URLEncodeISO(values) {
var params = [];
for(var k in values) params[params.length] = escape(k) + '=' + escape(values[k]);
return params.join('&');
}
The client side character coding is not completely up to you (immagine the usage of the page from different users all around the world: chinese, italian...) while on the server side you need to handle the coding for your purposes.
So, the data in the Ajax-POST can continue to be UTF8-encoded, but in your server side you coan to do the following:
PHP:
$name = utf8_decode($_POST['name']);
JSP:
request.setCharacterEncoding("UTF-8");
String name = request.getParameter("name");
I've been stuck on this for days now... Can't seem to figure out why I keep getting an undefined index error for this code, also why won't $text echo out anything?
$http({
url: url, url2,
method: "POST",
data: "data=" + window.btoa(encodeURIComponent(jsonData)),
headers: {'Content-Type': 'application/x-www-form-urlencoded'}
}).success(function (id, status, headers, config) {
console.log(data);
$scope.message = "From PHP file : "+data;
PHP:
$data = file_get_contents(json_decode(urldecode(base64_decode($_POST["data"]))));
$text = $data->text;
There's so much wrong with this it's hard to figure out where to begin. First and foremost, why are you using base64 encoding and uri encoding and json encoding?
Next, if you are going to use all those encoding functions, you need to call the exact equivalent decoding functions in the exact reverse order. This is not happening right now.
Next, url2 does not do anything here.
Then, file_get_contents needs a path as it's argument, not a string. file_get_contents is literally used to get the contents of a file, so it doesn't make any sense here. In addition file_get_contents returns a string, so it would never have a ->data member.
Last, undefined index implies that there is no data value in the $_POST object. This is probably because you are not sending the data across as form data. By default angular will encode data as json, which means that from PHP's perspective $_POST will be empty, and you can only get to the data via php://input.
All of this tells me that you basically slapped a whole bunch of stuff together from different sources hoping it would at one point work. The one advice I can give you is not just read my answer, but actually take the time to learn the stuff you are using. You're not going to solve your problems by guessing.
Also learn how to debug. The error you got is about an item not existing in an array. You could have validated this by checking out the full contents of $_POST. At that point you might not yet know why it went wrong, but the fact that you asked about this error and not about $_POST not containing data tells me that even accessing data in arrays is still foreign to you.
I have a limited experience with Angular, but that "url2" value seems off place to me:
$http({
url: url, url2,
I've been trying to pull data from the steam api, and have had no luck because I always get the above error. Here is the code I am using:
var steamurl = "https://api.steampowered.com/IDOTA2Match_570/GetMatchHistory/V001/?key=[keyomitted]&account_id=38440257&Matches_Requested=10";
function populate_api(){
var json;
$.ajax({
'url': steamurl,
'dataType': "jsonp",
'success': function (data) {
alert('success');
json = data;
}
});
}
I omitted my API key.
I have looked at many other posts, and cannot figure out where the problem is. I have tried using Jsonp, regular json, I have also tried using "&callback=?" after steamurl, but to no avail.
The solution for this is to add a local proxy that your jQuery code will call. Your proxy will be server side code (PHP, Python, Ruby, etc) that forwards the query on to Valve and then returns it to your jQuery call. You will, however, have to use one of their supported formats (of which JSONP is not one).
A high level view of what you'll be doing:
Create PHP file that accepts the parameters jQuery will be passing. In this case, it looks like account ID and matches you want to receive. Do not pass the API key, this should be stored in the PHP file
In PHP, build your steamurl using the stored API key, and the two passed values
Issue a call to the Valve servers using this steamurl and retrieve the results.
Return this response to your ajax call
Your PHP will look something like this (and should include more error checking since I am just taking the $_GET values as gospel:
$matches = $_GET['matches'];
$acct = $_GET['accountid'];
$APIKEY = <YOURKEYHERE>;
$steamurl = "https://api.steampowered.com/IDOTA2Match_570/GetMatchHistory/V001/?key=$APIKEY&account_id=$acct&Matches_Requested=$matches&format=json";
$json_object= file_get_contents($steamurl);
header('Content-Type: application/json');
echo $json_object;
Now you can use jQuery to parse this JSON response.