I am sending an AJAX POST request using jQuery on a chrome extension but the data doesn't arrive as expected, accented characters turn out malformed.
The text "HÄGERSTEN" becomes "HÄGERSTEN".
The text appears fine in the console etc, only via AJAX to this other page it appears as mentioned. My AJAX call is basic, I send a data-object via jQuery $.ajax. I've tried both with and without contentType, UTF-8 and ISO-8859-1. No difference.
This is how I make my AJAX call:
var newValues = {name: 'HÄGERSTEN'}
$.ajax({
url: POST_URL,
type: 'POST',
data: newValues,
success: function() ...
});
The newValues object has more values but I retrieve them from a form. However, I have tried to specify these values manually as newValues['name'] = 'ÄÄÄÄ'; and still would cause the same problem.
The original form element of the page that I am sending the AJAX to contains attribute accept-charset="iso-8859-1". Maybe this matters.
The target website is using Servlet/2.5 JSP/2.1. Just incase it might make a difference.
I assume this is an encoding issue and as I've understood it should be because Chrome extensions require the script files to be UTF-8 encoded which probably conflicts with the website the plugin is running on and the target AJAX page (same website) which is using an ISO-8859-1 encoding, however I have no idea how to deal with it. I have tried several methods of decoding/encoding it to and from UTF-8 to ISO-8859-1 and other tricks with no success.
I have tried using encodeURIComponent on my values which only makes them show that way exactly on the form that displays the values I have sent via POST, as e.g. H%C3%84GERSTEN.
I have no access to the websites server and cannot tell you whether this is a problem from their side, however I would not suppose so.
UPDATE
Now I have understood POST data must be sent as UTF-8! So a conversion is not the issue?
Seems like the data is UTF-8 encoded when it is sent and not properly decoded on the server side. It has to be decoded on the server side. Test it out with the following encode and decode functions:
function encode_utf8(s) {
return unescape(encodeURIComponent(s));
}
function decode_utf8(s) {
return decodeURIComponent(escape(s));
}
var encoded_string = encode_utf8("HÄGERSTEN");
var decoded_string = decode_utf8(encoded_string);
document.getElementById("encoded").innerText = encoded_string;
document.getElementById("decoded").innerText = decoded_string;
<div>
Encoded string: <span id="encoded"></span>
</div>
<div>
Decoded string: <span id="decoded"></span>
</div>
We too faced the same situation but in our case we always sent the parameters using JSON.stringify.
For this you have to make changes, 1) While making call to the page via AJAX you have to add content-type tag defining in which encoding data is sent
$.ajax
({
type: "POST",
url: POST_URL,
dataType: 'json',//In our case the datatype is JSON
contentType: "application/json; charset=utf-8",
data: JSON.stringify(newValues),//I always use parameters to be sent in JSON format
EDIT
After reading your question more clearly I came to know that your server side JSP uses ISO-8859-1 encoding and reading some posts, I came to know that all POST method data will be transmitted using UTF-8 as mentioned.
POST data will always be transmitted to the server using UTF-8 charset, per the W3C XMLHTTPRequest standard
But while reading post jquery-ignores-encoding-iso-8859-1 there was a workaround posted by iappwebdev which might be useful and help you,
$.ajax({
url: '...',
contentType: 'Content-type: text/plain; charset=iso-8859-1',
// This is the imporant part!!!
beforeSend: function(jqXHR) {
jqXHR.overrideMimeType('text/html;charset=iso-8859-1');
}
});
Above code is taken from Code Posted by iappwebdev
I don't know if it could have been solved using POST-data and AJAX. Perhaps if I made a pure JavaScript XHR AJAX call, I might be able to send POST-data encoded the way I like. I have no idea.
However, in my desperation I tried my final option (or what seemed like it); send the request as GET-data. I was lucky and the target page accepted GET-data.
Obviously the problem still persisted as I was sending data the same way, being UTF-8 encoded. So instead of sending the data as an object I parsed the data into a URL friendly string with my own function using escape, making sure they are ISO-8859-1 friendly (as encodeURIComponent encodes the URI as UTF-8 while escape encodes strings making them compatible with ISO-8859-1).
The simple function that cured my headaches:
function URLEncodeISO(values) {
var params = [];
for(var k in values) params[params.length] = escape(k) + '=' + escape(values[k]);
return params.join('&');
}
The client side character coding is not completely up to you (immagine the usage of the page from different users all around the world: chinese, italian...) while on the server side you need to handle the coding for your purposes.
So, the data in the Ajax-POST can continue to be UTF8-encoded, but in your server side you coan to do the following:
PHP:
$name = utf8_decode($_POST['name']);
JSP:
request.setCharacterEncoding("UTF-8");
String name = request.getParameter("name");
Related
let quotesData;
function getQuotes(){
return $.ajax({
headers:{
Accept:"application/json"
},
url:'https://gist.githubusercontent.com/camperbot/5a022b72e96c4c9585c32bf6a75f62d9/raw/e3c6895ce42069f0ee7e991229064f167fe8ccdc/quotes.json',
success: function(jsonQuotes){
if(typeof jsonQuotes==="string"){
quotesData = JSON.parse(jsonQuotes);
console.log('quotesData');
console.log(quotesData);
}
}
})
}
getQuotes()
I was trying to complete project on fcc.Didn't know how to add quotes in html.Looked up in their premade project found this piece.Can someone explain what is going on in this code?
It's a hack to get around an incorrect Content-Type.
gist.githubusercontent.com isn't designed to host JSON data. It serves up plain text documents.
The success function tests to response to see if jQuery parsed it from JSON (which it will do if the Content-Type response header says it is) and if the response is a string it assumes it is unparsed JSON.
This is risky. A valid JSON text might encode only a single string (and trying to double parse it will error) and the document might not be JSON in the first place (which would also cause an error).
A better solution would be to use an appropriate hosting service.
I hope you are fine guys. I am trying to consume an API written by my colleague with the use of AngularJS $http. It is a PATCH method type. One of the parameters is passed in the URL (SomeMethod/User/FHD&TE) and the other one in the body (JSON). The one in the body is obviously fine, but there is an issue with the one that is passed in the URL, due to some special characters coming up in there from time to time. The API response code is 400 and it says "A potentially dangerous Request.Path value was detected from the client". I tried to use escape and encoding JS functions but none of them worked. What is more, there is an option params in the angular.http that I use to make the requests. As far as I know it takes care of encoding the parameters (solves the special characters issue), but I cannot use it, cause the URL would be different from the one that API expects. It would be "SomeMethod/User?FHDTE%" instead of "SomeMethod/User/FHD&TE" and obviously reply with 404. Do you now if this can be solved on the client side and if yes how can I do it? Or does it need an update of the backend?
Sample Request:
return $http({
method: 'PATCH',
url: ...SomeMethod/User/firstURLParameter,
headers: { 'Content-Type': 'application/json' },
data: { "secondBodyParameter": true }
});
If server side is written in Nodejs try using unescape function for your recieved parameters.
var str = Need%20tips%3F%20Visit%20W3Schools%21;
var str_1 = unescape(str)
where str1 result will be "Need tips? Visit W3Schools!"
Examples from w3schools https://www.w3schools.com/jsref/jsref_unescape.asp
I am using Java 1.7 and struts 1.3 framework. I am working for Japanese client. Currently my requirement is to send Search key (containing Japanese string) to the Action class using JQuery Ajax call. But at the action side I found some Japanese character are corrupted.
My code:
var searchKey = $('#searchtxt').val();
// some Japanese string value for search.
var data = {
// other properties
"searchKey": searchKey,
// Other properties
};
$.ajax({
type: 'POST',
url: url,
data: data,
contentType: "application/x-www-form-urlencoded;charset=UTF-8",
success: function (resultData){//dostuff}
});
I am quite new to SO I don't know how to format.
I have tried many solution on SO but not work for me. Any help will be appreciated.
Thank you for any help.
To solve this Japanese encoding problem, use URL Encoding mechanism and then send data through ajax call. And then at the Struts action side simply you need to decode it by using URL decoder mechanism.
It will solve this problem.
For the more clarity see the below code.
At the Java script side while fetching data from the hidden field use URL encoding method:
var searchKey = encodeURIComponent($('#searchtxt').val().trim());
// It will encode the Japanese string before send from Ajax call.
At the Struts Action side use URLDecoder class to decode the string value:
String searchKey=form.getSearchKey();
if(!searchKey.isEmpty()) //Check for empty or null string
{
// Decode the string using URLDecoder class from java.net package
form.setSearchKey(URLDecoder.decode(searchKey, "UTF-8"));
}
I have written some jQuery + PHP code that takes the HTML from an element on a webpage and saves it on server. Here is the code that I am using:
var page = {
'html': document.querySelector("article").innerHTML,
'url': 'path/current/webpage.php' or (<?php echo "'$current_page'"; ?>)
// Both give same 'url' value. This is not an issue.
};
$.ajax({
url:'https://example.com/update.php',
type:'post',
data: page,
success:function(data){
window.location.reload();
}
});
Here is my code for update.php:
$content = $_REQUEST['html'];
$page = $_REQUEST['url'];
file_put_contents($page, $content, LOCK_EX);
I am not very comfortable with dataType and contentType so I skipped them initially. However the request succeeded sometimes but gave 403() error other times.I did a little research and found that this might be due to lack of dataType and contentType. So, I used the following values:
contentType: 'text/plain; charset=utf-8',
dataType: 'html'
I no longer get any errors but the pages are not actually updating. I also tried setting the values to:
contentType:'application/json',
dataType: 'html'
This time too, I did not get any 403() errors but the page would not actually update.
Does the post data needs to be accessed differently based on the value of contentType like 'application/json' or 'text/plain; charset=utf-8'? Because the updates don't seem to show up on the webpage even with a 200 response code.
Using application/x-www-form-urlencoded; charset=UTF-8 updates some pages but gives 403() error for others.
As Rory said (as did I, in an answer I wrote then deleted when I saw his comment; he was right to comment instead), a 403 response code probably doesn't mean there's a problem with either dataType or contentType. You should look for other reasons the server would refuse to satisfy the request. For instance, as you're posting HTML, perhaps you (or your web host) has some kind of anti-script-injection protection going on. You'll have to track that down, perhaps with your hosting company.
But two things: Some info for completeness, and a potential workaround:
dataType is the type you expect back from the server. contentType is the type of data you're sending to the server.
For the request you're sending, leaving off contentType is correct, because the default jQuery will use is what PHP will expect to see.
You shouldn't have to specify dataType at all; instead, you should ensure the response carries the correct Content-Type header. That means ensuring that your server is configured correctly (for static content) and that your PHP code sets the correct header if necessary via header("Content-Type: data/type-here") The only reason for specifying dataType is if you don't control the server and you know it sends back the wrong type.
If you need to try to work around it, first ask: What if someone sends me malicious HTML directly, not through my web page? The answer is: You need to be careful with what you do with the HTML. For example: If you are going to store this HTML and then display it (as HTML) to a user, that's a Cross-Site Scripting vulnerability and you have to rigorously sanitize that HTML before doing that.
Do not proceed with any workaround until you've answered that question for yourself.
Okay, so in terms of working around it (once you have robust safeguards in place): You might send JSON rather than a standard form, in hopes that whatever is rejecting the forms won't look at it. To do that, you'd change your ajax call:
var page = {
html: document.querySelector("article").innerHTML,
url: <?php echo "'$current_page'"; ?>
};
$.ajax({
url:'https://example.com/update.php',
type:'post',
data: JSON.stringify(page),
contentType: 'application/json; charset=UTF8',
success:function(data){
window.location.reload();
}
});
Then on the PHP side, you'd read that JSON and parse it (reading code below taken from this answer):
$entityBody = json_decode(stream_get_contents(STDIN));
$content = $entityBody['html'];
$page = $entityBody['url'];
file_put_contents($page, $content, LOCK_EX);
Again: Please do not use this unless you have robust anti-XSS safeguards in place. And again, if you do haev robust anti-XSS safeguards in place, you might be able to just use a normal form by changing your server config.
The service API I am consuming has a given GET method that requires the data be sent in the body of the request.
The data required in the body is a list of id's separated by hypen and could potentially be very large and thus it must be sent in the body otherwise it will likely foobar somewhere in the browsers/proxies/webservers etc chain. Note I don't have control over the service or API so please don't make suggestions to change it.
I am using the following jQuery code however observing the request/response in fiddler I can see that the "data" I am sending is ALWAYS converted and appended to the query string despite me setting the "processData" option to false...
$.ajax({
url: "htttp://api.com/entity/list($body)",
type: "GET",
data: "id1-id2-id3",
contentType: "text/plain",
dataType: "json",
processData: false, // avoid the data being parsed to query string params
success: onSuccess,
error: onError
});
Anyone know how I can force the "data" value to be sent in the body of the request?
In general, that's not how systems use GET requests. So, it will be hard to get your libraries to play along. In fact, the spec says that "If the request method is a case-sensitive match for GET or HEAD act as if data is null." So, I think you are out of luck unless the browser you are using doesn't respect that part of the spec.
You can probably setup an endpoint on your own server for a POST ajax request, then redirect that in your server code to a GET request with a body.
If you aren't absolutely tied to GET requests with the body being the data, you have two options.
POST with data: This is probably what you want. If you are passing data along, that probably means you are modifying some model or performing some action on the server. These types of actions are typically done with POST requests.
GET with query string data: You can convert your data to query string parameters and pass them along to the server that way.
url: 'somesite.com/models/thing?ids=1,2,3'
we all know generally that for sending the data according to the http standards we generally use POST request.
But if you really want to use Get for sending the data in your scenario
I would suggest you to use the query-string or query-parameters.
1.GET use of Query string as.
{{url}}admin/recordings/some_id
here the some_id is mendatory parameter to send and can be used and req.params.some_id at server side.
2.GET use of query string as{{url}}admin/recordings?durationExact=34&isFavourite=true
here the durationExact ,isFavourite is optional strings to send and can be used and req.query.durationExact and req.query.isFavourite at server side.
3.GET Sending arrays
{{url}}admin/recordings/sessions/?os["Windows","Linux","Macintosh"]
and you can access those array values at server side like this
let osValues = JSON.parse(req.query.os);
if(osValues.length > 0)
{
for (let i=0; i<osValues.length; i++)
{
console.log(osValues[i])
//do whatever you want to do here
}
}
Just in case somebody ist still coming along this question:
There is a body query object in any request. You do not need to parse it yourself.
E.g. if you want to send an accessToken from a client with GET, you could do it like this:
const request = require('superagent');
request.get(`http://localhost:3000/download?accessToken=${accessToken}`).end((err, res) => {
if (err) throw new Error(err);
console.log(res);
});
The server request object then looks like {request: { ... query: { accessToken: abcfed } ... } }
You know, I have a not so standard way around this. I typically use nextjs. I like to make things restful if at all possible. If I need to make a get request I instead use post and in the body I add a submethod parameter which is GET. At which point my server side handles it. I know it's still a post method technically but this makes the intention clear and I don't need to add any query parameters. Then the get method handles a get request using the data provided in the post method. Hopefully this helps. It's a bit of a side step around proper protocol but it does mean there's no crazy work around and the code on the server side can handle it without any problems. The first thing present in the server side is if(subMethod === "GET"){|DO WHATEVER YOU NEED|}