first of all it is a userscript and I can't change the server-side encoding.
My problem is that when using encodeURIComponent() for encoding POST params (later sent via xhr.setRequestHeader), the characters are encoded in utf-8, but the server needs to receive iso-8859-1 data. Is there an alternative to encodeURIComponent() that would encode in iso-8859-1 ?
.
To make sure you understand, here is an exemple:
A classic form on the website send é like this: yournewmessage:%E9
Ajax via xhr.send('yournewmessage='+encodeURIComponent('é')) sends this: yournewmessage:%E9%80%80
The server needs the former. Thanks to anyone who can help me.
So, I’ve since figured out this problem. What I did was searching for an equivalence between utf-8 and iso-8859-1, what I found was between utf-8 and cp1252 (Windows-1252) so there are two conversions, utf-8 to cp1252 and cp1252 to iso-8859-1 (these two having a lot of similarities)
http://pastebin.com/jTDqR2PQ
Ugly code, comments left in French, and unelegant solution, but I feel bad seeing this question unansered while I actually found a solution that works.
Related
I have to encode string in c# and decode it with javascript unescape function.
the javascript unescape is the only option since I am sending the string with get request to some api that using unescape to decoed it.
i tried almost everything
server.urlencode
WebUtility.HtmlEncode
and a lot other encoding! I even tried Uri.EscapeDataString using jscript
Nothing isn't encode like the "escape" function
Any idea How to make it work?
EDIT:
this is my code
string apiGetRequest = String.Format("http://212.00.00.00/Klita?name={0}&city={1}&CREATEBY=test ", Uri.EscapeDataString(name), Uri.EscapeDataString(city));
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(apiGetRequest);
req.GetResponse();
Can you give an example of the string you want do encode and the encoded result?
URLencoding is the correct encoding-type you need. Make sure, you don't double encode your string somewhere in your code.
You might need to use decodeURIComponent instead of unescape, since unescape is not UTF-8 aware, thus might result in in broken string after decoding.
See http://xkr.us/articles/javascript/encode-compare/ for more information.
EDIT:
I don't know much about asp, but it looks like your trying to access the url not with a browser but with your ASP-server-side application. Well, your server does not run any JS code. You will just retrieve the HTML markup and maybe some JS code as a big string. This code would be parsed and executed within a browser but not within ASP.
I'm dealing with a JSON file that i cannot modify, i have to keep it AS IS.
it contains text, with all the apostrophes converted to ’, and other special chars here and there...
what is that? unicode? how can i convert to the regular apostrophe?
i placed already the META tag utf-8 on the header but it doesn't seem to change anything...
What mime type is your JSON response being sent with? (Look in the headers in FireBug or the Developer Console.) It seems that you one of these steps is using a different encoding:
The JSON string generated by the web server
The mime type encoding sent along with the response
The mime type of your HTML page
The mime type for your JavaScript code
If you supply the community with actual code, or better yet a working reproducible test case, then the community can better help you.
what is that?
It is an attempt to interpret data stored in one character encoding as data stored in a different character encoding.
To ensure everything displays correctly you need to:
Pick an encoding (UTF-8 is a good bet)
Store everything in that encoding
Configure your editor to use it!
Configure your database (if applicable) to use it!
Ensure any server side code you use expects UTF-8 input and gives UTF-8 output
Configure your webserver to include charset=utf-8 on the Content-Type HTTP header
The W3C has a good introductory article on the subject, which has links to lots of useful further reading at the end.
I have a javascript script which is calling a php page to supply an ajax form with suggestions. The suggestions are returned fine by the php page, but for some reason, when i set the responsetext of the javascript object request as an element in my HTML page, all the special characters (ie. á or ã) show up as this question mark. Is there a function II must run on the response text of the request to make sure these are read properly?
Thanks.
If you are not serving your HTML pages as UTF-8, the browser will guess an encoding, typically a single-byte Windows codepage depending on the user's locale.
But this doesn't happen for AJAX. With XMLHttpRequest, unless you specifically state an encoding in the Content-Type: ...; charset= parameter, the browser will treat it as UTF-8. That means if you are actually serving Windows code page 1252 (Western European) content, you will get an invalid UTF-8 sequence and consequent question mark.
You don't want to be using a non-UTF-8 encoding! Make sure you are using UTF-8 throughout your application. Serve all your pages with Content-Type: text/html; charset=utf-8, store your data in UTF-8 tables, use mysql_set_charset() to choose UTF-8, etc.
In any case consider passing AJAX responses using JSON. The function json_encode() will create a JSON string that uses JavaScript escape sequences for non-ASCII characters, which avoids any problem of encoding mismatch. Also this is easier to extend to add functionality than returning raw HTML.
I would try, in your php script, to encode everything as html entities.
This can be easily tested by doing something like this before returning the results to javascript:
$results = htmlentities($htmlstring);
There's also the htmlspecialchars function you might try.
More about this here:
http://php.net/manual/en/function.htmlentities.php
I'm trying to display the pound symbol in HTML (from PHP) but all I get is a symbol with a question mark.
The following are things that I've tried.
In PHP:
header('Content-type: text/html; charset=utf-8');
In HTML, put this in the head tag:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
I tried displaying it using a javascript function which converts it to:
�
I suppose it would help if I knew what I was doing... but I guess that's why I'm asking this question :)
Educated guess: You have a ISO-8859-1 encoded pound sign in a UTF-8 encoded page.
Make sure your data is in the right encoding and everything will work fine.
Use £. I had the same problem and solved it using jQuery:
$(this).text('£');
If you try this and it does not work, just change the jQuery methods,
$(this).html('£');
This always work in all contexts...
1st: the pound symbol is a "special" char in utf8 encoding (try saving £$ in a iso-8859-1 (or iso-8859-15) file and you will get ä when encoding using header)
2nd: change your encoding to utf8 form the file.
there are plenty of methods to do it.
notepad and notepad++ are great sugestions.
3rd: use ob_start(); (in php) BEFORE YOU MAKE ANY OUTPUT if you are getting weird encoding errors, like missing the encoding sometimes.
and YES, this solves it!
this kind of errors occurs when a page is encoded in windows-1252(ANSI),ASCII,iso-8859-1(5) and then you have all the others in utf8.
this is a terrible error and can cause weird things like session_start(); not working.
4th: other php solutions:
utf8_encode('£');
htmlentities('£');
echo '£';
5th: javascript solutions:
document.getElementById('id_goes_here').innerText.replace('£','£');
document.getElementById('id_goes_here').innerText.replace('£',"\u00A3");
$(this).html().replace('£','£'); //jquery
$(this).html().replace('£',"\u00A3"); //jquery
String.fromCharCode('163');
you MUST send £, so it will repair the broken encoded code point.
please, avoid these solutions!
use php!
these solutions only show how to 'fix' the error, and the last one only to create the well-encoded char.
Have you tried displaying a £ ?
Here is an overwhelming list.
You could try using £ or £ instead of embedding the character directly; if you embed it directly, you're more likely to run into encoding issues in which your editor saves the file is ISO-8859-1 but it's interpreted as UTF-8, or vice versa.
If you want to embed it (or other Unicode characters) directly, make sure you actually save your file as UTF-8, and set the encoding as you did with the Content-Type header. Make sure when you get the file from the server that the header is present and correct, and that the file hasn't been transcoded by the web server.
Or for other code equivalents try:
£
£
You need to save your PHP script file in UTF-8 encoding, and leave the <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> in the HTML.
For text editor, I recommend Notepad++, because it can detect and display the actual encoding of the file (in the lower right corner of the editor), and you can convert it as well.
This works in all chrome, IE, Firefox.
In Database > table > field type .for example set the symbol column TO varchar(2) utf8_bin
php code:
$symbol = '£';
echo mb_convert_encoding($symbol, 'UTF-8', 'HTML-ENTITIES');
or
html_entity_decode($symbol, ENT_NOQUOTES, 'UTF-8');
And also make sure set the HTML OR XML encoding to encoding="UTF-8"
Note: You should make sure that database, document type and php code all have a same encoding
How ever the better solution would be using £
I have an html page were i can fill in some text and send (with javascript) this to an sql-database.
On my pc, everything works fine, but on another one (a french windows), it doesn't save my chars correctly.
french chars like é, è, â,.. were saved as 'É', or something like that.
I googled a lot but still did not found any solution, i'm also not able to reproduce the problem on my own pc..
"É" occurs when a character encoded in utf-8 (2 bytes) is read as latin (1 byte). The problem can be on the client side (e.g. by the use of escape) or on the server side (wrong parsing of the form's POST data, database encoding).
Make sure that your html pages encoding is set to something like UTF-8, UTF-16, etc... Also make sure that your strings are escaped properly in javascript.
You need to encode the file in ANSI. I do this my self. For example in notepad 2 you would click File->Encoding->ANSI and then save.