We Keep Coding

Why is split and join doing this?

I'm getting some very weird behaviour that I don't understand using JavaScript split and join. I'm sending names with spaces in an API call but using %3 as the delimiter for spaces, as in, if I'm sending "Ammar Ahmed", the API call would look like: api/v1?q=Ammar%3Ahmed. In the server code, when I split it up again with q.split("%3").join(" ") because the database contains names with spaces, for one name in particular: "Ashwini Bettahalsoor", I'm getting "Ashwini;ettahalsoor". I'm very confused why its doing this, its splitting it including the B and joining it with a ; but it works perfectly normal for all names that the last name does not start with B. I'm sure it has something to do with the letter B but first of all I'm curious as to why this is happening and secondly, I'm wondering what I should use instead of %3 for spaces in the API call.

%3 is not the correct encoding for a space. You're getting ; because %3B is the encoding for that character. URI encoding always uses 2 hex digits.
You should use encodeURICompnent() to generate the correct encoding.
let url = 'api/v1?q=' + encodeURIComponent('Ashwini Bettahalsoor');
And on the server you should use middleware that decodes the query parameters for you, rather than using split() and join() explicitly.

NodeJS escaping back slash

I am facing some issues with escaping of back slash, below is the code snippet I have tried. Issues is how to assign a variable with escaped slash to another variable.
var s = 'domain\\username';
var options = {
user : ''
};
options.user = s;
console.log(s); // Output : domain\username - CORRECT
console.log(options); // Output : { user: 'domain\\username' } - WRONG
Why when I am printing options object both slashes are coming?
I had feeling that I am doing something really/badly wrong here, which may be basics.
Update:
When I am using this object options the value is passing as it is (with double slashes), and I am using this with my SOAP services, and getting 401 error due to invalid user property value.
But when I tried the same with PHP code using same user value its giving proper response, in PHP also we are escaping the value with two slashes.

When you console.log() an object, it is first converted to string using util.inspect(). util.inspect() formats string property values as literals (much like if you were to JSON.stringify(s)) to more easily/accurately display strings (that may contain control characters such as \n). In doing so, it has to escape certain characters in strings so that they are valid Javascript strings, which is why you see the backslash escaped as it is in your code.

The output is correct.
When you set the variable, the escaped backslash is interpreted into a single codepoint.
However, options is an object which, when logged, appears as a JSON blob. The backslash is re-escaped at this point, as this is the only way the backslash can appear validly as a string value within the JSON output.
If you re-read the JSON output from console.log(options) into javascript (using JSON.parse() or similar) and then output the user key, only one backslash will show.
(Following question edit:)
It is possible that for your data to be accepted by the SOAP consuming service, the data needs to be explicitly escaped in-band. In this case, you will need to double-escape it when assigning the value:
var s = 'domain\\\\user'
To definitively determine whether you need to do this or not, I'd suggest you put a proxy between your working PHP app and the SOAP app, and inspect the traffic.

Converting textarea value into a valid JSON string

I am trying to make sure input from user is converted into a valid JSON string before submitted to server.
What I mean by 'Converting' is escaping characters such as '\n' and '"'.
Btw, I am taking user input from HTML textarea.
Converting user input to a valid JSON string is very important for me as it will be posted to the server and sent back to client in JSON format. (Invalid JSON string will make whole response invalid)
If User entered
Hello New World,
My Name is "Wonderful".
in HTML <textarea>,
var content = $("textarea").val();
content will contain new-line character and double quotes character.
It's not a problem for server and database to handle and store data.
My problem occurs when the server sends back the data posted by clients to them in JSON format as they were posted.
Let me clarify it further by giving some example of my server's response.
It's a JSON response and looks like this
{ "code": 0, "id": 1, "content": "USER_POSTED_CONTENT" }
If USER_POSTED_CONTENT contains new-line character '\n', double quotes or any characters that are must be escaped but not escaped, then it is no longer a valid JSON string and client's JavaScript engine cannot parse data.
So I am trying to make sure client is submitting valid JSON string.
This is what I came up with after doing some researches.
String.prototype.escapeForJson = function() {
return this
.replace(/\b/g, "")
.replace(/\f/g, "")
.replace(/\\/g, "\\")
.replace(/\"/g, "\\\"")
.replace(/\t/g, "\\t")
.replace(/\r/g, "\\r")
.replace(/\n/g, "\\n")
.replace(/\u2028/g, "\\u2028")
.replace(/\u2029/g, "\\u2029");
};
I use this function to escape all the characters that need to be escaped in order to create a valid JSON string.
var content = txt.val().escapeForJson();
$.ajax(
...
data:{ "content": content }
...
);
But then... it seems like str = JSON.stringify(str); does the same job!
However, after reading what JSON.stringify is really for, I am just confused. It says JSON.stringify is to convert JSON Object into string.
I am not really converting JSON Object to string.
So my question is...
Is it totally ok to use JSON.stringify to convert user input to valid JSON string object??
UPDATES:
JSON.stringify(content) worked good but it added double quotes in the beginning and in the end. And I had to manually remove it for my needs.

Yep, it is totally ok.
You do not need to re-invent what does exist already, and your code will be more useable for another developer.
EDIT:
You might want to use object instead a simple string because you would like to send some other information.
For example, you might want to send the content of another input which will be developed later.
You should not use stringify is the target browser is IE7 or lesser without adding json2.js.

I don't think JSON.stringify does what you need. Check the out the behavior when handling some of your cases:
JSON.stringify('\n\rhello\n')
*desired : "\\n\\rhello\\n"
*actual : "\n\rhello\n"
JSON.stringify('\b\rhello\n')
*desired : "\\rhello\\n"
*actual : "\b\rhello\n"
JSON.stringify('\b\f\b\f\b\f')
*desired : ""
*actual : ""\b\f\b\f\b\f""
The stringify function returns a valid JSON string. A valid JSON string does not require these characters to be escaped.
The question is... Do you just need valid JSON strings? Or do you need valid JSON strings AND escaped characters? If the former: use stringify, if the latter: use stringify, and then use your function on top of it.
Highly relevant: How to escape a JSON string containing newline characters using javascript?

Complexity. I don't know what say.
Take the urlencode function from your function list and kick it around a bit.
<?php
$textdata = $_POST['textdata'];
///// Try without this one line and json encoding tanks
$textdata = urlencode($textdata);
/******* textarea data slides into JSON string because JSON is designed to hold urlencoded strings ******/
$json_string = json_encode($textdata);
//////////// decode just for kicks and used decoded for the form
$mydata = json_decode($json_string, "true");
/// url decode
$mydata = urldecode($mydata['textdata']);
?>
<html>
<form action="" method="post">
<textarea name="textdata"><?php echo $mydata; ?></textarea>
<input type="submit">
</html>
Same thing can be done in Javascript to store textarea data in local storage. Again textarea will fail unless all the unix formatting is deal with. The answer is take urldecode/urlencode and kick it around.
I believe that urlencode on the server side will be a C wrapped function that iterates the char array once verses running a snippet of interpreted code.
The text area returned will be exactly what was entered with zero chance of upsetting a wyswyg editor or basic HTML5 textarea which could use a combination of HTML/CSS, DOS, Apple and Unix depending on what text is cut/pasted.
The down votes are hilarious and show an obvious lack of knowledge. You only need to ask yourself, if this data were file contents or some other array of lines, how would you pass this data in a URL? JSON.stringify is okay but url encoding works best in a client/server ajax.

Why is the length of a string returned by SQL different from that returned by other languages?

I have a web application that users use to send messages.
The problem is that the number of characters in a message
determine the cost of sending the message.
I have noticed that the javascript UI code counts the
characters just fine but the DBMS's in built functions
sometimes return a higher number of character.
Here is an example of a string that exhibit's this anomalous
behaviour:
String with different lengths..
This string has different lengths depending on the programming language use to count the characters.
Transact SQL LEN() and MySQL LENGTH() return 217.
Python len() returns 212.
The standard string length functions in Javascript and Python return similar values but lower than the values return by Transact-SQL's LEN() and DATALENGTH() and MySQL's LENGTH() (which also return values similar to each other).
So why the different values?

I noticed that this only happens when the strings have newline characters in them.
SQL server counts '\r\n' as two characters.
My solution was to count the characters using something like
LEN(REPLACE(the_string, CHAR(13), ''))
to get rid of carriage return before counting the length of the string.
This stackoverflow entry and this one helped me a lot.

I can't speak to MySQL. For SQL Server:
LEN() function tells you how many characters exist in the string, excluding trailing white space.
DATALENGTH() tells you how much space a given string takes up. For varchar data types, this will be 1 byte per character. For nvarchar data types, it's two bytes per character. Note that it DOES count white space when using DATALENGTH(). Here are some examples showing the use of the two functions with different strings and data types
select
LenTrailingSpace = len(' abc '),
LenNoTrailingSpace = len(' abc'),
DatalengthTrailingSpace = datalength(' abc '),
DatalengthNoTrailingSpace = datalength(' abc'),
UnicodeDatalengthTrailingSpace = datalength(N' abc '),
UnicodeDatalengthNoTrailingSpace = datalength(N' abc')

How could I process the quotation mark in the input box of a web page? [duplicate]

This question already has answers here:
What are the common defenses against XSS? [closed]
(4 answers)
Closed 9 years ago.
The users may input some special chars in the input box：
<input type="text" name="task_description" id="task_description" value="<?cs var:Query.task_description?>">
double quotation marks and single quotation marks for example. I need to get their input text and insert the task_description into my database table.
On my server, I write my cgi using C++, and my database is mySQL.
the user's input will be transferred to the server in JSON.
the problem is on the server side, when I want to get the strings input by the users in
JSON, I just can not get the right one. for example:
if the user input:
hello " " hello
on the server side I get the input string in the JSON like this:
static string get_escape_string(const string& src)
{
static char escape_buffer[1024*1024];
mysql_escape_string(escape_buffer, src.c_str(), src.length());
string dst(escape_buffer, strlen(escape_buffer));
return dst;
}
//here is how I get the user's input
string remarks = get_escape_string(record[i]["remarks"].asString());
the "record" is the JSON data, after the operation I can only get: hello
it is truncated at the first double quoation mark.
I tried to use the function "escape" in the front end javascript code, but "escape" can not
code double quotation mark.
How could I deal with the double and single quotation marks?
thanks in advance!

How could I deal with the double and single quotation marks?
The same way you deal with all DB input: Using parameterized queries. Never build up SQL strings using string concatenation. (Obligatory link to xkcd.)
How you do that will depend on what server-side technology you're using, which you haven't listed.
Similarly, when outputting the text of something into the value attribute in the HTML, you'll need to use a function (which is probably provided in your server-side environment) that correctly encodes that text as HTML (since attribute values are HTML), which will turn double quotes into " entities and similar.