What does colon ':' in API request mean? - javascript

i am using firebase for my project ,the documentation gives me the endpoint for signing in users as:
https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=[API_KEY]
i want to know what does the colon : mean, for example the word key after the question mark shows its a parameter likewise what does the notion accounts:signInWithPassword mean.The reason:I have an axios instance with config:
axios.create(
{
baseURL:"https://identitytoolkit.googleapis.com/v1",
params:{
apiKey:"somekey"
}
})
now since the baseUrl shown above remains same for firebase signing in with email and password or signing up with email and password. I want to dynamically embed accounts:signInWithPassword and accounts:signUp for respective requests and i am not sure if specifying accounts:respectiveUsecase in params object would work.

A colon doesn't have any special meaning in an URL path. It's just a convention those APIs tend to use in their paths.
There are a handful of metacharacters that do:
question marks (?) and hashes (#) delimit the query or search parts
% is used for escaping characters (e.g. %0A)
+ is sometimes an encoding for a space instead of %20.
& generally separates query parameters (e.g. foo=bar&baz), though this is not a part of the standard. Some server software could expect e.g. semicolon-separated parameters.
As #deceze pointed out, colons do have a special meaning in the host part, e.g. https://user:pass#host/path:where:colons:do:not:matter.

It is a dynamic value (like a parameter where you in pass in a value directly)
:nounId: The colon (:) before the word indicates that we don't mean the literal string "nounId" as part of the endpoint, but rather that we are expecting some dynamic data to be inside there. From the above example of /ski/:skiId, one actual endpoint might be something like /ski/1234 (where 1234 is the unique ID number of one of the skis in our database.
source: https://coursework.vschool.io/rest-api-design/#:~:text=%3AnounId%20%3A%20The%20colon%20(%3A)%20before,data%20to%20be%20inside%20there.

Related

Are there problems that I can run into by minifying graphql queries

I have a graphql query that I'm sending to the server (uses apollo-server) using fetch. I strip all extra whitespace from the query string before I send it. Should I be worried?
const query = `
{
thing {
id
name
relatedThing {
id
name
createdAt
}
}
}`
after query.replace(/\s+/g, ' ') I've got...
"{ thing { id name relatedThing { id name createdAt } } }"
I haven't had any complains from the server or any weird behaviour, but I don't understand the server demands and whether there's a possibility this could break some queries. Is it possible that I can break some queries by doing this?
From the spec:
White space is used to improve legibility of source text and act as separation between tokens, and any amount of white space may appear before or after any token. White space between tokens is not significant to the semantic meaning of a GraphQL Document, however white space characters may appear within a String or Comment token... Like white space, line terminators are used to improve the legibility of source text, any amount may appear before or after any other token and have no significance to the semantic meaning of a GraphQL Document. Line terminators are not found within any other token.
In other words, there's nothing wrong with what you are doing. There's just two things to keep in mind:
If your queries include String literals, that particular regex express will also change the value of the String literal as well if it includes more than one space.
GraphQL returns errors with a location that includes both the line number and character number where the error occurred. By transforming your queries like this, that information will reflect the transformed query and not your original one.

NodeJS escaping back slash

I am facing some issues with escaping of back slash, below is the code snippet I have tried. Issues is how to assign a variable with escaped slash to another variable.
var s = 'domain\\username';
var options = {
user : ''
};
options.user = s;
console.log(s); // Output : domain\username - CORRECT
console.log(options); // Output : { user: 'domain\\username' } - WRONG
Why when I am printing options object both slashes are coming?
I had feeling that I am doing something really/badly wrong here, which may be basics.
Update:
When I am using this object options the value is passing as it is (with double slashes), and I am using this with my SOAP services, and getting 401 error due to invalid user property value.
But when I tried the same with PHP code using same user value its giving proper response, in PHP also we are escaping the value with two slashes.
When you console.log() an object, it is first converted to string using util.inspect(). util.inspect() formats string property values as literals (much like if you were to JSON.stringify(s)) to more easily/accurately display strings (that may contain control characters such as \n). In doing so, it has to escape certain characters in strings so that they are valid Javascript strings, which is why you see the backslash escaped as it is in your code.
The output is correct.
When you set the variable, the escaped backslash is interpreted into a single codepoint.
However, options is an object which, when logged, appears as a JSON blob. The backslash is re-escaped at this point, as this is the only way the backslash can appear validly as a string value within the JSON output.
If you re-read the JSON output from console.log(options) into javascript (using JSON.parse() or similar) and then output the user key, only one backslash will show.
(Following question edit:)
It is possible that for your data to be accepted by the SOAP consuming service, the data needs to be explicitly escaped in-band. In this case, you will need to double-escape it when assigning the value:
var s = 'domain\\\\user'
To definitively determine whether you need to do this or not, I'd suggest you put a proxy between your working PHP app and the SOAP app, and inspect the traffic.

What is the right way to deal with special characters in resource URIs? How to properly escape?

Our application implements APIs to access entity details via AJAX GET requests - it once used POST but we decided to change our standards.
Unfortunately we hit a design flaw recently.
Suppose our API is http://localhost/app/module/entity/detail/{id} where {id} is a Spring #PathVariable that matches that entity's primary key.
If the primary key is a numeric surrogate key (auto-increment) there is no problem then. But we thought that this all worked with String primary keys.
It happened that we found some valid production data contain slashes, backslashes and semicolons for primary key. Our tables cannot use auto-increment surrogate keys because of their gargantuar size.
More in general, we discovered that we were unprepared to handle non-alphanumeric characters.
What the defective code was originally
Here is how our application used to reach to display entity data in a single page form the entity list:
User navigates table.jsp where an AJAX list is retrieved via POST
An Angular expression constructs the link to the detail page of that entity by means of detail?entityId={{::row:idAttribute}}
A valid link is generated by Angular
Examples:
detail?entityId=5903475934 //numeric case, we have several details page with same navigation pattern
detail?entityID=AAABBBCCC
detail?entityID=DO0000099101\test
The user clicks on the link and the browser points to the corresponding address
http://localhost/blablabla/detail?entityId=DO0000099101\test //Could be escaped.... read more later
The page needs the ID code from the parameters to issue the correct AJAX call
Need to retrieve the Entity ID from the query string. The Angular controller is in a separate file that doesn't see the query string (and is included dynamically, having the same page name)
<script type="text/javascript">
var ____entityId = '${param.entityId}';
</script>
Which gets translated to
<script type="text/javascript">
var ____entityId = 'DO0000099101\test'; //Yes, I know it's incorrect because now we have a TAB
</script>
Angular fetches the Entity ID into the page scope
In the Angular controller, we do the following
$scope.entityId = ___entityId;
$http.get($const.urlController+'detail/'+$scope.entityId)
Ajax call is issued
http://localhost/blablablabla/entity/detail/DO0000099101est //URL is bad
Spring MVC decodes the #PathVariable
Unfortunately, the parameter is treated as DO0000099101est
What we tried to fix
We have tried to fix the mistake by escaping the \t as it is clear that its presence in the HTML content constitutes a bug. Javascript interprets it as a tab.
Trying to URL-escape the ID
We tried to manually navigate to http://localhost/blablabla/detail?entityId=DO0000099101%5Ctest by escaping the backslash to its URL entity. The purpose is that if this worked we could modify Angular code in the table page
The result is that the \t appeared again as it is in the Javascript fragment. From that point, the sequence is the same
<script type="text/javascript">
var ____entityId = 'DO0000099101\test';
</script>
Trying to Java-escape + URL-Escape the ID in the URL
What if the Angular table page escaped the URL by this way?
http://localhost/blablabla/detail?entityId=DO0000099101%5C%5Ctest
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test'; //Looks great
</script>
Unfortunately Angular now reverses the backslash into a forward slash when performing the Ajax request
http://localhost/blablabla/detail/DO0000099101/test
Trying to perform the Ajax call manually with escaped URL
So know that the REST url is http://localhost/app/module/entity/detail/{id}, let's try to see how Spring expects the backslash to be escaped
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
http://localhost/app/module/entity/detail/DO0000099101\\test //Chrome reversed the backslash into a forward slash and the result is 404 as expected for http://localhost/app/module/entity/detail/DO0000099101//test
Using encodeURIComponent
We already tried this (but I didn't mention in the original post). To #Thomas comment, we tried again to encode the primary key by Javascript escaping in the controller itself.
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
The problem with this approach is that the backslash-T sequence in the URL is encoded to %09, which results in a tab decoded on the server side
Using both encodeURIComponent and a Java escape
Now we tried to use both approach #4 and fixing the hardcoded Java-escaped (i.e. \\t) into the Javascript for testing purposes.
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test';
</script>
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
This time the Ajax call resulted in 400 error as case #3
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
Question time
I want to ask this question in two ways, one specific and one more general.
Specific: how can I properly format a #PathVariable so that it can preserve special characters?
General: in the REST world, what care is to be done when dealing with resource IDs that may contain special characters?
For the second, let me be more clear. If you control your REST application and generation of IDs, you can choose to allow only alphanumeric identifier (user-generated or random or whatever) and sleep happy.
But in our case we need to browse entities that come supplied from external systems that have broader restrictions on character sets allowed for entity primary identifiers.
And again, we cannot use auto-increment surrogate keys.

Javascript replace plus sign with space (but not %20)

I am trying to use the League of Legends API and request data on a certain user. I use the line
var user = getUrlVars()["username"].replace("+", " ");
to store the username. However, when I do the XMLHttpRequest with that username, it'll put %20 instead of a space.
y.open("GET", "https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/"+user, false);
Edit: When I run this code with a user that has no space in their name it works, however when they have a space in their name it says the user is undefined.
For example, if I was looking for the user "the man", it would do a get at
https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/the%20man
But the correct request URL is
https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/the man
When you're creating a URL, you should use encodeURIComponent to encode all the special characters properly:
y.open("GET", "https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/"+encodeURIComponent(user), false);
Actually there are no "spaces" in the summoner names on Riot's side. So:
https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/the man
Becomes:
https://na.api.pvp.net/api/lol/na/v1.4/summoner/by-name/theman
Have a look at this: https://developer.riotgames.com/discussion/community-discussion/show/jomoRum7
I am unsure how + are handled (in fact I don't think you're able to have a + in your name). All you have to do is remove the spaces.
For "funny" characters, just request them with the funny character in them, and Riot returned it fine.
https://euw.api.pvp.net/api/lol/euw/v1.4/summoner/by-name/Trøyer?api_key=<insert your own>
will auto correct to
https://euw.api.pvp.net/api/lol/euw/v1.4/summoner/by-name/Tr%C3%B8yer?api_key=<insert your own>
and you generally don't even have to decode it. (I used JS as my language to fetch it, if you use something else your results may require the decoded value)
What you're experiencing is correct behaviour and is called URL encoding. HTTP requests have to conform to certain standards. The first line is always made up of three parts delimited by a space:
Method (GET, POST, etc.)
Path (i.e. /api/lol/na/v1.4/summoner/by-name/the%20man)
HTTP version (HTTP/1.1, HTTP/1.0, etc.)
This is usually followed by HTTP headers which I'll leave out for the time being since it is beyond the scope of your question (if interested, read this https://www.rfc-editor.org/rfc/rfc7230). So a normal request looks like this:
GET /api/lol/na/v1.4/summoner/by-name/the%20man HTTP/1.1
Host: na.api.pvp.net
User-Agent: Mozilla
...
With regards to your original question, the reason the library is URL encoding the space to %20 is because you cannot have a space character in the request line. Otherwise, you would throw off most HTTP message parsers because the man would replace the HTTP version line like so:
GET /api/lol/na/v1.4/summoner/by-name/the man HTTP/1.1
Host: na.api.pvp.net
User-Agent: Mozilla
...
In most cases, servers will return a 400 bad request response because they wouldn't understand what HTTP version man refers to. However, nothing to fear hear, most server-side applications/frameworks automatically decode the %20 or + to space prior to processing the data in the HTTP request. So even though your URL looks unusual, the server side will process it as the man.
Finally, one last thing to note. You shouldn't be using the String.replace() to URL decode your messages. Instead, you should be using decodeURI() and encodeURI() for decoding and encoding strings, respectively. For example:
var user = getUrlVars()["username"].replace("+", " ");
becomes
var user = decodeURI(getUrlVars()["username"]);
This ensures that usernames containing special characters (like / which would be URL encoded as %2f) are also probably decoded. Hope this helps!

Why use encodeURIComponent() when writing json to a cookie

In particular, when saving a JSON to the cookie is it safe to just save the raw value?
The reason I dopn't want to encode is because the json has small values and keys but a complex structure, so encoding, replacing all the ", : and {}, greatly increases the string length
if your values contain "JSON characters" (e.g. comma, quotes, [] etc) then you should probably use encodeURIComponent so these get escaped and don't break your code when reading the values back.
You can convert your JSON object to a string using the JSON.stringify() method then save it in a cookie.
Note that cookies have a 4000 character limit.
If your Json string is valid there should be no need to encode it.
e.g.
JSON.stringify({a:'foo"bar"',bar:69});
=> '{"a":"foo\"bar\"","bar":69}' valid json stings are escaped.
This is documented very well on MDN
To avoid unexpected requests to the server, you should call encodeURIComponent on any user-entered parameters that will be passed as part of a URI. For example, a user could type "Thyme &time=again" for a variable comment. Not using encodeURIComponent on this variable will give comment=Thyme%20&time=again. Note that the ampersand and the equal sign mark a new key and value pair. So instead of having a POST comment key equal to "Thyme &time=again", you have two POST keys, one equal to "Thyme " and another (time) equal to again.
If you can't be certain that your JSON will not include reserved characters such as ; then you will want to perform escaping on any strings being stored as a cookie. RFC 6265 covers special characters that are not allowed in the cookie-name or cookie-value.
If you are encoding static content you control, then this escaping may be unnecessary. If you are encoding dynamic content such as encoding user generated content, you probably need escaping.
MDN recommends using encodeURIComponent to escape any disallowed characters.
You can pull in a library such as cookie to handle this for you, but if your server is written in another language you will need to ensure it uses a library or language utilities to encodeURIComponent when setting cookies and to decodeURIComponent when reading cookies.
JSON.stringify is not sufficient as illustrated by this trivial example:
const bio = JSON.stringify({ "description": "foo; bar; baz" });
document.cookie = `bio=${stringified}`;
// Notice that the content after the first `;` is dropped.
// Attempting to JSON.parse this later will fail.
console.log(document.cookie) // bio={\"description\":\"foo;
Cookie: name=value; name2=value2
Spaces are part of the cookie separation in the HTTP Cookie header. Raw spaces in cookie values could thus confuse the server.

Categories