We Keep Coding

What does colon ':' in API request mean?

i am using firebase for my project ,the documentation gives me the endpoint for signing in users as:
https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=[API_KEY]
i want to know what does the colon : mean, for example the word key after the question mark shows its a parameter likewise what does the notion accounts:signInWithPassword mean.The reason:I have an axios instance with config:
axios.create(
{
baseURL:"https://identitytoolkit.googleapis.com/v1",
params:{
apiKey:"somekey"
}
})
now since the baseUrl shown above remains same for firebase signing in with email and password or signing up with email and password. I want to dynamically embed accounts:signInWithPassword and accounts:signUp for respective requests and i am not sure if specifying accounts:respectiveUsecase in params object would work.

A colon doesn't have any special meaning in an URL path. It's just a convention those APIs tend to use in their paths.
There are a handful of metacharacters that do:
question marks (?) and hashes (#) delimit the query or search parts
% is used for escaping characters (e.g. %0A)
+ is sometimes an encoding for a space instead of %20.
& generally separates query parameters (e.g. foo=bar&baz), though this is not a part of the standard. Some server software could expect e.g. semicolon-separated parameters.
As #deceze pointed out, colons do have a special meaning in the host part, e.g. https://user:pass#host/path:where:colons:do:not:matter.

It is a dynamic value (like a parameter where you in pass in a value directly)
:nounId: The colon (:) before the word indicates that we don't mean the literal string "nounId" as part of the endpoint, but rather that we are expecting some dynamic data to be inside there. From the above example of /ski/:skiId, one actual endpoint might be something like /ski/1234 (where 1234 is the unique ID number of one of the skis in our database.
source: https://coursework.vschool.io/rest-api-design/#:~:text=%3AnounId%20%3A%20The%20colon%20(%3A)%20before,data%20to%20be%20inside%20there.

NodeJS escaping back slash

I am facing some issues with escaping of back slash, below is the code snippet I have tried. Issues is how to assign a variable with escaped slash to another variable.
var s = 'domain\\username';
var options = {
user : ''
};
options.user = s;
console.log(s); // Output : domain\username - CORRECT
console.log(options); // Output : { user: 'domain\\username' } - WRONG
Why when I am printing options object both slashes are coming?
I had feeling that I am doing something really/badly wrong here, which may be basics.
Update:
When I am using this object options the value is passing as it is (with double slashes), and I am using this with my SOAP services, and getting 401 error due to invalid user property value.
But when I tried the same with PHP code using same user value its giving proper response, in PHP also we are escaping the value with two slashes.

When you console.log() an object, it is first converted to string using util.inspect(). util.inspect() formats string property values as literals (much like if you were to JSON.stringify(s)) to more easily/accurately display strings (that may contain control characters such as \n). In doing so, it has to escape certain characters in strings so that they are valid Javascript strings, which is why you see the backslash escaped as it is in your code.

The output is correct.
When you set the variable, the escaped backslash is interpreted into a single codepoint.
However, options is an object which, when logged, appears as a JSON blob. The backslash is re-escaped at this point, as this is the only way the backslash can appear validly as a string value within the JSON output.
If you re-read the JSON output from console.log(options) into javascript (using JSON.parse() or similar) and then output the user key, only one backslash will show.
(Following question edit:)
It is possible that for your data to be accepted by the SOAP consuming service, the data needs to be explicitly escaped in-band. In this case, you will need to double-escape it when assigning the value:
var s = 'domain\\\\user'
To definitively determine whether you need to do this or not, I'd suggest you put a proxy between your working PHP app and the SOAP app, and inspect the traffic.

What is the right way to deal with special characters in resource URIs? How to properly escape?

Our application implements APIs to access entity details via AJAX GET requests - it once used POST but we decided to change our standards.
Unfortunately we hit a design flaw recently.
Suppose our API is http://localhost/app/module/entity/detail/{id} where {id} is a Spring #PathVariable that matches that entity's primary key.
If the primary key is a numeric surrogate key (auto-increment) there is no problem then. But we thought that this all worked with String primary keys.
It happened that we found some valid production data contain slashes, backslashes and semicolons for primary key. Our tables cannot use auto-increment surrogate keys because of their gargantuar size.
More in general, we discovered that we were unprepared to handle non-alphanumeric characters.
What the defective code was originally
Here is how our application used to reach to display entity data in a single page form the entity list:
User navigates table.jsp where an AJAX list is retrieved via POST
An Angular expression constructs the link to the detail page of that entity by means of detail?entityId={{::row:idAttribute}}
A valid link is generated by Angular
Examples:
detail?entityId=5903475934 //numeric case, we have several details page with same navigation pattern
detail?entityID=AAABBBCCC
detail?entityID=DO0000099101\test
The user clicks on the link and the browser points to the corresponding address
http://localhost/blablabla/detail?entityId=DO0000099101\test //Could be escaped.... read more later
The page needs the ID code from the parameters to issue the correct AJAX call
Need to retrieve the Entity ID from the query string. The Angular controller is in a separate file that doesn't see the query string (and is included dynamically, having the same page name)
<script type="text/javascript">
var ____entityId = '${param.entityId}';
</script>
Which gets translated to
<script type="text/javascript">
var ____entityId = 'DO0000099101\test'; //Yes, I know it's incorrect because now we have a TAB
</script>
Angular fetches the Entity ID into the page scope
In the Angular controller, we do the following
$scope.entityId = ___entityId;
$http.get($const.urlController+'detail/'+$scope.entityId)
Ajax call is issued
http://localhost/blablablabla/entity/detail/DO0000099101est //URL is bad
Spring MVC decodes the #PathVariable
Unfortunately, the parameter is treated as DO0000099101est
What we tried to fix
We have tried to fix the mistake by escaping the \t as it is clear that its presence in the HTML content constitutes a bug. Javascript interprets it as a tab.
Trying to URL-escape the ID
We tried to manually navigate to http://localhost/blablabla/detail?entityId=DO0000099101%5Ctest by escaping the backslash to its URL entity. The purpose is that if this worked we could modify Angular code in the table page
The result is that the \t appeared again as it is in the Javascript fragment. From that point, the sequence is the same
<script type="text/javascript">
var ____entityId = 'DO0000099101\test';
</script>
Trying to Java-escape + URL-Escape the ID in the URL
What if the Angular table page escaped the URL by this way?
http://localhost/blablabla/detail?entityId=DO0000099101%5C%5Ctest
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test'; //Looks great
</script>
Unfortunately Angular now reverses the backslash into a forward slash when performing the Ajax request
http://localhost/blablabla/detail/DO0000099101/test
Trying to perform the Ajax call manually with escaped URL
So know that the REST url is http://localhost/app/module/entity/detail/{id}, let's try to see how Spring expects the backslash to be escaped
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
http://localhost/app/module/entity/detail/DO0000099101\\test //Chrome reversed the backslash into a forward slash and the result is 404 as expected for http://localhost/app/module/entity/detail/DO0000099101//test
Using encodeURIComponent
We already tried this (but I didn't mention in the original post). To #Thomas comment, we tried again to encode the primary key by Javascript escaping in the controller itself.
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
The problem with this approach is that the backslash-T sequence in the URL is encoded to %09, which results in a tab decoded on the server side
Using both encodeURIComponent and a Java escape
Now we tried to use both approach #4 and fixing the hardcoded Java-escaped (i.e. \\t) into the Javascript for testing purposes.
<script type="text/javascript">
var ____entityId = 'DO0000099101\\test';
</script>
$http.get($const.urlController+'detail/'+encodeURIComponent($scope.entityId))
This time the Ajax call resulted in 400 error as case #3
http://localhost/app/module/entity/detail/DO0000099101%5Ctest //results in 400 error
Question time
I want to ask this question in two ways, one specific and one more general.
Specific: how can I properly format a #PathVariable so that it can preserve special characters?
General: in the REST world, what care is to be done when dealing with resource IDs that may contain special characters?
For the second, let me be more clear. If you control your REST application and generation of IDs, you can choose to allow only alphanumeric identifier (user-generated or random or whatever) and sleep happy.
But in our case we need to browse entities that come supplied from external systems that have broader restrictions on character sets allowed for entity primary identifiers.
And again, we cannot use auto-increment surrogate keys.

Regular Expression - Extract subdomain & domain

I'm trying to form a regular expression (javascript/node.js) which will extract the sub-domain & domain part from any given URL. This is what I ended up with:
[^(?:http:\/\/|www\.|https:\/\/)]([^\/]+)
Right now, I'm just considering http, https for protocol & exclude "www." portion from the subdomain+domain portion of an URL. I checked the expression & it almost works. But, here is the issue:
Success
'http://mplay.google.co.in/sadfask/asdkfals?dk=10'.match(/[^(?:http:\/\/|www\.|https:\/\/)]([^\/]+)/i)
'http://lplay.google.co.in/sadfask/asdkfals?dk=10'.match(/[^(?:http:\/\/|www\.|https:\/\/)]([^\/]+)/i)
Failure
'http://play.google.co.in/sadfask/asdkfals?dk=10'.match(/[^(?:http:\/\/|www\.|https:\/\/)]([^\/]+)/i)
'http://tplay.google.co.in/sadfask/asdkfals?dk=10'.match(/[^(?:http:\/\/|www\.|https:\/\/)]([^\/]+)/i)
I just use the first element from the result array. I'm not able to understand why "play." & "tplay." doesn't work. Could anyone please help me in this regard?
Does "/p" and "/t" have any meaning for the regular expression evaluator?
Is there any other way of extracting sub-domain & domain from any given URL using a regular expression?
Edit -
Example:
https://play.google.com/store/apps/details?id=com.skgames.trafficracer => play.google.com
https://mail.google.com/mail/u/0/#inbox => mail.google.com

Your regex doesn't seem correct. Try this regex:
/^(?:https?:\/\/)?(?:[^#\n]+#)?(?:www\.)?([^:\/\n?]+)/img
RegEx Demo

You are about the one millionth person to try to parse URLs in JavaScript. I'm a little bit surprised you didn't see any of the existing questions on SO dating back years. The last thing you want to do is write yet another broken regexp, with all due respect to those that provided answers to your question.
There are many well documented libraries and approaches to handling this. Google it. The simplest way is to create an a element in memory, assign it an href, and then access its hostname and other properties. See http://tutorialzine.com/2013/07/quick-tip-parse-urls/. If that does not float your boat, then use a library like uri.js.
If you really don't want to use a library, and insist on reinventing the wheel, then at least do something like the following:
function get_domain_from_url(url) {
var a = document.createElement('a').
a.setAttribute('href', url);
return a.hostname;
}
Essentially, you are delegating the extraction of the subdomain/domain part of the URL to the browser's URL parsing logic, which is MUCH better than anything you will ever write.
Also see Parse URL with jquery/ javascript?, Parse URL with Javascript, How do I parse a URL into hostname and path in javascript?, or parse URL with JavaScript or jQuery. How did you miss those? Sorry, I have to vote to close this as a duplicate.

The same RegExp as in anubhava's answer, only added support for protocol-relative URLs like //google.com:
/^(?:https?:)?(?:\/\/)?(?:[^#\n]+#)?(?:www\.)?([^:\/\n]+)/im
RegEx Demo

Here's a solution ignoring everything before ://
.*\://?([^\/]+)
Incase you want to ignore www.
.*\://(?:www.)?([^\/]+)

Your regex expression works pretty well. You only need to remove the brackets. The final expression is:
^(?:http:\/\/|www\.|https:\/\/)([^\/]+)
Hope it's useful!

I know I am late to the party but I want to answer the question with some extra useful info.
Get the domain name from a link using regex.
^(https?:\/\/)?(www\.)?([^\/]+)
Here is the link to above regex.
If you want to get the subdomain, split the result from one of the matches of above regex with the first occurrence of .
Note: regex is faster than language built-in modules. check below examples, regex comes out to be 15x faster than the built-in module
javascript Example with Regex:
console.time('time2');
const pttrn = /^(https?:\/\/)?(www\.)?([^\/]+)/gm
const urlInfo = pttrn.exec("https://www.google.co.in/imghp");
console.timeEnd('time2');
//time2: 0.055ms
console.log(urlInfo[0]) // https://www.google.co.in
console.log(urlInfo[1]) // https://
console.log(urlInfo[2]) // www.
console.log(urlInfo[3]) // google.co.in
Nodejs with built-in url module
console.time('time');
const url = require('url');
const urlInfo = url.parse("https://www.google.co.in/imghp");
console.timeEnd('time');
//time: 0.840ms;
console.log(urlInfo.hostname) //www.google.co.in

Why use encodeURIComponent() when writing json to a cookie

In particular, when saving a JSON to the cookie is it safe to just save the raw value?
The reason I dopn't want to encode is because the json has small values and keys but a complex structure, so encoding, replacing all the ", : and {}, greatly increases the string length

if your values contain "JSON characters" (e.g. comma, quotes, [] etc) then you should probably use encodeURIComponent so these get escaped and don't break your code when reading the values back.

You can convert your JSON object to a string using the JSON.stringify() method then save it in a cookie.
Note that cookies have a 4000 character limit.
If your Json string is valid there should be no need to encode it.
e.g.
JSON.stringify({a:'foo"bar"',bar:69});
=> '{"a":"foo\"bar\"","bar":69}' valid json stings are escaped.

This is documented very well on MDN
To avoid unexpected requests to the server, you should call encodeURIComponent on any user-entered parameters that will be passed as part of a URI. For example, a user could type "Thyme &time=again" for a variable comment. Not using encodeURIComponent on this variable will give comment=Thyme%20&time=again. Note that the ampersand and the equal sign mark a new key and value pair. So instead of having a POST comment key equal to "Thyme &time=again", you have two POST keys, one equal to "Thyme " and another (time) equal to again.

If you can't be certain that your JSON will not include reserved characters such as ; then you will want to perform escaping on any strings being stored as a cookie. RFC 6265 covers special characters that are not allowed in the cookie-name or cookie-value.
If you are encoding static content you control, then this escaping may be unnecessary. If you are encoding dynamic content such as encoding user generated content, you probably need escaping.
MDN recommends using encodeURIComponent to escape any disallowed characters.
You can pull in a library such as cookie to handle this for you, but if your server is written in another language you will need to ensure it uses a library or language utilities to encodeURIComponent when setting cookies and to decodeURIComponent when reading cookies.
JSON.stringify is not sufficient as illustrated by this trivial example:
const bio = JSON.stringify({ "description": "foo; bar; baz" });
document.cookie = `bio=${stringified}`;
// Notice that the content after the first `;` is dropped.
// Attempting to JSON.parse this later will fail.
console.log(document.cookie) // bio={\"description\":\"foo;

Cookie: name=value; name2=value2
Spaces are part of the cookie separation in the HTTP Cookie header. Raw spaces in cookie values could thus confuse the server.