We Keep Coding

Node.js: Authenticate client using unique public key (Similar to Github SSH key authentication)

I have a central web-server (Node.js) and also have several client machines (also running Node). These clients are going to be pushing data into the web-server and so, for security, the clients will initially register with the main server - generate it's own unique SSL private/public keys and store the public key on the server.
Now, after the registration, every time a client tries to communicate with the server, it needs to present it's Public key(?) to authenticate itself.
I am trying to understand how to go about this. Should I store the contents of all the public keys of the clients in a regular database collection?
How do I actually make the clients "present" their certificates along with their data? I've seen a few packages like client-certificate-auth but I don't think this will help me for my use-case.
EDIT: I forgot to mention, I will be communicating over HTTPS (port 443) so I think that takes care of the actual security part. I just want to make sure that no rogue, unregistered client pushes data into the server using pub/private encryption / SSL certs.

If you're going down this path, use the Subtle Crypto API. Most likely, you shouldn't be doing this.
You want the private keys generated in javascript kept secret at the clients, and a public key sent and stored securely on the server.
During authentication, the client says what it's public key is by some unique identifier associated with it's public key, e.g. id or common name or user name.
The server then issues a challenge with a message payload encrypted with the client's public key.
The user decrypts the message with his private key, and then responds with the correct decrypted response.
This provides no security, you will have to perform this over an already established SSL connection.
It is an alternative to the traditional cookie and session token, and it's just different, harder, and not better.
The private key never arrives at the server at any time. The client does not encrypt using it's private key, that's not how public-key cryptography works.
It can however send a digital signature, and the server can verify that. That is an alternative to a unique identifier; make sure you specify a true and sufficiently long nonce to sign though.
To perform a digital signature challenge, the server sends a challenge string to the client, the client signs the challenge using it's private key and responds to the server, the server then verifies the signature using the public key. The public key must be part of the signature, and the server will have to know to trust the public key, so it'll have to have a way to look it up quickly.
Another alternative is to rely on the knowledge of the user in setting up their own client certificates, in which case passport-client-cert as mentioned in Paul's answer can handle that.

I'd strongly recommend not doing this yourself, there are a lot of things you can get wrong if you don't implement it correctly.
That said, the way this kind of thing works architecturally is not to "present the client's public key" but instead to have the client encrypt something with its private key and send that encrypted data to the server, which then decrypts with the client's public key. If it's able to do that, it knows the client is the client it claims to be (in theory; the reliability of this can vary depending on your keyspace).
What does the client encrypt? Well, that depends a lot on (again) how secure you want to be / paranoid you are. One way to do this (quasi-kerberos) would be for the client to send a request to the server for the information to encrypt. This is done over SSL, so the client can ensure that the server is who it says it is (and can check a CA and such if it wants). That thing passed back is some data that includes a random value, an expiry, and possibly a nonce of some kind. It's also stored by the server for lookup later verification and of course evicted on time. The client would typically take that exact object (possibly a JSON object) and encrypt it with its private key and send it back to the server to some login endpoint. As I said before, the server would then check its storage location for that object based on the client's ID, and do the comparison to see if the two objects are identical. If they are, then it can accept the client is who it says it is.
Normally, this handshake is done only to exchange a session key that the client can use to encrypt all its traffic, since a symmetric key encryption is typically easier on the CPU than PKI.
Honestly, though, I've only scratched the surface. You're probably much better off looking at what PassportJS has to offer for strategies and picking one of those. Perhaps passport-api-key or passport-client-cert will do what you need.

OAuth: How to hide API Secret Key from javascript

We're in the process of migrating our MVC-based server application and making a REST-ful API through which calls will be handled.
I've been reading up on AES encryption and OAuth2 and decided to implement a solution grown form those concepts as follows:
Client sends a request to log in providing a UserID or Email. This request is HMAC'd using an API Secret Key.
The server checks if the UserID/Email matches an existing account and if it finds one, creates and stores a server nonce which it sends as part of the response to the client.
The client creates their own client nonce and creates a new temporary key from the API Secret key and both nonces. It then sends a login request with a password encrypted using this temporary key [for added entropy and to avoid ever sending a password in plaintext].
The server decrypts the password and HMAC using the latest nonce it has stored for this client on this platform [a mobile and a web client can have their own distinct nonces and sessions] and the client nonce which was sent in the clear, if the HMAC checks out it then validates the password against the database [PBKDF2 hashing and salting].
If the request is valid and the password and UserID match records, a new Session Secret Key is created for that UserID on that platform and this Secret key is sent to the client and will be used to HMAC every API request fromt hat client henceforth.
Any new non-login request would include an HMAC signature computed from the Session Secret key and randomized IV's.
All communication is handled through TLS so this is added security and not the only line of defense.
On the mobile apps this would work since you can hide the Mobile App's Secret Key on a config file and this gives some decent measure of security - [perhaps not a lot I'm not fully sure] but if we try to convert all the requests from our webpage to this form this would mean using Javascript to handle the client-side AES encryption and authentication and ... well as this article clearly explains, " if you store your API key in a JavaScript web app you might as well just print it out in big bold letters across the homepage as the whole world now has access to it through their browser’s dev tools."
I could use only the nonces as the API Secret key -- or forgo using AES encryption for those requests altogether and try to validate through other means such as CSRF tokens and making sure all the requests come form our own front end in some way - but this wouldn't work if we wanted to create an API that allows integration with other pages or services and even then, how would I go about securing the client's secret Session key?
The article suggests generating single-use cookies as a tokens but that's a limited solution that works for the poster's services but wouldn't for us. I want to be able to HMAC every request the user sends with a user-specific key that can expire and be reset and since the service will eventually handle money, I want request authentication to be locked down tight.
So what are my options?
Do I just ditch Javascript since it is doomed? Is there some way to store a secret key without exposing it clear as day hardcoded into the .js script? Should I generate a new temporary Secret key to be used for login calls only and send that to the user when they request the server nonce?
Also, the post I linked to first suggests using a cookie to store the Session key for the client and then access the key from JS. Is this ok or would that provide more holes than it seals?

It's good to know which measures prevent which security holes.
You are correct that JavaScript is not well suited for encryption because there is no place to store a secret. There are also no good encryption libraries because you shouldn't be doing encryption in JavaScript.
The session key can serve as the authentication key. If you're using TLS your connection is secure and an attacker can't know the session key.
Additionally, JavaScript doesn't need to know the session key. Cookies, by default, are sent with every request. And you can set the cookie to be an http-only cookie. You don't have to do this, but it does add another layer of security.
You can give the session cookie a very long expiration time so that it essentially works like a secret API key. The browser will take care of storing the cookie securely. It is advised to rotate the session key often, typically at the start of every new session and when authentication information changes (like a password reset).
CSRF-tokens prevent replay attacks. It's definitely recommend to secure a modification request with a CSRF-token. You don't need a CSRF-check for every request, just requests that modify sensitive information (such as your login credentials, or in your case: transactions).
For CSRF-tokens you can use the same approach as the session key: store it in a cookie.
The key part is that JavaScript doesn't need to know about any of this.
One important thing that I'm sure you realize as well is that any keys or nonces you generate must be cryptographically safe. Don't use low entropy functions.
So:
You don't need to encrypt the userid or email, TLS does that for you already. Additionally you can send the password as well, you don't need to send it separately in step 3. We're not going to do any encryption in JavaScript. All encryption is handled by TLS/HTTPS alone.
If you have a separate authentication server (like a single sign on), this approach is fine. Else you can skip this step.
You don't need this.
The server doesn't need to decrypt anything, encryption is handled by TLS. How you store the password is a topic on it's own but I think you've got it.
Ok. Again, the client shouldn't encrypt anything.
Send just the session key. It's is enough.
Revised is:
Client sends login credentials. Connection must be secure.
Server verifies credentials and sends authentication token as cookie and keeps track of the authentication token is a session list.
For every request:
Client includes authentication token. This happens automatically if you use cookies.
Server verifies authentication token and possibly generates a fresh token that the client will use from then on.

Mobile apps should be considered as public clients. This means they should not store any secret. Whatever the encryption algorithm you will use, nothing prevent the client credentials from being compromised.
That is why the OAuth2 Framework protocol defines the Implicit grant type flow which allow public client interaction and do not need any client authentication. You may also consider the RFC7636 to protect the issuance of the access token.

Maintaining private key between https and http

I'm working on a new site that utilizes a service-oriented architecture, with pure JavaScript on the front-end, accessing web services for data via RESTful AJAX calls.
It shouldn't be of particular importance but my stack is:
javascriptMVC
jQuery
Bootsrap
ASP.NET Web API (C# on .NET 4.0)
MS SQL
From this article I've figured out some good ways of securing my web service calls once I have a private key shared between the client (JavaScript) and server (REST services via Web API). However, I'm struggling with how to establish the private key to be used for encryption.
Bad Idea #1
The initial though was to set it at log in which would occur over HTTPS, then store it on the client in a cookie for reuse. The problem is that our SSL cert is for https://secure.example.com, while our site is on http://www.example.com - so I wouldn't be able to access the secure.example.com cookie from www.example.com.
Bad Idea #2
My next thought was to pass it encrypted and signed via a URL parameter from the HTTPS login to the HTTP post-login page like so:
http://www.example.com/processLogin?key=[encryptedKey]&sig=[encryptedSig]&user=[userid]
encryptedKey and encryptedSig would both be encrypted with another private key that only exists just for that transaction. It would be created at log-in and assigned to that user in the database. On the HTTP side, all of this gets passed to the server which decrypts it, validates the signature, removes that private key (to guard against replay attacks - essentially a nonce) and returns the decrypted private key ([encryptedKey] decrypted).
From then on out, the decrypted value of [encryptedKey] would be used for all future transactions. The problem is that the decrypted private key would have to be sent over the line via HTTP, which sucks.
Bad Idea #3
It also briefly occurred to me to have a hard-coded key in the JavaScript that's used to decrypt this value but no matter how I try and obfuscate it, it could be found and used by a hacker.
Bad Idea #4
I've also considered some sort of key exchange using Public-key cryptography at the initial handshake, but as noted elsewhere, you can't really be confident on the client-side that there isn't tampering during this initial handshake unless it's over SSL - putting me back at square one.
The Big Question
So, how do you guys manage such things without everything going over HTTPS? Do I have to have the same domain name for my HTTP and HTTPS so that I can store this private key in a cookie?
Note that the non-SSL portions of the site wouldn't be sharing credit card or login information or the like. I just don't want to leave this sucker wide open.

You can not have secure and encrypted communication between a javascript client and a server without implementing SSL. It is impossible. If what you really want to accomplish is not to encrypt the traffic but simply insure the client you are talking to has been authorized to make the request and that the client is not an impersonator, then OAuth may be sufficient. See http://www.dotnetopenauth.net/ for the standard OAuth .net implementation.
If OAuth is not what you want to get involved in and you simply want to build on what you already have built, you should distribute a token and a public and a private key to the javascript client. The public key and the token is what gets sent back and forth for every request while the private key is never sent back and forth and is instead used to generate some type of signature hash. Every request should have this signature and a time-based nonce to prevent replays. You should also expire the token on a very frequent basis and require the client to request a "refresh" token with their sig and their public key. In essence, what I have described is OAuth 1.0a, and if you do want to take this route, I would refer back to DotNet OpenAuth instead of trying to roll it yourself.
However, to reiterate, without SSL, you will still be vulnerable to other types of attacks. Also, unless you SSL encrypt the initial token request, a hacker could always sniff the initial delivery of the token/public/private key pair, therefore, eliminating all your hard work to make things secure in the first place.
An alternative to the above is to have a proxy server sitting between your client and the REST API. Requests to the API can only go through the proxy server. The client logs in and gets a cookie from https://secure.example.com using basic auth. The client then continues to make requests to secure.example.com and secure.example.com then makes requests to your API and returns the data back to the client.
Anyway, hopefully enough info to give you food for thought.

You can view how to work with sub domains and cookies by checking out this answer: Creating a javascript cookie on a domain and reading it across sub domains

Regarding Bad Idea #3:
I've known for awhile that I can use http://jsbeautifier.org to deobfuscate anything that is obfuscated using http://dean.edwards.name/packer/ with the "Shrink variables" checkbox &/or the "Base62 encode" checkbox. So JavaScript is totally insecure & shouldn't be relied upon for saving any sort of SSL encryption, nor user auth tokens, nor editable account stats in the browser. Otherwise someone would simply try to edit their game account & give themselves +10 million game coins.
When everything goes over SSL it only protects against "man-in-the-middle" attacks. It's really a "server-bot-in-the-middle" attack. It doesn't prevent the end-user from being a hacker themselves.
In this next illustration, SSL would prevent servers a through e from seeing any data that's being passed from the client to the terminus server, but without SSL server C would steal data. This is how server hops work, without encryption, where the client + all servers can read the data:
client > a > b > server c's bot sniffs http traffic > d > e > terminus server
Server c's bot logs a credit card number, which is an encrypted bank account number. (Most people don't realize that a credit card number is an encrypted & transformed bank account number. If the credit card number is compromised, it's easy for a bank to re-issue a new encrypted CC# from a bank account number & send out a new card in the mail. They don't have to change the original bank account number nor printing new checks, which have the bank account number printed on the bottom of them.)
Server hops with TLS/SSL/https encryption would work like this, where only the client & server could read anything:
client > all servers from a-e are blind & pass the data through > terminus server
Server c's bot sees junk like: as65a89as7df08 instead of 1234-5678-9012-..., if they can read anything at all using SSL.
What's cool about iOS, is that it makes it harder to read the JS code when it's used with HTML 5 & CSS. User can't right-click to inspect on their iPhone, like they can in a desktop browser. It's easy to hide a password in the terminus server using a back-end language.
It's currently impossible to prevent JavaScript from being hacked by the end-user (client). Every end-user can be a hacker. If I figure something else out, I can post it here for future developers to read.

Is there a way to securely send information in Ajax?

I'm currently developing an application in HTML+JS based almost entirely in ajax connections (using the jQuery.ajax() call to ease the process).
I was thinking about the best practice to make secure calls without using HTTPS (at least at this time. I can't afford paying for a certificate right now).
At this point, the only thing that concerns me is the registration and login steps. Maybe the login is a bit easier. I thought of sending the username and a timestamp, and then encrypt them using the user's password. So, by doing this, I wouldn't be sending any password (keeping as a secret like in OAuth). The server should check the user, decrypt using the password and pairing the recieved timestamp with the decrypted result. The server should keep the nonce-like number into a database (to avoid repetition attacks) and then give back to the user another unique id (encrypted with the user's password). At that point the user should start using that key to encrypt all his information (and probably another nonce) and send it to the server. Please correct me if you find any mistake or leak.
The very big problem to me is the registration. I can't encrypt with a regular password the information, because if I do that in the javascript, any could know the password. If I serve temporary generated passwords to encrypt and I send it from the server to the client, any sniffer could get it and use to decrypt the info.
I know HTTPS could save my life at this point (and maybe that's the only solution), but at this point I'm not able to use it.
Is there any other solution, or should I wait until I can use HTTPS? Bear in mind that if I could skip the wait, it would be better. Thanks mates!

Short answer: You can't do it without HTTPS
Longer answer: If you try to do it without HTTPS, you will find yourself trying to reproduce everything that HTTPS was designed to do. You could reach at some point, but it is unrealistic to believe that you will succeed in implementing even the 1% that HTTPS offers. The only benefit you will have would be an obscure security mechanism (security through obscurity), which may be OK for not critical systems, but would fail miserably in a real critical situation.
You could create your own certificate you know and then work with Ajax the same way as with regular HTTP calls. The only drawback is that the users will get a warning message.

Using an SSL Certificate is the only way really, if you encrypt it in javascript anyone can read the code and decrypt it.
http://www.startssl.com/

Generate a public/private key pair on the server, along with a randomly-generated salt.
Attach the key pair and salt to the user session object.
Send the public key and the salt to the client-side code.
Use the public key and salt to encrypt the AJAX requests.
This would not be a trivial task. You'll probably find that it's cheaper and more effective to just buy a certificate.
EDIT: This also means that all the regular HTTP traffic (HTML, images, CSS, etc) is sent in the clear. That could be a problem, since it might allow an eavesdropper to indirectly figure out what the user is doing.

I think you should have a look at :
http://assl.sullof.com/assl/
Here is the description of the project :
aSSL is a library distributed under MIT License thats implements a technology similar to SSL without HTTPS.
aSSL enables the client to negotiate a secret random 128-bit key with the server using the RSA algorithm. Once the connection has been established, the data will be sent and received using AES algorithm.
aSSL is composed of some Javascript files and a server side component. Because I have recently changed the negotiation algoritm from RC4 to RSA, only a pure Javascript (ASP) server component is currently available. I will do a porting for the main web languages (PHP, Java, Perl, Python, TKL, etc.) as soon as possible once the library has passed the beta phase.

How can I design a javascript API that allows for cross-domain scripting securely?

I like the way Google Maps' api is consumed, using a script include, but I'm worried:
My api is "semi-private", that is, accessible over the internet but should allow for secure transmission of data and some kind of authentication. The data should remain private over the wire, and one consumer shouldn't be able to get at another's data.
How can I use SSL and some kind of authentication to keep the data secure, but still accessible "horizontally" from a plain HTML page with no server-side proxy required? Do I need to manage keys? How will the keys be posted to the server without being intercepted? Can I use OpenId (or some other 3rd-party authentication) to authenticate api users, or do I have to create my own authentication mechanism? I've been all over Google and can't find a good guide to designing and deploying my API securely.
Right now I'm using REST and AJAX to consume them, but cross-domain calls are impossible. Any help or a pointer in the right direction would be much appreciated.

I'd probably use a dynamically-generated script tag with an SSL URL that included a key in the query string that was public-key encrypted. The server would use the private key to decrypt the query string parameter and return script that included the relevant information (or didn't, if the key was invalid). Or something along those lines. But I'll admit that I haven't actually had to do it in practice.
I'd also look for prior art, like Amazon's S3 service.
So:
User provides secret
Client-side code uses public key to encrypt the secret
JavaScript appends a script tag that includes the URL
Server handles the script request, decrypts the secret, checks it, and sends back the relevant response.
You may well need two cycles, because otherwise the request to the server could be re-used via a man-in-the-middle attack. That would be:
JavaScript appends a script tag that requests a unique key (probably with some confounding information, like the source IP and some random further key)
Server responds with a one-time key tied to that IP
User provides secret
Client-side code uses public key to encrypt the secret, including the unique key from #1
JavaScript appends a script tag that includes the URL
Server handles the script request, decrypts the secret, checks it, and sends back the relevant response.
The response could well be encrypted (to some degree) using the random key included in #1
None of which I've actually done. (Or have I? BWAa-ha-ha-ha...) FWIW.

OAuth might help with this situation by having the user login to the 3rd-party application and allowing your application to access the 3rd-party on their behalf by using a request token when you make xhr requests. http://oauth.net/documentation/getting-started/
========
The reason for using a server-side proxy boils down to the Same-origin policy built into web browsers: http://en.wikipedia.org/wiki/Same_origin_policy
Essentially the browser only allows requests to be made to the address in which the page comes from (e.g. facebook.com can only make requests to facebook.com URIs). A server-side proxy solves this issue by making requests to servers outside the current origin. Server-side proxies are also the best practice for making requests like this.

Check out the opensource javascript Forge project. It provides a javascript TLS implementation that allows secure cross-domain xhr requests. It might be of use to you:
http://digitalbazaar.com/2010/07/20/javascript-tls-1/
http://digitalbazaar.com/2010/07/20/javascript-tls-2/
https://github.com/digitalbazaar/forge
One potential solution:
Set up an Apache server to run your site.
Get an SSL certificate for your site.
Install the apache mod that comes with Forge to setup a cross-domain policy that allows other sites to access yours.
Host Forge's TLS implementation on your site along with your site's certificate in PEM format.
Tell other sites to include the javascript from your site and use it to make secure calls to your site to do whatever it is you want to.

(3rd party) Page uses OAUTH or something similar to authenticate the user and get a token from your server.
Page loads an IFRAME from your server via SSL passing the token along for authentication.
The IFRAME can communicate securely to your server via SSL
Use easyXDM or something similar to communicate between the IFRAME and the 3rd party page, using some limited RPC-like or socket-like API you create.
Or if you really don't trust the third party - do your authentication inside the iframe (no need for oauth then, just use a plain html form) and communicate anything the outer page needs to know about the user using easyXDM.

Not too sure of what the question is exactly, I take it you're attempting to do a jsonp-like call to [https://secure.com] in order to process/display data on [http://regular.com]?
Can the two servers talk to each other? How about something like this:
User logs in on [https://secure.com]
Upon authentication, secure.com generates an token (lets call it syntoken) and passes it directly to regular.com (server-to-server), maybe like a session_id, some arbitrary message, and an otp cipher (lets call it syncipher).
Broswer receives a session_id cookie, and Secure.com then redirects the browser to http://regular.com/setcookieandredirect?session_id=blabla&otpencryptedsynmessage=blabla
Regular.com looks up otp cipher using session_id as a key, and decrypts otpencryptedmessage "blabla."
If decrypted message matches the original message in the syntoken, we can verify user is logged in [regular.com] and regular.com generates another token (lets call it acktoken, lolz) and passes it directly to [secure.com], consisting of session_id, some arbitrary ack message, and a different otp cipher (lets call it ackcipher).
Regular.com then sends the browser a cookie consisting of otpencryptedackmessage (let's name this cookie "verified_session").
Finish loading the page.
From there, you can do jsonp-like calls to
https://secure.com/getscript.js?query=dataname&verifiedtoken=(verified_sessions_cookie_value)
where secure.com/getscript.js will take the verifiedtoken, lookup the ackcipher based on the original cookie session_id sent by [secure.com] as the key, and decrypt the otpencrypedackmessage. If the decrypted message matches the ack message, render the script file.
It's kinda like a 3-way handshake. The secret sauce is that the servers have to be able to talk to each other directly to pass secret keys discretely. You don't have to use the same session_id for both servers, I was just using that as an easy point of reference to find a way to access the syn/ack otp ciphers. The ciphers must be completely hidden from public.