1) Client Access: Is there anyway to perform CRUD operations on DynamoDB using client side JavaScript (REST/Ajax/jQuery)?
I know Amazon has support for .NET and Java.
2) Server Access: Is there any way we can access DynamoDB using server side JavaScript (Node.js) without having to install Java/.NET on the server?
Update 2012-12-05
There is now an official AWS SDK for Node.js, see the introductory post AWS SDK for Node.js - Now Available in Preview Form for details, here are the initially supported services:
The SDK supports Amazon S3, Amazon EC2, Amazon DynamoDB, and the
Amazon Simple Workflow Service, with support for additional services
on the drawing board. [emphasis mine]
Update 2012-02-27
Wantworthy has implemented a Node.js module for accessing Amazon DynamoDB a week after its launch date, thus covering 2) as well, see dynode:
Dynode is designed to be a simple and easy way to work with Amazon's
DynamoDB service. Amazon's http api is complicated and non obvious how
to interact with it. This client aims to offer a simplified more
obvious way of working with DynamoDB, but without getting in your way
or limiting what you can do with DynamoDB.
Update 2012-02-11
Peng Xie has implemented a Node.js module for accessing Amazon DynamoDB at its launch date basically, thus covering 2) already, see dynamoDB:
DynamoDB uses JSON for communication. [...] This module wraps up the request
and takes care of authentication. The user will be responsible for
crafting the request and consuming the result.
Unfortunately there is no official/complete JavaScript SDK for AWS as of today (see AWS Software Development Kits and boto [Python] for the available offerings).
Fortunately decent coverage for several AWS services in JavaScript is provided by the Node.js library aws-lib already though, which would be a good starting point for adding DynamoDB accordingly. An as of today unresolved feature request to Add support for DynamoDB has been filed already as well.
Further, AWS forum user gmlvsk3 has recently implemented dedicated JavaScript interface for DynamoDB, but supposedly you need [a] Java runtime to run it, because it is based on the Mozilla Rhino JavaScript engine - I haven't reviewed the code in detail yet (at first sight it looks a bit immature though in comparison to e.g. aws-lib, but may cover your needs regardless of course), so you should check it out yourself.
Finally, you can implement JavaScript HTTP Requests to Amazon DynamoDB yourself of course (see the API Reference for Amazon DynamoDB for details):
If you don't use one of the AWS SDKs, you can perform Amazon DynamoDB
operations over HTTP using the POST request method. The POST method
requires you to specify the operation in the header of the request and
provide the data for the operation in JSON format in the body of the
request.
I created a module called Dino to make it easier to work with the AWS SDK in web applications. You can use something like Restify to expose your data to jQuery via a REST interface.
Suppose you wanted to display pages of blog posts for a user. Using Dino and Restify, you would do the following:
server.get('/posts/:user_id', function(req, res, next){
Post.find({
match: {
user_id: req.params.user_id
},
skip: req.params.skip || 0,
take: req.params.take || 10
}, function(err, posts){
return res.send(posts.toJSON());
});
});
Regarding 1), there is now the AWS SDK for JavaScript in the Browser that allows you to access services including DynamoDB.
as for 2) we've been working as well since DDB launch date. One of its key features are simplicity/performance and how close it is (retry behavior, etc) to Amazon official Java/PHP libraries:
https://github.com/teleportd/node-dynamodb
It's successfully used in production at various places with 100+ write/s (at teleportd). Additionally we're working on a a mocked version to enable efficient testing of the library's client code.
Related
I am building an application using ReactJS. I am trying to find out how to store data and to edit it. I tried to store it on my computer with 'fs, 'browserify-fs' but it didn't work.
Should I use express, or is there any other alternatives ?
If you are using React you are operating in the browser. Your option for storage is in local storage. This is explained here.
Examples of code are:
// setter
localStorage.setItem('myData', data);
// getter
localStorage.getItem('myData');
// remove
localStorage.removeItem('myData');
// remove all
localStorage.clear();
Note this is stored in the browser and can be easily cleared. You are going to realize that you need a back end solution. This is a server you can send requests to which has an API (a place you send requests to) which executes some form of operation (normally CRUD - Create Read Update Delete via a REST endpoint or GRAPHQL) to serve you back the data you are requesting from a database (MySQL, Postgres, MongoDB). This is a whole different discussion.
To store an array in local storage you will need to make it a string via JSON.stringify. An example would be:
localStorage.setItem("array", JSON.stringify(array));
In developer tools in Chrome you can go to Application -> Storage -> Local Storage and see what is saved. Here is an example:
If you want to share the data along multiple clients you should use server-side solution or if you just want to save the data for a client only you could use client-side solution provided by #diesel.
Create your own web-server
You need to create web server and a database to store your data. Database is used to store data. You could use: MySQL, PostgreSQL, SQLite3, MongoDB, ... You also need to create web service to make secure database calls.
To create web server you could use Express.js to write your web server easily.
Headless Content Management Systems (abbr: CMS)
If you don't want to spent time on creating your own web-server you could install a headless CMS to read/write your data using api endpoints provided by CMSs. Here's list of headless CMS softwares: headlesscms.org. I tried strapi which has lots of features you might need.
Here's some strapi features:
Open-source
Model builder
Extensible (plugin support)
Content editor (eg: to edit articles)
and many more
Firebase
If you don't want to spend your time on installing CMS software to your server and maintaining it regularly you could use Database service provided by Google Firebase. It is also feature rich too. Here's some features supported by Firebase.
NoSQL Database (to store your data)
Authentication (to authenticate users)
Storage (to store files)
Functions (to write serverless functions)
Machine Learning
and many more
I am new to AWS managed ES. I earlier worked on ES as local server. I am trying to build two-tier web app using the AWS JavaScript sdk (no Node.js). I have created an managed ES node, but not able to find out the way to connect for search and other add/update documents request. AWS SDK currently provides operation related classes and method but not for search and others. can some one help me to bridge this gap? I couldn't find any tutorial or sample code to connect for search operations?
Unfortunately, the SDK doesn't provide a way for you to do reads/writes. It's just done the basic way through HTTP requests but you should lock down your cluster.
This is a good blog post on how to lock down your cluster:
https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
So for example, you could setup an EC2 Instance and give the rights for the EC2 IP to access your elastic search cluster. Then, on the Elasticsearch page in AWS you should see an endpoint URL, just point your read/write requests to there, but lock it down to stay safe.
# Example write to Elasticsearch
curl -XPUT "https://yourESUrl.com/movies/movie/1" -d'
{
"title": "The Godfather",
"director": "Francis Ford Coppola",
"year": 1972
}'
If you need a brush up on the basics of Elasticsearch, check out this article.
In Node you can use the Requests module to do these commands if you like.
Elasticsearch.js is according to https://github.com/elastic/elasticsearch-js
The official low-level Elasticsearch client for Node.js and the browser.
You can use it from your browser to connect to classical Elasticsearch solution.
AWS Elasticsearch require you to sign your request. In node, you can use https://github.com/TheDeveloper/http-aws-es. It is written in ES6.
It is not your ideal solution but you can try to look at the source code to see how the connectors have been coded. https://github.com/TheDeveloper/http-aws-es/blob/master/connector-es6.js
By the way, is it not dangerous to expose your aws credential directly in your client ?
I wrote a javascript file manager to manage user files on an Amazon S3. It uses the AWS Javascript API. Developed it using hard-coded IAM user credentials, and now for production want to use temporary credentials Instead.
My plan is for our PHP server to generate the temp credentials from the IAM credentials, via AJAX callback from the JS code to PHP via STS GetSessionToken. Seems simple enough, but I can't seem to find any documentation on how to pass the IAM key/secret to GetSessionToken in the URL. The examples in the AWS docs all show something like:
https://sts.amazonaws.com/?Version=2011-06-15&Action=GetSessionToken&DurationSeconds=3600&AUTHPARAMS
Where I guess "AUTHPARAMS" is so obvious that I should not need any further explanation. But sadly, I do need further explanation. All I need from the PHP side of things is this one little call, so I didn't really want to install the whole AWS PHP SDK just for this. If I can just find out how to build the URL for this one call, then I can send it off via CURL and be all set. At least that was the plan.
Is there a way to call GetSessionToken directly via the REST api, and pass it the IAM key/secret, or is it really more complicated than that?
You need to learn about AWS API request signing. More specifics here. Note that AWS regions opened after January 30, 2014 require v4 signing, while earlier regions accept v2 or v4 signing.
Or just use the PHP SDK which makes it all much simpler.
For one of my projects I'd like to try out Gigya as my social network connection provider and am writing my app using Node.js. Has anyone done this?
Gigya provides a JavaScript API that is intended to be used on the client.
http://developers.gigya.com/020_Client_API
It should be possible to adapt that for server side use.
Gigya's client side javascript is intended to be run in the browser as much as possible, since they perform 2 part authentication using cookies set by their domains. You can try to port it to run server side, but none of the public methods will work as advertised.
I've written a wrapper for their REST API using their proprietary authentication that I've been using in a work project for a few weeks: https://github.com/jproulx/Gigya-Node-SDK -- note that not everything has been tested thoroughly as I've only needed to use a subset of the socialize services on the server side. It should serve as a good jumping off point to bootstrap something for your needs.
Gigya does not yet have an official Node SDK. However, I've written an SDK that implements the entire service.
In addition to the standard APIs, it contains special support for streaming data from Accounts & DS.
Git: https://github.com/scotthovestadt/node-gigya
Install with "npm install gigya".
I'm researching a possibility of using some cloud storage directly from client-side JavaScript. However, I ran into two problems:
Security - the architecture is usually build on per cloud client basis, so there is one API key (for example). This is problematic, since I need a security per my user. I can't give the same API key to all my users.
Cross-domain AJAX. There are HTTP headers that browsers can use to be able to do cross domain requests, but this means that I would have to be able to set them on the cloud-side. But, the only thing I need for this to work is to be able to add a custom HTTP response header: Access-Control-Allow-Origin: otherdomain.com.
My scenario involves a lots of simple queue messages from JS client and I thought I would use cloud to get rid of this traffic from my main hosting provider. Windows Azure has this Queue Service part, which seems quite near to what I need, except that I don't know if these problems can be solved.
Any thoughts? It seems to me that JavaScript clients for cloud services are unavoidable scenarios in the near future.
So, is there some cloud storage with REST API that offers management of clients' authentication and does not give the API key to them?
Windows Azure Blob Storage has the notion of a Shared Access Signature (SAS) which could be issued on the server-side and is essentially a special URL that a client could write to without having direct access to the storage account API key. This is the only mechanism in Windows Azure Storage that allows writing data without access to the storage account key.
A SAS can be expired (e.g., give user 10 minutes to use the SAS URL for an upload) and can be set up to allow for canceling access even after issue. Further, a SAS can be useful for time-limited read access (e.g., give user 1 day to watch this video).
If your JavaScript client is also running in a browser, you may indeed have cross-domain issues. I have two thoughts - neither tested! One thought is JSONP-style approach (though this will be limited to HTTP GET calls). The other (more promising) thought is to host the .js files in blob storage along with your data files so they are on same domain (hopefully making your web browser happy).
The "real" solution might be Cross-Origin Resource Sharing (CORS) support, but that is not available in Windows Azure Blob Storage, and still emerging (along with other HTML 5 goodness) in browsers.
Yes you can do this but you wouldn't want your azure key available on the client side for the javascript to be able to access the queue directly.
I would have the javascript talking to a web service which could check access rights for the user and allow/disallow the posting of a message to the queue.
So the javascript would only ever talk to the web services and leave the web services to handle talking to the queues.
Its a little too big a subject to post sample code but hopefully this is enough to get you started.
I think that the existing service providers do not allow you to query storage directly from the client. So in order to resolve the issues:
you can write a simple Server and expose REST apis which authenticate based on the APIKey passed on as a request param and get your specific data back to your client.
Have an embedded iframe and make the call to 2nd domain from the iframe. Get the returned JSON/XML on the parent frame and process the data.
Update:
Looks like Google already solves your problem. Check this out.
On https://developers.google.com/storage/docs/json_api/v1/libraries check the Google Cloud Storage JSON API client libraries section.
This can be done with Amazon S3, but not Azure at the moment I think. The reason for this is that S3 supports CORS.
http://aws.amazon.com/about-aws/whats-new/2012/08/31/amazon-s3-announces-cross-origin-resource-sharing-CORS-support/
but Azure does not (yet). Also, from your question it sounds like a queuing solution is what you want which suggests Amazon SQS, but SQS does not support CORS either.
If you need any complex queue semantics (like message expiry or long polling) then S3 is probably not the solution for you. However, if your queuing requirements are simple then S3 could be suitable.
You would have to have a web service called from the browser with the desired S3 object URL as a parameter. The role of the service is to authenticate and authorize the request, and if successful, generate and return a URL that gives temporary access to the S3 object using query string authentication.
http://docs.aws.amazon.com/AmazonS3/latest/dev/S3_QSAuth.html
A neat way might be have the service just redirect to the query string authentication URL.
For those wondering why this is a Good Thing, it means that you don't have to stream all the S3 object content through your compute tier. You just generate a query string authenticated URL (essentially just a signed string) which is a very cheap operation and then rely on the massive scalability provided by S3 for the actual upload/download.
Update: As of November this year, Azure now supports CORS on table, queue and blob storage
http://msdn.microsoft.com/en-us/library/windowsazure/dn535601.aspx
With Amazon S3 and Amazon IAM you can generate very fine grained API keys for users (not only clients!); however the full would be PITA to use from Javascript, even if possible.
However, with CORS headers and little server scripting, you can make uploads directly to the S3 from HTML5 forms; this works by generating an upload link on the server side; the link will have an embedded policy document on, that tells what the upload form is allowed to upload and with which kind of prefix ("directories"), content-type and so forth.