I have a bit of conceptual question regarding the structure of users and their documents.
Is it a good practice to give each user within CouchDB their own database which hold their document?
I have read that couchDB can handle thousands of Databases and that It is not that uncommon for each user to have their database.
Reason:
The reason for asking this question is that I am trying to create a system where a logged in user can only view their own document and can't view any other users document.
Any suggestions.
Thank you in advance.
It’s rather common scenario to create CouchDB bucket (DB) for each user. Although there are some drawbacks:
You must keep ddocs in sync in each user bucket, so deployment of ddoc changes across multiple buckets may become a real adventure.
If docs are shared between users in some way, you get doc and viewindex dupes in each bucket.
You must block _info requests to avoid user list leak (or you must name buckets using hashes).
In any case, you need some proxy in front of Couch to create and prepare a new bucket on user registration.
You better protect Couch from running out of capacity when it receives to many requests – it also requires proxy.
Per-doc read ACL can be implemented using _list functions, but this approach has some drawbacks and it also requires a proxy, at least a web-server, in front of CouchDB. See CouchDb read authentication using lists for more details.
Also you can try to play with CoverCouch which implements a full per-doc read ACL, keeping original CouchDB API untouched, but it’s in very early beta.
This is quite a common use case, especially in mobile environments, where the data for each user is synchronized to the device using one of the Android, iOS or JavaScript (pouchdb) libraries.
So in concept, this is fine but I would still recommend testing thoroughly before going into production.
Note that one downside of multiple databases is that you can't write queries that span multiple database. There are some workarounds though - for more information see Cloudant: Searching across databases.
Update 17 March 2017:
Please take a look at Cloudant Envoy for more information on this approach.
Database-per-user is a common pattern with CouchDB when there is a requirement for each application user to have their own set of documents which can be synced (e.g. to a mobile device or browser). On the surface, this is a good solution - Cloudant handles a large number of databases within a single installation very well. However ...
Source: https://github.com/cloudant-labs/envoy
The solution is as old as web applications - if you think of a mySQL database there is nothing in the database to stop user B viewing records belonging to user A - it is all coded in the application layer.
In CouchDB there is likewise no completely secure way to prevent user B from accessing documents written by user A. You would need to code this in your application layer just as before.
Provided you have a web application between CouchDB and the users you have no problem. The issue comes when you allow CouchDB to serve requests directly.
Using multiple database for multiple users have some important drawbacks:
queries over data in different databases are not possible with the native couchdb API. Analysis on your website overall status are quite impossible!
maintenance will soon becomes very hard: let's think of replicating/compacting thousands of database each time you want to perform a backup
It depends on your use case, but I think that a nice approach can be:
allow access only through virtual host. This can be achieved using a proxy or much more simply by using a couchdb hosting provider which lets you fine-tune your "domains->path" mapping
use design docs / couchapps, instead of direct document CRUD API, for read/write operations
2.1. using _rewrite handler to allow only valid requests: in this way you can instantly block access to sensible handlers like _all_docs, _all_dbs and others
2.2. using _list and _view handlers for read doc/role based ACLs as described in CouchDb read authentication using list
2.3. using _update handlers for write doc/role based ACLs
2.4. using authenticated rewriting rules for read/write role based ACL.
2.3. filtered _changes handler is another way of retrieving all user's data with read doc/role based ACL. Depending on your use case this can effectively simplify as much as possible your read API, letting you concentrate on your update API.
Related
So, I have been learning Vue.js as my first js framework for some time and after I made some simple SPA:s without much interactions with a server I started to wonder: What should a backend be like with Vue? For education purposes I gave it a try and came up with some pattern on my own and now I can't imagine anything else, maybe I got some wrong idea.
What I came up with: I made a simple API with PHP which was receiving requests from the frontend (Vue component methods reacting on UI events) and requesting data from the model or updating data through it.
There are a lot of different Backend Solutions and you should take what fits your websites purpose and also personal preference the best.
If Backend includes Hosting in your case then you basicly have the 2 big options:
a) A Server where you run it ex. via an Reverse Proxy (example: Digital Ocean)
b) A cloud computing Platform (example: AWS, Heroku, App Engine)
But you only need to host it that way if you actually run the app and retrieve dynamic updates on the page, new routes get added when you for example publish a new Post.
If that is not the case then a static hosting provider would be enough, there are 1000´s of them and they are pretty uncomplicated.
If you mean which Database to use, then it also comes down to preference, do you want a SQL Databse or a NoSQL Database like MongoDB? As a personal recommendation I would suggest you to use Firebase as your backend for your experimental app, the free plan is far than enough for testing purposes, you also have a smooth and easy to integrate Authentication System avaliable and you can also take quick advantage of things like Push Messages, Cloud Storage Buckets and more.
Note that Im not related to FB by any means and this is just a personal recommendation, but I feel like your Question is pretty opinion based so maybe be more specific about your goals or just comment down below if you got any more questions.
I am trying to implement caching in Apollo client with the AWS AppSync JavaScript SDK but I am struggling to understand first the best way to use the cache and second what if any changes I need to make to adapt the Apollo V2 tutorials to work with the AppSync SDK.
With regards to using the cache, I have a list of objects that I get, I then want to view and modify a single object from this list. There are lots of tutorials on how to update something in a list, but I would rather run a second query that gets a single object by its ID so that the page will always work without having to go through the list first.
Is the cache smart enough to know that object X got through queries Y and Z is the same object and will be updated at the same time? If not, is there any documentation on how to write an update that will update the object in the list and by itself at the same time?
If no documentation exists then I will try and work it out on my own and post the code (because it will most likely not work).
With regards to the second question I have got the application working and querying the API using Amplify for authentication but I am unsure as to how to correctly implement the cache. Do I need to specify the cache when creating the client or does the SDK have a built-in cache? How do I access the cache? Is it just by querying the client as in these tutorials? https://www.apollographql.com/docs/react/advanced/caching.html
I am going to answer your second question first:
With regards to the second question I have got the application working and querying the API using Amplify for authentication but I am unsure as to how to correctly implement the cache. Do I need to specify the cache when creating the client or does the SDK have a built-in cache? How do I access the cache? Is it just by querying the client as in these tutorials?
Ok. So here is where is gets a little hairy -- it looks like AppSync was deployed at a time when the major client libraries for GraphQL (Apollo, Relay, etc.) were getting an overhaul, so AWS actually created a wrapper around the Apollo Client (probably for stable API purposes) and then exposed their own way of doing things. Just a quick rundown through the code looks like they have their own proprietary and undocumented way of doing things that involves websockets, their authentication protocols, a redux store, offline functionality, ssr, etc). Thus, if it is not explicitly explained here or here, you're in uncharted territory.
Fortunately, all of the stuff that they provided (and much, much more) has now been implemented in the underlying Apollo Client in a documented way. Even more fortunately, it looks like the AppSync client forwards most of the actual GraphQL related stuff directly to the internal Apollo Cache and allows you to pass in config options under cacheOptions, so most of the configuration you can do with the Apollo Client you can do with the AppSync Client (more below).
Unfortunately, you cannot access the cache directly with the AppSync client (they've hidden it to make sure their public API remains stable in the fluctuating ecosystem). However, if you really need more control, most of the stuff they have implemented in the AppSync client could easily be replicated in your own instantiation of an Apollo Client wherein you'd unlock full control (you can use the open-source AppSync code as a foundation). Since GraphQL frontends and backends are decoupled, there is no reason why you couldn't use your own Apollo Client to connect with the AppSync server (for a large, serious project, this is what I would do as the Apollo Client is much better documented and under active development).
Is the cache smart enough to know that object X got through queries Y and Z is the same object and will be updated at the same time? If not, is there any documentation on how to write an update that will update the object in the list and by itself at the same time?
This first part pertains to both the Apollo Client and the AppSync client.
Yes! That's one of the great things about Apollo client - every time you make a query it tries to update the cache. The cache is a normalized key-value store where all of the objects are stored at the top level which the key being a combination of the __typename and id properties of the object. The Apollo client will automatically add __typename to all of your queries (though you will have to add id to your queries manually - otherwise it falls back to just the query path itself as the key [which is not very robust]).
The docs provide a very good overview of the mechanism.
Now, you may need to do some more advanced stuff. For example, if your GraphQL schema uses some unique object identifier other than id, you'll have to provide some function to the dataIdFromObject that maps to it.
Additionally, sometimes when making queries, it is difficult for the cache to know exactly what you are asking for to check the cache before making a network request. To alleviate this problem, they provide the cache redirect mechanism.
Finally, and perhaps most complicated, is how to work with updating the order of stuff in paginated queries (e.g., anything that is in an ordered list). To do this, you'll have to use the #connection directive. Since this is based on the relay connection spec, I'd recommend giving that a skim.
Bonus: To see the cache in action, I'd recommend the Apollo client dev tools. It's a little buggy, but it will at least give you some insight into what it actually happening to the cache locally -- this will not work if using AppSync.
So besides the above information which is all about setting up and configuring the cache, you can also control the data and access to the cache during the runtime of your app (if using Apollo Client directly and not the AppSyncClient).
The Direct Cache Access docs specify the available methods. However, since most of the updates happen automatically just based on the queries you make, you shouldn't have to use these often. However, one use for them is for complicated UI updates. For example, if you make a mutation that deletes an item from a list, instead of requerying for the entire list (which would update the cache, though at the expense of more network data, parsing, and normalization) you could defined a custom cache update using readQuery/writeQuery and the update mutation option. This also plays nicely with optimisticResponse which you should use if you're looking for optimistic UI.
Additionally, you can choose whether you want to use or bypass the cache (or some more advanced strategy) using one of the fetchPolicy or errorPolicy options.
I don't know much about databases, I've been asking a few questions about them lately to get a better understanding but I'm still a bit confused about what does and doesn't need one.
I'm making a simple application using HTML/CSS/JavaScript, it has a few quizzes and "tutorials" targeted towards children. I don't want the next tutorial/quiz to be unlocked until the previous one is completed.
So for that would I need a database so that it "saves" when one is completed? I don't need to save scores or anything like that, they just get to move on once they get a passing score.
Any other requirements such as saving to a profile or needing to persist between sessions (e.g. changing of device)?
Browsers have localStorage APIs now which allow you to save a lot of the data (and keep it for a set duration of time). There are also good'ol'fashioned cookies which allow you save pieces of information as well.
Keep in mind that both of the above mandate the user use the same browser and allow these mechanisms. Obviously using "private"/"incognito" browsing would also affect saving status.
It's up to what you feel the requirements are.
EDIT Just saw your mention of a mobile app. If you're planning on allowing the experience to transcend devices, you'll need a database. otherwise, you'll be relying heavily on if they use cross-device sync (like Chrome and Firefox do with bookmarks, passwords, etc.)
If you don't mind that people can do a "view source" on the webpage or use every browsers' developer tools to find out the answers or move on to the next tutorial or quiz, then you can use cookies to store the user's status. Or you can use the preferable Web Storage API.
You might want to look at Firebase. Using just simple JavaScript on the web browser, you can have users with logins (or just allow them to login via Facebook or other services) very easily. And then you can store and retrieve data very easily as well, like quizzes, tutorials and results. This way nobody can see the answers even if they're adept at analyzing the webpage.
When you don't use database, before any check, you have to load all data in your static page.
So My sloution: store students situation in a cookie. On each page check cookie status and then use Jquery remove() to remove (Client-side) those parts of page that he/she can not access.
EDIT
This wont work when JavaScript is disabled.
There seems to be a lot of ideas but no clarifying on the database subject.
TL;DR is: No.
Now for the specifics. A database is nothing more than a way to store information. While traditional "SQL" databases (it is pronounced "Sequel" as in "My Sequel" for MySQL) have concepts of tables, where you define columns with items to store and saves each row with its value, much like an Excel file, some databases like Redis store key-value pairs and others lide MongoDB store JavaScript Objects.
You can store information in the source code (As Variables possibly) or in a file. A database is a way to organize that information.
With that said, in your case, you probably need a backend or an API. An API is basically a means of communication with a server through AJAX (JavaScript in the browser asks for stuff). That would be your way to retrieve information from the server as needed, so that users wouldn't see the answers before they answer.
With that out of the way, there are some options. FireBase (As noted on other answer) and AppBase are easy ways to integrate this concept with little effort. But they tie you and your information to their system, and they are mostly targeting more resource intensive apps.
Since you are using JS and seem to be enjoying your learning experience, I would suggest you consider suing NodeJS and defining the data as either a JSON file or a variable in JS. You keep working on your problem but add options and get to learn some stuff.
If you decide to integrate a database and possibly do some neat stuff, you have most of the groundwork done already.
If NodeJS picks your interest, Mean.IO and KrakenJS are, in my opinion, the best places to start, though they may both seem overkill in your specific case.
Do consider though: A database is just a small possible piece in a puzzle, and it's mostly a horrible way to name some of the software that tries to organize your information. Consider first if you need to organize information, and what and how do you need to organize, then start thinking if databases are the best way to organize it.
I'm building a relatively complex and data heavy web application in AngularJS. I'm planning to use php as a RESTful backend (with symfony2 and FOSRESTbundle). I have spent weeks looking around for different solutions to on/off line synchronization solutions and there seem to be many half solutions (see list below for some examples). But non of them seem to fit my situation perfectly. How do I go about deciding which strategy will suite me?
What issues that might determine “best practices” for building an on/off line synchronization system in AngularJS and symfony 2 needs some research, but on the top of my head I want to consider things like speed, ease of implementation, future proof (lasting solution), extensibility, resource usage/requirements on the client side, having multiple offline users editing the same data, how much and what type of data to store.
Some of my requirements that I'm presently aware of are:
The users will be offline often and then needs to synchronize (locally created) data with the database
Multiple users share some of the editable data (potential merging issues needs to be considered).
User's might be logged in from multiple devices at the same time.
Allowing large amount of data to be stored offline(up to a gigabyte)
I probably want the user to be able to decide what he wants to store locally.
Even if the user is online I probably want the user to be able to choose whether he uses all (backend) data or only what's available locally.
Some potential example solutions
PouchDB - Interesting strategies for synchronizing changes from multiple sources
Racer - Node lib for realtime sync, build on ShareJS
Meteor - DDP and strategies for sync
ShareJS - Node.js operational transformation, inspired by Google Wave
Restangular - Alternative to $resource
EmberData - EmberJS’s ORM-like data persistence library
ServiceWorker
IndexedDB Polyfill - Polyfill IndexedDB with browsers that support WebSQL (Safari)
BreezeJS
JayData
Loopback’s ORM
ActiveRecord
BackBone Models
lawnchair - Lightweight client-side DB lib from Brian Leroux
TogetherJS - Mozilla Labs’ multi-client state sync/collaboration lib.
localForage - Mozilla’s DOMStorage improvement library.
Orbit.js - Content synchronization library
(https://docs.google.com/document/d/1DMacL7iwjSMPP0ytZfugpU4v0PWUK0BT6lhyaVEmlBQ/edit#heading=h.864mpiz510wz)
Any help would be much appreciated :)
You seem to want a lot of stuff, the sync stuff is hard... I have a solution to some of this stuff in an OSS library I am developing. The idea is that it does versioning of local data, so you can figure out what has changed and therefore do meaningful sync, which also includes conflict resolution etc. This is sort-of the offline meteor as it is really tuned to offline use (for the London Underground where we have no mobile data signals).
I have also developed an eco system around it which includes a connection manager and server. The main project is at https://github.com/forbesmyester/SyncIt and is very well documented and tested. The test app for the ecosystem will be at https://github.com/forbesmyester/SyncItTodoMvc but I have yet to write virtually any docs for it.
It is currently using LocalStorage but will be easy to move to localForage as it actually is using a wrapper around localStorage to make it an async API... Another one for the list maybe?
To work offline with your requeriments I suggest to divide problem into two scenarios: content (html, js, css) and data (API REST).
The content
Will be stored offline by appcache for small apps or for advanced cases with the awesome serviceworkers. Chrome 40+.
The data
Require solve the storage and synchronization and It becames a more difficult problem.
I suggest a deep reading of the Differential Synchronization algorimth, and take next tips in consideration:
Frontend
Store the resource and shadow (using for example url as key) into the localstorage for small apps or into more advanced alternatives (pouchdb,indexdb,...). With the resource you could work offline and when needs synchronize with the server use jsonpath to get diffs between the resource-shadow and to send it to server the PATCH request.
Backend
At backend take in consideration storage the shadow copies into redis.
The two sides (Frontend/Backend) needs to identify the client node, to do so you could use x- syn-token at HTTP header (send it in all request of the client with angular interceptors).
https://www.firebase.com/
it's reliable and proven, and can be used as a backend and sync library for what you're after. but, it costs, and requires some integration coding.
https://goinstant.com/ is also a good hosted option.
In some of my apps, I prefer to have both: syncing db source AND another main database. (mogno/express, php/mysql, etc..). then each db handles what's its best with, and it's features (real-time vs. security, etc...). This is true regardless to sync-db provider (be it Racer or Firebase or GoInstant ...)
The app I am developing has many of the same requirements and is being built in AngularJS. In terms of future proofing, there are two main concerns that I have found, one is hacking attempts requiring encryption and possible use of one time keys and an backend key manager and the other is support for WebSQL being dropped by the standards consortium in preference to indesedDB. So finding an abstraction layer that can support both is important. The solution set I have come up with is fairly straight forward. Where offline data is is loaded first into the UI and a request goes out to the REST Server if in an online state. As for resolving data conflicts in a multi user environment, that becomes a business rule decision. My decision was to simplify the matter and not delve into data mergers but to use a microtime stamp comparison to determine which version should be kept and pushed out to clients. When in offline mode, store data as a dirty write and the push to server when returning to an online state.
Or use ydn-db, which I am evaluating now as it has built in support for AWS and Google cloud storage built in.
Another suggestion:
Yjs leverages an OT-like algorithm to share a wide range of supported data types, and you have the option to store the shared data in IndexedDB (so it is available for offline editing).
A chat application can be implemented using a database (browser send a request conterminously with a particular period and get data from a table which keeps messages).
I want to know that, is there a way to implement a chat application using Ajax and jsp/servlets in HTTP and no database connection ? I know jsp,servlets. How can session,request,responses be handled internally in a jsp/servlet application ?
If you want the non-production, educational version you can use Application Scope:
You can have an application scoped variable holding the chat list
E.g. use <jsp:useBean scope="application"> (one instance per application)
And as long as you have thread safety goggles, and you synchronize where needed, you are fine
But as mentioned, try to check node.js, it seems like the natural candidate for that
Edit:
Note that the application context is per VM, e.g. not the most scaling approach
You can use also ServletContext.setAttribute (same syncronization and scaling issues)
A database is just a glorified file. If your data is simple enough and you don't want to deal with databases just write to a file?
If you are a java guy, what you need seems to me like a good fit with spire.io, a service that allows you to build server-less, database-less applications with a java client.