I'm starting a project that basically is a single-page app that downloads and shows a bunch of stats (using d3.js). The data layer is Mongo-powered, served through a RESTful API, and the client app will be coded in Ember.js. We want all data to be exchanged through the API, since we also have some mobile apps in the back burner that will hook to the same API.
I'm debating on whether write the API (using Express.js or other server-side MVC framework), or just serve the API use Deployd and not using a server-side framework at all, besides Deployd. I'll provide some hints about the project characteristics:
The main feature is basically a dashboard that shows aggregated stats that are already computed and stored in the Mongo database.
User interaction is minimal, enough only to allow users to customize their dashboards, but users never upload data (other that customization preferences).
Most of the app is a lot of d3.js to create and render a bunch of graphs, which can customized in many ways.
It requires a very rich and responsive user interface.
I proposed skipping completely the server-side framework, and simply go with a bunch of static HTML+CSS and do all the heavy lifting with a client-side MVC such as Ember.js. Since all data download and upload can be handled by Deployd, a pure static site would load much faster and is also easier to scale. Also, (I think) all user-related data and validation can be done with Deployd itself.
The thing is, some of my colleagues had a heart-stroke when I mentioned this idea. So I'd like a reality check: do I really need a server-side framework besides Deployd to cope with problems I cannot foresee yet? Are the benefits of having a pure static site a good enough tradeoff versus having, say, Express.js just in case?
I haven't worked with Deployd before, but from a quick skim of its docs, it is a server-side framework. It accepts requests and respond with json. It's just oriented to APIs and json and neglects html, unlike, say, default Ruby on Rails.
The main issues I can think of that might arise due to a lack of a traditional server-side framework are things like auth, CORS, and XSS/CSRF/other common security issues. You could cater for this through Deployd if it's built in or easily added, but that may be difficult.
Looking further into Deployd's docs, I see there's a guide for users and CORS. I can't find anything about XSS or CSRF.
Related
Currently, we have a number of GUIs that have been developed using PyQt4 and PyQtGraph.
We need to make these interfaces accessible to a number of remote users and are hoping to do so by serving the GUIs through an HTML/CSS/JavaScript Web-Application.
The link below provides a few suggestions for this task, but these seem to apply to a project that is in its initial stages of creation. I'm familiar with the development of Python-JavaScript Web-Applications using the Pyramid Server and would like to use this same method for providing users access to PyQt4 GUIs that have already been developed.
https://softwareengineering.stackexchange.com/questions/109572/can-i-create-a-desktop-python-gui-application-and-run-it-as-a-web-app
So my question is, can I serve a functional PyQt4 GUI as a widget using HTML/CSS/JavaScript?
This is hard. In theory you could use a VNC-like approach that sends an image of the window to the client browser and sends user events back to the server. This will probably have poor interactivity depending on the networking and would require a lot of server power, depending on the number of users. (For an example of this, see: https://github.com/campagnola/webqt)
You're probably better off distributing the application with something like py2exe, py2app, cx_freeze, etc.
I personally haven't tried it but you could look into Wt: http://www.webtoolkit.eu/wt/
It is a C++ framework that has a Qt flavour. It has a Java version called JWt. Hence, a Jython binding exists for us Python programmers: http://www.webtoolkit.eu/wt/other_language
It is not a direct replacement of Qt, and might need modification to your Qt code to work as a Wt web app, but it certainly seems worth it to look into.
I'm building a relatively complex and data heavy web application in AngularJS. I'm planning to use php as a RESTful backend (with symfony2 and FOSRESTbundle). I have spent weeks looking around for different solutions to on/off line synchronization solutions and there seem to be many half solutions (see list below for some examples). But non of them seem to fit my situation perfectly. How do I go about deciding which strategy will suite me?
What issues that might determine “best practices” for building an on/off line synchronization system in AngularJS and symfony 2 needs some research, but on the top of my head I want to consider things like speed, ease of implementation, future proof (lasting solution), extensibility, resource usage/requirements on the client side, having multiple offline users editing the same data, how much and what type of data to store.
Some of my requirements that I'm presently aware of are:
The users will be offline often and then needs to synchronize (locally created) data with the database
Multiple users share some of the editable data (potential merging issues needs to be considered).
User's might be logged in from multiple devices at the same time.
Allowing large amount of data to be stored offline(up to a gigabyte)
I probably want the user to be able to decide what he wants to store locally.
Even if the user is online I probably want the user to be able to choose whether he uses all (backend) data or only what's available locally.
Some potential example solutions
PouchDB - Interesting strategies for synchronizing changes from multiple sources
Racer - Node lib for realtime sync, build on ShareJS
Meteor - DDP and strategies for sync
ShareJS - Node.js operational transformation, inspired by Google Wave
Restangular - Alternative to $resource
EmberData - EmberJS’s ORM-like data persistence library
ServiceWorker
IndexedDB Polyfill - Polyfill IndexedDB with browsers that support WebSQL (Safari)
BreezeJS
JayData
Loopback’s ORM
ActiveRecord
BackBone Models
lawnchair - Lightweight client-side DB lib from Brian Leroux
TogetherJS - Mozilla Labs’ multi-client state sync/collaboration lib.
localForage - Mozilla’s DOMStorage improvement library.
Orbit.js - Content synchronization library
(https://docs.google.com/document/d/1DMacL7iwjSMPP0ytZfugpU4v0PWUK0BT6lhyaVEmlBQ/edit#heading=h.864mpiz510wz)
Any help would be much appreciated :)
You seem to want a lot of stuff, the sync stuff is hard... I have a solution to some of this stuff in an OSS library I am developing. The idea is that it does versioning of local data, so you can figure out what has changed and therefore do meaningful sync, which also includes conflict resolution etc. This is sort-of the offline meteor as it is really tuned to offline use (for the London Underground where we have no mobile data signals).
I have also developed an eco system around it which includes a connection manager and server. The main project is at https://github.com/forbesmyester/SyncIt and is very well documented and tested. The test app for the ecosystem will be at https://github.com/forbesmyester/SyncItTodoMvc but I have yet to write virtually any docs for it.
It is currently using LocalStorage but will be easy to move to localForage as it actually is using a wrapper around localStorage to make it an async API... Another one for the list maybe?
To work offline with your requeriments I suggest to divide problem into two scenarios: content (html, js, css) and data (API REST).
The content
Will be stored offline by appcache for small apps or for advanced cases with the awesome serviceworkers. Chrome 40+.
The data
Require solve the storage and synchronization and It becames a more difficult problem.
I suggest a deep reading of the Differential Synchronization algorimth, and take next tips in consideration:
Frontend
Store the resource and shadow (using for example url as key) into the localstorage for small apps or into more advanced alternatives (pouchdb,indexdb,...). With the resource you could work offline and when needs synchronize with the server use jsonpath to get diffs between the resource-shadow and to send it to server the PATCH request.
Backend
At backend take in consideration storage the shadow copies into redis.
The two sides (Frontend/Backend) needs to identify the client node, to do so you could use x- syn-token at HTTP header (send it in all request of the client with angular interceptors).
https://www.firebase.com/
it's reliable and proven, and can be used as a backend and sync library for what you're after. but, it costs, and requires some integration coding.
https://goinstant.com/ is also a good hosted option.
In some of my apps, I prefer to have both: syncing db source AND another main database. (mogno/express, php/mysql, etc..). then each db handles what's its best with, and it's features (real-time vs. security, etc...). This is true regardless to sync-db provider (be it Racer or Firebase or GoInstant ...)
The app I am developing has many of the same requirements and is being built in AngularJS. In terms of future proofing, there are two main concerns that I have found, one is hacking attempts requiring encryption and possible use of one time keys and an backend key manager and the other is support for WebSQL being dropped by the standards consortium in preference to indesedDB. So finding an abstraction layer that can support both is important. The solution set I have come up with is fairly straight forward. Where offline data is is loaded first into the UI and a request goes out to the REST Server if in an online state. As for resolving data conflicts in a multi user environment, that becomes a business rule decision. My decision was to simplify the matter and not delve into data mergers but to use a microtime stamp comparison to determine which version should be kept and pushed out to clients. When in offline mode, store data as a dirty write and the push to server when returning to an online state.
Or use ydn-db, which I am evaluating now as it has built in support for AWS and Google cloud storage built in.
Another suggestion:
Yjs leverages an OT-like algorithm to share a wide range of supported data types, and you have the option to store the shared data in IndexedDB (so it is available for offline editing).
I'm about to embark on a large project based around mobile web apps. I am going to create a mobile web app that will have multiple screens, a search system and a few other features.
I need to establish a primary framework for this application. It seems I have two options; multiple separate HTML pages, or alternatively, use AJAX to download the contents and move divs in and out of the viewport.
I don't have to worry about SEO. I'm primarily looking for good performance. I understand Stack Overflow is a site about questions and answers and this does somewhat involve opinion, but I feel that based on my criteria there should be a solid answer to this question.
I'd highly appreciate it if somebody could share what they think the best option would be for me on this project.
What I believe you're referring to by using AJAX to download the contents is commonly known as a single-page web app. These generally involve rendering data using templates on the client side and sending and receiving the data via AJAX.
For these, there are plenty of excellent JavaScript framework options, including:
Backbone.js
Knockout.js
Angular.js
Ember.js
Of these, Knockout is probably the easiest to learn (seriously, the tutorial is brilliant), but the lack of routing lets it down somewhat. I tend to use Backbone for this type of thing myself, although if I were starting over I would probably go for Angular as it's made by Google and seems to have more momentum. But try a few tutorials and see which one works best for you!
All of these offer a lot of very handy functionality for your use case, as they enable you to load data via AJAX and render it on the client side quite easily. There is a learning curve for all of them, but they result in a much more maintainable and testable client-side application. Believe me, I have tried to build such an application with just jQuery, and I would recommend you avoid it as it gets very painful, very quickly!
Your mileage may vary, but by and large a single-page web app like this should perform better than a more traditional web page because once the application has loaded, all the communication with the server is just sending and receiving JSON. It will also likely be simpler as you can use a lightweight REST framework like Slim, Sinatra, Express or Flask on the server side, and just serve up the static content on the home page, then do everything else via AJAX.
Choosing the "right" web framework is quite challenging task, at least in Java we have a lot of them. But looking at JavaScript frameworks like AngularJS I doubt if we really need something heavy at server. Usually web framework is responsible for routing, templating, building pretty URLs and some other stuff. With AngularJS we can move all these responsibilities to client side. Then the backend becomes nothing more than REST listener and data validator. A thin layer between your application logic and client view. So why do we need web frameworks now if all we want is a REST listener?
At the moment I found two points which must be handled by server side: authentication/authorization and things requiring 'pushing' like Comet. Are these criterias enough to choose the "right" framework?
I'll give you one more capability that I've seen require back-end server support. It's those pages which generate a file. For example, they get a file from a third party and then hand it to the client as though they produced it directly, or they are generating a JPEG/PNG/GIF image on the fly, or perhaps a CSV/XLS dump of data. There may be ways to generate those on the fly from the front end and make them available for download, but sometimes the back-end is just easier for those.
Other than that, your assessment is 100% correct. You literally need less server for apps built with AngularJS than was needed with the previous request/response model of ASP/JSP/PHP/etc.
However, just because you need less doesn't mean you need nothing. Issues like data caching and how user sessions are handled can still come up even for smaller servers as you scale. But it has definitely opened things up for tech like Node.js to be considered that I would not have given much thought to a few years back.
I am just starting to look at MVC structure, first i looked at how backbone.js worked, and now I have just completed rails for zombies, by Code School. I know that I haven't delved too far into any of this, but I had a question to begin with.
Can you use these libraries together?
I have learned how to create models, views, etc in both, but when creating a real application do you use both backbone and rails?
If so...
When do you use a backbone.js model vs. a rails model?
Maybe I am just getting ahead of myself and need to keep practicing and doing tutorials but I couldn't seem to find anything directly on this.
Thanks!
Before anything else I'd suggest taking a look at thoughtbot's Backbone.js on Rails book, which is a great starting point, although aimed at an intermediate to advanced audience. I bought this book having already worked with rails but as a total backbone.js beginner and it has served me very well.
Beyond that, there are some fundamental issues with combining these frameworks which go beyond the details covered in this book and other books. Below are some things I'd suggest you think about, from my own experiences pairing RoR and backbone.js. This is a long answer and strays a bit from the specifics of your question, but I hope it might help you out in the "big picture" sense of understanding the problem you're facing.
Rails: Web Framework vs API
The first thing you confront when using backbone.js on top of a rails application is what to do about views, but this is really just the surface of a much deeper issue. The problem goes to the very heart of what it means to create a RESTful web service.
Rails is set up out of the box to encourage its users to create RESTful services, by structuring routing in terms of a set of resources accessed at uniform URIs (defined in your routes.rb file) through standard HTTP actions. So if you have a Post model, you can:
Get all posts by sending GET request to /posts
Create a new post by sending a GET request to /posts/new, filling out the form and sending it (a POST request) to /posts
Update a post with id 123 by sending a GET request to /posts/123/edit, filling out the form and sending it (a PUT request) to posts/123
Destroy a post with id 123 by sending a DELETE request to /posts/123
The key thing to remember about this aspect of Rails is that it is fundamentally stateless: regardless of what I was doing previously, I can create a new Post simply by sending a POST request with a valid form data to the correct URI, say /posts. Of course there are caveats: I may need to be logged in (have a session cookie identifying me), but in essence Rails doesn't really care what I was doing before I sent that request. I could follow it up by updating another post, or by sending a valid action to whatever other resources are made available to me.
This aspect of how Rails is designed makes it relatively easy to turn a (Javascript-light) Rails web application into an API: the resources will be similar or the same, the web framework returning HTML pages while the API (typically) returns data in JSON or XML format.
Backbone.js: A new stateful layer
Backbone is also based on RESTful resources. Whenever you create, update or destroy a backbone.js model, you do so via the standard HTTP actions sent to URIs which assume a RESTful architecture of the kind described above. This makes it ideal for integrating with RESTful services like RoR.
But there is a subtle point to be stressed here: backbone.js integrates seamlessly with Rails as an API. That is to say, if you strip away the HTML views and just use Rails for serving RESTful resources, integrating with the database, performing session management, etc., then it integrates very nicely with the structure that backbone.js provides for client-side code. Many people argue that there's nothing wrong with using rails this way, and I think in many ways they are right.
The complications arise though from the issue of what to do with that other part of Rails that we've just thrown away: the views and what they represent.
Stateful humans, stateless machines
This is actually more important than it may initially seem. HTML views represent the stateless interface that humans use for accessing the RESTful resources your service provides. Doing away with them leaves you with two access points:
For humans: a rich, client-side interface provided by the backbone.js layer (stateful)
For machines: a resource-oriented RESTful API provided by the rails layer (stateless)
Notice that there is no longer a stateless (RESTful) interface for humans. In contrast, in a traditional rails app with an API, we had something closer to this:
HTML resources for humans (stateless)
JSON/XML resources (API) for machines (stateless)
The latter two interfaces for accessing resources are much closer in nature to each other than the previous two. Just think for example of rails' respond_with, which takes advantage of the similarities to wrap various RESTful responders in a unified method.
Working together
This might all seem very abstract and beside the point, I know. To try to make it more concrete, consider the following problem, which gets back to your question about getting rails and backbone.js to work together. In this problem, you want to:
Create a web service with a rich client-side experience using backbone.js, with rails as the back end serving resources in JSON format.
Use pushState to give each page in the app a URL (e.g. /posts/123) which can be accessed directly (by entering it into the browser bar).
For each of these URLs, also serve an HTML page for clients without javascript.
These are not unusual demands for a modern web service, but they create a complex challenge. To make a long story short, you now have to create two "human-oriented" layers:
Stateful client-side interface (backbone.js templates and views)
Stateless HTML resources (Rails HTML views)
The complexity of actually doing this leads many nowadays to abandon the latter of these two and just offer a rich client-side interface. What you decide to do depends on your goals and what you want to achieve, but it's worth thinking about this problem carefully.
As another possible reference for doing that, I'd suggest having a look at O'Reilly's RESTful Web Services. It might seem odd to be recommending a book on REST in a question about Rails and Backbone.js, but actually I think this is the key piece that fits these very different frameworks together, and understanding it more fully will help you take advantage of the strengths of both.
Yes, you can use both side by side. Backbone is for storing and manipulating data within the client browser. It generally needs a server to talk to and fetch the data from. This is where Rails comes in. You can have a web application without heavy client-side code. Backbone is for building out sites that feel more like apps--think of Gmail or Pandora.
I advise just learning Rails first. Once you can get static pages loading and styled as you wish, then understanding Backbone's place will make more sense
I've used rails as a backend server to serve a fairly large website, which included a few one-page apps (built in backbone).
I'd suggest the backbone-on-rails gem. The idea is that your rails server will serve up the backbone app as a script tag in one of your views. You keep your backbone app itself in the rails app/assets folder.
Backbone understands rails routing conventions, and you just need to give it a path to a json api that rails can almost generate for you with rails generate resource.
Other than the syncing between the models, your backbone apps and rails apps are fairly separate. Backbone and Rails don't have quite the same MVC model, but getting them to cooperate is pretty easy.