I have a simple offline html5/javascript single-html-file web application that I store in my dropbox. It's a sort of time tracking tool I wrote, and it saves the application data to local storage. Since its for my own use, I like the convenience of an offline app.
But I have several computers, and I've been trying to come up with any sort of hacky way to synchronize this app's data (which is currently using local storage) between my various machines.
It seems that chrome allows synchronization of data, but only for chrome extensions. I also thought I could perhaps have the web page automatically save/load its data from a file in a dropbox folder, but there doesn't appear to be a way to automatically sync with a specific file without user prompting.
I suppose the "obvious" solution is to put the page on a server and store the data in a database. But suppose I don't want a solution which requires me to maintain apps on a server - is there another way, however hacky, to cobble together synchronization?
I even looked for a while to see if there was a vendor offering a web database service - where I could, say, post/get a blob of json on demand, and then somehow have my offline app sync with this service, but the same-origin policy seems to invalidate that plan (and besides I couldn't find such a service).
Is there a tricky/sneaky solution to this problem using chrome, or google drive, or dropbox, or some other tool I'm not aware of? Or am I stuck setting up my own server?
I have been working on a Project that basically gives you versioned localStorage with support for conflict resolution if the same resource ends up being edited by two different clients. At this point there are no drivers for server or client (they are async in-memory at the moment for testing purposes) but there is a lot of code and abstraction to make writing your own drivers really easy... I was even thinking of doing a dropbox/google docs driver myself, except I want DynamoDB/MongoDB and Lawnchair done first.
The code is not dependent on jQuery or any other libraries and there's a pretty full features (though ugly) demo for it as are well.
Anyway the URL is https://github.com/forbesmyester/SyncIt
Apparently, I have exactly the same issue and invetigated it thoroghly. The best choice would be remoteStorage, if you could manage to make it work. It allows to use 3rd party server for data storage or run your own instance.
Related
A few days ago I decided to make my own "interface" to make it easier to organize (and work with) some of my personal files. You know when a lot of related data, pictures and links are right in front of you and you can change them in a couple of clicks, this is very convenient.
I started by studying HTML, CSS and JS, because I thought that the changes made to the local page would be saved somewhere on my PC so I can just run Index.html and do whatever I want. But they didn't. Refreshing the page erased all changes.
Using browser localstorage does not suit me, because if I change the browser, the data will be lost. I wanted it to just open with Index.html and work fine even if I change my browser or move the site folder to another computer.
Then I decided to learn more about server-side languages (such as PHP or Node.js) because they are directly related to databases, so I was hoping to save changes through them. But these languages required me to really open the server, with ip and port tracking. And I just wanted to open a local page through one file, without any ports or connections via the console. So this method scared me off quickly.
So is there an easy way to make a local page like this? Maybe I have not studied well one of the above methods and it has this opportunity?
Or the best I can hope for is a simple application that will use that local page as an interface to interact with data? I accidentally heard about this possibility a long time ago. Then I will ask you to give at least a hint as to which language to choose for this.
I don't understand well everything that lies outside of vanilla HTML, CSS and JS, so a complete study of a complex language like Java or Python will be too difficult for me, and the goal is not worth such a lot of effort.
I hope I understand correcly what you are trying to do.
If your goal is to make an application to manage your files, I think the simplest solution will be, as you said, to look into NodeJS and the File system api which will let you interact with your files through javascript code.
Your program will have to be in two part that will have to interact:
the "front" html page
the "back" nodejs script
The downside is that you'll have to go deeper into your study of the language to learn how to create the interactions you want between your html file and your NodeJS application.
However, there is no need to open your server to the web to make it work. The NodeJS application can be set to listen to requests from only the computer that runs it (localhost).
I obviously can't get too much into details without knowing precisely what you want to do but you'll probably have to learn to make a local server with node (search "nodejs http" or "nodejs express"), then make requests to it via the html page's scripts (search "ajax request").
What you need to look into are (web based) content management systems. like strapi or "grand old dame" WordPress.
I'm building a relatively complex and data heavy web application in AngularJS. I'm planning to use php as a RESTful backend (with symfony2 and FOSRESTbundle). I have spent weeks looking around for different solutions to on/off line synchronization solutions and there seem to be many half solutions (see list below for some examples). But non of them seem to fit my situation perfectly. How do I go about deciding which strategy will suite me?
What issues that might determine “best practices” for building an on/off line synchronization system in AngularJS and symfony 2 needs some research, but on the top of my head I want to consider things like speed, ease of implementation, future proof (lasting solution), extensibility, resource usage/requirements on the client side, having multiple offline users editing the same data, how much and what type of data to store.
Some of my requirements that I'm presently aware of are:
The users will be offline often and then needs to synchronize (locally created) data with the database
Multiple users share some of the editable data (potential merging issues needs to be considered).
User's might be logged in from multiple devices at the same time.
Allowing large amount of data to be stored offline(up to a gigabyte)
I probably want the user to be able to decide what he wants to store locally.
Even if the user is online I probably want the user to be able to choose whether he uses all (backend) data or only what's available locally.
Some potential example solutions
PouchDB - Interesting strategies for synchronizing changes from multiple sources
Racer - Node lib for realtime sync, build on ShareJS
Meteor - DDP and strategies for sync
ShareJS - Node.js operational transformation, inspired by Google Wave
Restangular - Alternative to $resource
EmberData - EmberJS’s ORM-like data persistence library
ServiceWorker
IndexedDB Polyfill - Polyfill IndexedDB with browsers that support WebSQL (Safari)
BreezeJS
JayData
Loopback’s ORM
ActiveRecord
BackBone Models
lawnchair - Lightweight client-side DB lib from Brian Leroux
TogetherJS - Mozilla Labs’ multi-client state sync/collaboration lib.
localForage - Mozilla’s DOMStorage improvement library.
Orbit.js - Content synchronization library
(https://docs.google.com/document/d/1DMacL7iwjSMPP0ytZfugpU4v0PWUK0BT6lhyaVEmlBQ/edit#heading=h.864mpiz510wz)
Any help would be much appreciated :)
You seem to want a lot of stuff, the sync stuff is hard... I have a solution to some of this stuff in an OSS library I am developing. The idea is that it does versioning of local data, so you can figure out what has changed and therefore do meaningful sync, which also includes conflict resolution etc. This is sort-of the offline meteor as it is really tuned to offline use (for the London Underground where we have no mobile data signals).
I have also developed an eco system around it which includes a connection manager and server. The main project is at https://github.com/forbesmyester/SyncIt and is very well documented and tested. The test app for the ecosystem will be at https://github.com/forbesmyester/SyncItTodoMvc but I have yet to write virtually any docs for it.
It is currently using LocalStorage but will be easy to move to localForage as it actually is using a wrapper around localStorage to make it an async API... Another one for the list maybe?
To work offline with your requeriments I suggest to divide problem into two scenarios: content (html, js, css) and data (API REST).
The content
Will be stored offline by appcache for small apps or for advanced cases with the awesome serviceworkers. Chrome 40+.
The data
Require solve the storage and synchronization and It becames a more difficult problem.
I suggest a deep reading of the Differential Synchronization algorimth, and take next tips in consideration:
Frontend
Store the resource and shadow (using for example url as key) into the localstorage for small apps or into more advanced alternatives (pouchdb,indexdb,...). With the resource you could work offline and when needs synchronize with the server use jsonpath to get diffs between the resource-shadow and to send it to server the PATCH request.
Backend
At backend take in consideration storage the shadow copies into redis.
The two sides (Frontend/Backend) needs to identify the client node, to do so you could use x- syn-token at HTTP header (send it in all request of the client with angular interceptors).
https://www.firebase.com/
it's reliable and proven, and can be used as a backend and sync library for what you're after. but, it costs, and requires some integration coding.
https://goinstant.com/ is also a good hosted option.
In some of my apps, I prefer to have both: syncing db source AND another main database. (mogno/express, php/mysql, etc..). then each db handles what's its best with, and it's features (real-time vs. security, etc...). This is true regardless to sync-db provider (be it Racer or Firebase or GoInstant ...)
The app I am developing has many of the same requirements and is being built in AngularJS. In terms of future proofing, there are two main concerns that I have found, one is hacking attempts requiring encryption and possible use of one time keys and an backend key manager and the other is support for WebSQL being dropped by the standards consortium in preference to indesedDB. So finding an abstraction layer that can support both is important. The solution set I have come up with is fairly straight forward. Where offline data is is loaded first into the UI and a request goes out to the REST Server if in an online state. As for resolving data conflicts in a multi user environment, that becomes a business rule decision. My decision was to simplify the matter and not delve into data mergers but to use a microtime stamp comparison to determine which version should be kept and pushed out to clients. When in offline mode, store data as a dirty write and the push to server when returning to an online state.
Or use ydn-db, which I am evaluating now as it has built in support for AWS and Google cloud storage built in.
Another suggestion:
Yjs leverages an OT-like algorithm to share a wide range of supported data types, and you have the option to store the shared data in IndexedDB (so it is available for offline editing).
I've been doing some research on the HTML5 local storage/offline capabilities and I'm beginning to get a better understanding of it. I've been designing a billing application with Grails and I'm just wondering if it is possible to set up a Grails application to be an offline web application, like you can with HTML5 (by utilizing a Manifest file). If so, would it be set up in the same way that you'd set it up with HTML5, or would there be any key differences to make a note of? Also, what is the best way to implement the local storage functionality of HTML5 in Grails? Is there any plugin or framwork available for this purpose . Has anyone ever used this plugin/framework before, and if so, do you know if there are any guides or examples that use it anywhere that would be really able help me get started with it?
I'd appreciate the information on making a Grails app work "offline
Thanks in advance!
I am developing an offline HTML5 app, myself. The biggest difference is that it is a single page rather than many small ones (but it might work that way if you tried it; you would need to be sure it got all the data it needs at startup).
Don't forget to add the application cache mime type to your configuration.
I never thought to look for a plugin for local storage. In my project, the local tables (I'm using WebSQL because it's in Webkit browsers) are different from the domain classes because they have a different function; that is, they have to hold the data for the domain tables untilthe device can sync with the host.
I hope there is something useful here.
Ed
I want to create an internally used web app that can be run with just a copy of the web app and the DB (anything from a text file to MS Access/Excel would work fine). Is this possible? I don't want users to have to setup a SQL server to get the app to work. Having the files necessary to run the web app stored on a shared network drive would be ideal, for example. The problem is that JS can't write to a DB. Is there anything that can use to do this?
Like mentioned, I can assume that Access/Excel are installed, if there's anything that might help there.
It's most certainly possible. W3 has put up the specs for a client side database that can be accessed by JavaScript. The modern browsers have good support for it, and since this is for an internal application, you would have some level of control I believe.
Checkout this slide that shows a live demo of Indexed Database. The full spec can be found here. See this link for browsers that currently support IndexedDB. Here's another set of slides showcasing how to use IndexedDB.
However, with this approach, each user's browser has its own DB locally. If you want a centralized DB, then you will need a server.
You can perform database transactions with JavaScript. This is generally discouraged, because it has terrible security implications. However, in a completely local environment, you are probably not causing any additional security risks. (Because, your database is already on the user's machine.) You can see an example of how to use ADO in JavaScript at How to connect to SQL Server database from JavaScript in the browser? .
Possible, yes, Like Making cars that can float in the sea but could not work on dry roads.
Use winforms or something similar. Use the right tool.
If you insists, Firefox plugins can behave in the way you mentioned, and there is a way to bundle a web application with it's server (check the beginner tutorials for RoR to have an example for something similar with webrick).
If I understand your requirements, you might look into ColdFusion.
For example, you can run a DB query pretty simply, check here, in Adobe
I was fascinated by Google Gears and its potential use in online game development, particularly massive online game development. One could take the game resources and store them locally using ResourceStore, thus reducing game load time, server bandwidth issues, etc. I have therefore welcomed the news that HTML5 supports offline storage.
However, from what I can tell it only supports a manifest file for resource caching, which looks like something ManagedResourceStore is supposed to offer. I didn't study either in detail.
I also didn't find something that would allow programatically loading resources and caching them (as apparently ResourceStore allows).
Is it possible to programatically control which resources should be cached? Or would I have to store each "map" on a separate page with separate manifest file (with this being in fact done through a server side script, instead of literally creating pages and manifests -- this is still fugly)?
If you're asking about design work, it seems like you're looking for Programmable HTTP Caching and Serving. If you actually wanted to know if there's something like this in shipping browsers, then I don't know, but doubt it.