I have a ReactJS application which is a personal/news site (half of both). It has a publications section which requires a database to list and store all of the entries. The only user that will be able to edit/add/delete publications is the admin. Also, the format of every publication will be the same. So a separate backend application would be like inventing a bicycle. Too complex for a simple thing like publications.
The first thing i tried to use is the Firebase from google. (https://firebase.google.com/). It is really simple, has real-time synchronization (which is useless in my case) and also does all the dirty job for you. That is really awesome, but i got a problem. The website contains many high-resolution images and many of them have to be loaded at the same time. I realised, that the website takes about 2-3 seconds to load and that's with only a couple of images.
Than i thought about a local storage on the same host. I did a few test with local storage and, of course, it's many times faster (3s vs milliseconds). So now i think about a JSON file which will contain all the information (Image paths, paragraph text, names e.t.c).
1 - will it be safe to store everything on the same host with such an easy to access DB? It's okay if users can read the files cause they will be available on the website anyway. But is there a possibility that they also will be able to change them?
2 - to manipulate the DB i think about making a simple app-interface which will allow to edit the publications visually. But will it be safe (x2) to store that interface on the same host? Of course it won't be running on a public IP and will be stored in another directory.
Related
Currently I'm trying to learn nativescript and for this I thought about doing an App like 'Anki'
But while thinking about the data storage I stumpled upon the problem on how to save my flash cards locally for keeping the app offline (for example with SQLite), save the users time when to reflect each card (e.g. to show again in 10 minutes or 1 day) AND have an update functionality to update the database with new cards without deleting the users data.
What's the best way to solve that problem, especially when I want to provide the updates with an App-Update and without fetching everything from an external database?
I don't have any code yet, therefore a recommendation on how to solve that would be nice.
There is several methods in NativeScript you can use:
NativeScript-Sqlite (disclaimer: I'm the author)
This allows full access to Sqlite for saving and loading items; you can have as big of databases as you need and Sqlite is very fast. Sqlite's biggest drawback is speed of writes; if you have a LOT of writing it can be slower than just writing to a file yourself.
NativeScript-LocalStorage (disclaimer again: I'm the author)
This is more geared to smaller data sizes; as when the app starts and saves it has to load the entire json backed data store into memory. This is really fast over all; but not something you want to use for 10's of thousands of records.
NativeScript-Couchbase
This uses sqlite for local storage and can use couchbase for the remote storage; very nice for having syncable storage - couchbase can be your own server or a leased or rented server.
NativeScript-Firebase
This is also very useful for having syncable storage; however Google charges for FireBase at a certain point.
Built in AppSettings.
This is really designed for a few application settings, not designed for lots of data. But useful for the smaller amounts of data.
Role your own to the file system.
I have done this in a couple of my projects; basically a hybrid between my localstorage plugin and a mini-sql type system. One project was very much write dependent so it made more sense to generate the 20 or so separate files on the phone for each table because I could save them much quicker than inserting/replacing > 100,000 records each time the app started up into sqlite. Had minimal searching needs.
Your storage really needs to be dependent upon what you are doing; it is a balancing act. Lots of searchable data; sqlite wins in almost all cases. Lots of frequent writing; something you create might be a lot faster.
I'm developing an Android application, typically with a fairly small database (hundreds to a thousand or so main entries, plus similar order of magnitude in joined tables) populated by the user. There is also a file associated with each main entry, but those are kept separate, and not part of this question.
In my current prototype, the structured data is stored in a SQLite database in the usual way, and synced between devices by uploading the file to an Application Folder on Google Drive. This allows me to avoid having to provide any user storage, which is something I hope to stick to.
Storing files here also has the happy side effect that I can access them via JavaScript code in a web page using the Google Drive API. But JavaScript interacting with a remote SQLite file is an incredibly ugly thing to attempt. (Downloading the database server-side and handling it there is another option which sounds not great either.) I am wondering if anyone has any recommendations of the best practice approach here, given I would like to have a web-based version of the app accessing the same data.
The obvious most sensible idea seems to be
Exporting from SQLite to some other structured format XXX (like XML, JSON, CSV, ...) first, uploading that XXX file to Google Drive, so that JavaScript can straightforwardly interact with it, and then on sync reimporting this XXX data back into SQLite.
though I'm happy to hear any criticisms/improvements on this idea.
I'm mainly interested in best-practice recommendations of efficient ways to approach these problems. Question: What file format XXX would be good; and do you have any recommended methods for (a) exporting and importing in Java from SQLite to XXX, (b) interacting in preferably client-side JavaScript with a remote Google Drive file in format XXX? (If maintaining server-side code to interact with XXX is substantially better practice do say.)
It depends on how many clients your app will manage
Exporting from SQLite to some other structured format XXX (like XML, JSON, CSV, ...) first, uploading that XXX file to Google Drive, so that JavaScript can straightforwardly interact with it, and then on sync reimporting this XXX data back into SQLite.
IMHO, this is quite a bad idea for quite a few reasons:
If multiple users try to change data at the same time, you will have a tough problem managing all those data entries and possible conflicts
Providing direct access to your data layer is ALWAYS a bad idea since any client can wipe out all your data
If you have very few users or are making it as a personal project, you can stick with SQLite.
Kripken, the developer of Emscripten (the C/C++ to Javascript LLVM compiler) has compiled the SQLite Library to Javascript to make the sql.js library. You can find it on GitHub. There's also a demo page of sql.js
But if you have a lot of users, you might want to use Firebase by Google. It gives you a database and a REST API to use it. There are libraries to access Firebase from different programming languages too.
Cheers!
I have a simple audio sequencer application using javascript and HTML5. The goal of the application is to let users make a beat and save it so other people can open it up, add on to it, then re save. The issue I'm having is the save/open feature. How Do I add this? Or at least where can I start looking fro this solution?
If you are asking for an only-JS solution to this:
Opening Files
You can use the FileAPI to open the files from your users local machine, as long as its your users who are selecting the files manually(you are not digging into the file system yourself).
Here is nice demo for this functionality: http://html5demos.com/file-api
Saving Files
However, saving files was a W3C recommendation that got ditched so your only choice here is to allow your users to download the file. See this relevant question I made some time ago
If I were doing this for a class or prototype I might use PUT or POST requests to save the beat sequence in json form to a web server. When it's time to save your work, take your beat sequence data, convert from a js (I presume) object to a json string and PUT it to an appropriate url on your server. Give it a name and use the name in the url path, like "myserver.com/beats/jpbeat1.json".
When you want to retrieve that beat, issue an HTTP get to the url you saved. You can, of course also maintain and rewrite a directory page in json or html, or you can just navigate the directory offered by your web server. Most developers will think this crude, I imagine, but in a race for time, crude can be good if it meets the requirements.
Choose a web server, the easiest one for you. I might use Amazon S3. Costs money, but available to the internet, has authentication, security features, and supports http GET, PUT of static files. There are many other site hosting services.
For a local install, I might use Node.js. Also very easy to set up to serve static content. For example, the Express framework has static content service and easy examples how to set it up if you are comfortable with Javascript. With Node, you can make the site a bit fancier, by writing server side code to maintain an index, for example. Any real world implementation would require at least some server side code.
There are many other choices. Most developers would want a database, but you don't have to have one if simple files are ok for your class project. It all depends on your requirements.
I hope those ideas are helpful.
I don't know much about databases, I've been asking a few questions about them lately to get a better understanding but I'm still a bit confused about what does and doesn't need one.
I'm making a simple application using HTML/CSS/JavaScript, it has a few quizzes and "tutorials" targeted towards children. I don't want the next tutorial/quiz to be unlocked until the previous one is completed.
So for that would I need a database so that it "saves" when one is completed? I don't need to save scores or anything like that, they just get to move on once they get a passing score.
Any other requirements such as saving to a profile or needing to persist between sessions (e.g. changing of device)?
Browsers have localStorage APIs now which allow you to save a lot of the data (and keep it for a set duration of time). There are also good'ol'fashioned cookies which allow you save pieces of information as well.
Keep in mind that both of the above mandate the user use the same browser and allow these mechanisms. Obviously using "private"/"incognito" browsing would also affect saving status.
It's up to what you feel the requirements are.
EDIT Just saw your mention of a mobile app. If you're planning on allowing the experience to transcend devices, you'll need a database. otherwise, you'll be relying heavily on if they use cross-device sync (like Chrome and Firefox do with bookmarks, passwords, etc.)
If you don't mind that people can do a "view source" on the webpage or use every browsers' developer tools to find out the answers or move on to the next tutorial or quiz, then you can use cookies to store the user's status. Or you can use the preferable Web Storage API.
You might want to look at Firebase. Using just simple JavaScript on the web browser, you can have users with logins (or just allow them to login via Facebook or other services) very easily. And then you can store and retrieve data very easily as well, like quizzes, tutorials and results. This way nobody can see the answers even if they're adept at analyzing the webpage.
When you don't use database, before any check, you have to load all data in your static page.
So My sloution: store students situation in a cookie. On each page check cookie status and then use Jquery remove() to remove (Client-side) those parts of page that he/she can not access.
EDIT
This wont work when JavaScript is disabled.
There seems to be a lot of ideas but no clarifying on the database subject.
TL;DR is: No.
Now for the specifics. A database is nothing more than a way to store information. While traditional "SQL" databases (it is pronounced "Sequel" as in "My Sequel" for MySQL) have concepts of tables, where you define columns with items to store and saves each row with its value, much like an Excel file, some databases like Redis store key-value pairs and others lide MongoDB store JavaScript Objects.
You can store information in the source code (As Variables possibly) or in a file. A database is a way to organize that information.
With that said, in your case, you probably need a backend or an API. An API is basically a means of communication with a server through AJAX (JavaScript in the browser asks for stuff). That would be your way to retrieve information from the server as needed, so that users wouldn't see the answers before they answer.
With that out of the way, there are some options. FireBase (As noted on other answer) and AppBase are easy ways to integrate this concept with little effort. But they tie you and your information to their system, and they are mostly targeting more resource intensive apps.
Since you are using JS and seem to be enjoying your learning experience, I would suggest you consider suing NodeJS and defining the data as either a JSON file or a variable in JS. You keep working on your problem but add options and get to learn some stuff.
If you decide to integrate a database and possibly do some neat stuff, you have most of the groundwork done already.
If NodeJS picks your interest, Mean.IO and KrakenJS are, in my opinion, the best places to start, though they may both seem overkill in your specific case.
Do consider though: A database is just a small possible piece in a puzzle, and it's mostly a horrible way to name some of the software that tries to organize your information. Consider first if you need to organize information, and what and how do you need to organize, then start thinking if databases are the best way to organize it.
I'm building my first site using this framework, i'm remaking a website i had done in PHP+mySQL and wish to know something about performance... In my website, i have two kinds of content:
Blog Posts (for 2 sections of the site) - these have the tendency to one day sum to thousands of records and, are more often updated
Static (sort of) data: this is information i keep in the database, like site section's data (title, metatags, header image url, fixed html content, javascript and css filenames to include in that section), that is rarely updated and it's very small in size.
While i was learning the basics on nodeJS, i started thinking of a way to improve the performance of the website, in a way i couldn't do with PHP. So, what i'm doing is:
When i run the app, the static content is all loaded into memory, i have a "model" Object for each content that stores the data in an array, has a method to refresh that data, ie, when the administrator updates something, i call refresh() to go get the new data from the database to that array. In this way, for every page load, instead of querying the database, the app queries the object in memory directly.
What i would like to know is if there should be any increase of performance, working with objects directly in memory or if constant queries to the database would work just as good or even better.
Any documentation supporting your answer will be much appreciated.
Thanks
In terms of the general database performance, MongoDB will keep your working set in memory - that's its basic method of operation.
So, as long as there is no memory contention to cause the data to get swapped out, and it is not too large to fit into your physical RAM, then the queries to the database should be extremely fast (in the sub millisecond range once you have your data set paged in initially).
Of course, if the database is on a different host then you have network latency to think about and such, but theoretically you can treat them as the same until you have a reason to question it.
I don't think there will be any performance difference. First thing is that this static data is probably not so big (up to 100 records?) and querying DB for it is not a big deal. Second thing (more important) is that most DB engines (including mongoDB) have caching systems built-in (although I'm not sure how they work in details). Third thing is that holding query results in memory does not scale well (for big websites) unless you use storage engine like Redis. And that's my opinion, although I'm not the expert.