How to save large data in local using reactJs? - javascript

Now I use ReactJs and I want to save large data in local, I try to look for a SQLite for reactJs but I can't find it, all about SQLite for react-native, Please give me information how to save large data in local using reactJs.
Thanks.

You're saying locally but in one of your comments you are concerned about the amount of data that can be saved..
If you're not looking for the overhead of a full-scale database solution, you could use Firebase RealTime Database; It's free for up to 1 gb of data and super easy to set up.
https://firebase.google.com/products/realtime-database/
You'll define a schema for your database through Firebase's console then copy and paste the config into your code then run an intitalization script and then use firebase refs to access the API (see below for simple example)
npm install firebase
import firebase from 'firebase';
var config = {
... {paste-from-firebase}
};
writeUserData(name, priority){
firebase.database().ref('users').push({
name,
priority
}).then((data)=>{
//success callback
console.log('data' , data)
}).catch((error)=>{
//error callback
console.log('error ' , error)
})
}
componentDidMount(){
// initialize firebase
if (!firebase.apps.length) {
firebase.initializeApp(config);
}
// an example of writing user data to the db
this.writeUserData('John', 'normal')
}
If you want to implement SQLite in your application, you will need to implement it server-side and not client-side for several reasons; the most important being security... Take a look at this SO answer:
Is it possible to access an SQLite database from JavaScript?
Apparently, there's also another relatively newer option called IndexedDB that some browsers support to locally store large amounts of structured data;
https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Using_IndexedDB
I would use firebase. It's well-supported, made by Google, and very easy to get set up with.

Related

How to handle CORS error, while fetching firebase realtime database API in vue project?

I am using vue-2.And want to do shallow query for firebase realtime database by fetching API.But While running on development server ,it shows CORS blocked. What should I do?
PS: I am also using vuefire
created(){
var apiUrl = 'https://console.firebase.google.com/u/4/project/enajori-45094/database/enajori-45094/data/Admin/Data%20Collection/Paying%20Guest';
fetch(apiUrl).then(response => {
return response.json();
}).then(data => {
console.log(data);
}).catch(err => {
console.log(err);
});
}
Cors error i am receiving
Firebase REST-Api only reacts to HTTPS-Requests. If your localhost doesen't have SSL enabled your requests will fail no matter what your try. The URI also need to end with the the table's name like something.json
And you realy should use the Firebase SDK - it's the smoother way.
It's entirely unclear to me why you would want to use a URL to the Firebase console to access the database. Console URLs are for human consumption, not for programs.
You should use the provided javascript SDK to access data. It will work around CORS issues for you automatically, and give you a much cleaner way to read and write data. If you can't use the SDK, you can always try the REST API. Just don't depend on those console URLs at all.

sync data from mongoDB to firebase and vice-versa

My current situation:
I have created an application using React, NodeJS and Electron. Most of the users are a kind of offline users. They use my application offline.
Next plans:
Now, I am planning to create a mobile application for them. I plan to create that application using React-Native.
Since their database is offline, I planned to give them a sync to firebase button in desktop application. When he clicks on sync to firebase button, the data in their local mongodb should syncronize with firebase.
My thoughts:
when a new record is added to mongodb, I will store a new key with that record which will look like: new: true.
when a record is updated I will store a key named updated: true
similarly for delete...
And then when user presses Sync to firebase, I will search for those records and add/update/delete respective records on firebase and then I will remove those keys from mongodb database.
Problems in executing my thoughts:
At first it does not smell me a good thing as I think it is time consuming because I will perform operations on firebase as well as mongodb.
Another problem with this approach is that if I think the other way round, that when user add/update/delete a record from React-Native app, firebase will have those keys line new/updated/deleted and then when user presses sync button in desktop application, I will have to do same thing but in reverse.
Yet another problem is that if user accidently uninstalled my application and then reinstalls it, then what should I do?
And the biggest problem is managing all the things.
My Expectations:
So, I want a clean and maintainable approach. Does any one have any idea on how to sync data from mongodb to firebase and vice-versa?
Both database systems supports for some sort of operation log or trigger system. You can use these to live update changes to databases to sync them almost real time.
For MongoDB
You can use Oplog to see what changes made to database (insert/update/delete) and run a suitable function to sync firebase.
oplog
A capped collection that stores an ordered history of logical writes
to a MongoDB database. The oplog is the basic mechanism enabling
replication in MongoDB.
There are small libraries that help you easily subscribe to these events.
Example (mongo-oplog)
import MongoOplog from 'mongo-oplog'
const oplog = MongoOplog('mongodb://127.0.0.1:27017/local', { ns: 'test.posts' })
oplog.tail();
oplog.on('op', data => {
console.log(data);
});
oplog.on('insert', doc => {
console.log(doc);
});
oplog.on('update', doc => {
console.log(doc);
});
oplog.on('delete', doc => {
console.log(doc.o._id);
});
For Firebase
You can use Cloud Functions. With Cloud Functions you can watch triggers like Cloud Firestore Triggers or Realtime Database Triggers and run a function to sync MongoDB database.
With Cloud Functions, you can handle events in the Firebase Realtime
Database with no need to update client code. Cloud Functions lets you
run database operations with full administrative privileges, and
ensures that each change to the database is processed individually.
// Listens for new messages added to /messages/:pushId/original and creates an
// uppercase version of the message to /messages/:pushId/uppercase
exports.makeUppercase = functions.database.ref('/messages/{pushId}/original').onWrite((event) => {
// Grab the current value of what was written to the Realtime Database.
const original = event.data.val();
console.log('Uppercasing', event.params.pushId, original);
const uppercase = original.toUpperCase();
// You must return a Promise when performing asynchronous tasks inside a Functions such as
// writing to the Firebase Realtime Database.
// Setting an "uppercase" sibling in the Realtime Database returns a Promise.
return event.data.ref.parent.child('uppercase').set(uppercase);
});

Getting data from nest api not working anymore

I am working with firebase to get data from the nest api. I had a working test setup where I was able to receive data, and set values. Now I am working in python to get a little server running that automatic updates my database. I had data a few days back. But today I wanted to work on it, and i didn't receive any data anymore. So I checked my test setup. And that one is neither getting any data, But I am still able to send data.
This is the javascript I use for my test setup.
var dataRef = new Firebase('wss://developer-api.nest.com/');
dataRef.authWithCustomToken("{{user.token}}", function(error, authData) {
if (error) {
console.log("Login Failed!", error);
} else {
console.log("Login Succeeded!", authData);
}
});
dataRef.on('value', function(snapshot){
console.log(snapshot.val());
});
It should work. But can it be that something broke with the update?
I also created an new client but that did not work either.
An other python application that i am running with nest is using the websocket-client an that one still works. But for the application i am now working on i need to get data from different accounts. If i use the websocket-client it wil use to much resources. so that is not an option.
Hope someone can help me.
Firebase upgraded their client recently. Since Nest runs its own instance of Firebase, make sure that you are using one of the supported Firebase versions available here and not that latest version.
https://developer.nest.com/documentation/cloud/firebase-client-libraries

Ember Data with Websockets

In the ember guides on models it says (1) :
Ember Data is also designed to work with streaming APIs like socket.io, Firebase, or WebSockets. You can open a socket to your server and push changes to records into the store whenever they occur.
I tried writing a custom adapter that uses a websocket but i'm not getting very far. I couldn't find any working examples anywhere.
This is my totally unfinished prototype:
DS.WSAdapter = DS.Adapter.extend(Ember.Evented, {
websocket: undefined,
init: function () {
if(this.websocket === undefined)
{
this.websocket = new WebSocket('ws://localhost:8887');
this.websocket.onopen = function(e) {
console.log("Connection established!");
};
this.websocket.onmessage = function(e) {
// What to do here?
};
}
this._loadData();
},
//....
Can somone please help me with the websocket adapter?
My main problem is that I have no clue what to do when the websocket.onmessage() gets executed. I can't even access the store (using DS.get('defaultStore')) or anything
I don't have experience working directly with sockets in Ember, however I have recently completed an Ember Data + Firebase adapter which should follow very similar methodologies.
You should, at the least, be able to use it as inspiration:
https://github.com/sandersonet/ember-data-firebase
Firebase does provide an additional layer of abstraction from the sockets underneath, but the methodologies are very similar.
Have a look at http://emberjs.com/guides/models/frequently-asked-questions/#toc_how-do-i-inform-ember-data-about-new-records-created-on-the-backend
Some applications may want to add or update records in the store
without requesting the record via store.find. To accomplish this you
can use the DS.Store's push, pushPayload, or update methods. This is
useful for web applications that have a channel (such as SSE or Web
Sockets) to notify it of new or updated records on the backend.
Basically, you need to deserialize data you receive in your onmessage hook and push new objects to the data store using store.push('model', record) or alternative methods.

Adding couchdb persistence to a socketio json feed in node

I'm currently researching how to add persistence to a realtime twitter json feed in node.
I've got my stream setup, it's broadcasting to the client, but how do i go about storing this data in a json database such as couchdb, so i can access the stores json when the client first visits the page?
I can't seem to get my head around couchdb.
var array = {
"tweet_id": tweet.id,
"screen_name": tweet.user.screen_name,
"text" : tweet.text,
"profile_image_url" : tweet.user.profile_image_url
};
db.saveDoc('tweet', strencode(array), function(er, ok) {
if (er) throw new Error(JSON.stringify(er));
util.puts('Saved my first doc to the couch!');
});
db.allDocs(function(er, doc) {
if (er) throw new Error(JSON.stringify(er));
//client.send(JSON.stringify(doc));
console.log(JSON.stringify(doc));
util.puts('Fetched my new doc from couch:');
});
These are the two snippets i'm using to try and save / retrieve tweet data. The array is one individual tweet, and needs to be saved to couch each time a new tweet is received.
I don't understand the id part of saveDoc - when i make it unique, db.allDocs only lists ID's and not the content of each doc in the database - and when it's not unique, it fails after the first db entry.
Can someone kindly explain the correct way to save and retrieve this type of json data to couchdb?
I basically want to to load the entire database when the client first views the page. (The database will have less than 100 entries)
Cheers.
You need to insert the documents in the database. You can do this by inserting the JSON that comes from the twitter API or you can insert one status at a time (for loop)
You should create a view that exposes that information. If you saved the JSON directly from Twitter you are going to need to emit several times in your map function
There operations (ingestion and querying) are not the same thing, so you should really do them at the different times in your program.
You should consider running a bg process (maybe in something as simple as a setInterval) that updates your database. Or you can use something like clarinet (http://github.com/dscape/clarinet) to parse the Twitter streaming API directly.
I'm the author of nano, and here is one of the tests that does most of what you need:
https://github.com/dscape/nano/blob/master/tests/view/query.js
For the actual query semantics and for you learn a bit more of how CouchDB works I would suggest you read:
http://guide.couchdb.org/editions/1/en/index.html
I you find it useful I would suggest you buy the book :)
If you want to use a module to interact with CouchDB I would suggest cradle or nano.
You can also use the default http module you find in Node.js to make requests to CouchDB. The down-side is that the default http module tends to be a little verbose. There are alternatives that give you an better API to deal with http requests. The request is really popular.
To get data you need to make a GET request to a view you can find more information here. If you want to create a document you have to use PUT request to your database.

Categories