Right now I am replicating my entire device database over to my remote database.
Once that is complete, I grab all my data that is not older than 1 month from my remote database, using a filter, and bring it to my device.
FILTER
{
_id: '_design/filters',
"filters": {
"device": function(doc, req) {
if(doc.type == "document" || doc.type == "signature") {
if(doc.created >= req.query.date) return true;
else return false;
}
else return true;
}
}
}
REPLICATION
device_db.replicate.to(remote_db)
.on('complete', function () {
device_db.replicate.from(remote_db, {
filter: "filters/device",
query_params: { "date": (Math.floor(Date.now() / 1000)-2419200) }
})
.on('complete', function () {
console.log("localtoRemoteSync replicate.to success");
callback(true);
});
});
My question:
I want to be able to periodically delete data from my device that is older than 3 months (old enough data where I already know it's been sync'd)
But just because I delete it from my device, when I replicate the data back to my remote_db, I don't want it to be deleted on there too.
How can I delete specific data on my device but not have that deletion translated when I replicate?
FILTERS
Here, we have 2 filters:
noDeleted : This filter doesn't push _deleted documents.
device : Filter to get the latest data only.
{
_id: '_design/filters',
"filters": {
"device": function(doc, req) {
if (doc.type == "document" || doc.type == "signature") {
if (doc.created >= req.query.date) return true;
else return false;
}
return true;
},
"noDeleted": function(doc, req) {
//Document _deleted won't pass through this filter.
//If we delete the document locally, the delete won't be replicated to the remote DB
return !doc._deleted;
}
}
}
REPLICATION
device_db.replicate.to(remote_db, {
filter: "filters/noDeleted"
})
.on('complete', function() {
device_db.replicate.from(remote_db, {
filter: "filters/device",
query_params: { "date": (Math.floor(Date.now() / 1000) - 2419200) }
})
.on('complete', function() {
console.log("localtoRemoteSync replicate.to success");
callback(true);
});
});
Workflow
You push all your documents without pushing the deleted document.
You get all the updates for the latest data
You delete your old documents
You could either query the remote DB to get the ids of the documents that are too old and delete them locally. Note that the documents will still be there as _deleted. To completely remove them, a compaction will be required.
You could also totally destroy your local database after step1 and start from scratch.
callback(true);
Add a one-way filtered replication. However anything you need back on the server you will need to use a put request with the server's _rev.
For example
Replicate from server to client, then add a filter mechanism, like transfer:true to the docs you want to replicate. replication
db.replicate.from(remoteDB, {
live: true,
retry: true,
selector: {transfer:true}// or any other type of selector
});
To delete a doc on the client, set transfer to false, then delete it on the client. it won't meet your filter criteria so it won't replicate.
Anything you want to put back to the server use a put request instead of replicate.
If you want the document back on the client just set transfer to true in the doc.
Related
I am new to IndexedDB and serviceworkers and am having a very difficult time understanding how to turn these into a funcitonal application. I've done extensive reading on both, but even the "complete" examples don't incorporate the two.
I am tasked with creating an application that will allow users to work offline. The first time they connect to the site, I want to pull specific information from the database and store it in IndexedDB. When they go offline, I need to use that data to display information on the page. Certain interactions will cause the data to update, then to be synced later once an internet connection is reestablished. From a high-level, I udnerstand how this works.
It is my understanding that we cannot call functions from the serviceworker.js file due to the asynchronous nature of serviceworkers. Additionally, serviceworkers.js cannot directly update the DOM. However, the examples I have seen are creating and managing the IndexedDB data within the serviceworkers.js file.
So let's say I have a file:
<!-- index.html -->
<html>
<body>
Hello <span id="name"></span>
</body>
</html>
And a serviceworker.js:
var CACHE_NAME = 'my-cache-v1';
var urlsToCache = [
'/'
// More to be added later
];
self.addEventListener('install', function(event) {
// Perform install steps
event.waitUntil(
caches.open(CACHE_NAME)
.then(function(cache) {
console.log('Opened cache');
return cache.addAll(urlsToCache);
})
);
});
self.addEventListener('activate', function(event) {
event.waitUntil(
createDB() //Use this function to create or open the database
);
});
self.addEventListener('fetch', function(event) {
event.respondWith(
caches.match(event.request)
.then(function(response) {
// Cache hit - return response
if (response) {
return response;
}
return fetch(event.request).then(
function(response) {
// Check if we received a valid response
if(!response || response.status !== 200 || response.type !== 'basic') {
return response;
}
var responseToCache = response.clone();
caches.open(CACHE_NAME)
.then(function(cache) {
cache.put(event.request, responseToCache);
});
return response;
}
);
})
);
});
function createDB() {
idb.open('mydata', 1, function(upgradeDB) {
var store = upgradeDB.createObjectStore('user', {
keyPath: 'id'
});
store.put({id: 1, name: 'John Doe'}); //This can be updated with an AJAX call to the database later
});
}
How do I now update the element "name" with the value for key = 1 from the "user" objectstore in the "mydata" database?
Depending on your use case, you've got several options :
You dont need the service worker. Just pull your data from iDB directly from the page. The DOM has access to iDB.
Set a template for your index.html. At the activate step in service worker, pre-render the page with the value from iDB and cache it.
I have a sample code that goes like this:
Client Helper:
getUsername: function (userId) {
Meteor.call("getUsername", userId, function (err, result) {
if(!err) {
Session.set("setUsername", result);
else {
console.log(err);
}
});
return Session.get("setUsername");
}
Server
Meteor.methods({
"getUsername": function (userId) {
var x = Meteor.users.find({_id: userId}, {fields: {username:1}}).fetch()[0];
return x.username;
}
});
The result of this code is an infinite loop of username passing to the client. Is there a way to stop the loop and pass only the data that is needed on the client? I believe the reactivity is causing the data to loop infinitely and I am not sure how to stop it. I tried using "reactive":false on my query in the server but it does not work.
If you want to access username everywhere in client templates (so thats why you put it into session), I would not set it in template helper. I would set it on startup and get username from session in template helpers (without calling server method)
If you need username just in one template, so you want to return its value from your template helper, do not put it into session, just return it in your server method callback.
Based on your sample code, I assume, you have a set of posts and you are retrieving user name based on user id for each post. Then instead of doing it this way, you should use publish composite package to publish related users as well.
Meteor.publishComposite('getPosts', function (postIds) {
return [{
find: function() {
return Posts.find({ _id: { $in: postIds }});
// you can also do -> return Posts.find();
// or -> return Posts.find({ /* or what ever your selector is to get the posts you need*/ });
},
children: [{
find: function(post) {
return Meteor.users.find({
id: post.userId //or the correct field in your post document to get user id
}, {
fields: {
"profile": 1
}
});
}
}}
}]
});
This way your publication will take care of publishing related users along with posts. You don't need to use methods and call them each time.
I'm looking for a way to determine if Meteor.user() is set in a function that can be called both from the server and client side, without raising an error when it is not.
In my specific case I use Meteor server's startup function to create some dummy data if none is set. Furthermore I use the Collection2-package's autoValue -functions to create some default attributes based on the currently logged in user's profile, if they are available.
So I have this in server-only code:
Meteor.startup(function() {
if (Tags.find().fetch().length === 0) {
Tags.insert({name: "Default tag"});
}
});
And in Tags-collection's schema:
creatorName: {
type: String,
optional: true,
autoValue: function() {
if (Meteor.user() && Meteor.user().profile.name)
return Meteor.user().profile.name;
return undefined;
}
}
Now when starting the server, if no tags exist, an error is thrown: Meteor.userId can only be invoked in method calls. Use this.userId in publish functions.
So in other words calling Meteor.user() on the server startup throws an error instead of returning undefined or null or something. Is there a way to determine whether it will do so prior to calling it?
I cannot solve this simply by wrapping the call with if (Meteor.isServer) within the autoValue function, as the autoValue functions are normally called from server side even when invoked by the user, and in these cases everything in my code works fine.
Note that this is related to How to get Meteor.user() to return on the server side?, but that does not address checking if Meteor.user() is available in cases where calling it might or might not result in an error.
On the server, Meteor.users can only be invoked within the context of a method. So it makes sense that it won't work in Meteor.startup. The warning message is, unfortunately, not very helpful. You have two options:
try/catch
You can modify your autoValue to catch the error if it's called from the wrong context:
autoValue: function() {
try {
var name = Meteor.user().profile.name;
return name;
} catch (_error) {
return undefined;
}
}
I suppose this makes sense if undefined is an acceptable name in your dummy data.
Skip generating automatic values
Because you know this autoValue will always fail (and even if it didn't, it won't add a useful value), you could skip generating automatic values for those inserts. If you need a real name for the creator, you could pick a random value from your existing database (assuming you had already populated some users).
Been stuck with this for two days, this is what finally got mine working:
Solution: Use a server-side session to get the userId to prevent
"Meteor.userId can only be invoked in method calls. Use this.userId in publish functions."
error since using this.userId returns null.
lib/schemas/schema_doc.js
//automatically appended to other schemas to prevent repetition
Schemas.Doc = new SimpleSchema({
createdBy: {
type: String,
autoValue: function () {
var userId = '';
try {
userId = Meteor.userId();
} catch (error) {
if (is.existy(ServerSession.get('documentOwner'))) {
userId = ServerSession.get('documentOwner');
} else {
userId = 'undefined';
}
}
if (this.isInsert) {
return userId;
} else if (this.isUpsert) {
return {$setOnInsert: userId};
} else {
this.unset();
}
},
denyUpdate: true
},
// Force value to be current date (on server) upon insert
// and prevent updates thereafter.
createdAt: {
type: Date,
autoValue: function () {
if (this.isInsert) {
return new Date;
} else if (this.isUpsert) {
return {$setOnInsert: new Date};
} else {
this.unset();
}
},
denyUpdate: true
},
//other fields here...
});
server/methods.js
Meteor.methods({
createPlant: function () {
ServerSession.set('documentOwner', documentOwner);
var insertFieldOptions = {
'name' : name,
'type' : type
};
Plants.insert(insertFieldOptions);
},
//other methods here...
});
Note that I'm using the ff:
https://github.com/matteodem/meteor-server-session/ (for
ServerSession)
http://arasatasaygin.github.io/is.js/ (for is.existy)
Working on my social app I've found a strange behavior in the collection Meteor.users, this problem does not occur with other Collections using the same methodologies
I would like to have an initial list of users downloading a minimum number of information for everyone and when I open the panel to a specific user I subscribe a different showing more information if the specified user is a friend of mine.
But after subscribe the client collection Meteor.users is not updated!
CLIENT
Meteor.startup(function() {
Meteor.subscribe('usersByIds', Meteor.user().profile.friends, function() {
//... make users list panel using minimal fields
});
//performed when click on a user
function userLoadInfo(userId) {
Meteor.subscribe('userById', userId, function() {
var userProfile = Meteor.users.findOne(userId).profile;
//...
//make template user panel using full or minimal user fields
//...
//BUT NOT WORK!
//HERE Meteor.users.findOne(userId) keep minial user fields!!
//then if userId is my friend!
});
}
});
SERVER
//return minimal user fields
getUsersByIds = function(usersIds) {
return Meteor.users.find({_id: {$in: usersIds} },
{
fields: {
'profile.username':1,
'profile.avatar_url':1
}
});
};
//return all user fields
getFriendById = function(userId) {
return Meteor.users.find({_id: userId},
{
fields: {
'profile.username':1,
'profile.avatar_url':1
//ADDITIONAL FIELDS
'profile.online':1,
'profile.favorites':1,
'profile.friends':1
}
});
};
//Publish all users, with minimal fields
Meteor.publish('usersByIds', function(userId) {
if(!this.userId) return null;
return getUsersByIds( [userId] );
});
//Publish user, IF IS FRIEND full fields
Meteor.publish('userById', function(userId) {
if(!this.userId) return null;
var userCur = getFriendById(userId),
userProfile = userCur.fetch()[0].profile;
if(userProfile.friends.indexOf(this.userId) != -1) //I'm in his friends list
{
console.log('userdById IS FRIEND');
return userCur; //all fields
}
else
return getUsersByIds( [userId] ); //minimal fields
});
This is a limitation or bug in DDP. See this.
A workaround is to move data out of users.profile.
Like this:
//limited publish
Meteor.publish( 'basicData', function( reqId ){
if ( this.userId ) {
return Meteor.users.find({_id: reqId },{
fields: { 'profile.username':1,'profile.avatar_url':1}
});
}
else {
this.ready();
}
});
//friend Publish
Meteor.publish( 'friendData', function( reqId ){
if ( this.userId ) {
return Meteor.users.find( {_id: reqId, 'friendProfile.friends': this.userId }, {
fields: {
'friendProfile.online':1,
'friendProfile.favorites':1,
'friendProfile.friends':1
}
});
}
else {
this.ready();
}
});
//example user
var someUser = {
_id: "abcd",
profile: {
username: "abcd",
avatar_url: "http://pic.jpg"
},
friendProfile: {
friends: ['bcde', 'cdef' ],
online: true,
favorites: ['stuff', 'otherStuff' ]
}
}
As given in a comment, this link reveals your problem. The current DDP Protocol does not allow publishing of subdocuments. One way to get around this is to create a separate collection with your data but a better way would probably to just remove some of the data and make it a direct object off of your user.
The best way to do this is add the data to your user's profile upon insert and then in the onCreateUser move the data onto the user directly:
Accounts.onCreateUser(function(options, user) {
if (options.profile) {
if (options.profile.publicData) {
user.publicData = options.profile.publicData;
delete options.profile.publicData;
}
user.profile = options.profile;
}
return user;
});
If you are allowing clients to perform user inserts make sure you validate the data better though. This way you can have the online, favorites, and friends in the profile and publish that specifically when you want it. You can then have username and avatar_url in the publicData object directly on the user and just always publish all-the-time.
Just hit an insanely frustrating roadblock in prototyping. I need to update and increment values an array inside of a collection. To do this, I'm accessing the collection using the MongoDB syntax like so:
Players.update({_id: Session.get('p1_id'), 'opponents.$.id' : Session.get('p2_id')},
{$inc: {
'games_played' : 1
}}
);
When this runs I get an error saying: Uncaught Error: Not permitted. Untrusted code may only update documents by ID. [403]
Now, I searched the hell out of this and I know that it came down in an update and why they only allow update by id's. But my problem is that I can't seem to find a way around it. I tried forcing it by adding this to if (Meteor.isServer):
Players.allow({
insert: function(userId, doc, fields, modifier){
return true;
},
update: function(userId, doc, fields, modifier){
return true;
},
remove: function(userId, doc, fields, modifier){
return true;
}
});
Nothing seems to work, and all the examples I find talk about using a Meteor method (not really sure what that is) or are doing userId validation (I dont have any users and don't want to add them right now). I'm just prototyping/sketching and I'm not concerned about security. How can I proceed here?
Here's how you can make this into a method:
Meteor.methods({
incrementGames: function (player1Id, player2Id) {
check(player1Id, Meteor.Collection.ObjectID);
check(player2Id, Meteor.Collection.ObjectID);
Players.update({
_id: player1Id,
'opponents.$.id': player2Id
}, {
$inc: {
'games_played' : 1
}
}, function(error, affectedDocs) {
if (error) {
throw new Meteor.Error(500, error.message);
} else {
return "Update Successful";
}
});
}
});
And on your client:
Meteor.call("incrementGames", Session.get('p1_id'), Session.get('p2_id'), function(error, affectedDocs) {
if (error) {
console.log(error.message);
} else {
// Do whatever
}
});
You just got the update wrong. The first parameter of the update method should be the id. the second parameter is an object containing the modifiers.
Players.update(playerId, {$inc:{games_played:1}});
Optionally you can add a callback containing error as the first parameter and response as the second parameter.