My code is as follows (usually naming convention for the well-known objects):
var DBOpenRequest = window.indexedDB.open("messages", 6);
//...
DBOpenRequest.onupgradeneeded = function(event) {
console.log("Need to upgrade.");
var db = event.target.result;
console.log(db);
db.onerror = function(event) {
console.log("Error upgrading.");
};
// Create an objectStore for this database
var objectStore = db.createObjectStore("messages", { keyPath: "id", autoIncrement: true });
};
This ran fine for versions 3 and 4. When it came to version 5, I get the error:
Failed to execute 'createObjectStore' on 'IDBDatabase': An object store with the specified name already exists.
at IDBOpenDBRequest.DBOpenRequest.onupgradeneeded
Isn't the createObjectStore operating on a new version of the database which is empty? How do I fix the error?
I happened to log the db object and the details are below:
I am curious why the version number is different in the summary line and when expanded.
Isn't the createObjectStore operating on a new version of the database which is empty?
When you get upgradeneeded the database is in whatever state you left it in before. Since you don't know what versions of your code a user will have visited, you need to look at the event's oldVersion to find out what that was. The typical pattern is something like this:
var rq = indexedDB.open('db', 5);
rq.onupgradeneeded = function(e) {
var db = rq.result;
if (e.oldVersion < 1) {
// do initial schema creation
db.createObjectStore('users');
}
if (e.oldVersion < 2) {
// do 1->2 upgrade
var s = db.createObjectStore('better_users');
s.createIndex('some_index', ...);
db.deleteObjectStore('users'); // migrating data would be better
}
if (e.oldVersion < 3) {
// do 2->3 upgrade
rq.transaction.objectStore('better_users').createIndex('index2', ...);
}
if (e.oldVersion < 4) {
// do 3->4 upgrade
db.createObjectStore('messages', ...);
}
if (e.oldVersion < 5) {
// do 4->5 upgrade
// ...
}
}
I am curious why the version number is different in the summary line and when expanded.
That one is subtle... I believe at the point where the 5 was logged the database had started the upgrade. But because an exception was thrown in the upgradeneeded handler the upgrade was aborted, and the version number was rolled back to 4 before the details were logged.
The best way to upgrade the DB is checking if the store name is already there. In this example I'm using https://npmjs.com/idb
openDB('db-name', version, {
upgrade(db, oldVersion, newVersion, transaction) {
if(!db.objectStoreNames.contains('messages')) {
db.createObjectStore('messages', { keyPath: "id", autoIncrement: true })
}
}
})
If you need to check if an indexName already exist, you can get the objectStore and check for the indexNames property if it contains the indexName you need.
openDB('db-name', version, {
upgrade(db, oldVersion, newVersion, transaction) {
const storeName = transaction.objectStore('storeName')
if(!storeName.indexNames.contains('indexName')) {
storeName.createIndex('indexName', 'propertyName', { unique: false });
}
}
})
Using indexDB API with indexNames and objectStoreNames to check if something is either there or not makes my code way more reliable and easy to maintain, it is also briefly mentioned on Working with IndexDB Using database versioning
Related
I currently have a collection in Mongodb say "Collection1".
I have the following array of objects that need to be into inserted into MongoDB. I am using Mongoose API. For now, I am iterating through the array and inserting each of them into mongo.
This is ok for now, but will be a problem when the data is too big.
I need a way of inserting the data in bulk into MongoDB without repetition.
I am not sure how to do this. I could not find a bulk option in Mongoose.
My code below
myData = [Obj1,Obj2,Obj3.......]
myData.forEach(function(ele){
//console.log(ele)
saveToMongo(ele);
});
function saveToMongo(obj){
(new Collection1(obj)).save(function (err, response) {
if (err) {
// console.log('Error while inserting: ' + obj.name + " " +err);
} else {
// console.log('Data successfully inserted');
}
});
return Collection1(obj);
}
You might want to use the insertMany() method here if you're using the latest Mongoose version 4.4.X and greater, which essentially uses Model.collection.insertMany() under the hood and the driver might handle parallelizing >= 1000 docs for you.
myData = [Obj1, Obj2, Obj3.......];
Collection1.insertMany(myData, function(error, docs) {});
or using Promises for better error handling
Collection1.insertMany(myData)
.then(function(docs) {
// do something with docs
})
.catch(function(err) {
// error handling here
});
It works by creating a bunch of documents, calls .validate() on them in parallel, and then calls the underlying driver's insertMany() on the result of toObject({ virtuals: false }); of each doc.
Although insertMany() doesn't trigger pre-save hooks, it has better performance because it only makes 1 round-trip to the server rather than 1 for each document.
For Mongoose versions ~3.8.8, ~3.8.22, 4.x which support MongoDB Server >=2.6.x, you could use the Bulk API as follows
var bulk = Collection1.collection.initializeOrderedBulkOp(),
counter = 0;
myData.forEach(function(doc) {
bulk.insert(doc);
counter++;
if (counter % 500 == 0) {
bulk.execute(function(err, r) {
// do something with the result
bulk = Collection1.collection.initializeOrderedBulkOp();
counter = 0;
});
}
});
// Catch any docs in the queue under or over the 500's
if (counter > 0) {
bulk.execute(function(err,result) {
// do something with the result here
});
}
you can pass an array of objects to mongoose model create function
var Collection1 = mongoose.model('Collection1');
Collection1.create(myData,function(err){
if(err) ...
});
The ultimate goal is to detect changes between an existing Parse object and the incoming update using the beforeSave function in Cloud Code.
From the Cloud Code log available through parse.com, one can see the input to beforeSave contains a field called original and another one called update.
Cloud Code log:
Input: {"original": { ... }, "update":{...}
I wonder if, and how, we can access the original field in order to detect changing fields before saving.
Note that I've already tried several approaches for solving this without success:
using (object).changedAttributes()
using (object).previousAttributes()
fetching the existing object, before updating it with the new data
Note on request.object.changedAttributes():
returns false when using in beforeSave and afterSave -- see below for more details:
Log for before_save -- summarised for readability:
Input: { original: {units: '10'}, update: {units: '11'} }
Result: Update changed to { units: '11' }
[timestamp] false <--- console.log(request.object.changedAttributes())
Log for corresponding after_save:
[timestamp] false <--- console.log(request.object.changedAttributes())
There is a problem with changedAttributes(). It seems to answer false all the time -- or at least in beforeSave, where it would reasonably be needed. (See here, as well as other similar posts)
Here's a general purpose work-around to do what changedAttributes ought to do.
// use underscore for _.map() since its great to have underscore anyway
// or use JS map if you prefer...
var _ = require('underscore');
function changesOn(object, klass) {
var query = new Parse.Query(klass);
return query.get(object.id).then(function(savedObject) {
return _.map(object.dirtyKeys(), function(key) {
return { oldValue: savedObject.get(key), newValue: object.get(key) }
});
});
}
// my mre beforeSave looks like this
Parse.Cloud.beforeSave("Dummy", function(request, response) {
var object = request.object;
var changedAttributes = object.changedAttributes();
console.log("changed attributes = " + JSON.stringify(changedAttributes)); // null indeed!
changesOn(object, "Dummy").then(function(changes) {
console.log("DIY changed attributes = " + JSON.stringify(changes));
response.success();
}, function(error) {
response.error(error);
});
});
When I change someAttribute (a number column on a Dummy instance) from 32 to 1222 via client code or data browser, the log shows this:
I2015-06-30T20:22:39.886Z]changed attributes = false
I2015-06-30T20:22:39.988Z]DIY changed attributes =
[{"oldValue":32,"newValue":1222}]
I am trying to add hashtags in the post's hashtag[] array as a object with a num:1 variable to the users hashtagseen[] array if it is not already in it else add 1 the num if the hashtag is already in the hashtagseen[] array. How do I fix my code? Here is the code, thanks in advanced.
edit: I think I am not finding post.hashtag with this.hashtag and that is why it will not go to else. Just a guess.
The user object
Accounts.createUser({
username: username,
password: password,
email: email,
profile: {
hashtagsl:[],
}
});
collections/post.js
var post = _.extend(_.pick(postAttributes, 'title', 'posttext','hashtags'), {
userId: user._id,
username: user.username,
submitted: new Date().getTime(),
commentsCount: 0,
upvoters: [], votes: 0,
});
calling it
Meteor.call('addposthashtags',this.hashtags,Meteor.user().profile.hashtagsl);
lib/usershash
Meteor.methods({
addposthashtags: function (hashtags,hashtagsl) {
//supposed to make hashtagseen a array with the names from the hashtagsl object in it
var hashtagseen = _.pluck(hashtagsl, 'name');
//supposed to run once per each hashtag in the posts array.
for (var a = 0; a < hashtags.length; a++) {
//supposed set hashtagnumber to the number indexOf spits out.
var hashnumber=hashtagseen.indexOf(hashtags[a]);
//supposed to check if the current hashtag[a] === a idem in the hashtagseen.
if(hashnumber===-1){
var newhashtag = this.hashtags[a];
//supposed to make the object with a name = to the current hashtags
Meteor.users.update({"_id": this.userId},{"$push":{"profile.hashtagsl": {name: newhashtag, num: 1}}})
} else {
var hashi = hashtagseen[hashnumber];
//supposed to ad one to the num variable within the current object in hashtagsl
Meteor.users.update({"_id": this.userId, "profile.hashtagsl.name":hashi},{"$inc":{"profile.hashtagsl.num":1}});
}
}
}
});
Your addposthashtags function is full of issues. You also haven't provided a "schema" for hashtag objects.
addposthashtags: function () {
for (a = 0; a < this.hashtag.length; a++) {
// Issue1: You're querying out the user for every iteration of the loop!?
for (i = 0; i < Meteor.user().profile.hashtagseen.length; i++) {
// Issue2: You're comparing two _objects_ with ===
// Issue3: Even if you use EJSON.equals - the `num` property wont match
// Issue4: You're querying out the user again?
if (this.hashtag[a] === Meteor.user().profile.hashtagseen[i]) {
// Issue5 no `var` statement for hashtagseeni?
// Issue6 You're querying out the user again??
hashtagseeni = Meteor.user().profile.hashtagseen[i];
//Issue7 undefined hashtagsli?
//Issue8 Calling multiple methods for the one action (eg in a loop) is a waste of resources.
Meteor.call('addseen', hashtagsli);
} else {
//Issue9 no `var` statement for newhashtag?
newhashtag = this.hashtag[a];
newhashtag.num = 1;
//Issue8b Calling multiple methods for the one action (eg in a loop) is a waste of resources.
Meteor.call('updateUser', newhashtag, function (err, result) {
if (err)
console.log(err);
});
}
}
}
}
Also, the method has similiar issues:
addseen: function (hashtagseeni) {
// Issue10: var `profile` is undefined
// Issue11: should use `this.userId`
// Issue12: hashtagseeni wouldn't match profile.hashtagseen due to "num" field.
Meteor.users.update({"_id": Meteor.userId, "profile.hashtagseen": profile.hashtagseeni}, {"$inc":{"profile.hashtagseen.$.num":1}});
}
New issues with your new set of code:
Meteor.methods({
addposthashtags: function (hashtags,hashtagsl) {
//Issue1 `hashtag` is undefined, guessing you mean `hashtags`
//Issue2 no `var` for a
for (a = 0; a < hashtag.length; a++) {
//Issue3 no `var` for i
//Issue4 Why are you looping through both?
// don't you just want to check if hashtag[a] is in hashtagsl?
for (i = 0; i < hashtagsl.length; i++) {
if (hashtags[a] === hashtagsl[i].name) {
var hashi = hashtagsl[i].name;
//supposed to ad one to the num variable within the current object in hashtagsl.
// Issue5: This query wont do what you think. Test until you've got it right.
Meteor.users.update({"_id": Meteor.userId, 'profile.hashtagsl':hashi}, {"$inc":{"num":1}});
} else {
// Issue6 `this.hashtag` isn't defined. guessing you mean `hashtags[a]`
var newhashtag = this.hashtag[a];
// Issue7 superfluous statement
var newhashtagnum = num = 1;
// Issue8 Obvious syntax errors
// Perhaps try Meteor.users.update({"_id": this.userId},{"$push":{"profile.hashtagsl": {name: newhashtag, num: 1}}})
Meteor.users.update({"_id": Meteor.userId, 'profile'},{"$addToSet":{"hashtagsl"[newhashtag]=newhashtagnum}})
};
};
};
};
});
I'd suggest you trying the following
1) Assuming that after newhashtag=hashtag[a] you get a JSON object in newhashtag variable, try replacing newhashtag:{num:1}; with newhashtag.num = 1 - this will add the num variable to the object and set the value.
1.a) For debugging purposes try adding some console.log(JSON.stringify(newhashtag)); after each of the two lines where you're setting and changing the newhashtag variable - this way you'll know exactly what you're trying to add to the mongoDB document.
2) The update to increment the views also doesn't seem to me that will work. Couple of things to note here - $set:{'profile.hashtagseen[i]':num++} - MongoDB won't be able to identify the 'i' in 'profile.hashtagseen[i]' and 'num++' is not how increments are done in Mongo.
I'd suggest you look into the $inc and to the positional update documentation of MongoDB.
Your final increment update statement will look something like
Meteor.users.update({"_id": Meteor.userId, "profile.hashtagseen": profile.hashtagseen[i]}, {"$inc":{"profile.hashtagseen.$.num":1}});
I see that executing addposthashtags is in the client, and you must to pay attention because this function will execute in minimongo and doesn't work all operations. First you try execute this operation under mongo if it's work you must to create one function inside the folder server.
Add text of the documentation of Minimongo
In this release, Minimongo has some limitations:
$pull in modifiers can only accept certain kinds of selectors.
findAndModify, aggregate functions, and map/reduce aren't supported.
All of these will be addressed in a future release. For full Minimongo
release notes, see packages/minimongo/NOTES in the repository.
Minimongo doesn't currently have indexes. It's rare for this to be an
issue, since it's unusual for a client to have enough data that an
index is worthwhile.
You try create one method on the server, with the same operation.
Server:
Meteor.methods({
updateUser: function (newhashtag) {
Meteor.users.update(this.userId,
{
$addToSet: {'profile.$.hashtagseen': newhashtag}
});
}
});
Client:
Meteor.call('updateUser',newhashtag,function(err,result){
if (err)
console.log(err);// there you can print the erro if there are
});
Minimongo doesn't support alls operation, for test you can to execute in the console for testing the method if supported. After that you can to execute the operation under mongo directly, that clears your doubts.
I'm developing a small Chrome extension that would allow me to save some records to chrome.storage and then display them.
I've managed to make the set and get process work as I wanted (kinda), but now I'd like to add a duplicate check before saving any record, and I'm quite stuck trying to find a nice and clean solution.
That's what I came up for now:
var storage = chrome.storage.sync;
function saveRecord(record) {
var duplicate = false;
var recordName = record.name;
storage.get('records', function(data) {
var records = data.records;
console.log('im here');
for (var i = 0; i < records.length; i++) {
var Record = records[i];
if (Record.name === recordName) {
duplicate = true;
break;
} else {
console.log(record);
}
}
if (duplicate) {
console.log('this record is already there!');
} else {
arrayWithRecords.push(record);
storage.set({ bands: arrayWithRecords }, function() {
console.log('saved ' + record.name);
});
}
});
}
I'm basically iterating on the array containing the records and checking if the name property already exists. The problem is it breaks basic set and get functionality -- in fact, when saving it correctly logs 'im here' and the relative record object, but it doesn't set the value. Plus, after a while (generally after trying to list the bands with a basic storage.get function) it returns this error:
Error in response to storage.get: TypeError: Cannot read property
'name' of null
I'm guessing this is due to the async nature of the set and get and my incompetence working with it, but I can't get my head around it in order to find a better alternative. Ideas?
Thanks in advance.
I am very new to IndexedDB Concepts. I am trying to Store a list of movies in the IndexedDB and retrieve it. But for some reason when i try to retrieve it there is a DOM IDBDatabase Exception 11 in chrome browser. I try to retrieve it by using a simple alert. I also tried to retrieve the data by putting the alert inside an onComplete event, but this too seems to be a failure. Could someone please let me know what wrong i am doing. Below is my code.
const dbName = "movies";
var request = indexedDB.open(dbName, 1);
request.onerror = function(event) {
alert("Seems like there is a kryptonite nearby.... Please Check back later");
};
request.onsuccess = function(event) {
var db = event.target.result;
var transaction = db.transaction(["movies"],"readwrite");
var objectStore = transaction.objectStore("movies");
var request1 = objectStore.get("1");
request1.result.oncomplete=function(){
alert("The movie is"+request1.result.name);//This is the place where i get the error
}
};
request.onupgradeneeded = function(event) {
db = event.target.result;
var objectStore = db.createObjectStore("movies", { keyPath: "movieid" });
objectStore.createIndex("name", "name", { unique: false });
objectStore.createIndex("runtime", "runtime", { unique: false });
for (var i in movieDataToStore) {
objectStore.add(movieDataToStore[i]);
}};
I still do not know what was wrong with the last program. i re-wrote the above program and it worked like a charm. here is the code. Hope this helps anyone who is stuck with this problem. Also if anyone figures out what went wrong the last time please share your thoughts.
var db; //database will be stored in this value when success is called
var movieDataToStore = [{ movieid: "1", name: "Keep my Storage Local", runtime:"60"},
{ movieid: "2", name: "Rich Internet Conversations", runtime:"45"},
{ movieid: "3", name: "Applications of the Rich and Famous", runtime:"30"},
{ movieid: "4", name: "All Jump All eXtreme", runtime:"45"}];
window.query = function() {
db.transaction("movies").objectStore("movies").get("1").onsuccess = function(event) {
alert("QUERY: CThe first movie is" + event.target.result.name);
};};
window.onload = function() {
if (!window.indexedDB) {
window.alert("Your browser doesn't support a stable version of IndexedDB. Such and such feature will not be available.")
}
else{
var request = indexedDB.open("movies", 1);
request.onerror = function(event) {
alert("Seems like there is a kryptonite nearby.... Please Check back later");
};
request.onsuccess = function(event) {
db = this.result;
query();
};
request.onupgradeneeded = function(event) {
var db = event.target.result;
if(db.objectStoreNames.contains("movies")) {
db.deleteObjectStore("movies");
}
var objectStore = db.createObjectStore("movies", { keyPath: "movieid"});
objectStore.createIndex("name", "name", { unique: false });
objectStore.createIndex("runtime", "runtime", { unique: false });
for (var i in movieDataToStore) {
objectStore.add(movieDataToStore[i]);
}
};
}
};
I think it is bad practice to insert data in the onupgradeneeded context. You should be doing this separately in an unrelated function at some other time. In fact, attempting to insert the data on a database whose version was incremented since last page load will automatically trigger the upgradeneeded event for you.
While many of the online examples shove the database connection handle (your db var) into some global scope variable, this is also a bad practice that will lead to errors down the road. Only access the db var within your callbacks as a parameter. In other words, your openRequest.onsuccess function should pass the db variable to the query function. This also reduces the chances of any garbage collection issues later and leaving database connections open (which the designers of indexedDB allow for, but should generally be avoided).
If your movie ids are integers, it isn't clear to me why you are storing and retrieving them as strings. You can store integer values. You can pass an integer to store.get.
You are using for...in inappropriately. It will work, but for...in is intended for looping over the keys of object literals (like var x = {key:value}). Use a normal for loop or use array.foreach when iterating over your movies array.
As you found out in your fixed code, it is better to use request.onsuccess and request.onerror. There is also a transaction.oncomplete. But I am not sure there is a request.oncomplete. What happens is you are setting the oncomplete property of an IDBRequestObject but this does nothing since the code never triggers it.
DOM 11 usually signals you tried to access a table that does not exist or is in an incorrect state. Usually this happens due to mistakes elsewhere, like onupgradeneeded never getting called when connecting. It is confusing, but given the way your code is setup, basically the db gets created the first time your page loads, but then never gets created again, so while developing, if you made changes, but do not increment your db version, you will never see them.