I've just started learning on MEAN stack and need to generate dynamic forms on the fly.
The requirement is to import document (excel/csv/xml/xls etc..) and generate dynamic forms using it so a user can update their data and again export it into the respective file.
So to accomplish this I'm converting documents to JSON format and storing JSON data into the MongoDB database.
Ex: Consider this xlsx data:
ID Name dob Gender
1 user1 7-Dec-87 m
2 user2 8-Dec-87 f
3 user3 9-Dec-87 f
3 user4 4-Dec-87 m
And I'm converting this using xlsx-to-json module to JSON format and storing it into Mongodb.
app.post('/myapp', function (req, res) {
//console.log("===========" + req.file.path);
converter({
input: req.file.path,
output: "output.json"
}, function (err, result) {
if (err) {
console.error(err);
} else {
console.log(result);
db.collection('test').insert(result, function (err, doc) {
console.log(err);
res.json(doc);
});
}
});
});
Here I'm fetching above data from Mongodb & express.js
app.get('/myapp', function (req, res) {
db.collection('test').find(function (err, docs) {
console.log(docs);
res.json(docs);
});
});
app.get('/birthdaylist/:id', function (req, res) {
var id = req.params.id;
console.log(id);
db.collection('test').findOne({_id: mongojs.ObjectId(id)}, function (err, doc) {
console.log(JSON.stringify(doc));
res.json(doc);
});
});
and here's the JSON output:
[
{ dob: '7-Dec-87', ID: '1', Name: 'user1' },
{ dob: '8-Dec-87', ID: '2', Name: 'user2' },
{ dob: '9-Dec-87', ID: '3', Name: 'user3' },
{ dob: '4-Dec-87', ID: '4', Name: 'user4' }
]
So, I've few queries:
Is this the correct approach I'm doing to generate dynamic form from xlsx/csv..etc ? If yes, then how can I generate dynamic form from above JSON.
While exploring on google I've found mongodb generates form automatically (https://github.com/GothAck/forms-mongoose)
So will it help because there may be chance of huge data on excel files.
Any help would be really appreciated.
Do you actually need to analyze an arbitrary spreadsheet and dynamically extract the schema, or do you know the schema ahead of time? If you know the schema, then the Mongoose form generating example is straightforward. But make sure that is actually a requirement because it is tough.
You are never going to be 100% because spreadsheets are created by users and users do weird things. But you can make something that works most of the time.
You need something that takes a JSON object and extracts the schema and puts that in a Mongoose schema format.
So you want to add an interesting module to Mongoose schema. I searched node-modules.com and this came up: https://github.com/Nijikokun/generate-schema
Form generation is not a trivial task. You may want to consider using a library for this. Here are a few that might be useful to you:
http://schemaform.io/
https://github.com/jdorn/json-editor/
Also, if you need help generating JSON schema from JSON:
http://jsonschema.net/#/
and of course: http://json-schema.org/
Related
I'm trying to use Mongoose (MongoDB JS library) to create a basic database, but I can't figure out how to delete the documents / items, I'm not sure what the technical term for them is.
Everything seems to work fine, when I use Item.findById(result[i].id), it returns a valid id of the item, but when I use Item.findByIdAndDelete(result[i].id), the function doesn't seem to start at all.
This is a snippet the code that I have: (Sorry in advance for bad indentation)
const testSchema = new schema({
item: {
type: String,
required: true
},
detail: {
type: String,
required: true
},
quantity: {
type: String,
required: true
}
})
const Item = mongoose.model("testitems", testSchema)
Item.find()
.then((result) => {
for (i in result) {
Item.findByIdAndDelete(result[i].id), function(err, result) {
if (err) {
console.log(err)
}
else {
console.log("Deleted " + result)
}
}
}
mongoose.connection.close()
})
.catch((err) => {
console.log(err)
})
I'm not sure what I'm doing wrong, and I haven't been able to find anything on the internet.
Any help is appreciated, thanks.
_id is a special field on MongoDB documents that by default is the type ObjectId. Mongoose creates this field for you automatically. So a sample document in your testitems collection might look like:
{
_id: ObjectId("..."),
item: "xxx",
detail: "yyy",
quantity: "zzz"
}
However, you retrieve this value with id. The reason you get a value back even though the field is called _id is because Mongoose creates a virtual getter for id:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.
The key takeaway is that when you get this value with id it is a string, not an ObjectId. Because the types don't match, MongoDB will not delete anything.
To make sure the values and types match, you should use result[i]._id.
I'm running a Node.js server, connecting to a MongoDB database with mongoose.
Inside my controller, I have several methods that make operations to the database. One of them is this one:
async findMultiple(req, res) {
const [baseSkillsArray] = Array(req.body);
try {
// if there is not baseSkillsArray, skip
if (!baseSkillsArray) {
return res.status(200).send([]);
}
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({
_id: { $in: [baseSkillsArray.baseSkillArray] } //
});
console.log('test ' + allBaseSkills);
res.status(200).send(allBaseSkills);
} catch (error) {
console.error(error.message);
res.status(500).send('Server error find BaseSkills');
}
}
However, this returns me nothing. I did some debugging and I found the reason is the find id $in the array. So I tried hard coding a value, like '2', for instance.
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({ _id: { $in: ['2'] } });
No success. So I went to MongoDB Atlas, where my DB is stored. I tried filtering using the same line of code in my collections.
{ _id: { $in: ['2'] } }
Surprisingly, it returns my document as I wanted!
The issue is that I need to make it work with mongoose. Any ideas? Is this a known bug?
There is nothing wrong with the query, nor a bug regarding $in.
In fact, what's wrong is the actual collection name. I manually created a collection in MongoDB Atlas, called "baseSkills". However, mongoose by default transforms your collection name into lowercase and adds an "s" if your collection's name is not in the plural.
So every time I started my server, I noticed that there was a new collection called "baseskills". I assumed it was a bug and deleted it. Only after making this post that I realized the collection was there again.
So I exported the documents to this collection and my query was working fine.
FYI, there is a way to enforce the collection's name in mongoose. When you declare you model, add a second parameter to the Schema function called "collection". Here is an example:
const BaseSkillSchema = new mongoose.Schema({
_id: {
type: String,
required: true
}, ...
}, { collection: 'baseSkills' })
That's it! Sorry for the mess and thank you for your help!
you want to query over mongo db object ids. So you should create a new ObjectId to do that.
import {Types} from 'mongoose';
{ _id: { $in: [new Types.Object("2")] } }
Or if you have 2 ids one generated and one custom created as id then you can query without creating a new object.
{ id: { $in: ['2'] } }
on serverside, i get a simple json file via REST with a lot of IDs, something like that:
[ {
"_id": "5825a49dasdasdasd8417c1b6d5",
}
"_id": "dfsdfsdf4960932218417c1b6d5",
}
"_id": "23434344960932218417c1b6d5",
},]
For that i have written in the main:
main.post('/..../add', Controller.addEvent);
In the Controller, i want to recieve the request and search in the mongodb for these ID's, because i need some informations about these IDs
exports.addEvent = function(req, res) {
var collection = db.get().collection('events');
My question is, if someone send me over "localhost:8080/events/add" the given simple json file, how do i have to handle this json? i need the ID's and want to search with them.
Thanks for help!
----------ADDED------------
I am bit further now. In my controller i have the following function
exports.addevent = function(req, res)
{
var ids = req.body;
console.log(ids);
}
Now i'm getting all IDs, which i have posted with "Postman" from chrome.
The output in the console is:
[ { _id: '5825a49dasdasdasd8417c1b6d5' },
{ _id: 'dfsdfsdf4960932218417c1b6d5' },
{ _id: '23434344960932218417c1b6d5' } ]
How can i have every single ID?
Ok, the request is an object and not a string. That was the error :-/
for(var i in ids) {
console.log(ids[i]._id);
}
and it works, now i can connect to the database and search for the id
Sorry if I'm not getting the terminology right. Here's what I have currently my MongoDB user docs db.users:
"liked" : [
"EBMKgrD4DjZxkxvfY",
"WJzAEF5EKB5aaHWC7",
"beNdpXhYLnKygD3yd",
"RHP3hngma9bhXJQ2g",
"vN7uZ2d6FSfzYJLmm",
"NaqAsFmMmnhqNbqbG",
"EqWEY3qkeJYQscuZJ",
"6wsrFW5pFdnQfoWMs",
"W4NmGXyha8kpnJ2bD",
"8x5NWZiwGq5NWDRZX",
"Qu8CSXveQxdYbyoTa",
"yLLccTvcnZ3D3phAs",
"Kk36iXMHwxXNmgufj",
"dRzdeFAK28aKg3gEX",
"27etCj4zbrKhFWzGS",
"Hk2YpqgwRM4QCgsLv",
"BJwYWumwkc8XhMMYn",
"5CeN95hYZNK5uzR9o"
],
And I am trying to migrate them to a new key that also captures the time that a user liked the post
"liked_time" : [
{
"postId" : "5CeN95hYZNK5uzR9o",
"likedAt" : ISODate("2015-09-23T08:05:51.957Z")
}
],
I am wondering if it might be possible to simply do this within the MongoDB Shell with a command that iterates over each user doc and then iterates over the liked array and then updates and $push the new postId and time.
Or would it be better to do this in JavaScript. I am using Meteor.
I almost got it working for individual users. But want to know if I could do all users at once.
var user = Meteor.users.findOne({username:"atestuser"});
var userLiked = user.liked;
userLiked.forEach(function(entry) {
Meteor.users.update({ username: "atestuser" },
{ $push: { liked_times: { postId: entry, likedAt: new Date() }}});
console.log(entry);
});
Still a bit of a newbie to MongoDB obviously......
Here is something i made real quick you should run this on the server side just put it into a file e.g. "migrate.js" in root meteor and run the meteor app
if (Meteor.isServer) {
Meteor.startup(function () {
var users = Meteor.users.find().fetch();
users.forEach(function (doc) {
liked.forEach(function (postId) {
Meteor.users.update(doc._id, { $push: { liked_times: { postId: postId, likedAt: new Date() } } });
});
});
console.log('finished migrating');
});
}
p.s I didn't test it
If this is a one time migration i would do something like this in a one time js script.
Get all users
Iterate over each user
Get all likes
Iterate over them, get likedAt
var liked_times = _.collect(likes, function (likeId) {
return {
'postId' : likeId,
'likedAt': // get post liked time from like id.
}
});
Insert the above in the collection of choice.
Note:
The above example makes use of lodash
I would rather just save likedAt as a timestamp.
I use Mongoose.js and cannot solve problem with 3 level hierarchy document.
There 2 ways to do it.
First - without refs.
C = new Schema({
'title': String,
});
B = new Schema({
'title': String,
'c': [C]
});
A = new Schema({
'title': String,
'b': [B]
});
I need to show C record. How can i populate / find it, knowing only _id of C?
I was try use:
A.findOne({'b.c._id': req.params.c_id}, function(err, a){
console.log(a);
});
But i dont know how to get from returnet a object only c object that i need.
Second if working with refs:
C = new Schema({
'title': String,
});
B = new Schema({
'title': String,
'c': [{ type: Schema.Types.ObjectId, ref: 'C' }]
});
A = new Schema({
'title': String,
'b': [{ type: Schema.Types.ObjectId, ref: 'B' }]
});
How to populate all B, C records to get hierarchy?
I was try to use something like this:
A
.find({})
.populate('b')
.populate('b.c')
.exec(function(err, a){
a.forEach(function(single_a){
console.log('- ' + single_a.title);
single_a.b.forEach(function(single_b){
console.log('-- ' + single_b.title);
single_b.c.forEach(function(single_c){
console.log('--- ' + single_c.title);
});
});
});
});
But it will return undefined for single_c.title. I there way to populate it?
Thanks.
As of Mongoose 3.6 the ability to recursively populate related documents in a query has been added. Here is an example of how you might do it:
UserList.findById(listId)
.populate('refUserListItems')
.exec(function(err, doc){
UserListItem.populate(doc.refUserListItems, {path:'refSuggestion'},
function(err, data){
console.log("User List data: %j", doc);
cb(null, doc);
}
);
});
In this case, I am populating an array of id's in 'refUserListItems' with their referenced documents. The result of the query then gets passed into another populate query that references the field of the original populated document that I want to also populate - 'refSuggestion'.
Note the second (internal) populate - this is where the magic happens. You can continue to nest these populates and tack on more and more documents until you have built your graph the way you need it.
It takes a little time to digest how this is working, but if you work through it, it makes sense.
in Mongoose 4 you can populate multilevel like this (even in different database or instance)
A
.find({})
.populate({
path: 'b',
model: 'B',
populate: {
path: 'c',
model: 'C'
}
})
.exec(function(err, a){});
In Mongoose 4 you can populate documents across multiple levels:
Say you have a User schema which keeps track of the user's friends.
var userSchema = new Schema({
name: String,
friends: [{ type: ObjectId, ref: 'User' }]
});
Firstly populate() lets you get a list of user friends. But what if you also wanted a user's friends of friends? In that case, you can specify a populate option to tell mongoose to populate the friends array of all the user's friends:
User.
findOne({ name: 'Val' }).
populate({
path: 'friends',
// Get friends of friends - populate the 'friends' array for every friend
populate: { path: 'friends' }
});
Taken from: http://mongoosejs.com/docs/populate.html#deep-populate
I'm late to this, but I wrote a Mongoose plugin that makes it extremely simple to perform deep model population. For your example, you can do this to populate b and c:
A.find({}, function (err, docs) {
A.deepPopulate(docs, 'b.c', cb)
}
You can also specify Mongoose populate options for each of the populated paths, like this:
A.deepPopulate(docs, 'b.c', {
b: {
select: 'name'
}
}, cb)
Check out the plugin documentation for more information.