I am a beginner in Backend developing, and I have this array called (Movie). I use expressJS and I want to save the array on MongoDB.I will use Mongodb atlas for my database. I appreciate your help
I tried to follow this instruction on this website:
https://medium.com/#lavitr01051977/node-express-js-aea19636a500
I ignored the first steps and starts from ( Connecting to the Database ) title but It doesn't work.
var express = require('express')
var app = express()
app.get('/', (req, res) => {
res.send('ok')
});
//movie array
const movies = [
{ title: 'Jaws', year: 1975, rating: 8 },
{ title: 'Avatar', year: 2009, rating: 7.8 },
{ title: 'Brazil', year: 1985, rating: 8 },
{ title: 'الإرهاب والكباب', year: 1992, rating: 6.2 }
]
//read the array movie
app.get('/movies/read/',(req,res) => {
res.send({status:200, data:movies})
})
//add elements to array movies
app.get('/movies/add',(req,res) => {
var t = req.query.title
var y = req.query.year
var r = req.query.rating
if(t == undefined || y == undefined || y.length > 4 || isNaN(y)) {
res.send({status:403, error:true, message:'you cannot create a movie without providing a title and a year'})
}
if (r == "") {
r = 4
}
movies.push({title: t, year: y, rating: r})
res.send({status:200, data:movies})
})
//delete elements from array movies
app.get('/movies/delete/:ID',(req,res) => {
var d = req.params.ID
if (d > 0 && d < movies.length ) {
movies.splice(d-1, 1)
res.send({status:200, message: movies})
}
else {
res.send({status:404, error:true, message:'the movie <ID> does not exist'})
}
})
//update elements from array movies
app.get('/movies/update/:ID',(req,res) => {
let c = req.params.ID
let x = req.query.title
let y = req.query.year
let z = req.query.rating
function update(a, b) {
if(a != undefined || a == "") {
movies[c-1][b] = a
}
}
if(c > 0 && c < movies.length ) {
update(x, 'title')
update(y, 'year')
update(z, 'rating')
res.send({status:200, message: movies})
}
else {
res.send({status:404, error:true, message:'the movie <ID> does not exist'})
}
})
app.listen(3000, () => console.log('listinig on port 3000'))
I expect the answer is like the link that I put it above on medium.com website
mongoose is a framework that facilitates interacting with MongoDB. Actually you basically never want to do all the validation, casting, and logic boilerplate on your own, so why reinvent the wheel.
And since you're a beginner, don't be afraid of frameworks. There are many useful frameworks for many areas of backend and frontend to make life easier for you.
The article you shared is self-explanatory, but I will sum up only the database part for you (I won't go deep into your code, no donkey work. the rest is up to you):
1) First of all install mongoose.
npm install mongoose
The article has --save which is no need to add anymore, as "npm install saves any specified packages into dependencies by default."(ref.)
2) to able to access and use mongoose, you need to import it, in node way, that is require().
const express = require(‘express’)
const mongoose = require(“mongoose”);
const bodyParser = require(‘body-parser’);
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
And what's body-parser there for?
When dealing with database in express, you'll sooner or later come across errors like this one.
and the reason why we need one after all is best explained in this answer.
Also, recent versions of express has its own body parser now, so you can use app.use(express.json()) instead of app.use(bodyParser.json()).
Important: body-parser must be before your routes.
3) use mongoose.connect(url).
url argument is what you find in your MongoDB Atlas. :
Location: clusters tab -> connect -> connect your application -> driver node.js
Which gives you, something like this:
mongodb+srv://<user>:<password>#,cluster>.mongodb.net/test?retryWrites=true&w=majority
Important: you use user and password of the user you made within Database Access tab, not your own user and password.
You can set up your environment variables to secure sensitive and changeable data. But I prefer using config.js for simplicity, and which usually resides in the root of app.
Not only you can secure them (like for using .gitignore), but also easily modify them as there are variables that might change from one environment to another environment, making them available in one place and easy to find, instead of looking for them to change all over your app.
For .env file approach, read this article.
Important: in case you want to put your code in github or anywhere online, which one reason we use config.js, don't forget to add this file in .gitignore to avoid such sensitive data get leaked and exposed to others online.
in config.js you can do so:
exports.username = 'your user';
exports.pass = 'your pass';
exports.myCluster = 'your cluster's name';
Then import them so:
const { username, pass, myCluster } = require('./config'); <- the path might be different for you!
Tip: You can use back-tick (` `) to easily insert those variables for const url, through interpolation.
That is:
const url = `mongodb+srv://${username}:${password},${myCluster}.mongodb.net/test?retryWrites=true&w=majority`
Important: make sure to whitelist your IP from MongoDB side (atlas), otherwise you will get connection error.
Under security: Network Access -> IP Whitelist
You could use 0.0.0.0/0 to whitelist all IPs.
Also, when using VPN, your IP will change too.
4) last but not least, after connecting to database, you need to define your schema:
const moviesSchema = new mongoose.Schema({
title: String,
year: Number,
rating: Number
});
And
const Movies = mongoose.model(“Movies”, moviesSchema);
Tip: A common mistake many newbies make is that they forgot to use new:
new mongoose.Schema({...})
If you want to create your model in a separate file (which is the best practice), you will need to modify your const Movies so:
module.exports = mongoose.model(“Movies”, moviesSchema);
Don't forgot to add const mongoose = require('mongoose'); in that separate js model file.
And in wherever you use want to use this model, you import so:
const Movies= require('../models/movies'); (the path may different for your app)
The rest, my friend, is up to you. What you want to do with your database and how to use it.
Note to others: I put so much time and mind to this as I was writing this. Please, if you see something wrong, or think you can add something, feel free to edit and improve my answer.
I would suggest you to take look at mongoose framework to interact with a Mongo database using NodeJS.
However, in the code you've provided you're not interacting with any database. You would need to define a Schema and then you could save a new doc or do any other action with your collection. Please follow some 'Get started' guide on how to do it.
Hope it helps!
I ll explain step by step. NOTE THAT THIS PROCESS SHOULD BE RUN ONLY ONCE. SO YOU SHOULD ADD THIS CODE TO A SEPARATE MODULE AND RUN IT ONCE. otherwise you will keep adding more items to the db. lets name this module
movies.js
//you need to connect to mongodb through mongoose.
const mongoose = require("mongoose");
mongoose
.connect("mongodb://127.0.0.1:27017/movies", { //will create movies db automatically
useNewUrlParser: true,
useCreateIndex: true
})
.catch(err => {
console.log(err.message);
process.exit(1);
})
.then(() => {
console.log("connected");
});
//next create a schema and model:
const movieSchema = new mongoose.Schema({
title: String,
year: Number,
rating: Number
});
const Movie = mongoose.model("Movie", movieSchema);
//create movies array based on `Movie model` so u can use save() method.
const movies = [
new Movie({ title: "Jaws", year: 1975, rating: 8 }),
new Movie({ title: "Avatar", year: 2009, rating: 7.8 }),
new Movie({ title: "Brazil", year: 1985, rating: 8 }),
new Movie({ title: "الإرهاب والكباب", year: 1992, rating: 6.2 })
];
//Last step save it.
movies.map(async (p, index) => {
await p.save((err, result) => {
if (index === movies.length - 1) {
console.log("DONE!");
mongoose.disconnect();
}
});
});
map is an array method.it iterates over array and save each item inside the array. Once every item in the array is saved we need to disconnect from db. array method takes 2 arguments. on is each item inside the array, second one is the index of each item. index in array starts from 0, so last item's index in movies array is 3 but length of the array is 4 so once 4-1=3 that means we saved every item in the array.
Now run the code
node movies.js
Related
I've looked everywhere and I cannot figure out why I get this error while I try to create and save documents with Mongoose.
It has worked to save individual documents with await fruit.save();, but when I run the script to add multiple documents with .insertMany() I get the following message in terminal and I have no clue what to do with it.
/Users/FruitsProjectMongoose/node_modules/mongoose/lib/model.js:3519
for (let i = 0; i < error.writeErrors.length; ++i) {
^
TypeError: Cannot read properties of undefined (reading 'length')
at /Users/FruitsProjectMongoose/node_modules/mongoose/lib/model.js:3519:47
at collectionOperationCallback (/Users/FruitsProjectMongoose/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:194:24)
at /Users/FruitsProjectMongoose/node_modules/mongodb/lib/utils.js:349:66
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
It would be an understatement to say that I have tried everything that I could think of/find across the web. I’ve messaged a few devs. and they’ve recommended me to try some things but no luck. I really need some help with this. I start to think it might be something wrong with my system.
I’ve installed MongoDB through Homebrew and Mongoose through npm in the past two days so everything is up-to-date.
The fruitsDB exists and I am able to access and view the collections and the individually added documents through the MongoDB shell mongosh.
Here is my simple JS script:
const mongoose = require("mongoose");
mongoose.set('strictQuery', false);
// Connect to MongoDB by port and catch errors.
main().catch(err => console.log(err));
async function main() {
await mongoose.connect('mongodb://127.0.0.1:27017/fruitsDB')
.then(() => console.log('Connected!'));
// Defining a Model Schema.
const Schema = mongoose.Schema;
const fruitSchema = new Schema({
name: {
type: String,
require: true
},
rating: {
type: Number,
require: true
},
review: {
type: String,
require: true
}
});
const peopleSchema = new Schema({
name: String,
age: Number
});
// Create a Model.
const Fruit = new mongoose.model("Fruit", fruitSchema);
const People = new mongoose.model("People", peopleSchema);
// Create & Save a Document.
const fruit = new Fruit({
name: "Banana",
rating: 10,
review: "Perfection!"
});
// await fruit.save();
const people = new People({
name: "Eduard",
age: 25
});
// await people.save();
// Create & Save docs. in Bulk.
const kiwi = new Fruit({
name: "Kiwi",
rating: 9,
review: "Great, kinda expensive!"
});
const orange = new Fruit({
name: "Orange",
rating: 6,
review: "Too sweet."
});
const apple = new Fruit({
name: "Apple",
rating: 7,
review: "Great fruit!"
});
Fruit.insertMany([kiwi, orange, apple], function(err) {
if (err) {
console.log(err);
} else {
console.log("Succesfully saved to fruitsDB");
}
});
mongoose.connection.close();
};
MongoDB server is running on brew services start mongodb-community.
This error is likely caused by an issue with the Mongoose library. To fix it, try updating the Mongoose library to the latest version. You can do this by running the following command in your terminal:
npm install mongoose#latest --save
If the "error" object is not defined, make sure that it is properly initialized before attempting to access its properties.
I've missed a set of curly braces. So the correct syntax is this.
await Fruit.insertMany([kiwi, orange, apple], {function(err) {
if (err) {
console.log(err);
} else {
console.log("Succesfully saved to fruitsDB");
}
}});
I'm running a Node.js server, connecting to a MongoDB database with mongoose.
Inside my controller, I have several methods that make operations to the database. One of them is this one:
async findMultiple(req, res) {
const [baseSkillsArray] = Array(req.body);
try {
// if there is not baseSkillsArray, skip
if (!baseSkillsArray) {
return res.status(200).send([]);
}
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({
_id: { $in: [baseSkillsArray.baseSkillArray] } //
});
console.log('test ' + allBaseSkills);
res.status(200).send(allBaseSkills);
} catch (error) {
console.error(error.message);
res.status(500).send('Server error find BaseSkills');
}
}
However, this returns me nothing. I did some debugging and I found the reason is the find id $in the array. So I tried hard coding a value, like '2', for instance.
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({ _id: { $in: ['2'] } });
No success. So I went to MongoDB Atlas, where my DB is stored. I tried filtering using the same line of code in my collections.
{ _id: { $in: ['2'] } }
Surprisingly, it returns my document as I wanted!
The issue is that I need to make it work with mongoose. Any ideas? Is this a known bug?
There is nothing wrong with the query, nor a bug regarding $in.
In fact, what's wrong is the actual collection name. I manually created a collection in MongoDB Atlas, called "baseSkills". However, mongoose by default transforms your collection name into lowercase and adds an "s" if your collection's name is not in the plural.
So every time I started my server, I noticed that there was a new collection called "baseskills". I assumed it was a bug and deleted it. Only after making this post that I realized the collection was there again.
So I exported the documents to this collection and my query was working fine.
FYI, there is a way to enforce the collection's name in mongoose. When you declare you model, add a second parameter to the Schema function called "collection". Here is an example:
const BaseSkillSchema = new mongoose.Schema({
_id: {
type: String,
required: true
}, ...
}, { collection: 'baseSkills' })
That's it! Sorry for the mess and thank you for your help!
you want to query over mongo db object ids. So you should create a new ObjectId to do that.
import {Types} from 'mongoose';
{ _id: { $in: [new Types.Object("2")] } }
Or if you have 2 ids one generated and one custom created as id then you can query without creating a new object.
{ id: { $in: ['2'] } }
Summary
I have some JSON coming back me from this:
// Now Get Only New Listings
finalArray = _.difference(existingResults, items);
and I need to insert those values into this:
Some of the data coming in is not available to fill in all columns. In that case, I would like to leave them blank.
Here is what the data looks like coming in (one of many presumably)...
{
price: '$6,000',
title: '1999 Jeep Wrangler SE Sport Utility 2D',
location: 'Auburndale, Florida',
miles: '12K miles',
imgUrl: 'http://www.example.com/pic.jpg',
itemURL: '/products/item/36427/'
}
So I need to figure out how to go through every JSON object, and insert into the correct columns.
var insertQuery = 'Your code here! :) ';
I suppose that query needs to be in a loop somewhere? Or one big insert statement?
Any advice or code would be really appreciated. Thanks.
Note
Using: NodeJS and JawsDB MySQL ( in case that info is important )
The following approach will only work if your payload doesn't have any nested objects.Used ES6 Maps to do the work. I have used connection.escape() to remove any injectable scripts within the JSON to prevent SQL injection attacks. I hope that you have already install mysql connector for node.js.
const mysql = require('mysql');
const connection = mysql.createConnection({
host : 'localhost',
user : 'me',
password : 'secret',
database : 'my_db'
});
connection.connect();
const payload = {
price: '$6,000',
title: '1999 Jeep Wrangler SE Sport Utility 2D',
location: 'Auburndale, Florida',
miles: '12K miles',
imgUrl: 'http://www.example.com/pic.jpg',
itemURL: '/products/item/36427/'
}
const map = new Map(Object.entries(payload));
const keys = map.keys();
const values = map.values();
const insertQuery = `
INSERT INTO TABLE_NAME (${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)},
${connection.escape(keys.next().value)})
VALUES (${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)},
${connection.escape(values.next().value)});
`
connection.query(insertQuery, function (error, results, fields) {
if (error) throw error;
// ...
});
console.log(insertQuery);
I want to write a query that would change the type of a project field from string to object.
So, if project field has the value abcd now, I want it to have an object like this:
{id: 'abcd'}
So:
project: 'abcd'
Turns to:
project: {id: 'abcd'}
I have no problems doing it in mongo:
db.hello.find({}).forEach((project) => {
project.project = {
id: x.project
}
db.hello.save(x)
})
But I don't know how to do it in Meteor. So far I have:
Projects.update($set: { client: ??? } }, { multi: true });
My 2 main problems are:
I don't know how to get the current value of client
I don't know how to change type
First of all, if you already ran the query, then you are aware that the db has already been adjusted yes? Because if you did run that, it would have updated all of the documents in that collection!
Please note that this should be ran server-side, I don't think that the $type is supported by all versions of minimongo.
// grab the cursor all string typed `project` fields
const cursor = Projects.find({ project: { $type : "string" } });
// grab the data from the cursor
const projects = cursor.fetch();
// Loop on each project and update
projects.forEach( project => Projects.update(project._id, {
$set: {
project: { id: project }
}
}) )
Sorry if I'm not getting the terminology right. Here's what I have currently my MongoDB user docs db.users:
"liked" : [
"EBMKgrD4DjZxkxvfY",
"WJzAEF5EKB5aaHWC7",
"beNdpXhYLnKygD3yd",
"RHP3hngma9bhXJQ2g",
"vN7uZ2d6FSfzYJLmm",
"NaqAsFmMmnhqNbqbG",
"EqWEY3qkeJYQscuZJ",
"6wsrFW5pFdnQfoWMs",
"W4NmGXyha8kpnJ2bD",
"8x5NWZiwGq5NWDRZX",
"Qu8CSXveQxdYbyoTa",
"yLLccTvcnZ3D3phAs",
"Kk36iXMHwxXNmgufj",
"dRzdeFAK28aKg3gEX",
"27etCj4zbrKhFWzGS",
"Hk2YpqgwRM4QCgsLv",
"BJwYWumwkc8XhMMYn",
"5CeN95hYZNK5uzR9o"
],
And I am trying to migrate them to a new key that also captures the time that a user liked the post
"liked_time" : [
{
"postId" : "5CeN95hYZNK5uzR9o",
"likedAt" : ISODate("2015-09-23T08:05:51.957Z")
}
],
I am wondering if it might be possible to simply do this within the MongoDB Shell with a command that iterates over each user doc and then iterates over the liked array and then updates and $push the new postId and time.
Or would it be better to do this in JavaScript. I am using Meteor.
I almost got it working for individual users. But want to know if I could do all users at once.
var user = Meteor.users.findOne({username:"atestuser"});
var userLiked = user.liked;
userLiked.forEach(function(entry) {
Meteor.users.update({ username: "atestuser" },
{ $push: { liked_times: { postId: entry, likedAt: new Date() }}});
console.log(entry);
});
Still a bit of a newbie to MongoDB obviously......
Here is something i made real quick you should run this on the server side just put it into a file e.g. "migrate.js" in root meteor and run the meteor app
if (Meteor.isServer) {
Meteor.startup(function () {
var users = Meteor.users.find().fetch();
users.forEach(function (doc) {
liked.forEach(function (postId) {
Meteor.users.update(doc._id, { $push: { liked_times: { postId: postId, likedAt: new Date() } } });
});
});
console.log('finished migrating');
});
}
p.s I didn't test it
If this is a one time migration i would do something like this in a one time js script.
Get all users
Iterate over each user
Get all likes
Iterate over them, get likedAt
var liked_times = _.collect(likes, function (likeId) {
return {
'postId' : likeId,
'likedAt': // get post liked time from like id.
}
});
Insert the above in the collection of choice.
Note:
The above example makes use of lodash
I would rather just save likedAt as a timestamp.