I am sending 12 cell values from a google sheet to a mongodb database. The reason I'm doing it is because I want to concatenate the 12 cells and do some transforms on the data and output it on a front end somewhere after. I'm also doing this because sheets limit each cell to 50k characters and I have around 500k characters I need to POST to the database each time. My initial assumption here is that I need to create 1 record with all 12 cell values in MondoDB, ( potentially a different avenue would be to just post 12 separate records in a collection ). So the way I'm doing it is doing a POST request, and then potentially 11 PATCH requests on the initially posted record all in one google scripts function inside google sheets. The problem I'm having is that I'm not sure how to form the model & route & request. Everytime I do a patch (shown below) it overrides the previous POST data. I want it to only update the part of the JSON that is being sent via the PATCH request. Currently the PATCH request is changing every other record to null and deleting the previous POSTs data. I know there is a way to patch only a specific record in a collection, but how to patch a specific part of the json in a collection I don't get.
EXPRESS POST ROUTE:
router.post('/', async (req,res) => {
const post = new Post({
title: req.body.title,
en1: req.body.en1,
en2: req.body.en2,
en3: req.body.en3,
en4: req.body.en4,
fr1: req.body.fr1,
fr2: req.body.fr2,
fr3: req.body.fr3,
fr4: req.body.fr4,
de1: req.body.de1,
de2: req.body.de2,
de3: req.body.de3,
de4: req.body.de4
});
try {
const savedPost = await post.save();
res.json(savedPost);
} catch (err) {
res.json({ message: err })
}
})
EXPRESS UPDATE ROUTE:
router.patch('/:postId', async (req,res) => {
try {
const updatedPost = await Post.updateOne(
{ title:req.params.postId },
{ $set: {
title: req.body.title,
en1: req.body.en1,
en2: req.body.en2,
en3: req.body.en3,
en4: req.body.en4,
fr1: req.body.fr1,
fr2: req.body.fr2,
fr3: req.body.fr3,
fr4: req.body.fr4,
de1: req.body.de1,
de2: req.body.de2,
de3: req.body.de3,
de4: req.body.de4
} }
)
res.json(updatedPost)
}catch(err){
res.json({ message: err })
}
})
MONGOOSE MODEL:
const mongoose = require('mongoose')
const PostSchema = mongoose.Schema({
title: {
type: String,
required: false
},
en1: {
type: String,
required: false
},
en2: {
type: String,
required: false
},
en3: {
type: String,
required: false
},
en4: {
type: String,
required: false
},
fr1: {
type: String,
required: false
},
fr2: {
type: String,
required: false
},
fr3: {
type: String,
required: false
},
fr4: {
type: String,
required: false
},
de1: {
type: String,
required: false
},
de2: {
type: String,
required: false
},
de3: {
type: String,
required: false
},
de4: {
type: String,
required: false
},
date: {
type: Date,
default: Date.now
}
})
module.exports = mongoose.model('Posts', PostSchema)
GOOGLE SCRIPT POST REQUEST FROM SHEETS (with only one patch request for now):
function sendInfoToApi() {
const randomId = Math.random()*100000000000000000;
var en1 = SpreadsheetApp.getActive().getSheetByName("Final").getRange('A3').getValues()[0][0];
var en2 = SpreadsheetApp.getActive().getSheetByName("Final").getRange('A4').getValues()[0][0];
// 11111 POST REQUEST //
var data1 = {
"title": randomId,
"en1": en1
}
var payload1 = JSON.stringify(data1)
var url1 = 'https://ag-sheets-api.herokuapp.com/posts';
var fetchParameters1 = {
'method': 'post',
'contentType': 'application/json',
'payload' : payload1,
'muteHttpExceptions' : false
};
try {
var response = UrlFetchApp.fetch(url1, fetchParameters1);
} catch(e){
Logger.log(e)
}
// 22222 PATCH REQUEST //
var data2 = {
"title": randomId,
"en2": en2
}
var payload2 = JSON.stringify(data2)
var url2 = `https://ag-sheets-api.herokuapp.com/posts/${randomId}`;
var fetchParameters2 = {
'method': 'patch',
'contentType': 'application/json',
'payload' : payload2,
'muteHttpExceptions' : false
};
try {
var response = UrlFetchApp.fetch(url2, fetchParameters2);
} catch(e){
Logger.log(e)
}
}
RESULTING RECORD IN MONGODB:
_id:60072da8c52278001791e22e
title:"42612001948065650"
en1:null
date:2021-01-19T19:06:16.052+00:00
__v:0
de1:null
de2:null
de3:null
de4:null
en2:"<div class="qw">
<div class="qe" data-country="gq.svg"> <p cla..."
en3:null
en4:null
fr1:null
fr2:null
fr3:null
fr4:null
Here you can see how the PATCH request is overwriting the data in en1 to null, how can I make it only update en2, and skip the other values?
The issue is that you retrieve all the parameter even if they don't exist in the request. Thus, returning null in all properties.
en1: req.body.en1,
en2: req.body.en2,
en3: req.body.en3,
en4: req.body.en4,
fr1: req.body.fr1,
fr2: req.body.fr2,
fr3: req.body.fr3,
fr4: req.body.fr4,
de1: req.body.de1,
de2: req.body.de2,
de3: req.body.de3,
de4: req.body.de4
You only sent one of them, not all of them.
You should loop on the request keys and those only.
router.patch('/:postId', async (req,res) => {
try {
const updatedObj = {};
for(let i in req.body) {
if(req.body.hasOwnProperty(i) && i.match(/^(title|(en|fr|de)[1-4])$/)) updatedObj[i] = req.body[i];
}
const updatedPost = await Post.updateOne(
{ title:req.params.postId },
{ $set: updatedObj }
)
res.json(updatedPost)
}catch(err){
res.json({ message: err })
}
})
I added some regex validation to the properties send to avoid setting other keys you don't want the web request to update. You can remove that if the request is trusted.
Related
I am working in creating a todo list and one of the action I want users to do is delete all completed todos in one click. I have my models here, and the code I have, I have been reading trying to figure it out, but can't find it anywhere. Thanks in advance.
User Model
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true,
},
email: {
type: String,
required: true,
unique: true,
},
password: {
type: String,
required: true,
},
todos: [{
type: mongoose.Types.ObjectId,
ref: 'Todo'
}]
});
Todo model:
const TodoSchema = new mongoose.Schema({
creator: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'User',
},
content: {
type: String,
required: true,
},
completed: {
type: Boolean,
default: false
},
});
This is what I have now. I get a "Cast to ObjectId failed for value true at path todos.
router.delete('/delete/completed', auth, async (req, res) => {
try {
const userTodos = await User.update(
{'_id':req.user.id},
{ $pull: { 'todos': { 'completed': true } } },
{ multi: true }
);
if (!userTodos) {
return res.status(400).send('Server error');
}
return res.json({ userTodos });
//return res.json({msg: 'All completed removed'})
} catch (err) {
console.error(err.message);
return res.status(404).json({ msg: 'Something went wrong, try again' });
}
});
If you are (as it seems from your code) using mongoose, you could use mongoose's populate feature:
const userTodos = await User.find(
{'_id':req.user.id}).populate('todos', {
match: {completed: true}
});
please note, however, that you'll need to delete both the documents in the todos collection, AND the todo reference in the user's todos array. You may consider to remove one side of the reference, see the pros and cons of two-way referencing here
I have a mongoose schema for stories that looks like this:
{
id: {
type: Number,
default: 0
},
title: {
type: String,
maxLength: 60
},
author: {
userid: {
type: Number
},
username: {
type: String
}
}
chapters: [chapter],
numchapters: {
type: Number,
default: 1
},
favs: {
type: Number,
default: 0
},
completed: {
type: Boolean,
default: false
}
}
What I'm trying to do is reference a document in a separate collection (users), and use the values of its userid and username fields in the author field.
how do I do this?
current code:
storyobj.populate('author', {path: 'author', model: 'users', select: 'userid username'}, (err) => {
if (err) {
console.log(err)
}
})
just in case it's relevant, the structure of the users collection looks like this:
{
username: {
type: String,
},
email: {
type: String,
},
password: {
type: String,
},
userid: {
type: Number
},
isAdmin: {
type: Boolean,
default: false
},
banned: {
type: Boolean,
default: false
}
}
EDIT:
I've changed the author field in the Stories model to look like this:
author: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
}
This is so I tell Mongoose, "Hey, I want this field to reference a user in the User collection".
Here are some more details that I hope will be of help.
Full code:
var storydb = require('../models/stories/story');
var chapterdb = require('../models/stories/chapter');
var userdb = require('../models/user');
const file = JSON.parse(fs.readFileSync('test.json')); // this is a file with the data for the stories I am trying to insert into my database
for (const d in file) {
var storyobj = new storydb({
id: d,
chapters: []
});
for (let e = 0; e < file[d].length; e++) {
var abc = file[d][e];
var updatey = {
chaptertitle: abc.chapter,
chapterid: parseInt(abc.recid),
words: abc.wordcount,
notes: abc.notes,
genre: abc.gid.split(', '),
summary: abc.summary,
hidden: undefined,
loggedinOnly: undefined,
posted: new Date(Date.parse(abc.date)),
bands: abc.bandnames.split(', ')
};
var kbv = getKeyByValue(userlookup, abc.uid);
storyobj.title = abc.title;
storyobj.numchapters = file[d].length;
storyobj.favs = file[d][0].numfavs;
updatey.characters = abc.charid.split(/, |,/g);
storyobj.chapters.push(updatey)
}
storyobj.save();
}
In file, there's a unique ID representing the author of each story. kbv returns the userid associated with that unique ID (note that they're NOT the same).
Now, here's where I'm having trouble:
What I want to do is find a user matching the userid in kbv, and make that the author property in the story model.
The code I'm currently using to try and achieve that:
storydb.findOne({storyobj}, 'author').populate("author", (f) => console.log(f));
const Stories = require("./path/to/model");
Stories
.find({ /* query */ }, { /* projection */ })
.populate("author.username", ["userid", "username"])
.then(/* handle resolve */)
.catch(/* handle rejection */)
For this to work, you have to add a ref key to the userid key in your model, where the ref value is the name of the model it's referencing.
Story.model.js
const StorySchema = new Schema({
author: {
userid: { type: Schema.Types.ObjectId, ref: "users", required: true },
/* other props */
}
/* other props */
});
I have the object array's ID and I want to update the info with information I already have on my backend.
My code deletes every array and creates a new one with the new info.
I want to access the array with the variable ID and then change it's values:
const ProdcutSchema = new mongoose.Schema({
name:{
type: String,
required: true
},
productDescription:{
type: String,
required: true
},
pricePerUnit:{
type: Number,
required: true
},
productAmmount:{
type:Number,
required: true
},
/*productImagePath:{
type:String,
required: true
}*/
});
const UserSchema = new mongoose.Schema({
name:{
type: String,
},
email:{
type: String,
},
password:{
type: String,
},
date:{
type: Date,
default: Date.now
},
products:[ProdcutSchema]
});
//Update products
router.put('/dashboard/:id', (req, res)=>{
const ID = req.params.id;
const {product_name, price_PerUnit, product_Description, product_Ammount} = req.body; //Get access to ajax data using body parser
if(!product_name || !price_PerUnit || !product_Description || !product_Ammount){
res.send('Please fill out all fields with valid content');
}else{
User.products.findOneAndUpdate(
{ _id : ID },
{ $set: { products: {
name :product_name,
productDescription : product_Description,
pricePerUnit : price_PerUnit,
productAmmount : product_Ammount
} } },
(err) => {
if (err) throw console.log('found errors');
console.log('no errors');
})
}
});
If you have ObjectId of that item you want to update, code should look like this:
User.products.findOneAndUpdate(
{ _id: ID, "products._id": <here goes id for an array element> },
{
$set: {
"products.$": {
name: product_name,
productDescription: product_Description,
pricePerUnit: price_PerUnit,
productAmmount: product_Ammount
}
}
},
err => {
if (err) throw console.log("found errors");
console.log("no errors");
}
);
Also u gotta be aware that u need to supply an specific ID of an array element for this kind of situation when you want to update subdocument.
Recently, I have been working with mongodb with one single model. When I tried to add a second model, I noticed that I might face some issues.
First, here's the code with one single model:
riskRating.js
const mongoose = require('mongoose')
const Schema = mongoose.Schema;
let riskRatingRow = new Schema({
securitycause: {
type: String
},
operationalrisk: {
type: String
},
riskid: {
type: String
},
riskstatements: {
type: String
},
businessline: {
type: String
},
supportingasset: {
type: String
},
category: {
type: String
},
frequency: {
type: String
},
impact: {
type: String
},
inherentriskrating: {
type: String
},
controleffectiveness: {
type: String
},
residualrisk: {
type: String
}
});
module.exports = mongoose.model('riskRating', riskRatingRow);
Here's how I use it in the server code:
server.js
const RiskRatingRow= require('./models/riskRating');
router.route('/table').get((req, res) => {
RiskRatingRow.find((err, tableData) => {
if (err)
console.log(err);
else
res.json(tableData);
});
});
router.route('/table/add').post((req, res) => {
console.log('REQ.body is ', req.body);
const riskRatingRow = new RiskRatingRow(req.body);
riskRatingRow.save()
.then(issue => {
res.status(200).json({
'tableRow': 'Added successfully'
});
})
.catch(err => {
res.status(400).send('Failed to create new record');
});
});
First question: Is there anything wrong so far?
Now, when I add the second model:
twoModels.js
const mongoose = require('mongoose')
const Schema = mongoose.Schema;
let riskRatingRow = new Schema({
//1st model defintion
});
const businessLineDashboardRow = new Schema({
//2nd model defintion
});
module.exports = mongoose.model('businessLineDashboard', businessLineDashboardRow);
module.exports = mongoose.model('riskRating', riskRatingRow);
I have noticed that in server.js, when I am using the first model, I am not referencing it directly. I'm rather referecing the singleModel.js file. Particularly in these two lines:
const RiskRatingRow = require('./models/riskRating');
// Here I am using directly the file reference RiskRatingRow
const riskRatingRow = new RiskRatingRow(req.body);
// Same thing here
RiskRatingRow.find((err, tableData) => {
if (err)
console.log(err);
else
res.json(tableData);
});
So, when I was about to make use of the second model, I found myself blocked since as I explained when I used the first model, I didn't reference it directly. I just referenced the file.
Thing is, that actually works fine.
But, I don't know if the model file contains two models, how am I supposed to make use of them both in the server file.
So here's my two other questions:
1/ How come that code works even though I am just referecing the model defintion file?
2/ Should I define the second model in a separate file, and reference it in order to be able to use it?
Thank you, I hope I was clear enough.
module.exports can be an object containing multiple things as properties:
module.exports = {
RiskRatingRow: mongoose.model('businessLineDashboard', businessLineDashboardRow),
BusinessLineDashboardRow: mongoose.model('riskRating', riskRatingRow),
}
Since it's an empty object ({}) by default you can also assign the exports individually:
module.exports.RiskRatingRow = mongoose.model('businessLineDashboard', businessLineDashboardRow)
module.exports.BusinessLineDashboardRow = mongoose.model('riskRating', riskRatingRow)
You can now destructure the models out of the object inside server.js:
const { RiskRatingRow, BusinessLineDashboardRow } = require('./models/twoModels')
Or, if you want to do it the old-school way:
const models = require('./models/twoModels')
const RiskRatingRow = models.RiskRatingRow
const BusinessLineDashboardRow = models.BusinessLineDashboardRow
Niklas's answer is on point. However, I have found a more complete solution:
riskRating.js
const mongoose = require('mongoose')
const Schema = mongoose.Schema;
module.exports = function(mongoose) {
let riskRatingRow = new Schema({
securitycause: {
type: String
},
operationalrisk: {
type: String
},
riskid: {
type: String
},
riskstatements: {
type: String
},
businessline: {
type: String
},
supportingasset: {
type: String
},
category: {
type: String
},
frequency: {
type: String
},
impact: {
type: String
},
inherentriskrating: {
type: String
},
controleffectiveness: {
type: String
},
residualrisk: {
type: String
}
});
let businessLineDashboardRow = new Schema({
ref: {
type: String
},
riskstatements: {
type: String
},
maximpact: {
type: String
},
controleffectiveness: {
type: String
},
recommendedriskrating: {
type: String
},
frequency: {
type: String
},
impact: {
type: String
},
validatedreviewriskrating: {
type: String
},
rationalforriskadjustment: {
type: String
}
});
var models = {
BusinessLineDashboard : mongoose.model('BusinessLineDashboard', businessLineDashboardRow),
RiskRating : mongoose.model('RiskRating', riskRatingRow)
};
return models;
}
server.js
router.route('/riskRating2').get((req, res) => {
models.RiskRating.find((err, tableData) => {
if (err)
console.log(err);
else
res.json(tableData);
analyseDeRisqueService.determineMaxImpactForEachRisk(tableData)
});
});
I've tried almost everything splice, pop, shift, remove but I can't remove the user id which simple means downvote the post.
Here is my code:
// #type POST
// #route /api/question/upvote/:id
// #desc route for upvoting answers to questions
// #access PRIVATE
router.post('/upvote/:id', passport.authenticate('jwt', {session:false}), (req, res)=> {
Profile.findOne({user: req.user.id})
.then(profile => {
Question.findById(req.params.id)
.then(question => {
if(question.upvotes.filter(upvote => upvote.user.toString() === req.user.id.toString()).length > 0){
return res.status(400).json({noUpvote : 'User is downvoted the question'});
}
question.upvotes.unshift({user : req.user.id});
question.save()
.then(question => res.json(question))
.catch(err => console.log('Error on saving the upvote user id :' + err));
})
.catch(err => console.log('Error on getting the question : ' + err));
})
.catch(err => console.log('Error on finding the user : ' + err));
});
There are the three models in my application:
//Load the Person Model
const Person = require('../../models/Person');
//Load the Profile Model
const Profile = require('../../models/Profile');
//Load the Questions Model
const Question = require('../../models/Questions');
So the upvote is in question model.
The Person model contains the registration and login information.
The Profile model contains the Person details. The Question model contains question, answer, comment, and upvotes.
Here is the question model:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const QuestionSchema = new Schema({
user : {
type: Schema.Types.ObjectId,
ref: 'myPerson'
},
textone: {
type: String,
required: true
},
texttwo: {
type: String,
required: true
},
name:{
type: String
},
upvotes: [{
user : {
type: Schema.Types.ObjectId,
ref: 'myPerson'
}
}],
answers:[{
user : {
type: Schema.Types.ObjectId,
ref: 'myPerson'
},
answer: {
type: String,
required: true
},
name:{
type: String
},
date: {
type: Date,
default: Date.now
}
}],
comments: [{
user : {
type: Schema.Types.ObjectId,
ref: 'myPerson'
},
name:{
type: String
},
text: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
}],
date: {
type:Date,
default: Date.now
}
});
module.exports = Questions = mongoose.model("myQuestion", QuestionSchema);
you can remove the id using filter.
question.upvotes = question.upvotes.filter(upvote => upvote.user.toString() !== req.user.id.toString());
question.save();
add it inside the if condition if this is where you want to remove it.