We Keep Coding

Why I am getting Query was already executed error?

I have the following Schema and Model:
const userSchema = new mongoose.Schema({
name: String,
})
const UserModel = mongoose.model('User', userSchema, 'users')
and I have written the following express middleware, which simply takes one argument, awaits that argument, set the returned value from that awaiting job to the req object on a property called gottenDocs, (i.e.: req.gottenDocs)
function getDocumentsMw(query) {
return async (req, res, next) => {
const dbRes = await query
req.gottenDocs = dbRes
next()
}
}
and I have the following route:
app.get(
'/users',
getDocumentsMw(UserModel.find({})),
(req, res, next) => {
const gottenDoc = req.gottenDocs
res.status(200).json({
status: 'success',
data: gottenDoc,
})
})
That's all I have, now, when I request [ GET " /users " ] I recieve the following response which is great, and nothing is wrong:
{
"status": "success",
"data": []
}
but, the weird thing is when I request this route again, it throws this error:
MongooseError: Query was already executed: User.find({})
What could be the problem? is it a bug in Nodejs? which could be hmmm, something like, that it is not removing the function call from the call stack after the response has been sent?
any help appreciated.

The problem is on this line
getDocumentsMw(UserModel.find({})),
Here you create a query once the application start, because of that the query is created once but executed multiple times.
You may need to refactor your code to something like that
getDocumentsMw(() => UserModel.find({})),
Now you are passing a function and not a query. The function creates a query, aka factory. Next step is to refactor getDocumetnsMw to call the function to create a query when it needs to do something with it.
function getDocumentsMw(queryFactory) {
return async (req, res, next) => {
const dbRes = await queryFactory()
req.gottenDocs = dbRes
next()
}
}

How to send data from NodeJS server side to the JS client side, only when data is ready?

On my website, when the user clicks on a button, some user's data will be stored in a database and after that I want the server to send notification data to the Javascript frontend file to change the UI.
Right now, the Js file (index.js) receives data right after the website loads (always false). I want it to be received only when the data is ready on the server.
I searched a lot but couldn't find an answer to my problem?
I appreciate any help :)
server.js
var requestValidation = false;
app.post("/", function(req, res){
var name = req.body.personName;
var email = req.body.personEmail;
var collabTopic = req.body.collabTopic;
const newUser = new User({ //mongoDB schema
name: name,
email: email,
collabTopic: collabTopic
});
newUser.save(function(err){ //adding data to mongoDB
if(!err){
requestValidation = true;
}
});
});
app.get("/succ", function(req, res){
res.json(requestValidation);
});
index.js
const url = "http://localhost:3000/succ";
const getData = async (url) => {
try {
const response = await fetch(url);
const json = await response.json();
console.log(json);
} catch (error) {
console.log(error);
}
};
getData(url);

I'm not sure this is completely the answer you're looking for, but it's definitely a tool/feature to consider as you rework your approach.
app.post("/", async (req, res) => {
let result = await INSERT MONGODB UPDATE OR INSERT FUNCTION;
res.render("YOUR TEMPLATE", result);
});
You probably can't plug and play this, but when you finish a MongoDB operation, it returns a json object with some details on whether or not there was success. For example, a MongoDB insert operation returns something like this (stored in the variable result that I created)
{ "acknowledged" : true, "insertedId" : ObjectId("5fd989674e6b9ceb8665c57d") }
and then you can pass this value on as you wish.
Edit: This is what tkausl referred to in a comment.

Here is an example if you want to pass the content of a txt file to the client with express and jquery:
in express:
app.get('/get', (req, res) => {
fs.readFile('test.txt', (err, data) => {
if (err) throw err;
return res.json(JSON.parse(data));
})
})
jquery in client side:
$.getJSON( "http://localhost:3000/get", function( data ) {
geojsondata1 = JSON.stringify(data)
}
now you can do anything you want with the variable data

Node/ Express Rest API keeps entering same controller function inspite of correct mapping

I am writing a node/express rest api.
Hitting,
http://localhost:5000/api/news
and
http://localhost:5000/api/news/?id=c5f69d56be40e3b56e55d80
both give me all the news objects because it enters the same .getNews function on for both the urls.
My controller:
const NewsController = {};
const News = require('../models/news.model');
// This implementation of getNews is using Promises
NewsController.getNews = function(req, res) {
console.log('Inside getNews');
sendResponse = function(arg) {
res.json(arg);
}
const allnews = News.find({}, function(err, ns) {
sendResponse(ns);
});
};
// ES6 style
NewsController.getSingleNews = async (req, res) => {
console.log("Inside getSingleNews");
const news = await News.findById(req.params.id);
res.json[news];
};
NewsController.createNews = async (req, res) => {
const news = new News(req.body);
await news.save();
res.json[{
'status': 'item saved successfully'
}];
};
NewsController.deleteNews = async (req, res) => {
await News.findByIdAndRemove(req.params.id);
res.json[{
'status': 'item deleted successfully'
}]
};
module.exports = NewsController;
My routes.js (I am using the router at /api. So app.js has // use Router
app.use('/api', newsRoutes);
)
const express = require('express');
const router = express.Router();
var newsController = require('../controllers/NewsController')
router.get('/news', newsController.getNews);
router.get('/news/:id', newsController.getSingleNews);
router.post('/news', newsController.createNews);
router.delete('news/:id', newsController.deleteNews);
module.exports = router;
My Model
const mongoose = require('mongoose');
const { Schema } = mongoose;
const newsSchema = new Schema({
title: { type: String, required: true },
content: { type: String, required: true },
author: { type: String },
image: { type: String },
source: { type: String }
});
module.exports = mongoose.model('news', newsSchema);

The issue with your code is the way you are trying to call your endpoint. Express routes don't match query string parameters.
Having said that, your call to the news endpoint that looks like this:
http://localhost:5000/api/news/?id=c5f69d56be40e3b56e55d80
Should look like this instead:
http://localhost:5000/api/news/c5f69d56be40e3b56e55d80
That way the id parameter will get mapped to the req.params.id property inside your getSingleNews controller.
Being that the expected behavior for the way you declared your route:
router.get('/news/:id', newsController.getSingleNews);
For more information on how express routes work, check the documentation here: https://expressjs.com/en/guide/routing.html

Use /news/:id first. Your request will be redirected to the first matched url following the declaration order.
So /api/news satisfies app.get(/news)? Yep, gets redirected to that controller.
/api/news/?id=c5f69d56be40e3b56e55d80 satisfies app.get(/news)? Yep, also gets redirected to /news controller.
By the way, as your getting the id from req.params you should use /news/c5f69d56be40e3b56e55d80. If you were to get it from req.query you wouldn't need another route. /news?id=c5f69d56be40e3b56e55d80 would be perfect, you'd just need to check the existence of req.query.

How to take clientside Javascript arrays and POST through a Node.js API into a MongoDB database?

I have a single webpage that initially has two form inputs, one for a list of names and another for the title of a game. I've written some javascript/jquery that takes the X names and creates X more form inputs meant for each person's specific score. The javascript then creates the following variables upon the clicking of the names/scores form's submit button:
gameTitle = Monopoly
users = [Bob, Bill, Jim, Janet]
scores = [100, 110, 90, 80]
positions = [2, 1, 3, 4]
I then have a MongoDB schema set up as such:
const SessionSchema = new mongoose.Schema({
gameTitle: String,
users: [],
scores: [],
positions: []
});
And a Node.js handler as such:
const express = require('express');
const router = express.Router();
const bodyParser = require('body-parser');
const timestamps = require('mongoose-timestamp');
router.use(bodyParser.urlencoded({ extended: true }));
router.use(bodyParser.json());
const Session = require('./Session');
//Post a session to the database
router.post('/', function(req, res) {
Session.create({
gameTitle : req.body.gameTitle,
users : req.body.user,
scores : req.body.score,
positions : req.body.position
},
function (err, session) {
if (err) return res.status(500).send("There was a problem adding the information to the database");
res.status(200).send(session);
});
});
Using Postman I can see that posting works when I use this format:
Postman POST
Postman GET
How do I take the created javascript variables and, also upon the clicking of the names/scores form's submit button, POST them through the API and into the MongoDB database?
Apologies if I have missed any important information/code - I haven't fully wrapped my head around how the backend stuff works.

You need to register your Schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const SessionSchema = new mongoose.Schema({
gameTitle: String,
users: [],
scores: [],
positions: []
});
module.exports = mongoose.model('session', SessionSchema);
And here you need to use the mongo schema model, like this:
const express = require('express');
const router = express.Router();
const bodyParser = require('body-parser');
const timestamps = require('mongoose-timestamp');
router.use(bodyParser.urlencoded({ extended: true }));
router.use(bodyParser.json());
const SessionSchema = require('./Session'); // register the mongo model
const mongoose = require('mongoose');
const Session = mongoose.model('session');
//Post a session to the database
router.post('/', function(req, res) {
const new_session = {
gameTitle : req.body.gameTitle,
users : req.body.user,
scores : req.body.score,
positions : req.body.position
};
new_session.save((err, saved_session) => {
if(err) {
res.json(err);
} else {
res.json(saved_session);
}
});
});

Sounds like you have the backend working. What you're missing is the API request. Since your website is not under the same host:port than your API server, when doing it from the browser you'll face CORS issues. Let's get to that later:
First, you'll be making an API call. You can use axios or fetch. Let's go with fetch here:
fetch(url, {
body: JSON.stringify(yourJavascriptVariablesAsAnObject),
headers: {
'Content-Type': 'application/json'
},
method: 'POST',
})
.then(response => {
// here you can check for status if you want.
// ...
return response.json(); // assuming server returns JSON.
})
.then(responseBody => {
// Do something with the server's response body
});
Now for the CORS problem, if your client app is from create-react-app or at least you're using webpack-dev-server, you can proxy request really easy.
If you're not, then you need to allow CORS on your nodeJS server. The simplest way is to use a library.
PS: CORS basically means you can't do requests from a browser to a service living in a different `url:port:, unless that service explicitly says it's ok.
A third option would be putting both UI and server project behind a Web server like Nginx and proxy the requests, but that sounds too complex for what you need.

Javascript scraper logging in

I seem to be doing something wrong.
I have a student website that I want to scrape, but first I need to log in. Currently I have a python scraper that does it. The website logs in with a post request to a url containing a sid and PIN.
var login_url = 'https://example.com';
var formData = {
sid: 'username',
PIN: 'password'
}
How would I go about creating the same scraper but with javascript? I have seen the request library, which seems like what I want to use but cannot get it to work.

You need to use the request module to POST the form data to your endpoint. The response from the server will be in the call back to the .post() method.
const request = require('request');
// do not reassign "request", if you need to set properties us a different variable
// use the action= value from the form for the URL
const url = 'https://central.carleton.ca/prod/twbkwbis.P_ValLoginn';
const data = {
sid: 'username',
PIN: 'password',
};
request.post({ url: url, formData: data }, (err, response, body) => {
if (err) {
console.log('failed', err);
} else {
console.log('the response', body);
}
});
If you are interesting in parsing the resulting HTML I recommend using CheerioJS - much like jQuery but server side.