I am building a REACT note taking app.
I only track the changes the user makes to the note, and not what the current state of the note is.
If there are no changes to a specific property, the property will be sent as an empty string.
I am handling this in NODE (node-postgres) with the following function:
const updateNote = async (req, res) => {
const { category, title, body, nid } = req.body;
let noteStatement = "";
let valueStatement = "";
for (const key in req.body)
{
if (req.body[key] !== "" && key !== "nid") {
noteStatement = noteStatement + key + ", ";
valueStatement = valueStatement + `'${req.body[key]}', `;
}
}
try {
const result = await pool.query(
`UPDATE notes SET (${noteStatement}last_updated)
= (${valueStatement}(to_timestamp(${Date.now()} / 1000.0)))
WHERE nid = ${nid} RETURNING *`
);
const note = result.rows;
return res.status(200).send({ note });
} catch (err) {
return res.send({error: err});
}
};
I may be overthinking, but the goal was to send the smallest bit of data to the DB as possible.
I have spent a fair amount of time on this and this is the most pragmatic approach I came up with.
Is writing this type of query bad practice?
Would it make more sense to send all the note data including properties that have not been updated from React and have a fixed query updating all properties?
EDIT: Updated Query
const updateNote = async (req, res) => {
const { category, title, body, nid } = req.body;
const text = `UPDATE notes SET (category, title, body, last_updated)
= ($1, $2, $3, (to_timestamp(${Date.now()} / 1000.0)))
WHERE nid = $4 RETURNING *`;
const values = [category, title, body, nid];
try {
const result = await pool.query(text, values);
const note = result.rows;
return res.status(200).send({ note });
} catch (err) {
return res.send({ error: err });
}
};
I wanted to leave a comment but soon realized I approach the character limit so I would just leave it as a response.
First of, I want to make sure I understand what you are trying to accomplish. I assume you just wanna update your DB with only the fields that has been provided from the client. With that in mind I just want to underline the fact that most people are trying to overcomplicate things that should not. Everything in software is a tradeoff, in your case your data isn't that big to worry about updating just certain fields. It can be done but not the way you are doing it right now, you can have a utility function that would build a parameterized query based on the values that are not empty/null depending on how would you send the data that did not change from the client
Which brings me to the 2nd thing, you should never write a SQL query the way you have done it, by concatonating a string, leaves you vulnerable to SQL injections. Instead you must always use parameterized query unless you use some kind of library that abstracts writing the queries (ORMs).
As a sidenote, never trust data that comes from the client, so always, always validate the data on the server before you make any changes to the DB, even if you already did validation on the client. You can do it using a middleware like celebrate or other validation libraries. Never trust anything that comes from the client.
Related
I am using Couchbase in a node app. Every time I insert a document, I am using a random UUID.
It inserts fine and I could retrieve data based on this id.
But in reality, I actually want to search by a key called url in the document. To be able to get or update or delete a document.
I could possibly add the url as the id I suppose but that is not what I see in any database concepts. Ids are not urls
or any unique names. They are typically random numbers or incremental numbers.
How could I approach this so that I can use a random UUID as id but be able to search by url?
Cos lets say the id was 56475-asdf-7856, I am not going to know this value to search for right.
Whereas if the id was https://www.example.com I know about this url and searching for it would give me what I want.
Is it a good idea making the url the id.
This is in a node app using Couchbase.
databaseRouter.put('/update/:id', (req, res) => {
updateDocument(req)
.then(({ document, error }) => {
if (error) {
res.status(404).send(error);
}
res.json(document);
})
.catch(error => res.status(500).send(error));
});
export const updateDocument = async (req) => {
try {
const result = await collection.get(req.params.id); // Feels like id should be the way to do this, but doesn't make sense cos I won't know the id beforehand.
document.url = req.body.url || document.url;
await collection.replace(req.params.id, document);
return { document };
} catch (error) {
return { error };
}
};
I think it's okay to use URLs as IDs, especially if that's the primary way you're going to lookup documents, and you don't need to change the URL later. Yes, often times IDs are numbers or UUIDs, but there is no reason you have to be restricted to this.
However, another approach you can take is to use a SQL query (SQL++, technically, since this is a JSON database).
Something like:
SELECT d.*
FROM mybucket.myscope.mydocuments d
WHERE d.url = 'http://example.com/foo/baz/bar'
You'll also need an index with that, something like:
CREATE INDEX ix_url ON mybucket.myscope.mydocuments (url)
I'd recommend checking out the docs for writing a SQL++ query (sometimes still known as "N1QL") with Node.js: https://docs.couchbase.com/nodejs-sdk/current/howtos/n1ql-queries-with-sdk.html
Here's the first example in the docs:
async function queryPlaceholders() {
const query = `
SELECT airportname, city FROM \`travel-sample\`.inventory.airport
WHERE city=$1
`;
const options = { parameters: ['San Jose'] }
try {
let result = await cluster.query(query, options)
console.log("Result:", result)
return result
} catch (error) {
console.error('Query failed: ', error)
}
}
I am using Node.js and mongoose Module, I am looking for a way to count how many documents a user has and then if they have more then 0 documents it would edit their existing document and add the input so at the end it would have the previous text + the text that the user sent, so far this is how much I gotten.
const List = require('../Models/list.js')
List.countDocuments({}, function(err, count) {
if(count>0){
//edit document
}
else if(count=0){
const input = List.create({
User: User.name,
Songlist: args[0],
})
}
})
console.log('done')
here is how I think the code would look like
List.update(User.name) => update Songlist into List.Songlist + '|' + args[0]
I have never seen an update method like that. I am a nodejs developer. well, maybe there's a way like that.
Here's how I do to update a document
await Product.findByIdAndUpdate(id, //here where I have written "id" you have to write the id of the document you want to update.
{ //here in this object you have to put the variables of updated values
title: title,
description:description,
product_id:product_id,
category,
price,
});
there is also another method
await Product.findOneAndUpdate(name: 'asim', //let's suppose
{ //updated values
title:title,product: product
})
you can also read the documentation here https://mongoosejs.com/docs/tutorials/findoneandupdate.html
The Problem:
I have been unable to use Firebase (Google) Cloud Functions to collect and utilize device tokens for the cloud messaging feature.
Context:
I am a self-taught android-Java developer and have no JavaScript experience. Despite that, I believe I have code that should work and am not sure what the problem is. To my understanding, it could be one of three things:
Somehow my Firebase Realtime Database references are being called incorrectly and I am not retrieving data as expected.
I may need to use Promises to wait for all calls to be made before proceeding, however I don't really understand how I would incorporate that into the code I have.
I may be using multiple return statements incorrectly (which I am also fuzzy on).
My error message on the Firebase Realtime Database console is as follows:
#firebase/database: FIREBASE WARNING: Exception was thrown by user callback. Error: Registration token(s) provided to sendToDevice() must be a non-empty string or a non-empty array.
at FirebaseMessagingError.FirebaseError [as constructor] (/srv/node_modules/firebase-admin/lib/utils/error.js:42:28)
at FirebaseMessagingError.PrefixedFirebaseError [as constructor] (/srv/node_modules/firebase-admin/lib/utils/error.js:88:28)
at new FirebaseMessagingError (/srv/node_modules/firebase-admin/lib/utils/error.js:254:16)
at Messaging.validateRegistrationTokensType (/srv/node_modules/firebase-admin/lib/messaging/messaging.js:729:19)
at Messaging.sendToDevice (/srv/node_modules/firebase-admin/lib/messaging/messaging.js:328:14)
at admin.database.ref.once.snapshot (/srv/index.js:84:12)
at onceCallback (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:4933:51)
at /srv/node_modules/#firebase/database/dist/index.node.cjs.js:4549:22
at exceptionGuard (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:698:9)
at EventList.raise (/srv/node_modules/#firebase/database/dist/index.node.cjs.js:9684:17)
The above indicates I am not retrieving data either at all or by the time the return is called. My JavaScript function code is:
'use strict';
const admin = require('firebase-admin');
admin.initializeApp();
exports.pushNotification = functions.database.ref('/Chat Messages/{chatId}/{pushID}').onCreate((snapshot, context) => {
const valueObject = snapshot.after.val();
return admin.database().ref(`/Chat Basics/${valueObject.chatKey}/Chat Users`).once('value', statusSnapshot => {
var index = 0;
var totalkeys = statusSnapshot.numChildren();
var msgIDs = [];
statusSnapshot.forEach(msg=>{
msgIDs.push(msg.key.toString());
if(index === totalkeys - 1){
const payload = {
notification : {
title: valueObject.userName,
body: valueObject.message,
sound: "default"
}
}
sendNotificationPayload(valueObject.uid, payload);
}
index++;
});
});
});
function sendNotificationPayload(uid, payload){
admin.database()
.ref(`/User Token Data/${uid}`)
.once('value', snapshot=> {
var tokens = [];
//if(!snapshot.exists())return;
snapshot.forEach(item =>{
tokens.push(item.val())
});
admin.messaging()
.sendToDevice(tokens, payload)
.then(res => {
return console.log('Notification sent')
})
.catch(err => {
return console.log('Error in sending notification = '+err)
});
});
}
This code is mostly inspired by what was said to be a working example here from another Stack Overflow question here. I have successfully tested sending a notification to a single device by manually copying a device token into my function, so the function does run to completion. My Java code seems to be irrelevant to the problem, so I have not added it (please ask in the comments if you would like it added for further context).
What I Have Tried:
I have tried implementing promises into my code, but I don't think I was doing it properly. My main reference for this was here. I have also looked at the documentation for literally everything related to this topic, however my knowledge of JS is not sufficient to really apply barebones examples to my code.
My Firebase Realtime Database Nodes:
#1: Loop through chat members to collect user IDs:
"Chat Basics" : {
"1607801501690_TQY41wIfArhHDxEisyupZxwyHya2" : {
"Chat Users" : {
"JXrclZuu1aOwEpCe6KW8vSDea9h2" : true,
"TQY41wIfArhHDxEisyupZxwyHya2" : true
},
#2: Collect user tokens from collected IDs (ignore that tokens are matching):
"User Token Data" : {
"JXrclZuu1aOwEpCe6KW8vSDea9h2" : "duDR3KH3i3I:APA91bH_LCeslZlqL8akYw-LrM9Dv__nx4nU1TquCS0j6bGF1tlIARcheREuNdX1FheC92eelatBC8LO4t6gt8liRdFHV-NDuNLa13oHYxKgl3JBPPlrMo5rB5XhH7viTo4vfYOMftRi",
"TQY41wIfArhHDxEisyupZxwyHya2" : "duDR3KH3i3I:APA91bH_LCeslZlqL8akYw-LrM9Dv__nx4nU1TquCS0j6bGF1tlIARcheREuNdX1FheC92eelatBC8LO4t6gt8liRdFHV-NDuNLa13oHYxKgl3JBPPlrMo5rB5XhH7viTo4vfYOMftRi"
}
Conclusion:
Concrete examples would be much appreciated, especially since I am crunching right now. Thanks for your time and help!
Update:
After some more testing, it looks like the problem is definitely due to my lack of understanding of promises in two areas. Firstly, only one user is collected before the final return is called. Secondly, the final return is called before the 2nd forEach() loop can store snapshot data to an array.
For this code then, how may I modify (or rebuild) it so that it collects all keys before proceeding to retrieve token data from all keys - ultimately before returning the notification?
Just as with every question I post, I managed to figure out how to do it (tentatively) a few hours later. Below is a full example of how to send a notification to chat users based on a message sent (although it does not yet exclude the sender) to a given chat. The order of operations are as such:
User message is saved and triggers event. Relevant data the message contains are:
username, chat key, message
These are retrieved, with (username + message) as the (title + body) of the
notification respectively, and the chat key is used for user id reference.
Loop through chat user keys + collect.
Loop through array of chat user keys to collect array of device tokens.
Send notification when complete.
The code:
//Use firebase functions:log to see log
exports.pushNotification = functions.database.ref('/Chat Messages/{chatId}/{pushId}').onWrite((change, context) => {
const valueObject = change.after.val();
return admin.database().ref(`/Chat Basics/${valueObject.chatKey}/Chat Users`).once('value', statusSnapshot => {
var index = 0;
var totalkeys = statusSnapshot.numChildren();
var msgIDs = [];
statusSnapshot.forEach(msg=>{
msgIDs.push(msg.key.toString());
if(index === totalkeys - 1){
const payload = {
notification : {
title: valueObject.userName,
body: valueObject.message,
sound: "default"
}
}
let promises = [];
var tokens = [];
for(let i=0; i < msgIDs.length; i++){
let userId = msgIDs[i];
let promise = admin.database().ref(`/User Token Data/${userId}`).once('value', snapshot=> {
tokens.push(snapshot.val());
})
promises.push(promise);
}
return Promise.all(promises).then(() => {
return admin.messaging().sendToDevice(tokens, payload);
});
}
index++;
return false;
});
});
});
I have almost completed pub/sub fake-server, that requests user password and email (from the client), compares this info with database and returns data back. It has 'api_in' and 'api_out' frames and then the JSON.
The publisher takes and processes all the info without a hitch, but it doesn't seem to send anything back to the client (subscriber) and I don't know why, cause it is connected to the subsequent port.
And I know that this implementation is not a classic PUB/SUB pattern, but that was prerequisite, to do it like that.
I tried different pub/sub options, but nothing has changed.
Server
let zmq = require('zeromq');
const sqlite3 = require('sqlite3').verbose();
const DBSOURCE = "./db.sqlite";
let db = new sqlite3.Database(DBSOURCE, (err) => {
if(err) {
console.error(err.message);
throw err;
} else {
console.log('Connected to SQLite database');
db.run(`CREATE TABLE users (
user_id INTEGER,
email TEXT,
passw TEXT)`,
(err) => {
if (err) {
// Table already created
} else {
// Creating rows
let insert = 'INSERT INTO users (user_id, email, passw) VALUES (?,?,?)';
db.run(insert, [123098, 'phillCollins#gmail.com','5502013']);
db.run(insert, [42424242,'dukenukem3d#mustdie.com','RodriguesShallLiveLong']);
db.run(insert, [5,'yourchick#yandex.ru','semolinaAndPain666']);
}
})
}
});
const args = require('minimist')(process.argv.slice(2));
const pubSocket = zmq.socket('pub', null);
pubSocket.bindSync(`tcp://127.0.0.1:${args['pub']}`);
const subSocket = zmq.socket('sub', null);
subSocket.subscribe('api_in');
subSocket.on('message', function(data) {
let message = data.toString().replace(/api_in/g, '');
let mes = JSON.parse(message);
let api_out = 'api_out';
let errorWrongPWD = 'WRONG_PWD';
let errorWrongFormat = 'WRONG_FORMAT';
if(mes.type = 'login') {
db.get(`SELECT user_id from users WHERE email = ? and passw = ?`, [mes.email, mes.pwd], function(err, row) {
if(err) {
console.log(err);
} else {
if(row) {
let msg = {
msg_id: mes.msg_id,
user_id: row.user_id,
status: 'ok'
}
let outMessage = api_out + JSON.stringify(msg);
console.log(outMessage);
subSocket.send(outMessage);
} else {
let msg = {
msg_id: mes.msg_id,
status: 'error',
error: mes.email == '' || mes.pwd == '' ? errorWrongFormat : errorWrongPWD
}
console.log(msg);
let outMessage = api_out + JSON.stringify(msg);
subSocket.send(outMessage);
}
}
});
}
});
subSocket.bindSync(`tcp://127.0.0.1:${args['sub']}`);
client
let zmq = require('zeromq');
let uniqid = require('uniqid');
let readline = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
const args = require('minimist')(process.argv.slice(2));
const pubSocket = zmq.socket('pub', null);
let pubSocketTCP = `tcp://127.0.0.1:${args['sub']}`;
pubSocket.connect(pubSocketTCP);
const subSocket = zmq.socket('sub', null);
let subSocketTCP = `tcp://127.0.0.1:${args['pub']}`;
subSocket.connect(subSocketTCP);
let api_in = 'api_in';
let secondFrame = {
type: 'login',
email: '',
pwd: '',
msg_id: uniqid()
}
readline.question('What is your email? \n', (email) => {
secondFrame.email = email;
readline.question('What is your password \n', (pwd) => {
secondFrame.pwd = pwd;
let msg = api_in + JSON.stringify(secondFrame);
console.log(msg);
pubSocket.send(msg);
});
});
subSocket.subscribe('api_out');
subSocket.on('message', (response) => {
/* let res = response.toString().replace('api_out');
let responseParsed = JSON.parse(res);
console.log(responseParsed.status);
if(response.status == 'error') console.log(response.error); */
console.log(response);
});
I want the server side to send info back.
Well, first of all, welcome to the Zen-of-Zero domain. ZeroMQ is a powerful tool for smart signaling / messaging, so if you pay attention to all its internal beauties, there will be no such thing you will not be able to do with it ( nightmares still not avoided on this way forward ). If feeling to be new to this domain, one may enjoy a short read into "ZeroMQ Principles in less than Five Seconds" before diving into further details on subject, or re-use some of tricks posted here
Q : it doesn't seem to send anything back to the client (subscriber) and I don't know why
There are two pieces of code, that seem to use both PUB and SUB Scalable Formal Communication Pattern Archetypes, yet have some glitches on how these get configured.
The server-code :
seems to try to instantiate PUB-archetype and equips that instance with a single AccessPoint, using .bindSync()-method and cli-parameter args['pub'] for accepting connections over a plain and commontcp://-transport-class.
After defining the event-handler .on( 'message', ... ) for the second instance, being the SUB-archetype, this one becomes .bindSync()-equipped with a single AccessPoint, using tcp://-transport-class, for receiving connections using tcp://-transport-class.
If you indeed have to make a .send() over a SUB-alike archetype, you have to use XSUB alternative, where you can send data and perform some tricks with the actual payload on the PUB-side or XPUB-side ( ref. API documentation for details about ZMQ_XPUB_MANUAL mode capabilities and limits for some wilder data-mangling on the XPUB-side )
ZMQ_XSUB
Same as ZMQ_SUB except that you subscribe by sending subscription messages to the socket. Subscription message is a byte 1 (for subscriptions) or byte 0 (for unsubscriptions) followed by the subscription body. Messages without a sub/unsub prefix may also be sent, but have no effect on subscription status.
The client-code :
Seems to instantiate and .connect() both the client-local PUB and SUB Archetypes over tcp://-transport-class to the server-side AccessPoints ( which both ought to set ZMQ_LINGER to 0, so as to avoid infinitely hanging orphans ( version dependent defaults do not have other solution but an explicit setting on this ) ).
Possible improvements :
XPUB/XSUB with ZMQ_XPUB_MANUAL may solve the sending via SUB-archetype
XPUB/SUB with ZMQ_XPUB_MANUAL may solve the sending via SUB-archetype with less comfort of masquerading the messages to be sent via the .subscribe()-method
PUB/SUB with making all .send()-s strictly via a local PUB-archetype instance.
be explicit with ZMQ_SNDHWM and ZMQ_RCVHWM parameters if still loosing messages
be explicit on setting .subscribe() only after a successfully completed { .bind() + .connect() }-methods ( systematically using original zmq_errno() and zmq_strerror() functions for actively detecting and repairing the potential colliding states )
may request to work with only completed connections ( distributed systems have zero-warranty of an order of operations autonomous distributed (many) agents perform ) using .setsockopt( ZMQ_IMMEDIATE, 1 )
Your server is trying to send on the sub socket
subSocket.send(outMessage);
You can't send on the sub socket. It should be sending on a pub socket.
I'm building an application using Node/Express/MongoDB (first time with all these) that will let me pull data from the DB and display it on an Express page. This is what the GET request looks like:
var str = "";
app.route('/view-reports').get(function(req, res) {
var cursor = collections.find({});
cursor.each(function(err, item) {
if (item != null) {
console.log(str);
str = str + "Substance: " + item.substance + "<br>";
}
if (err) {
console.log(err);
}
});
console.log(str);
res.send(str);
str = "";
});
I would expect this to return something like this:
Substance: a
Substance: b
Substance: c
However, the initial request does not return anything at all. The second request will return the above. If I enclose res.send(str) in an if conditional it simply will not load until a second request is made.
cursor.each() is asynchronous. That means it runs sometimes LATER, after your res.send(str), thus you get the previous version of str. You need to collect all the data first and then send your response only when you have all the data.
If you want all the data, then you could use promises and .toArray() like this:
app.route('/view-reports').get(function(req, res) {
collections.find({}).toArray().then(data => {
let result = data.map(item => {
return "Substance: " + item.substance + "<br>";
}).join("");
res.send(result);
}).catch(err => {
// database error
console.log(err);
res.sendStatus(500);
});
});
Note: This also wisely gets rid of the str variable which was outside the scope of the request and thus could easily lead to a concurrency bug when multiple requests were in flight at the same time (from different users).
Create a router specifically for substances and use it in app. Instead of breaks, you can create a ul, also, that processing should happen on the front end. Separate your concerns. The server shouldn't have to worry about any rendering and etc. One purpose per process.
The routers can be created per resource. Create a router for substances, for cats, for dogs. Each individual router has it's own get post delete and puts that allow you to modify that resource. app can use all the routers at once.
app.use(catRouter);
app.use(mooseRouter);
app.use(platypusRouter);
const { Router } = require('express');
const createError = require('http-errors');
let substanceRouter = new Router();
function buildElement(arr)
{
let start = '';
arr.forEach(val => {
if(!val) return;
start += `Substance : ${val}<br>`;
});
return start;
}
subtanceRouter.get('/endpoint/whatever', function(req, res, next) {
collectios.find({})
.then(results => {
if(!results) throw new Error('Resource Not Found');
let output = buildElement(results);
res.json(output);
next();
})
.catch(err => next(createError(404, err.message)));
})
app.use(substanceRouter);
Alternately we can write :
let output = results
.filter(sub => !!sub)
.join('<br>');
res.json(output);
But be advised this will add memory overhead, generates a completely new array to house the results, consuming at worst, O(n) memory.