Requirement
I want to create an email scheduling system in which user set the time to schedule a email sending. Now in this case every user can set their appropriate time to send an email everyday. Now how can I tackle this for individual user ?
Where I stuck ?
If I run the cron job every minute to verify the scheduled time set by the users and if the previous call of Cron() function is not finished then it will not run for the second time until the previous execution completes. and hence same task for the other users will not start. So do I need to create a separate cron job for each user ? And if so then How can I implement that ?
Cron Function
Inside Cron function I am fetching all the users whose time is matching with the current time and then sending them email.
const job = new CronJob({
cronTime: `* * * * *`,
onTick: function () {
Cron().catch((err) => console.error(`Error --> ${err.stack}`))
},
start: false,
timeZone: `Asia/Kolkata`
})
job.start()
You Events in Nodejs (multithreading). With the help of MultiThreading, you can run multiple users tasks without stopping in between.
Related
What I intend to do is a program that sends congratulatory emails for their birthday to several users, then the program will take today and execute a query to the database (it is an Excel file), in which it will take the date of the users and compare their date of birth with the current date, if the month and day coincide, mail will be sent. I think it can be done with a setInterval(), but I don't know if it affects the performance of the program. Since it will be uploaded on a windows server 2012 server of my company.
My code:
const express = require("express");
const app = express();
const excel = require('./public/scripts/readExcel.js');
const email = require('./services/email/sendEmail.js');
app.post('/send-email',(req, res)=>{
setInterval(() => {
email.sendEmail()
.then((result) => {
res.status(200).jsonp(req.body);;
console.log(result)
}).catch((err) => {
res.status(500).jsonp(req.body);;
console.log(err);
});
}, 3600000);//1 hour
});
app.listen(4000, ()=>{
console.log("Serven on -> http://localhost:4000");
})
Basically what it does is call the sendEmail function every hour which reads the Excel file or the database and extracts the date fields, compares with the current day, and sends the mail with Nodemailer to those who have birthdays. Also, the setInterval would go in the route of "/send-email" or how would the request be made?
For that, you can also run a cron job at every hour using npm package cron-job
using
var cron = require('node-cron');
cron.schedule('* * 1 * *', () => {
console.log('running a task every hour');
});
I'll break down my answer in two parts
What you need to do to make your solution work
How can you optimise the performance
1. What you need to do to make your solution work
There are two essential problems you need to resolve.
a. Your solution will work as it is, only thing you need to do is to call /send-email endpoint once after starting your server. BUT... this comes with side effects.
As setInterval will call the email.sendEmail... code block every hour, and this code block calls res.status(200).jsonp(req.body) every time. If you don't know this res.status.. sets the response for the request you receive. In this case, your request to /send-email. For the first time, it will work fine because you are returning the response to your above request. But when second time call to this code block kicks in, it has nothing to respond to because request has already been responded. Remember, HTTP protocol responds to a request once, then the request has been completed. So for this reason, your code block res.status... becomes invalid. So first thing, call res.status only once. So I'd remove this line out of the setInterval code block as follows
app.post('/send-email',(req, res)=>{
setInterval(() => {
email.sendEmail()
.then((result) => {
console.log(result)
}).catch((err) => {
console.log(err);
});
}, 3600000);//1 hour
res.status(200).jsonp(req.body);
})
b. Also I don't think you'd want the hastle of calling /send-email every time you start server, so I'd also make sure that this code block for birthday wishes gets kicked off every time you start server automatically. So I'd then just remove the line app.post('/send-email',(req, res)=>{. Also not that I'm not calling this for a request, I don't have any request to send response to so I can also remove the res.status.. line. And your code looks like this now
const express = require("express");
const app = express();
const email = require('./services/email/sendEmail.js');
(function(){
// Birthday wish email functionality
setInterval(() => {
email.sendEmail()
.then((result) => {
console.log(result)
}).catch((err) => {
console.log(err);
});
}, 3600000);//1 hour
})() // Immediately invoked function
That's it, your solution works now. Now to send birthday wish emails, you don't need to do anything else other than just starting your server.
Let's move on to second part now
2. How can you optimise the performance
a. Set interval to be 24hrs instead of 1 hr
Why do you need to check every hour for birthday? If you don't have a good answer here, I'd definitely change the interval to be 24hrs
b. Making the code more robust to deal with large data
As long as you have only 100s of entries in your excels and they are not going to grow much in future, I wouldn't go into making it more complex for performance.
But if your entries are destined to grow to 1000s and further. I'd suggest to use database(such as mongodb, postgres or mysql, etc.) to store your data and query only the entries with the birthday matching the particular date.
I'd also implement a queuing system to process query and send emails in batches instead of doing all of that at once.
I have a route like this to execute a function:
router.post('/send-verification-again', sendVerificationAgain);
At the function above we have the email or phone number of the account in which we want to verify.
This will execute a function to send an email or sms to the user with his/her verification code ok?
Apparently we don't want the user to be able to send more than one request per minute to this route, right?
How can I limit execution of a function for specified user at one per minute?
function sendVerificationAgain(email) {
// for each specific email here the function can be executed one time per minute
}
This is very common functionality, is there any convention to do this?
You can simply store the email for a minute and check if it's stored:
const recentlySent = new Set();
function sendVerificationAgain(email) {
if (recentlySent.has(email)) return;
recentlySent.add(email);
setTimeout(() => recentlySent.delete(email), 60e3);
// ...
}
You might also want to first normalize the email, e.g. lowercase it.
For more information, look up "debounce" in the programming sense
I have a product table, and an expired_date field.
I want to use cron-job, to send node-mailer about 1 week or 2 weeks before a product get expired, based from the expired_date field.
is this possible to do with cron?
Yes, it can be done.
I suggest you create one CRON-JOB which execute once daily.
The CRON Job execute the task about checking your table about which product is expiring before 1 week or 2 weeks, then send the email if expiring product existed.
You can use npm cron to get this done. Run a daily job where you will be querying database to fetch all the expired results, send an email to users after fetching data.
const CronJob = require('cron').CronJob;
const job = new CronJob('* * * */1 * *', () => {
console.log('You will see this message every second');
}, null, true, 'America/Los_Angeles');
job.start();
I'm building a chat bot (using MeteorJS/NodeJS) which interacts with about 2,000 active users everyday. I know the exact number of people who chat with the bot everyday because I store the users active information in a MongoDB collection called ActiveReports.
This is a scenario in my app: if a user A chat with the bot 100 times (= 100 messages) in a day, these steps will be executed:
- receive message from users
- check if this user is marked as 'active' today ? // high cost
- if yes => don't do anything
- if no => mark this user as 'active' for today
As you can see, step 2 is executed for every messages. This step is technically equivalent to accessing the ActiveReports collection, find the one with timestamp = today, user = user A. Since the ActiveReports collection has a lot of documents (about 100,000 documents), this is a fairly heavy task. This negatively affects the app's performance.
NOTE 1: This is the ActiveReports collection schema:
SimpleSchema({
// _id must be set `type` as String and `optional` as true
// to avoid ObjectId(_id) after insert in to database
_id: {
type: String,
optional: true,
},
date: {
type: Date, // Note: date is always the timestamp of the start of the current day, so 1AM timestamp and 9PM timestamp will be changed to 0AM timestamp (before the insert)
},
userId: {
type: String,
},
});
And this is how I indexed this collection:
ActiveReports._ensureIndex({ date: 1, userId: 1 }, { unique: true });
NOTE 2: A user is active in a day means he interacts with the bot at least 1 time (e.g send a message to the bot) that day.
Any ideas how I can improve this ? Please tell me if you need further information. Thank you.
Add a field last_active_date to a User schema and update it every time you get a message. If the date matches today, you are done. If it's not, you need to update the field and add a record to ActiveReports collection.
Actually, it seems to me that you are trying to use Mongo here in a way you would use a relational database. I mean, that there is no need in ActiveReports if you just want to mark a user as active.
If you are trying to build some sort of report for showing app usage per user per day, you can do it in the background. You can have a job that will run once a day (actually, if you have users in different time zones and you want to tolerate their time, you may want to run it few times a day). This job will query the User collection and add records to ActiveReports for each user it finds where last_active_date is current_date.
If you're building a stateless server application the minimum you need to do is pull the user's record to check active.
You might consider having have a daemon task process the ActiveReports and update user dates in the background. That way you only process those records once and the user info is ready to go. Also, that process can have state so it can be more optimal that updating every user for every record.
I have a chat room and I want to show how many people are online exactly in this chat at this moment. User can join room with or without registation.
This is strongly dependant on the way you implement the chat room.
You could assign a chat-session id and timeout to each visitor, which gets expired over time and removed from a list.
This list will contain details on visitors, including the count.
Just a quick idea that has come to my mind (can be heavily customized and improved):
1) Call a PHP script periodically (for instance, once per a minute) through an AJAX call with a unique per-user ID. Like this, for example:
var visitorCounter = function() {
$.get('audience_checker.php', {
id: get_random_id() // inspiration below
});
}
setInterval(visitorCounter, 60000); // it gets called every 60000 ms = 1 minute
Take an inspiration how to build a random ID generation here. Or use the IP address.
2) Now write the PHP script that will store IDs from $_GET super-global variable in a database, with timestamp. If the ID already exists, just update the timestamp.
3) And finally, another script statistics.php can just select those data from the database which are not older than a minute bases on the timestamp.
Of course will depend on your chat application logic but that's something I am using to count the users in my application. It is not perfect because you never know about your users if they don't log off.
You can add a new table to handle sessions:
`id`, `expire`, `data`, `user_id`, `last_write`
then change the configuration to save the sessions into this table instead of files.
'session' => [
'class' => 'yii\web\DbSession',
'writeCallback' => function ($session) {
return [
'user_id' => Yii::$app->user->id,
'last_write' => time(),
];
},
],
then you can check the sessions in the last 5 minutes for instance
Hope it helps