I'm writing a Discord bot and having it schedule a cron job when it starts up using node-cron. I've defined it as follows (very basic so far):
cron.schedule('15 1,3,5,7,9,11,13,15,17,19,21,23 * * *', () => {
GameUpdater.sendGameUpdatesToLeagueChannels(this.guilds);
});
The problem is, Discord bots will often reconnect which will rerun the portion of the code where this has to be. I'm considering having a check of an external value, perhaps in my database, that manages whether or not it's run recently or is already running or something like that, but I was wondering if there was a proper way to make a cron job unique using node-cron?
First, you need somewhere to save the name of the scheduled task, which defaults to a UUID V4 but can be provided as an option using the 3rd argument of the schedule method you are calling above.
With a reference to this name, either from a database or a constant that is hard-coded in the script you have wrote, you can use cron.getTasks to list all of the scheduled tasks to ensure that the current task doesn't already exist before scheduling it.
If you are referencing a hard-coded name, then calling the schedule method will replace the existing scheduled task, rather than creating a new task.
UPDATE
To set the name of the scheduled task, simply pass in an options object as a third argument to the schedule method you are calling, with a name for your task.
Example:
cron.schedule('15 1,3,5,7,9,11,13,15,17,19,21,23 * * *', () => {
GameUpdater.sendGameUpdatesToLeagueChannels(this.guilds);
}, { name: "example-task-1234" });
if the cron schedule is fired on startup from Client.on('ready') have your cronjob to be scheduled at a separate event Client.once('ready'), this way it won't fire the routine again when the bot reconnects. Only if the entire shard/cluster process is terminated and restarted.
You can you use logger to keep track of restaring or starting time of your server.
You can use winston Click HERE
It will help to generate your server log so that you can read it before starting of your server, so that you can take your action. Like say adjust the time etc
Related
I am having a project in Laravel. In database I have a status column, which shows if exam is started or not. I had an idea in the waiting room checking every single second if the status was changed or not, if changed to 1, when the exam starts, but I am so new to Laravel and everything else, that I even don't get the main idea how I could do this, I don't ask for any code, just for the lead, to move on. yeah, hope someones gets me. Thanks if someone answers me.
Check about laravel cron jobs. You will need a class implementing ShouldQueue interface and using Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
With regards to the storage of the jobs i do recommend Redis or SQS.
In order to keep monitoring the queue in production think about installing supervisor.
Further information here: Queues
Your plan can work, it is called polling.
Basically, you will want to call
setInterval(function() {
//your code here
}, 1000);
setInterval is a function that receives two parameter. The first is a callback function, that will periodically be executed and the second is the length of the period in milliseconds (1000 milliseconds is a second).
Now, you will need to implement your callback function (Javascript, of course) to send an AJAX request to a Laravel action. You will need to look into XMLHttpRequest and its usages, or you can use some libraries to simplify your task, like jQuery or Axios.
On Laravel's side you will need to implement an action and a Route for it. (read this: https://appdividend.com/2022/01/22/laravel-ajax/)
Your Laravel will need to load data from your database, you can use Eloquent for this purpose or raw queries and then respond the POST request with the result.
Now, in your Javascript at the AJAX request's code you will need to have a callback function (yes, a callback inside a callback) which will handle the response and the the changes.
What about leveraging Observers? Also instead of having a status boolean, you could take a similar approach that Laravel has done for soft deletes and set exam_started_at. This way you can also keep track of time stamp and state all in one column. Also, observers are immediate rather than pushing them into a queue. Then generate a websocket event that can report back to your front end, if needed.
check out Laravel observer and soft delete documentation.
I know you specified "when the column on db changes..." but if it's not a strict-requirement you might want to consider implementing event-based architecture. Laravel has support for model events, which essentially allows you to run certain assertions and controls when a model created, updated, deleted etc.
class Exam extends Model
protected static function booted()
{
static::updated(function ($exam) {
if($exam->status=='your-desired-status'){
//your actions
}
//you can even in cooperate change controls
if ($exam->isDirty('status')){
//means status column changed
});
}
}
Of course this solution applies only if Database in question is in Laravel's reach. If database data changes outside the Laravel application these event listeners won't help at all.
I am having an issue with a lightweight gRPC server I'm writing in NodeJS. I'm referencing the documentation here. I have been able to compile my proto files representing messages and services, and have successfully stood up a gRPC server with a server-side stream method I am able to trigger via BloomRPC.
I have a proto message called parcel, which has one field: parcel_id. I want this method to stream a parcel of data every second. My first rudimentary pass at this was a loop that executes every second for a minute, and applies a new parcel via call.write(parcel). I've included the method below, it executes with no errors when I invoke it via gRPC.
/**
* Implements the updateParcel RPC method.
* Feeds new parcel to the passed in "call" param
* until the simulation is stopped.
*/
function updateParcels(call) {
console.log("Parcels requested...");
// Continuously stream parcel protos to requester
let i = 0;
let id = 0;
while(i < 60){
// Create dummy parcel
let parcel = new messages.Parcel();
parcel.setParcelId(id);
id++;// Increment id
// Write parcel to call object
console.log("Sending parcel...");
call.write(parcel);
// Sleep for a second (1000 millis) before repeating
sleep(1000);
}
call.end();
}
My issue is that, although I am able to call my methods and receive results, the behavior is that I receive the first result immediately on the client (for both NodeJS client code and BloomRPC calls), but receive the last 59 results all at once only after the server executes call.end(). There are no errors, and the parcel objects I receive on the client are accurate and formatted correctly, they are just batched as described.
How can I achieve a constant stream of my parcels in real time? Is this possible? I've looked but can't tell for sure - do gRPC server-side streams have a batching behavior by default? I've tried my best to understand the gRPC documentation, but I can't tell if I'm simply trying to force gRPC server-side streams to do something they weren't designed to do. Thanks for any help, and let me know if I can provide any more information, as this is my first gRPC related SO question and I may have missed some relevant information.
It could be unrelated to gRPC, but to the sleep implementation used there.
The default one provided by node is a promise, so for this to work you probably have to declare the function as async and call await sleep(1000);.
I would like to know if it is possible to detect that a thread is already running a Cloud Functions, and if possible to also detect if it is running on a particular ID's data. I think I could have a variable stored in firebase memory of the ID in Firebase Database that the function is being run on from the Database, and to remove the variable when the function is done running,but the concern is of two writes to the database happening subsequently and very rapidly, causing the initial thread to not be able to write to memory fast enough before the second thread checks if the variable is there, especially on a cold start from the firebase thread - which in my understanding is a variable amount of time in which either thread could potentially spin up first.
My use case is this:
Let's say a write to the realtime database happens from the client side that causes a trigger for Cloud Functions to run a handler. This handlers job is to loop through and do work with the snapshot of records that was just written to by the client, and using a loop will parse each record in the snapshot, and when it is done, delete them. The handler works great until another record is written to the same group of records in the database before the handler's job is done, which causes a second handler thread to spin up, and start moving through the records in the same group of records, which would cause records to be iterated over twice, and possibly the data to be handled twice.
I have other solutions for my particular case which I can use instead, but it involves just allowing each record to trigger a separate thread like normal.
Thanks in advance!
There is no way to track running instances "in-memory" for Cloud Functions, as each function invocation may be running in entirely different virtual infra. Instead, what you'd most likely want to do here is have some kind of lock persisted in e.g. the Firebase Realtime Database using a transaction. So you'd do something like:
When the function invocation starts, generate a random "worker ID".
Run a transaction to check a DB path derived from the file you're processing. If it's empty, or populated with a timestamp that is older than a function timeout, write your worker ID and the current timestamp to the location. If it's not empty or the timestamp is fresh, exit your function immediately because there's already an active worker.
Do your file processing.
Run another transaction that deletes the lock from the DB if the worker ID in the DB still matches your worker ID.
This will prevent two functions from processing the same file at the same time. It will mean, however, that any functions that execute while a path is locked will be discarded (which may or may not be what you want).
Is there a way to automate functions and call them at a specific mooment in time with Node.Js ? More precisely, I would like to create a reminder with node js. When I receive the date from the user, I have to send notification to remind him/her of something.
I know setTimeout() function but is it really good idea when I have large database of users ?
Thank you.
You can use setTimeout() and keep all the work inside your server, but for a large number of users, you would not necessarily want to set a timeout for every single user. All you really need is a single timeout for the next user that needs to be notified. When that timer fires, you then set a timer for the next user that needs to be notified and so on.
This can be done with an array of objects that you sort by the notification time. Each time you add something to the array, you cancel your current timer, add the new notification to the array, sort the array and set a timer for the earliest notification.
When a timer fires, you remove that item from the array and schedule the next one.
And, since you likely want this to survive a server restart, you save the notification array (probably in JSON format) to a file or to a database each time you modify it so you can reload that data upon a server restart.
FYI, there are scheduling modules for node.js that already offer this type of functionality if you'd prefer to pick up code someone else has already written.
Examples of some of these modules:
node-cron
agenda
node-schedule
What you are looking for is called a Scheduler or a cron job (its origin is from Linux OS).
In NodeJS you can find the node-schedule that implement the same idea.
So I'm working on this tutorial to learn how to make a RESTful API in Node.js, and one part of it suddenly got me a bit worried. In the app an object is instantiated to handle the RESTful requests called TaskRepository().
As per Gist related to the tutorial, you'll see this code snippet:
var taskRepository = new TaskRepository();
My question is, will this instantiate one TaskRepository() object per user? In that case, isn't there a chance you'll run rather quickly out of memory if there's high enough traffic?
What's best practice here?
Also, if that is the case, how would you get around it programmatically to avoid a future traffic jam?
In that specific API, there is an API to create a task and it returns a task ID. That task will exist until some future API call refers to that specific ID and uses the delete operation.
The TaskRepository is per server (created once for your server), not per-user.
These tasks are not particularly per-user, but when you create a task and return it to the requestor, it is only that requestor that will likely know the ID and use it, though since this example does not create random IDs, they are predictable so anyone could reference a specific ID.
If you do not delete tasks after they are created, they will accumulate over time. Usually, something like this would create some sort of inactivity timeout and would automatically delete tasks if they are not used in some period of time (say 30 minutes).
In the case of the Gist you linked, the answer is no. new TaskRepository() is called once when the server is setup (right next to creating the var app = express and then that one instance will be shared and used for all requests.
Now, if you had called new TaskRepository() inside a route (app.get('stuff/etc', function () {}))) then you'd be correct. It would create a new instance of TaskRepository per HTTP request.