Is there a way to automate functions and call them at a specific mooment in time with Node.Js ? More precisely, I would like to create a reminder with node js. When I receive the date from the user, I have to send notification to remind him/her of something.
I know setTimeout() function but is it really good idea when I have large database of users ?
Thank you.
You can use setTimeout() and keep all the work inside your server, but for a large number of users, you would not necessarily want to set a timeout for every single user. All you really need is a single timeout for the next user that needs to be notified. When that timer fires, you then set a timer for the next user that needs to be notified and so on.
This can be done with an array of objects that you sort by the notification time. Each time you add something to the array, you cancel your current timer, add the new notification to the array, sort the array and set a timer for the earliest notification.
When a timer fires, you remove that item from the array and schedule the next one.
And, since you likely want this to survive a server restart, you save the notification array (probably in JSON format) to a file or to a database each time you modify it so you can reload that data upon a server restart.
FYI, there are scheduling modules for node.js that already offer this type of functionality if you'd prefer to pick up code someone else has already written.
Examples of some of these modules:
node-cron
agenda
node-schedule
What you are looking for is called a Scheduler or a cron job (its origin is from Linux OS).
In NodeJS you can find the node-schedule that implement the same idea.
Related
I am having a project in Laravel. In database I have a status column, which shows if exam is started or not. I had an idea in the waiting room checking every single second if the status was changed or not, if changed to 1, when the exam starts, but I am so new to Laravel and everything else, that I even don't get the main idea how I could do this, I don't ask for any code, just for the lead, to move on. yeah, hope someones gets me. Thanks if someone answers me.
Check about laravel cron jobs. You will need a class implementing ShouldQueue interface and using Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
With regards to the storage of the jobs i do recommend Redis or SQS.
In order to keep monitoring the queue in production think about installing supervisor.
Further information here: Queues
Your plan can work, it is called polling.
Basically, you will want to call
setInterval(function() {
//your code here
}, 1000);
setInterval is a function that receives two parameter. The first is a callback function, that will periodically be executed and the second is the length of the period in milliseconds (1000 milliseconds is a second).
Now, you will need to implement your callback function (Javascript, of course) to send an AJAX request to a Laravel action. You will need to look into XMLHttpRequest and its usages, or you can use some libraries to simplify your task, like jQuery or Axios.
On Laravel's side you will need to implement an action and a Route for it. (read this: https://appdividend.com/2022/01/22/laravel-ajax/)
Your Laravel will need to load data from your database, you can use Eloquent for this purpose or raw queries and then respond the POST request with the result.
Now, in your Javascript at the AJAX request's code you will need to have a callback function (yes, a callback inside a callback) which will handle the response and the the changes.
What about leveraging Observers? Also instead of having a status boolean, you could take a similar approach that Laravel has done for soft deletes and set exam_started_at. This way you can also keep track of time stamp and state all in one column. Also, observers are immediate rather than pushing them into a queue. Then generate a websocket event that can report back to your front end, if needed.
check out Laravel observer and soft delete documentation.
I know you specified "when the column on db changes..." but if it's not a strict-requirement you might want to consider implementing event-based architecture. Laravel has support for model events, which essentially allows you to run certain assertions and controls when a model created, updated, deleted etc.
class Exam extends Model
protected static function booted()
{
static::updated(function ($exam) {
if($exam->status=='your-desired-status'){
//your actions
}
//you can even in cooperate change controls
if ($exam->isDirty('status')){
//means status column changed
});
}
}
Of course this solution applies only if Database in question is in Laravel's reach. If database data changes outside the Laravel application these event listeners won't help at all.
The requirement is the user can take many actions at the same time (call API), but there some actions need to run sequentially for each user.
etc: user hit button add item, and clear cart the same time, but on the backend we need do it's sequentially, done every phase in action add item before start any new phase of action clear cart.
A phase can be called rest API to another service, query database or execute some javascripts code.
I am thinking about creating a queue job for every user's to archive this. Is that a good idea? What is the best way to do it, and are there any libraries for this (I have already searched but couldn't find anything)?
P/s: I'm using expressJS and MongoDB (> 4.0).
Edited:
If using a queue, how can i detect that user already has a queue or not to create a new one? If using Redis we can meet the race condition, or variable in ram can't help when I run nodejs as multi-cluster.
bluebird will solve this problem
code example:
action1()
.then(phase1)
.then(phase2)
.then(phase3);
action2()
.then(phase1)
.then(phase2);
I would like to know if it is possible to detect that a thread is already running a Cloud Functions, and if possible to also detect if it is running on a particular ID's data. I think I could have a variable stored in firebase memory of the ID in Firebase Database that the function is being run on from the Database, and to remove the variable when the function is done running,but the concern is of two writes to the database happening subsequently and very rapidly, causing the initial thread to not be able to write to memory fast enough before the second thread checks if the variable is there, especially on a cold start from the firebase thread - which in my understanding is a variable amount of time in which either thread could potentially spin up first.
My use case is this:
Let's say a write to the realtime database happens from the client side that causes a trigger for Cloud Functions to run a handler. This handlers job is to loop through and do work with the snapshot of records that was just written to by the client, and using a loop will parse each record in the snapshot, and when it is done, delete them. The handler works great until another record is written to the same group of records in the database before the handler's job is done, which causes a second handler thread to spin up, and start moving through the records in the same group of records, which would cause records to be iterated over twice, and possibly the data to be handled twice.
I have other solutions for my particular case which I can use instead, but it involves just allowing each record to trigger a separate thread like normal.
Thanks in advance!
There is no way to track running instances "in-memory" for Cloud Functions, as each function invocation may be running in entirely different virtual infra. Instead, what you'd most likely want to do here is have some kind of lock persisted in e.g. the Firebase Realtime Database using a transaction. So you'd do something like:
When the function invocation starts, generate a random "worker ID".
Run a transaction to check a DB path derived from the file you're processing. If it's empty, or populated with a timestamp that is older than a function timeout, write your worker ID and the current timestamp to the location. If it's not empty or the timestamp is fresh, exit your function immediately because there's already an active worker.
Do your file processing.
Run another transaction that deletes the lock from the DB if the worker ID in the DB still matches your worker ID.
This will prevent two functions from processing the same file at the same time. It will mean, however, that any functions that execute while a path is locked will be discarded (which may or may not be what you want).
There is an event after which comes feedback on my server. I have to send a message to exactly one hour after the event.
I can use a setTimeout() but it seems it will not be good.
Perhaps it would be better if I start cron which will check the status of an event from database(such as every minute)
Does anyone know of a greater good option for the implementation of such a long delay?
If you require 100% reliability use a cron job. The problem with a cron job is that your code will be spread around the place and it would require configuration each time you set up a new server. An alternate approach would be to use an npm package like cron.
The issue with setTimeout() is that if your application crashes/restarts the future function call will be lost. However, each time your application starts, it could query the DB and see all missing timeouts and set them appropriately.
My app's framework is built around collapsing backbone models sending the data via websockets and updating models on other clients with the data. My question is how should I batch these updates for times when an action triggers 5 changes in a row.
The syncing method is set up to update on any change but if I set 5 items at the same time I don't want it to fire 5 times in a row.
I was thinking I could do a setTimeout on any sync that gets cleared if something else tries to sync within a second of it. Does this seem like the best route or is there a better way to do this?
Thanks!
i haven't done this with backbone specifically, but i've done this kind of batching of commands in other distributed (client / server) apps in the past.
the gist of it is that you should start with a timeout and add a batch size for further optimization, if you see the need.
say you have a batch size of 10. what happens when you get 9 items stuffed into the batch and then the user just sits there and doesn't do anything else? the server would never get notified of the things the user wanted to do.
timeout generally works well to get small batches. but if you have an action that generates a large number of related commands you may want to batch all of the commands and send them all across as soon as they are ready instead of waiting for a timer. the time may fire in the middle of creating the commands and split things apart in a manner that causes problems, etc.
hope that helps.
Underscore.js, the utility library that Backbone.js uses, has several functions for throttling callbacks:
throttle makes a version of a function that will execute at most once every X milliseconds.
debounce makes a version of a function that will only execute if X milliseconds elapse since the last time it was called
after makes a version of a function that will execute only after it has been called X times.
So if you know there are 5 items that will be changed, you could register a callback like this:
// only call callback after 5 change events
collection.on("change", _.after(5, callback));
But more likely you don't, and you'll want to go with a timeout approach:
// only call callback 30 milliseconds after the last change event
collection.on("change", _.debounce(30, callback));