We Keep Coding

How to stop a recursive setTimeout with an API call?

I have a NextJS application that runs a recursive setTimeout when the server is started. I need to create an API endpoint that can start and stop this loop (to have more control over it in production). This loop is used to process items in a database that are added from another API endpoint.
import { clearTimeout } from "timers";
var loopFlag = true;
export function loopFlagSwitch(flag: boolean) {
loopFlag = flag;
}
export async function loop() {
try {
// Retrieve all unprocessed transactions
const unprocessedTransactions = await prisma.transaction.findMany({
take: 100,
where: { status: "UNPROCESSED" },
});
// Loop through transactions and do stuff
for (const transaction of unprocessedTransactions) {
//stuff
}
} catch (e) {
// handle error
}
if (loopFlag === true) {
setTimeout(loop, 1000); //if flag changes, this will stop running
}
}
if (require.main === module) {
loop(); // This is called when server starts, but not when file is imported
}
The reason I use setTimeout and not setInterval is because many errors can occur when processing items retrieved from DB. These errors, however, are solved by waiting a few milliseconds. So, the benefit of the pattern below is that if an error happens, the loop immediately restarts and the error will not appear because a ms has passed (it's due to concurrency problems -- let's ignore this for now).
To attempt to start and stop this loop, I have an endpoint that simply calls the loopFlagSwitch function.
import { NextApiRequest, NextApiResponse } from "next";
import { loopFlagSwitch } from "services/loop";
async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
loopFlagSwitch(req.body.flag);
} catch (error) {
logger.info({ error: error });
}
}
export default handler;
Problem is, even when this endpoint is called, the setTimeout loop keeps going. Why isn't it picking the change in flag?

clearTimeout()
The global clearTimeout() method cancels a timeout previously established by calling setTimeout().
To clear a timeout, use the id returned from setTimeout():
Usage
const myTimeout = setTimeout(function, milliseconds);
//Then you can to stop the execution by calling clearTimeout():
clearTimeout(myTimeout);
loopFlag as a condition
...
if (loopFlag === true) {
myTimeout();
} else {
clearTimeout(myTimeout)
}
...
Add abortTimer function
Full code
export function loopFlagSwitch(flag) {
flag === true ? loop : abortTimer()
}
// set timeout
var myTimeout = setTimeout(loop, 1000);
function abortTimer() { // to be called when you want to stop the timer
clearTimeout(myTimeout);
}
export async function loop() {
try {
// Retrieve all unprocessed transactions
let d = "Retrieve all unprocessed transactions"
process.stdout.write(d + '\n');
// Loop through transactions and do stuff
for (let i = 0; i<10; i++) {
//stuff
let c = "second loop"
process.stdout.write(c + '\n');
}
} catch (e) {
// handle error
console.log("error ", e)
} finally {
myTimeout = setTimeout(loop, 1000); // repeat myself
}
}
if (require.main === module) {
loop(); // This is called when server starts, but not when file is imported
}

The flag will not work because node doesn't maintain the state of a file, the import only cares about the things it obtains from a file, it doesn't mind about the state of the variables declared in it.
Even though the clearTimeout() function may be sufficient, i think there is an even better option you can use to stop this loop.
Use a JS Class!
Instead of using just a function without state. You could instantiate a class that runs on the server with an internal boolean that can be called "shouldKeepLooping" for example:
class NextJsLooper { // or whatever better name you can use
private shouldKeepLooping: boolean
constructor () {
this.shouldKeepLooping = true
}
public shouldKeepLooping(value) { this.shouldKeepLooping = value }
public async loop () {
if (shouldKeepLooping) {
//... rest of the loop code
setTimeout(this.loop, 1000);
}
}
}
This way if you set the value to false it will automatically stop. Since this is a reference to the object.
Keep in mind that you would need to keep this instance alive as probably something global, and would need it to be accesible by nextJS.
You can use Symbol.for and the Node global to mantain the instance saved somewhere in the server!

My suggestion is to introduce the use of signals.
Service like Pusher will trigger the event that will be listened by the transaction processor.
your transaction processing api / code above or any other
Signal actions like "start" and "stop" will be triggered anytime even in production by either frontend or through the pusher portal that will be used to change the loop flag to either true or false.

you can retrieve the thread in charge of the operation and interrupt after a given expected time ,and log something to inform you about the time out in case that your operation took more than the necessary time .

How to run an async function in js with timeouts (for rate limiting)

I'm designing a system where a user will interact with my RESTful API through JS, and will replace the ts html element with a different SVG if the returned answer is true. If the API call returns false, the JS should wait for 5 seconds, before retrying the API call. again, if the API call returns true, the SVG will be changed, and the function will stop calling the API. How can I have the function "sleep" for 5 seconds before running again, and ending the function if the returned API call is true?
Code:
async function getTransaction(){
var lookup = $("meta[name='lookup']").attr("content");
axios.get('http://127.0.0.1:5000/client_api/transaction_status/' + lookup)
.then(function (response) {
var tStat = response.data.transactionStatus;
console
if(tStat){
document.getElementById("ts").innerHTML = 'drawing the new svg and replacing the old one'
console.log('transaction_found')
}
else if(!tStat){
console.log("transaction not found")
}
else{
console.log("error")
}
});
}console.log("hello"); while(true){setTimeout(getTransaction,5000);}

Don't use sleep sice it pauses your programm to pause. Instead use setTimeout like so. This creates an event listener which calls the function again after 5s if tStat is false.
In case of false the function will set an timeout after which the function is called again. After the timeout is completed it clears itself. This step repeats until the api request eventually resolves to true
async function getTransaction() {
var lookup = $("meta[name='lookup']").attr("content");
axios.get('http://127.0.0.1:5000/client_api/transaction_status/' + lookup)
.then(function(response) {
var tStat = response.data.transactionStatus;
console
if (tStat) {
document.getElementById("ts").innerHTML = 'drawing the new svg and replacing the old one'
console.log('transaction_found')
} else {
console.log("transaction not found")
setTimeout(getTransaction, 5000);
}
});
}
console.log("hello");

Loops in RxJS Observable don't stop after calling complete or error

When calling observer.complete() or observer.error() an observer stops sending data and is considered done. However, If there is a for loop in the observer, the loop will continue running even after observer.complete() is called. Can someone explain to me this behaviour? I expect that the loop would be cut short. The current behaviour means that an interval or a while loop will forever run in an Observable unless I unsubscribe in the code.
In the below snippet I added a console.log to see if the log will be called after observer.complete().
const testObservable = new Observable( observer => {
for (let count = 0; count < 11; count++){
observer.next(count);
if (count > 5) {
observer.complete()
console.log("test")
}
if (count > 7) {
observer.error(Error("This is an error"))
}
}}
);
let firstObservable = testObservable.subscribe(
next => {console.log(next)},
error => { alert(error.message)},
() => {console.log("complete")}
)

It is a responsibility of the create function to properly destroy resources. In your case there could be simply return statement after observer.complete().
But in general if needed, the create function should return the teardown function (TeardownLogic), that is called when the observable is finalised.
new Observable( observer => {
// observable logic, e.g. ajax request
return () => {
// teardown function, e.g. cancel ajax request
};
});
Observable just ignore future calls of next(...), complete() and error(...) once the observable is finalized. It means that complete() or error(...) function was called internally or the observable was unsubscribed externally.
new Observable( observer => {
observer.next('a');
observer.complete();
// next calls will be ignored
observer.next('b');
});
const observable = new Observable( observer => {
setTimeout(
() => observer.next('a'), // this will be ignored due to unsubscribe
10
);
});
const subscription = observable.subscribe(...);
subscription.unsubscribe();

Why is my code executing although it should pause?

I have an API which is limited regarding how many requests per minute (50/minute) I can send to any endpoint provided by that API.
In the following code-section, I filter the objects orders with an URL as property, every object with an URL that provides data should be stored in successfullResponses in my app.component.ts.
Promise.all(
orders.map(order => this.api.getURL(order.resource_url).catch(() => null))
).then(responses => {
const successfulResponses = responses.filter(response => response != null)
for(let data of successfulResponses) {
// some other requests should be sent with data here
}
});
There are more than 50 orders to check, but I just can check maximum 50 orders at once, so I try to handle it in my service. I set the first date when the first request is sent. After that I compare the dates of the new request with the first one. If the difference is over 60, I set the current date to the new one and set maxReq again to 50. If it is under 60, I check if there are requests left, if yes I send the request and if not I just wait one minute :
sleep(ms){
return new Promise(resolve => setTimeout(resolve, ms));
}
async getURL(){
if(!this.date){
let date = new Date();
this.date = date;
}
if((new Date().getSeconds() - this.date.getSeconds() > 60 )){
this.maxReq = 50;
this.date = new Date();
return this.http.get(url, this.httpOptions).toPromise();
} else {
if(this.maxReq > 0){
this.maxReq -= 1;
return this.http.get(url, this.httpOptions).toPromise();
} else{
console.log("wait");
await this.sleep(60*1000);
this.maxReq = 50;
this.date = new Date();
return this.http.get(url, this.httpOptions).toPromise();
}
}
}
However the code in app.component.tsis not waiting for the function getURL() and executes further code with requests which leads to the problem that I send ´too many requests too quickly´.
What can I do about that?

I had a similar problem while trying to use promises with multiple async functions. It's an easy thing to forget, but in order to make them all wait, you have to use await on the root line that calls the function in question.
I'm not entirely certain, but my presumption is that your await this.sleep(60*1000); line is indeed waiting for a timeout to occur, but whilst it is doing this, the code that called getURL() is executing the rest of its lines, because it did not have an await (or equivalent, like .then) before getURL().
The way I discovered this in my case was by using a good debugging tool (I used Chrome DevTools's own debugging features). I advise you do the same, adding breakpoints everywhere, and see where your code is going with each line.
Here is a short, rough example to show what I mean:
// This code increments a number from 1 to 2 to 3 and returns it each time after a delay of 1 second.
async function loop() {
for (i = 1; i <= 3; i++) {
console.log('Input start');
/* The following waits for result of aSync before continuing.
Without 'await', it would execute the last line
of this function whilst aSync's own 'await'
waited for its result.
--- This is where I think your code goes wrong. --- */
await aSync(i);
console.log('Input end');
}
}
async function aSync(num) {
console.log('Return start');
/* The following waits for the 1-second delay before continuing.
Without 'await', it would return a pending promise immediately
each time. */
let result = await new Promise(
// I'm not using arrow functions to show what it's doing more clearly.
function(rs, rj) {
setTimeout(
function() {
/* For those who didn't know, the following passes the number
into the 'resolved' ('rs') parameter of the promise's executor
function. Without doing this, the promise would never be fulfilled. */
rs(num);
}, 1000
)
}
);
console.log(result);
console.log('Return end');
}
loop();

How to call API every 2 seconds until desired result or timeout in async function

I need to call an api to get a status every 2 seconds if the response is running and first return when response is either complete or failed, or until 30 seconds have passed and the function times out.
This is what I have now which works, but I am sure it can be done much more efficient, but I simply can't figure it out at this point:
const getStatus = async (processId) => {
try {
const response = await fetch(`example.com/api/getStatus/${processId}`);
const status = await response.json();
return await status;
} catch(err) {
// handle error
}
}
Inside another async function using getStatus():
randomFunction = async () => {
let status = null;
let tries = 0;
let stop = false;
while (tries <= 15 && !stop) {
try {
status = await getStatus('some-process-id');
if (status === 'complete') {
stop = true;
// do something outside of loop
}
if (status === 'failed') {
stop = true;
throw Error(status);
}
if (tries === 15) {
stop = true;
throw Error('Request timed out');
}
} catch (err) {
// handle error
}
const delay = time => new Promise(resolve => setTimeout(() => resolve(), time));
if (tries < 15) {
await delay(2000);
}
tries++;
}
}
I would prefer to handle the looping inside getStatus() and in a more readable format, but is it possible?
EDIT:
I tried a solution that looks better and seems to work as I expect, see it here:
https://gist.github.com/AntonBramsen/6cec0faade032dfa3c175b7d291e07bd
Let me know if parts of the solution contains any solutions that are bad practice.

Your question is for javascript. Unfortunately I don't drink coffee, I can only give you the code in C#. But I guess you get the gist and can figure out how to translate this into java
Let's do this as a generic function:
You have a function that is called every TimeSpan, and you want to stop calling this function whenever the function returns true, you want to cancel, whenever some maximum time has passed.
For this maximum time I use a CancellationToken, this allows you to cancel processing for more reasons than timeout. For instance, because the operator wants to close the program.
TapiResult CallApi<TapiResult> <Func<TapiResult> apiCall,
Func<TapiResult, bool> stopCriterion,
CancellationToken cancellationToken)
{
TapiResult apiResult = apiCall;
while (!stopCriterion(apiResult))
{
cancellationToken.ThrowIfCancellationRequested();
Task.Delay(delayTime, cancellationToken).Wait;
apiResult = apiCall;
}
return apiResult;
}
ApiCall is the Api function to call. The return value is a TApiResult. In your case the status is your TApiResult
StopCriterion is a function with input ApiResult and output a boolean that is true when the function must stop. In your case this is when status equals complete or failed
CancellationToken is the Token you can get from a CancellationTokenSource. Whenever you want the procedure to stop processing, just tell the CancellationTokenSource, and the function will stop with a CancellationException
Suppose this is your Api:
Status MyApiCall(int x, string y) {...}
Then the usage is:
Timespan maxProcessTime = TimeSpan.FromSeconds(45);
var cancellationTokenSource = new CancellationTokenSource();
// tell the cancellationTokenSource to stop processing afer maxProcessTime:
cancellationTokenSource.CancelAfter(maxProcessTime);
// Start processing
Status resultAfterProcessing = CallApi<Status>(
() => MyApiCall (3, "Hello World!"), // The Api function to call repeatedly
// it returns a Status
(status) => status == complete // stop criterion: becomes true
|| status == failed, // when status complete or failed
cancellationTokenSource.Token); // get a token from the token source
TODO: add try / catch for CancellationException, and process what should be done if the task cancels
The function will stop as soon as the stopCriterion becomes true, or when the CancellationTokenSource cancels. This will automatically be done after maxTimeOut. However, if you want to stop earlier, for instance because you want to stop the program:
cancellationTokenSource.Cancel();