Background
As to my knowledge, Firebase-triggered functions can run more than once from just one triggering event (I think I can see that from the logs in Firebase).
Since this is potentially a data-corrupting behavior, I would like to implement some flag, which allows the function to stop if it finds that the flag is true.
Upon creation of that object in the database I would like this function (Typescript) to run only once.
For example:
adm.database().ref('/Users/{UserID}')
I plan to store a key value pair for each user node in the database:
{"alreadyTriggered":true}
Or store the 'trigger' in a separate node, it does not matter.
And then when Firebase decides to trigger the respective function, I will do a check and stop the execution of the function if it is not the first time it is being run:
data = snapshot.val();
if (data.alreadyTriggered) {
return;
}
// The function continues here
Implementation
(some parentheses and brackets can be missing, it's a psecudocode-Typescript written in the browser window)
index.ts:
import * as u from './users';
import * as ff from 'firebase-functions';
import * as adm from 'firebase-admin';
import * as types from './types';
// Triggered function
export const userCreated = ff.database.ref('/Users/{userId}').onCreate((sn, ctx) => {
let user: types.User = sn.val();
console.log(`New user created, User Id: ${ctx.params.userId}`);
return u.onCreated(user);
});
users.ts:
// Function implementation
export function onCreated(user: types.User): Promise<void> {
return adm.database().ref("/Triggers").once('value', snapshot => {
let data: any = snapshot.val();
const alreadyTriggered: boolean = data.alreadyTriggered;
}).then(alreadyTriggered => {
if (alreadyTriggered) {
// We would like to stop any execution here
return
}
else {
console.log(`Continuing processing`)
}
}).then(() => {
// This code should be reached only if alreadyTriggered is set to false
// do actual work with the user object
//...someNewData
//then save it
adm.database().ref('/Users').child(user.userId).update(someNewData)
})
}
Problem
The problem is that is that I don't know how reliably to stop executing the function. I have tried to use return (doesn't work), break (only for loops) and I came to using throw new Error('Stopping function execution'), but it does not seem to stop reliably - sometimes I see in the logs that the function does continue the execution although alreadyTriggered is definitely true.
Question
How can I stop execution of a Firebase Typescript function from within its code?
Your return statement isn't returning from the top-level function. It's just returning from the lambda function you passed to then(). You will have to propagate information down the chain of promises to tell the next callback whether or not to do its work.
return adm.database().ref("/Triggers").once('value', snapshot => {
let data: any = snapshot.val();
const alreadyTriggered: boolean = data.alreadyTriggered;
}).then(alreadyTriggered => {
if (alreadyTriggered) {
// We would like to stop any execution here
return false
}
else {
console.log(`Continuing processing`)
return true
}
}).then((continue) => {
// This code should be reached only if alreadyTriggered is set to false
// do actual work with the user object
//...someNewData
//then save it
if (continue) {
return adm.database().ref('/Users').child(user.userId).update(someNewData)
}
})
You also missed a return of the promise returned by update above, which would cause your function to time out.
Note that this is all lot easier with async/await syntax.
Related
I have a NextJS application that runs a recursive setTimeout when the server is started. I need to create an API endpoint that can start and stop this loop (to have more control over it in production). This loop is used to process items in a database that are added from another API endpoint.
import { clearTimeout } from "timers";
var loopFlag = true;
export function loopFlagSwitch(flag: boolean) {
loopFlag = flag;
}
export async function loop() {
try {
// Retrieve all unprocessed transactions
const unprocessedTransactions = await prisma.transaction.findMany({
take: 100,
where: { status: "UNPROCESSED" },
});
// Loop through transactions and do stuff
for (const transaction of unprocessedTransactions) {
//stuff
}
} catch (e) {
// handle error
}
if (loopFlag === true) {
setTimeout(loop, 1000); //if flag changes, this will stop running
}
}
if (require.main === module) {
loop(); // This is called when server starts, but not when file is imported
}
The reason I use setTimeout and not setInterval is because many errors can occur when processing items retrieved from DB. These errors, however, are solved by waiting a few milliseconds. So, the benefit of the pattern below is that if an error happens, the loop immediately restarts and the error will not appear because a ms has passed (it's due to concurrency problems -- let's ignore this for now).
To attempt to start and stop this loop, I have an endpoint that simply calls the loopFlagSwitch function.
import { NextApiRequest, NextApiResponse } from "next";
import { loopFlagSwitch } from "services/loop";
async function handler(req: NextApiRequest, res: NextApiResponse) {
try {
loopFlagSwitch(req.body.flag);
} catch (error) {
logger.info({ error: error });
}
}
export default handler;
Problem is, even when this endpoint is called, the setTimeout loop keeps going. Why isn't it picking the change in flag?
clearTimeout()
The global clearTimeout() method cancels a timeout previously established by calling setTimeout().
To clear a timeout, use the id returned from setTimeout():
Usage
const myTimeout = setTimeout(function, milliseconds);
//Then you can to stop the execution by calling clearTimeout():
clearTimeout(myTimeout);
loopFlag as a condition
...
if (loopFlag === true) {
myTimeout();
} else {
clearTimeout(myTimeout)
}
...
Add abortTimer function
Full code
export function loopFlagSwitch(flag) {
flag === true ? loop : abortTimer()
}
// set timeout
var myTimeout = setTimeout(loop, 1000);
function abortTimer() { // to be called when you want to stop the timer
clearTimeout(myTimeout);
}
export async function loop() {
try {
// Retrieve all unprocessed transactions
let d = "Retrieve all unprocessed transactions"
process.stdout.write(d + '\n');
// Loop through transactions and do stuff
for (let i = 0; i<10; i++) {
//stuff
let c = "second loop"
process.stdout.write(c + '\n');
}
} catch (e) {
// handle error
console.log("error ", e)
} finally {
myTimeout = setTimeout(loop, 1000); // repeat myself
}
}
if (require.main === module) {
loop(); // This is called when server starts, but not when file is imported
}
The flag will not work because node doesn't maintain the state of a file, the import only cares about the things it obtains from a file, it doesn't mind about the state of the variables declared in it.
Even though the clearTimeout() function may be sufficient, i think there is an even better option you can use to stop this loop.
Use a JS Class!
Instead of using just a function without state. You could instantiate a class that runs on the server with an internal boolean that can be called "shouldKeepLooping" for example:
class NextJsLooper { // or whatever better name you can use
private shouldKeepLooping: boolean
constructor () {
this.shouldKeepLooping = true
}
public shouldKeepLooping(value) { this.shouldKeepLooping = value }
public async loop () {
if (shouldKeepLooping) {
//... rest of the loop code
setTimeout(this.loop, 1000);
}
}
}
This way if you set the value to false it will automatically stop. Since this is a reference to the object.
Keep in mind that you would need to keep this instance alive as probably something global, and would need it to be accesible by nextJS.
You can use Symbol.for and the Node global to mantain the instance saved somewhere in the server!
My suggestion is to introduce the use of signals.
Service like Pusher will trigger the event that will be listened by the transaction processor.
your transaction processing api / code above or any other
Signal actions like "start" and "stop" will be triggered anytime even in production by either frontend or through the pusher portal that will be used to change the loop flag to either true or false.
you can retrieve the thread in charge of the operation and interrupt after a given expected time ,and log something to inform you about the time out in case that your operation took more than the necessary time .
Ok the goal is to be equal a global scope variable to an inner information I need and then export it. But I am not able to do it (I keep getting undefined even though when I console out data, I get the info I want). I have looked at several documentations and other peoples questions NodeJS Async/Await Exporting a Variable , but it still doesn't answer my question. I know I have to use asynchronous JavaScript (async, await) but because I fairly new to JS. NOTE: GETAPRODUCTAPI is a SpringBoot API and updateClick() is called inside another method.
The global variable called dataToExport I want to export and it equals data (then(data)).
export var dataToExport;
const updateClick = () => {
const editBtns = getQSelectorAll(".edit");
editBtns.forEach((btn) => {
btn.addEventListener("click", (e) => {
const currentClicked = e.currentTarget.dataset.editid;
const api = GETAPRODUCTAPI + currentClicked;
fetch(api).then((response) => {
return response.json();
}).then((data) => {
console.log(data); // this works fine
// TODO
dataToExport = data
});
});
});
};
console.log(dataToExport); // undefined output
This works but you have to ensure that reading of dataToExport happens after the asynchronous code that actually sets the variable.
That is, through whatever means necessary, you would want to wait to console.log(dataToExport) until after all of the following have occurred:
updateClick is called and the event listener has been bound to the buttons
The button(s) are actually clicked (programmatically or by user interaction)
The fetch(es) are sent out
The response to the fetch(es) are received
The response json is parsed and assigned to dataToExport.
As is you're trying to read dataToExport before any of those steps have occurred and therefore the variable has not been set yet. You can periodically poll to see if the variable has been assigned (e.g. setTimeout or setInterval) or otherwise have an event that triggers a check of this variable.
Based on #arcyqwerty explanation, I was able to figure out a logic. I created two global scopes a variable status and a function changedStatus() . Equalled status to null and then changedStatus() whose job is to check when status changes from null to something else (event that triggers a check of this variable). NOTE: I renamed dataToExport to status
let status = null;
let changedStatus = (value) => {
status = value;
if (status != null) {
console.log(status);
}
}
const updateClick = () => {
const editBtns = getQSelectorAll(".edit");
editBtns.forEach((btn) => {
btn.addEventListener("click", (e) => {
const currentClicked = e.currentTarget.dataset.editid;
const api = GETAPRODUCTAPI + currentClicked;
fetch(api).then((response) => {
return response.json();
}).then((data) => {
// TODO
changedStatus(data);
});
});
});
};
And as far as exporting dataToExport just like #Bergi said it doesn't make sense to export data before user clicks so instead I used localStorage. This example helped for better understanding.
i don't really understand async. i have a function like this:
function getTeam() {
let sir = setInterval(() => {
const teamsGrid = $('[class*="teamsgrid"]').find("p");
const firstTeam = $(teamsGrid[0]).text();
if (firstTeam != '') {
clearInterval(sir)
return firstTeam.trim()
}
}, 100)
}
im not js master. i just want to get that element when it loads in, this code is running in a userscript and // #run-at document-idle doesnt help either. i knew i would have to get into async js promises callbacks and whatever someday but i really dont understand how it works after pages of docs and other stackoverflow.
when i console.log this function it will print undefined once then if i have a console.log inside the if it will print the actual team name.
how do i wait for that result
Answer regarding the javascript language part if the question
You could modify your code to the following (but don't - see further below - I'm just providing this as your StackOverflow tags included async/await):
async function getTeam() {
return new Promise(resolve => {
const sir = setInterval(() => {
const teamsGrid = $('[class*="teamsgrid"]').find("p");
const firstTeam = $(teamsGrid[0]).text();
if (firstTeam != '') {
clearInterval(sir);
resolve(firstTeam.trim());
}
}, 100);
});
}
// ... and then anywhere else in your code:
doSomethingSynchronous();
const team = await getTeam();
soSomethingSynchronousWithTeam(team);
Note that this will only work with modern browsers supporting >= ECMAScript 2017:
https://caniuse.com/async-functions (but luckily that's most by now!)
Answer regarding the implicit "howto wait for an element part"
... you really shouldn't actively wait for an element because this is unnecessarily heavy on the CPU. Usually you'll have some kind of event that informs you as soon as the element you're waiting for has been created. Just listen for that and then run your code.
What to do, if there's currently no such event:
If you're in control of the code creating the element, then trigger one yourself (see https://api.jquery.com/trigger/ for example).
If the element is created by a third party lib or by something else you cannot easily modify, you could use a MutationObserver (see this StackBlitz answer to a related question) and run your getTeam code only whenever something has changed instead of every 100ms (smaller impact on performance!)
You can make it async if you want, but the main part us going to be using events instead. There is a special object called mutation observer. It will call a function you give it any time there's a change in the element you're observing.
Check the mutation observer docs to understand the code below: https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver
Without knowing much about your HTML, I can say as much as this should work:
function getTeam() {
const teamsGrid = $('[class*="teamsgrid"]').find("p");
const firstTeam = $(teamsGrid[0]).text();
if (firstTeam != '') {
return firstTeam.trim()
}
}
function getTeamWhenAvailable() {
// returning a promise allows you to do "await" and get result when it is available
return new Promise((resolve, reject) => {
// furst try if element is available right now
const teamText = getTeam();
if(teamText) {
// resolve() "returns" the value to whoever is doing "await"
resolve(teamText);
// resolve does not terminate this function, we need to do that using return
return;
}
// Mutation observer gives you list of stuff that changed, but we don't care, we just do our own thing
const observer = new MutationObserver(()=>{
const teamText = getTeam();
if(teamText) {
// stop observing
observer.disconnect();
// resolve the value
resolve(teamText);
}
});
observer.observe(document.body, { childList: true, subtree: true };
})
}
// usage, the extra brackets around the lambda cause it to invoke immediatelly
(async () => {
console.log("Waitinf for team...");
const teamName = await getTeamWhenAvailable();
console.log("Result team name: ", teamName)
})();
Now you might wanna narrow the scope of the mutation observer, in the example above it watches the entire document. Try to instead observe the deepest element that you can rely on not being removed.
If you need to receive team name multiple times, I think you should just go with the obsever alone without the async stuff.
function getTeam() {
let sir = new Promise((res, rej) => {
const teamsGrid = $('[class*="teamsgrid"]').find("p");
const firstTeam = $(teamsGrid[0]).text();
if (firstTeam != '') {
clearInterval(sir);
res(firstTeam.trim());
}
});
return sir();
}
From what I understood, you are looking for firstTeam. Also, we assume that there is always firstTeam, so there isnt a case when there would be no team name.
I am not sure where you are making a req that will take time to process honestly from this code. So far it looks that sync function should do just fine. Are you reaching out to any API?
I'm fetching some data from firebase and would like to run async/await function (to fetch data) only once upon the first page load. I'm used to React and lifecycle methods / hooks doing it but this little project is just too small to use React. I just need to run this function once, fetch the data, save it to a variable and do not make any further calls to firebase api in the same session.
async function getEntries() {
const snapshot = await firebase.firestore().collection('riders').get()
// Do my thing with the data, etc.
// console.log(snapshot.docs.map(doc => doc.data()));
}
Is there any js-only way of running this function only once when the page loads?
If you call a function just once, why do you need the function at all?
const snapshot = await firebase.firestore().collection('riders').get()
// Do my thing with the data, etc.
// console.log(snapshot.docs.map(doc => doc.data()));
This top level await only works in modules, and it blocks all depending modules to load. If that is not necessary (they don't depend on the data), or if you don't want write a module, you can wrap the code in an async IIFE, and store the returned promise in a variable:
const dataPromise = (async function() {
//...
return data;
})();
While the data is loading, you might want to show some loading icon or so. That can easily be done with the following hook:
function usePromise(p) {
const [state, setState] = useState(null);
useEffect(() => { p.then(setState); }, []);
return state;
}
// Inside a component:
const data = usePromise(dataPromise);
if(data === null)
return <Loading />;
// show data
Yes. You can use Self Invoking (self executing) Functions. Syntax is like:
(function(){})();
The last parentheses are for running function. the function is anonymous.
You can Implement it this way:
(async function () {
const snapshot = await firebase.firestore().collection('riders').get()
})();
in this way you can never call this function again and it will run only once.
Tutorial: https://blog.mgechev.com/2012/08/29/self-invoking-functions-in-javascript-or-immediately-invoked-function-expression/
And The question you asked is somehow duplicate and answered here: Function in JavaScript that can be called only once
What you are looking for is memoization of the function result. There are several libraries to supporting including react.
Theres also a handmade pattern you can use by changing the function implementation after it's called once, accoring to JavaScript: The Good Parts
async function getEntries() {
const snapshot = await firebase.firestore().collection('riders').get()
// Do my thing with the data, etc.
// console.log(snapshot.docs.map(doc => doc.data()));
getEntries = async function(){
return snapshot
}
return snapshot
}
I think you can load it with the load method when the page is first loaded and then set it to cookie or local stroge. You can check this value on next page loads. You can do this quickly using jQuery.
$(window).load(function() {
var item = localStorage.getItem('test');
if(item != null){
// your code
}
else {
localStorage.setItem('test', 1);
}
});
The simplest way is to make a global variable like:
let isCalled = false;
and in the function body do:
if(isCalled) return;
//the stuff the function would do
isCalled = true;
//Assign isCalled to true before using a return statement as it will make the program discard the lines below it.
Currently, I am trying to get the md5 of every value in array. Essentially, I loop over every value and then hash it, as such.
var crypto = require('crypto');
function userHash(userIDstring) {
return crypto.createHash('md5').update(userIDstring).digest('hex');
}
for (var userID in watching) {
refPromises.push(admin.database().ref('notifications/'+ userID).once('value', (snapshot) => {
if (snapshot.exists()) {
const userHashString = userHash(userID)
console.log(userHashString.toUpperCase() + "this is the hashed string")
if (userHashString.toUpperCase() === poster){
return console.log("this is the poster")
}
else {
..
}
}
else {
return null
}
})
)}
However, this leads to two problems. The first is that I am receiving the error warning "Don't make functions within a loop". The second problem is that the hashes are all returning the same. Even though every userID is unique, the userHashString is printing out the same value for every user in the console log, as if it is just using the first userID, getting the hash for it, and then printing it out every time.
Update LATEST :
exports.sendNotificationForPost = functions.firestore
.document('posts/{posts}').onCreate((snap, context) => {
const value = snap.data()
const watching = value.watchedBy
const poster = value.poster
const postContentNotification = value.post
const refPromises = []
var crypto = require('crypto');
function userHash(userIDstring) {
return crypto.createHash('md5').update(userIDstring).digest('hex');
}
for (let userID in watching) {
refPromises.push(admin.database().ref('notifications/'+ userID).once('value', (snapshot) => {
if (snapshot.exists()) {
const userHashString = userHash(userID)
if (userHashString.toUpperCase() === poster){
return null
}
else {
const payload = {
notification: {
title: "Someone posted something!",
body: postContentNotification,
sound: 'default'
}
};
return admin.messaging().sendToDevice(snapshot.val(), payload)
}
}
else {
return null
}
})
)}
return Promise.all(refPromises);
});
You have a couple issues going on here. First, you have a non-blocking asynchronous operation inside a loop. You need to fully understand what that means. Your loop runs to completion starting a bunch of non-blocking, asynchronous operations. Then, when the loop finished, one by one your asynchronous operations finish. That is why your loop variable userID is sitting on the wrong value. It's on the terminal value when all your async callbacks get called.
You can see a discussion of the loop variable issue here with several options for addressing that:
Asynchronous Process inside a javascript for loop
Second, you also need a way to know when all your asynchronous operations are done. It's kind of like you sent off 20 carrier pigeons with no idea when they will all bring you back some message (in any random order), so you need a way to know when all of them have come back.
To know when all your async operations are done, there are a bunch of different approaches. The "modern design" and the future of the Javascript language would be to use promises to represent your asynchronous operations and to use Promise.all() to track them, keep the results in order, notify you when they are all done and propagate any error that might occur.
Here's a cleaned-up version of your code:
const crypto = require('crypto');
exports.sendNotificationForPost = functions.firestore.document('posts/{posts}').onCreate((snap, context) => {
const value = snap.data();
const watching = value.watchedBy;
const poster = value.poster;
const postContentNotification = value.post;
function userHash(userIDstring) {
return crypto.createHash('md5').update(userIDstring).digest('hex');
}
return Promise.all(Object.keys(watching).map(userID => {
return admin.database().ref('notifications/' + userID).once('value').then(snapshot => {
if (snapshot.exists()) {
const userHashString = userHash(userID);
if (userHashString.toUpperCase() === poster) {
// user is same as poster, don't send to them
return {response: null, user: userID, poster: true};
} else {
const payload = {
notification: {
title: "Someone posted something!",
body: postContentNotification,
sound: 'default'
}
};
return admin.messaging().sendToDevice(snapshot.val(), payload).then(response => {
return {response, user: userID};
}).catch(err => {
console.log("err in sendToDevice", err);
// if you want further processing to stop if there's a sendToDevice error, then
// uncomment the throw err line and remove the lines after it.
// Otherwise, the error is logged and returned, but then ignored
// so other processing continues
// throw err
// when return value is an object with err property, caller can see
// that that particular sendToDevice failed, can see the userID and the error
return {err, user: userID};
});
}
} else {
return {response: null, user: userID};
}
});
}));
});
Changes:
Move require() out of the loop. No reason to call it multiple times.
Use .map() to collect the array of promises for Promise.all().
Use Object.keys() to get an array of userIDs from the object keys so we can then use .map() on it.
Use .then() with .once().
Log sendToDevice() error.
Use Promise.all() to track when all the promises are done
Make sure all promise return paths return an object with some common properties so the caller can get a full look at what happened for each user
These are not two problems: the warning you get is trying to help you solve the second problem you noticed.
And the problem is: in Javascript, only functions create separate scopes - every function you define inside a loop - uses the same scope. And that means they don't get their own copies of the relevant loop variables, they share a single reference (which, by the time the first promise is resolved, will be equal to the last element of the array).
Just replace for with .forEach.