We Keep Coding

Encountering a problem when trying to remove some code from a route to put it into a service - Node.js / Express.js

I'm having a problem right now when i want to remove some code out of my route to put it into a service. I'm just trying to follow the best practices of developing an application.
This is my route right now:
const express = require('express');
const cityRouter = express.Router();
const axios = require('axios');
const NodeCache = require('node-cache');
const myCache = new NodeCache();
cityRouter.get('/:cep', async (request, response) => {
try {
const { cep } = request.params;
const value = myCache.get(cep);
if (value) {
response.status(200).send({
city: value,
message: 'Data from the cache',
});
} else {
const resp = await axios.get(`https://viacep.com.br/ws/${cep}/json/`);
myCache.set(cep, resp.data, 600);
response.status(200).send({
city: resp.data,
message: 'Data not from the cache',
});
}
} catch (error) {
return response.status(400);
}
});
module.exports = cityRouter;
I'm using axios to retrieve data from an API, where i have a variable called "cep" as a parameter and then using node-cache to cache it.
And it works with out problems:
enter image description here
But, when i try to put the same code into a service, and then call it into my route:
My service:
const axios = require('axios');
const NodeCache = require('node-cache');
const myCache = new NodeCache();
function verificaCache(cep) {
return async function (request, response, next) {
const value = myCache.get(cep);
console.log(cep);
if (value) {
response.status(200).send({
city: value,
message: 'Data from the cache',
});
} else {
const resp = await axios.get(`https://viacep.com.br/ws/${cep}/json/`);
myCache.set(cep, resp.data, 600);
response.status(200).send({
city: resp.data,
message: 'Data not from the cache',
});
}
next();
};
}
module.exports = verificaCache;
My route using the service:
const express = require('express');
const cityRouter = express.Router();
const verificaCache = require('../services/VerificaCacheService');
cityRouter.get('/:cep', async (request, response) => {
const { cep } = request.params;
verificaCache(cep);
response.status(200);
});
module.exports = cityRouter;
By some reason, it doesn't work:
enter image description here
What is the problem that i can't see? I'm a beginner so i'm kinda lost right now.

You have created a high-order function by returning a function in verificaCache(), so to properly call it you need to do it like that await verificaCache(cep)(req, res), remember, the first time you call it, you have a function being returned, since you want the tasks inside of that function to be executed, you need to call it as well.
Take a reading about high-order functions here: https://blog.alexdevero.com/higher-order-functions-javascript/
My recommendation, you could just get rid of the other function you are returning to simplify your code, and let the service only handle business logic, all the http actions should be handled on the controller level:
// Service
function verificaCache(cep) {
const value = myCache.get(cep);
if (value) {
return { city: value, message: 'Data from the cache'})
}
// No need of an else statement because the
// execution will stop at the first return if the condition passes
const resp = await axios.get(`https://viacep.com.br/ws/${cep}/json/`);
myCache.set(cep, resp.data, 600);
return { city: resp.data, message: 'Data not from the cache'};
}
// Controller
cityRouter.get('/:cep', async (request, response) => {
const { cep } = request.params;
try {
const data = verificaCache(cep);
// Use json() instead of send()
response.status(200).json(data);
} catch(error) {
// Handle errors here
console.log(error);
}
});
Estamos juntos!

What does `suppressed_by_user` mean when using Google One Tap?

I am trying to get the one tap to appear when the user clocks sign in, The below code works just fine when It runs as soon as the page loads but if I try to get it run on click then I get the error suppressed_by_user error I don't have an advert blocking plugin running and I don't know what suppressed_by_user could mean?
docs here detail the error but don't explain what caused it or how to fix it.
<script src="https://accounts.google.com/gsi/client"></script>
...
const signin = () => {
const handleCredentialResponse = (response: any) => {
const credential = response.credential;
const getDetails = async () => {
const params = new window.URLSearchParams({ credential });
const url = `${process.env.REACT_APP_API_BASE_URL}/google?${params}`;
const response = await fetch(url, { method: "GET" });
const data = await response.json();
setLoggedIn(true);
setState({ ...state, participant: data.participant });
};
getDetails();
};
if (state && state.participant && state.participant.id) {
setLoggedIn(true);
} else {
const client_id = process.env.REACT_APP_GOOGLE_CLIENT_ID;
const callback = handleCredentialResponse;
const auto_select = false;
const cancel_on_tap_outside = false;
google.accounts.id.initialize({ client_id, callback, auto_select, cancel_on_tap_outside });
google.accounts.id.prompt((notification: any) => {
console.log(notification); // rl {g: "display", h: false, j: "suppressed_by_user"}
console.log(notification.getNotDisplayedReason()); // suppressed_by_user
});
}
};
...
<div className="center--main-join" onClick={signin}>Sign in or Join</div>

It means the user has manually closed the One Tap prompt before, which triggered the Cool Down feature (as documented at: https://developers.google.com/identity/one-tap/web/guides/features#exponential_cooldown).
During the cool-down period, One Tap prompt is suppressed.

FetchEvent for "[url]" resulted in a network error response: the promise was rejected

I am getting those errors and warning in my console after trying to create a PWA - Progressive Web App out of my website using this tutorial.
The FetchEvent for
"https://www.googletagmanager.com/gtag/js?id=UA-4562443-3" resulted in
a network error response: the promise was rejected. Promise.then
(async) (anonymous) # service-worker.js:228 service-worker.js:1
Uncaught (in promise) fetch failed 1:21 GET
https://www.googletagmanager.com/gtag/js?id=UA-4562443-3
net::ERR_FAILED The FetchEvent for
"https://fonts.googleapis.com/css?family=Open+Sans:300,400&display=swap&subset=cyrillic"
resulted in a network error response: the promise was rejected.
Promise.then (async) (anonymous) # service-worker.js:228
service-worker.js:1 Uncaught (in promise) fetch failed 1:28 GET
https://fonts.googleapis.com/css?family=Open+Sans:300,400&display=swap&subset=cyrillic
net::ERR_FAILED The FetchEvent for
"https://widget.uservoice.com/VuHfPZ0etI2eQ4REt1tiUg.js" resulted in a
network error response: the promise was rejected. Promise.then (async)
(anonymous) # service-worker.js:228 service-worker.js:1 Uncaught (in
promise) fetch failed 1:894 GET
https://widget.uservoice.com/VuHfPZ0etI2eQ4REt1tiUg.js net::ERR_FAILED
It actually works pretty well. I am able to get a fully working PWA icon in Audits in Chrome Dev Tools. Which is great, but after a refresh I am getting all those errors. My service-worker.js which is located at root of my website looks like this
"use strict";
const SERVICE_WORKER_VERSION = "REPLACED_WITH_SERVICE_WORKER_VERSION"; // updated with tools/service_worker_version.js (String)
const CACHE_VERSION = SERVICE_WORKER_VERSION;
//const fileNamesToSaveInCache = ["/"];
const HOME = "/";
const OFFLINE_ALTERNATIVE = "/offline";
const fileNamesToSaveInCache = [];
const fileNamesToSaveInCacheProd = [
OFFLINE_ALTERNATIVE,
"/",
"/publics/img/favicon/fav.gif",
"/publics/css/style.css",
"/publics/css/searchhelp.css",
"/publics/css/Helpa.css",
];
const rtcLength = 4; // "rtc/".length;
const rtcFetchDelay = 10000;//ms
const origin = location.origin;
const answerFromfileName = {};
const resolveFromfileName = {};
const rejectFromfileName = {};
const timeOutIdFromfileName = {};
let logLater = [];
// todo put all into single container
const resolveFetchFromPeerToPeer = function (fileName) {
clearTimeout(timeOutIdFromfileName[fileName]);
resolveFromfileName[fileName](answerFromfileName[fileName]);
delete answerFromfileName[fileName];//stop listening
delete resolveFromfileName[fileName];
delete rejectFromfileName[fileName];
};
const rejectFetchFromPeerToPeer = function (fileName, reason) {
if (rejectFromfileName[fileName]) {
rejectFromfileName[fileName](reason);
delete resolveFromfileName[fileName];
delete rejectFromfileName[fileName];
}
};
const fetchFromPeerToPeer = function (customRequestObject) {
/*asks all page for a fileName*/
const fileName = customRequestObject.header.fileName;
const promise = new Promise(function (resolve, reject) {
resolveFromfileName[fileName] = resolve;
rejectFromfileName[fileName] = reject;
if (answerFromfileName.hasOwnProperty(fileName)) {
resolveFetchFromPeerToPeer(fileName);
}
timeOutIdFromfileName[fileName] = setTimeout(function() {
rejectFetchFromPeerToPeer(fileName, "No answer after 10 seconds");
}, rtcFetchDelay);
});
self.clients.matchAll().then(function(clientList) {
clientList.forEach(function(client) {
client.postMessage(customRequestObject);
});
});
return promise;
};
const logInTheUI = (function () {
//console.log("logInTheUI function exists");
return function (what) {
console.log(what);
self.clients.matchAll().then(function(clientList) {
clientList.forEach(function(client) {
client.postMessage({LOG: JSON.parse(JSON.stringify(what))});
});
});
};
}());
const logInTheUIWhenActivated = function (what) {
logLater.push(what);
};
const fetchFromMainServer = function (request, options = {}) {
/*wrap over fetch. The problem with fetch here, it doesn't reject properly sometimes
see if statement below*/
return fetch(request, options).then(function (fetchResponse) {
// console.log("fetchFromMainServer:", fetchResponse.ok, fetchResponse);
// logInTheUI([request, options]);
if ((!fetchResponse) || (!fetchResponse.ok)) {
return Promise.reject("fetch failed");
}
return fetchResponse;
});
};
const fetchFromCache = function (request) {
return caches.open(CACHE_VERSION).then(function (cache) {
return cache.match(request).then(function (CacheResponse) {
//console.log("fetchFromCache:", CacheResponse.ok, CacheResponse);
if ((!CacheResponse) || (!CacheResponse.ok)) {
return Promise.reject("Not in Cache");
}
return CacheResponse;
});
});
};
const isLocalURL = function (url) {
return !(String(url).match("rtc"));
};
const fillServiceWorkerCache2 = function () {
/*It will not cache and also not reject for individual resources that failed to be added in the cache. unlike fillServiceWorkerCache which stops caching as soon as one problem occurs. see http://stackoverflow.com/questions/41388616/what-can-cause-a-promise-rejected-with-invalidstateerror-here*/
return caches.open(CACHE_VERSION).then(function (cache) {
return Promise.all(
fileNamesToSaveInCache.map(function (url) {
return cache.add(url).catch(function (reason) {
return logInTheUIWhenActivated([url + "failed: " + String(reason)]);
});
})
);
});
};
const latePutToCache = function (request, response) {
return caches.open(CACHE_VERSION).then(function(cache) {
cache.put(request, response.clone());
return response;
});
};
const deleteServiceWorkerOldCache = function () {
return caches.keys().then(function (cacheVersions) {
return Promise.all(
cacheVersions.map(function (cacheVersion) {
if (CACHE_VERSION === cacheVersion) {
//console.log("No change in cache");
} else {
//console.log("New SERVICE_WORKER_VERSION of cache, delete old");
return caches.delete(cacheVersion);
}
})
);
});
};
const useOfflineAlternative = function () {
return fetchFromCache(new Request(OFFLINE_ALTERNATIVE));
};
const isAppPage = function (url) {
/*appPage does not work offline, and we don't serve it if offline
returns Boolean*/
return (origin + HOME) === url;
};
self.addEventListener("install", function (event) {
/*the install event can occur while another service worker is still active
waitUntil blocks the state (here installing) of the service worker until the
promise is fulfilled (resolved or rejected). It is useful to make the service worker more readable and more deterministic
save in cache some static fileNames
this happens before activation */
event.waitUntil(
fillServiceWorkerCache2()
.then(skipWaiting)
);
});
self.addEventListener("activate", function (event) {
/* about to take over, other service worker are killed after activate, syncronous
a good moment to clear old cache*/
event.waitUntil(deleteServiceWorkerOldCache().then(function() {
//console.log("[ServiceWorker] Skip waiting on install caches:", caches);
return self.clients.claim();
}));
});
self.addEventListener("message", function (event) {
const message = event.data;
/*
if (message.hasOwnProperty("FUTURE")) {
console.log(message.FUTURE);
return;
}
*/
const fileName = message.fileName;
const answer = message.answer;
answerFromfileName[fileName] = answer;
//console.log(fileName, answer, resolveFromfileName);
if (resolveFromfileName.hasOwnProperty(fileName)) {//
resolveFetchFromPeerToPeer(fileName);
}
});
self.addEventListener("fetch", function (fetchEvent) {
/* fetchEvent interface FetchEvent
see https://www.w3.org/TR/service-workers/#fetch-event-interface
IMPORTANT: fetchEvent.respondWith must be called inside this handler immediately
synchronously fetchEvent.respondWith must be called with a response object or a
promise that resolves with a response object. if fetchEvent.respondWith is called
later in a callback the browser will take over and asks the remote server directly, do not do that
why have fetchEvent.respondWith( and not respond with the return value of the callback function ?
-->
It allows to do other thing before killing the service worker, like saving stuff in cache
*/
const request = fetchEvent.request;//Request implements Body;
//const requestClone = request.clone(); //no need to clone ?
const url = request.url;
if (logLater) {
logLater.forEach(logInTheUI);
logLater = undefined;
}
// logInTheUI(["fetch service worker " + SERVICE_WORKER_VERSION, fetchEvent]);
// Needs to activate to handle fetch
if (isLocalURL(url)) {
//Normal Fetch
if (request.method === "POST") {
// logInTheUI(["POST ignored", request]);
return;
}
// logInTheUI(["Normal Fetch"]);
fetchEvent.respondWith(
fetchFromCache(request.clone()).then(function (cacheResponse) {
/* cannot use request again from here, use requestClone */
//console.log(request, url);
return cacheResponse;
}).catch(function (reason) {
// We don't have it in the cache, fetch it
// logInTheUI(fetchEvent);
return fetchFromMainServer(request);
}).then(function (mainServerResponse) {
if (isAppPage(url)) {
return mainServerResponse;
}
return latePutToCache(request, mainServerResponse).catch(
function (reason) {
/*failed to put in cache do not propagate catch, not important enough*/
return mainServerResponse;
}
);
}).catch(function (reason) {
if (isAppPage(url)) {
//if it is the landing page that is asked
return useOfflineAlternative();
//todo if we are offline , display /offline directly
}
return Promise.reject(reason);
})
);
} else {
// Peer to peer Fetch
//console.log(SERVICE_WORKER_VERSION, "rtc fetch" url:", fetchEvent.request.url);
// request, url are defined
const method = request.method;
const requestHeaders = request.headers;
//logInTheUI(["Special Fetch"]);
const customRequestObject = {
header: {
fileName: url.substring(url.indexOf("rtc/") + rtcLength),
method
},
body: ""
};
requestHeaders.forEach(function (value, key) {
//value, key correct order
//is there a standard way to use Object.assign with Map like iterables ?
//todo handle duplicates
//https://fetch.spec.whatwg.org/#terminology-headers
customRequestObject.header[key] = value;
});
//console.log(request);
fetchEvent.respondWith(
/*should provide the peer the full request*/
request.arrayBuffer().then(function (bodyAsArrayBuffer) {
const bodyUsed = request.bodyUsed;
if (bodyUsed && bodyAsArrayBuffer) {
customRequestObject.body = bodyAsArrayBuffer;
}
}).catch(function (reason) {
/*console.log("no body sent, a normal GET or HEAD request has no body",
reason);*/
}).then(function (notUsed) {
return fetchFromPeerToPeer(customRequestObject);
}).then(function (response) {
const responseInstance = new Response(response.body, {
headers: response.header,
status: response.header.status || 200,
statusText : response.header.statusText || "OK"
});
return responseInstance;
}).catch(function (error) {
const responseInstance = new Response(`<html><p>${error}</p></html>`,
{
headers: {
"Content-type": "text/html"
},
status: 500,
statusText : "timedout"
});
return responseInstance;
})
);
}
/*here we could do more with event.waitUntil()*/
});
I am guessing the problem comes from loading those external libraries. So, this is my code loading those libraries.
// Include the UserVoice JavaScript SDK (only needed once on a page)
UserVoice = window.UserVoice || [];
(function() {
var uv = document.createElement('script');
uv.type = 'text/javascript';
uv.async = true;
uv.src = '//widget.uservoice.com/VuHfPZ0etI2eQ4REt1tiUg.js';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(uv, s)
})();
//
// UserVoice Javascript SDK developer documentation:
// https://www.uservoice.com/o/javascript-sdk
//
// Set colors
UserVoice.push(['set', {
accent_color: '#448dd6',
trigger_color: 'white',
trigger_background_color: 'rgba(46, 49, 51, 0.6)'
}]);
// Identify the user and pass traits
// To enable, replace sample data with actual user traits and uncomment the line
UserVoice.push(['identify', {
//email: 'john.doe#example.com', // User’s email address
//name: 'John Doe', // User’s real name
//created_at: 1364406966, // Unix timestamp for the date the user signed up
//id: 123, // Optional: Unique id of the user (if set, this should not change)
//type: 'Owner', // Optional: segment your users by type
//account: {
// id: 123, // Optional: associate multiple users with a single account
// name: 'Acme, Co.', // Account name
// created_at: 1364406966, // Unix timestamp for the date the account was created
// monthly_rate: 9.99, // Decimal; monthly rate of the account
// ltv: 1495.00, // Decimal; lifetime value of the account
// plan: 'Enhanced' // Plan name for the account
//}
}]);
// Add default trigger to the bottom-right corner of the window:
UserVoice.push(['addTrigger', {mode: 'contact', trigger_position: 'bottom-right'}]);
// Or, use your own custom trigger:
//UserVoice.push(['addTrigger', '#id', { mode: 'contact' }]);
// Autoprompt for Satisfaction and SmartVote (only displayed under certain conditions)
UserVoice.push(['autoprompt', {}]);
});//ready
#import url('https://fonts.googleapis.com/css?family=Open+Sans:300,400&display=swap&subset=cyrillic');
#font-face {
font-family: 'fa-solid-900';
font-display: swap;
src: url(https://use.fontawesome.com/releases/v5.8.2/webfonts/fa-solid-900.woff2) format('woff2');
}
#font-face {
font-family: 'fa-brands-400';
font-display: swap;
src: url(https://use.fontawesome.com/releases/v5.8.2/webfonts/fa-brands-400.woff2) format('woff2');
}
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-number-3"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-number-3');
</script>
What should i do in order to fix those errors. This is my first try in PWA so i am lost.

I end up using Workbox and everything is working great now.

Nested HTTP requests in Firebase cloud function

I'm using an HTTP-triggered Firebase cloud function to make an HTTP request. I get back an array of results (events from Meetup.com), and I push each result to the Firebase realtime database. But for each result, I also need to make another HTTP request for one additional piece of information (the category of the group hosting the event) to fold into the data I'm pushing to the database for that event. Those nested requests cause the cloud function to crash with an error that I can't make sense of.
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const request = require('request');
exports.foo = functions.https.onRequest(
(req, res) => {
var ref = admin.database().ref("/foo");
var options = {
url: "https://api.meetup.com/2/open_events?sign=true&photo-host=public&lat=39.747988&lon=-104.994945&page=20&key=****",
json: true
};
return request(
options,
(error, response, body) => {
if (error) {
console.log(JSON.stringify(error));
return res.status(500).end();
}
if ("results" in body) {
for (var i = 0; i < body.results.length; i++) {
var result = body.results[i];
if ("name" in result &&
"description" in result &&
"group" in result &&
"urlname" in result.group
) {
var groupOptions = {
url: "https://api.meetup.com/" + result.group.urlname + "?sign=true&photo-host=public&key=****",
json: true
};
var categoryResult = request(
groupOptions,
(groupError, groupResponse, groupBody) => {
if (groupError) {
console.log(JSON.stringify(error));
return null;
}
if ("category" in groupBody &&
"name" in groupBody.category
) {
return groupBody.category.name;
}
return null;
}
);
if (categoryResult) {
var event = {
name: result.name,
description: result.description,
category: categoryResult
};
ref.push(event);
}
}
}
return res.status(200).send("processed events");
} else {
return res.status(500).end();
}
}
);
}
);
The function crashes, log says:
Error: Reference.push failed: first argument contains a function in property 'foo.category.domain._events.error' with contents = function (err) {
if (functionExecutionFinished) {
logDebug('Ignoring exception from a finished function');
} else {
functionExecutionFinished = true;
logAndSendError(err, res);
}
}
at validateFirebaseData (/user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1436:15)
at /user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1479:13
at Object.forEach (/user_code/node_modules/firebase-admin/node_modules/#firebase/util/dist/index.node.cjs.js:837:13)
at validateFirebaseData (/user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1462:14)
at /user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1479:13
at Object.forEach (/user_code/node_modules/firebase-admin/node_modules/#firebase/util/dist/index.node.cjs.js:837:13)
at validateFirebaseData (/user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1462:14)
at /user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1479:13
at Object.forEach (/user_code/node_modules/firebase-admin/node_modules/#firebase/util/dist/index.node.cjs.js:837:13)
at validateFirebaseData (/user_code/node_modules/firebase-admin/node_modules/#firebase/database/dist/index.node.cjs.js:1462:14)
If I leave out the bit for getting the group category, the rest of the code works fine (just writing the name and description for each event to the database, no nested requests). So what's the right way to do this?

I suspect this issue is due to the callbacks. When you use firebase functions, the exported function should wait on everything to execute or return a promise that resolves once everything completes executing. In this case, the exported function will return before the rest of the execution completes.
Here's a start of something more promise based -
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const request = require("request-promise-native");
exports.foo = functions.https.onRequest(async (req, res) => {
const ref = admin.database().ref("/foo");
try {
const reqEventOptions = {
url:
"https://api.meetup.com/2/open_events?sign=true&photo-host=public&lat=39.747988&lon=-104.994945&page=20&key=xxxxxx",
json: true
};
const bodyEventRequest = await request(reqEventOptions);
if (!bodyEventRequest.results) {
return res.status(200).end();
}
await Promise.all(
bodyEventRequest.results.map(async result => {
if (
result.name &&
result.description &&
result.group &&
result.group.urlname
) {
const event = {
name: result.name,
description: result.description
};
// get group information
const groupOptions = {
url:
"https://api.meetup.com/" +
result.group.urlname +
"?sign=true&photo-host=public&key=xxxxxx",
json: true
};
const categoryResultResponse = await request(groupOptions);
if (
categoryResultResponse.category &&
categoryResultResponse.category.name
) {
event.category = categoryResultResponse.category.name;
}
// save to the databse
return ref.push(event);
}
})
);
return res.status(200).send("processed events");
} catch (error) {
console.error(error.message);
}
});
A quick overview of the changes -
Use await and async calls to wait for things to complete vs. being triggered in a callback (async and await are generally much easier to read than promises with .then functions as the execution order is the order of the code)
Used request-promise-native which supports promises / await (i.e. the await means wait until the promise returns so we need something that returns a promise)
Used const and let vs. var for variables; this improves the scope of variables
Instead of doing checks like if(is good) { do good things } use a if(isbad) { return some error} do good thin. This makes the code easier to read and prevents lots of nested ifs where you don't know where they end
Use a Promise.all() so retrieving the categories for each event is done in parallel

There are two main changes you should implement in your code:
Since request does not return a promise you need to use an interface wrapper for request, like request-promise in order to correctly chain the different asynchronous events (See Doug's comment to your question)
Since you will then call several times (in parallel) the different endpoints with request-promise you need to use Promise.all() in order to wait all the promises resolve before sending back the response. This is also the case for the different calls to the Firebase push() method.
Therefore, modifying your code along the following lines should work.
I let you modifying it in such a way you get the values of name and description used to construct the event object. The order of the items in the results array is exactly the same than the one of the promises one. So you should be able, knowing that, to get the values of name and description within results.forEach(groupBody => {}) e.g. by saving these values in a global array.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
var rp = require('request-promise');
exports.foo = functions.https.onRequest((req, res) => {
var ref = admin.database().ref('/foo');
var options = {
url:
'https://api.meetup.com/2/open_events?sign=true&photo-host=public&lat=39.747988&lon=-104.994945&page=20&key=****',
json: true
};
rp(options)
.then(body => {
if ('results' in body) {
const promises = [];
for (var i = 0; i < body.results.length; i++) {
var result = body.results[i];
if (
'name' in result &&
'description' in result &&
'group' in result &&
'urlname' in result.group
) {
var groupOptions = {
url:
'https://api.meetup.com/' +
result.group.urlname +
'?sign=true&photo-host=public&key=****',
json: true
};
promises.push(rp(groupOptions));
}
}
return Promise.all(promises);
} else {
throw new Error('err xxxx');
}
})
.then(results => {
const promises = [];
results.forEach(groupBody => {
if ('category' in groupBody && 'name' in groupBody.category) {
var event = {
name: '....',
description: '...',
category: groupBody.category.name
};
promises.push(ref.push(event));
} else {
throw new Error('err xxxx');
}
});
return Promise.all(promises);
})
.then(() => {
res.send('processed events');
})
.catch(error => {
res.status(500).send(error);
});
});

I made some changes and got it working with Node 8. I added this to my package.json:
"engines": {
"node": "8"
}
And this is what the code looks like now, based on R. Wright's answer and some Firebase cloud function sample code.
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const request = require("request-promise-native");
exports.foo = functions.https.onRequest(
async (req, res) => {
var ref = admin.database().ref("/foo");
var options = {
url: "https://api.meetup.com/2/open_events?sign=true&photo-host=public&lat=39.747988&lon=-104.994945&page=20&key=****",
json: true
};
await request(
options,
async (error, response, body) => {
if (error) {
console.error(JSON.stringify(error));
res.status(500).end();
} else if ("results" in body) {
for (var i = 0; i < body.results.length; i++) {
var result = body.results[i];
if ("name" in result &&
"description" in result &&
"group" in result &&
"urlname" in result.group
) {
var groupOptions = {
url: "https://api.meetup.com/" + result.group.urlname + "?sign=true&photo-host=public&key=****",
json: true
};
var groupBody = await request(groupOptions);
if ("category" in groupBody && "name" in groupBody.category) {
var event = {
name: result.name,
description: result.description,
category: groupBody.category.name
};
await ref.push(event);
}
}
}
res.status(200).send("processed events");
}
}
);
}
);

NodeJS script not actioning with async/await

Cannot figure out why the below script won't run. It is likely the script is not going to do what I want but using
node ./contentful/contentful-assets.js
in the terminal, it does nothing - No errors, nothing logged for me to even start debugging. However, if I remove async it will attempt the script and shoot back an error.
./contentful/contentful-assets.js
const contentful = require('contentful-management');
const iterator = require('make-iterator');
const assets = require('./assetObject.js');
async resolve => {
console.log('Creating Contentful client');
const client = contentful.createClient({
accessToken: 'token',
logHandler: (level, data) => console.log(`${level} | ${data}`)
});
const iterableAssets = iterator(assets);
const space = await client.getSpace('space');
const environment = await space.getEnvironment('enviroment');
const cmsAssets = [];
const assetProcessingTimes = [];
const inProcess = new Map();
let processedAssetsCounter = 0;
const createAndPublishSingleAsset = async ({ asset, done, index }) => {
if (done) {
if (inProcess.size > 0) return false;
return resolve(cmsAssets);
}
const start = Date.now();
const id = '' + start + Math.round(Math.random() * 100);
inProcess.set(id, true);
let cmsAsset;
try {
cmsAsset = await environment.createAssetWithId(asset.postId, {
fields: {
title: {
'en-US': asset.title
},
description: {
'en-US': asset.description
},
file: {
'en-US': {
contentType: 'image/jpg',
fileName: asset.filename,
upload: asset.link
}
}
}
});
} catch (e) {
console.log(`Asset "${asset.title}" failed to create, retrying...`);
createAndPublishSingleAsset({
asset,
done,
index
});
}
try {
const processedCMSAsset = await cmsAsset.processForAllLocales();
const publishedCMSAsset = await processedCMSAsset.publish();
cmsAssets.push(publishedCMSAsset);
assetProcessingTimes.push((Date.now() - start) / 1000);
inProcess.clear(id);
const eta = Math.floor(
assetProcessingTimes.reduce((a, b) => a + b, 0) /
assetProcessingTimes.length *
(assets.length - index) /
60
);
processedAssetsCounter += 1;
console.log(
`Processed asset ${processedAssetsCounter}/${assets.length} - eta: ${eta}m`
);
createAndPublishSingleAsset(iterableAssets.next());
} catch (e) {
console.log(`Asset "${asset.title}" failed to process, retrying...`);
await cmsAsset.delete();
createAndPublishSingleAsset({
asset,
done,
index
});
}
};
console.log('Starting to create assets');
createAndPublishSingleAsset(iterableAssets.next());
createAndPublishSingleAsset(iterableAssets.next());
createAndPublishSingleAsset(iterableAssets.next());
};
assetObject.js
[
{
link: 'https://example.com/example1.jpg',
title: 'Example 1',
description: 'Description of example 1',
postId: '1234567890',
filename: 'example1.jpeg'
}, ... // Many more
]
What have I missed here?

I fear that you are not calling the function, could you try, the following?
const contentful = require('contentful-management');
const iterator = require('make-iterator');
const assets = require('./assetObject.js');
const doWork = async resolve => {
console.log('Creating Contentful client');
...
}
doWork();
You are just declaring a function that is async and does all of the code defined, but you are not actually calling it.

In this code snippet you are declaring a function, but never invoking it:
//declaring an async function, with "resolve" as the argument
async resolve => {
//function definition
}
In order to be able to later reference the function to invoke you can assign it to const/let/etc.:
const createAssets = async resolve => { }
//now, invoke
createAssets()