This question already has an answer here:
Why is my PDF not saving intermittently in my Node function?
(1 answer)
Closed last year.
As described in the firebase docs, it is required to
"resolve functions that perform asynchronous processing (also known as
"background functions") by returning a JavaScript promise."
(https://firebase.google.com/docs/functions/terminate-functions?hl=en).
otherwise it might happen, that
"the Cloud Functions instance running your function does not shut down
before your function successfully reaches its terminating condition or
state. (https://firebase.google.com/docs/functions/terminate-functions?hl=en)
In this case I am trying to adapt a demo-code for pdf-generation written by Volodymyr Golosay on https://medium.com/firebase-developers/how-to-generate-and-store-a-pdf-with-firebase-7faebb74ccbf.
The demo uses 'https.onRequest' as trigger and fulfillis the termination requirement with 'response.send(result)'. In the adaption I need to use a 'document.onCreate' trigger and therefor need to find a different termination.
In other functions I can fulfill this requirement by using async/await, but here I am struggling to get a stable function with good performance. The shown function logs after 675 ms "finished with status: 'ok' ", but around 2 minutes later it logs again that the pdf-file is saved now (see screenshot of the logger).
What should I do to terminate the function properly?
// adapting the demo code by Volodymyr Golosay published on https://medium.com/firebase-developers/how-to-generate-and-store-a-pdf-with-firebase-7faebb74ccbf
// library installed -> npm install pdfmake
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const db = admin.firestore();
const Printer = require('pdfmake');
const fonts = require('pdfmake/build/vfs_fonts.js');
const fontDescriptors = {
Roboto: {
normal: Buffer.from(fonts.pdfMake.vfs['Roboto-Regular.ttf'], 'base64'),
bold: Buffer.from(fonts.pdfMake.vfs['Roboto-Medium.ttf'], 'base64'),
italics: Buffer.from(fonts.pdfMake.vfs['Roboto-Italic.ttf'], 'base64'),
bolditalics: Buffer.from(fonts.pdfMake.vfs['Roboto-Italic.ttf'], 'base64'),
}
};
exports.generateDemoPdf = functions
// trigger by 'document.onCreate', while demo uses 'https.onRequest'
.firestore
.document('collection/{docId}')
.onCreate(async (snap, context) => {
const printer = new Printer(fontDescriptors);
const chunks = [];
// define the content of the pdf-file
const docDefinition = {
content: [{
text: 'PDF text is here.',
fontSize: 19 }
]
};
const pdfDoc = printer.createPdfKitDocument(docDefinition);
pdfDoc.on('data', (chunk) => {
chunks.push(chunk);
});
pdfDoc.on('end', async () => {
const result = Buffer.concat(chunks);
// Upload generated file to the Cloud Storage
const docId = "123456789"
const bucket = admin.storage().bucket();
const fileRef = bucket.file(`${docId}.pdf`, {
metadata: {
contentType: 'application/pdf'
}
});
await fileRef.save(result);
console.log('result is saved');
// NEEDS PROPER TERMINATION HERE?? NEEDS TO RETURN A PROMISE?? FIREBASE DOCS: https://firebase.google.com/docs/functions/terminate-functions?hl=en
// the demo with 'https.onRequest' uses the following line to terminate the function properly:
// response.send(result);
});
pdfDoc.on('error', (err) => {
return functions.logger.log('An error occured!');
});
pdfDoc.end();
});
I think everything is fine in your code. It seems it takes 1m 34s to render the file and save it to storage.
Cloud function will be terminated automatically when all micro and macro tasks are done. Right after you last await.
To check how long does it takes and does it terminate right after saving, you can run the firebase emulator on your local machine.
You will see logs in the terminal and simultaneously watch on storage.
I suspect you did terminate properly - that's the nature of promises. Your function "terminated" with a 200 status, returning a PROMISE for the results of the PDF save. When the PDF save actually terminates later, the result is logged and the promise resolved. This behavior is WHY you return the promise.
Related
I'm using electron to develop an app. after some encryption operations are done, I need to show a dialog to the user to save the file. The filename I want to give to the file is a random hash but I have no success also with this. I'm trying with this code but the file will not be saved. How I can fix this?
const downloadPath = app.getPath('downloads')
ipcMain.on('encryptFiles', (event, data) => {
let output = [];
const password = data.password;
data.files.forEach( (file) => {
const buffer = fs.readFileSync(file.path);
const dataURI = dauria.getBase64DataURI(buffer, file.type);
const encrypted = CryptoJS.AES.encrypt(dataURI, password).toString();
output.push(encrypted);
})
const filename = hash.createHash('md5').toString('hex');
console.log(filename)
const response = output.join(' :: ');
dialog.showSaveDialog({title: 'Save encrypted file', defaultPath: downloadPath }, () => {
fs.writeFile(`${filename}.mfs`, response, (err) => console.log(err) )
})
})
The problem you're experiencing is resulting from the asynchronous nature of Electron's UI functions: They do not take callback functions, but return promises instead. Thus, you do not have to pass in a callback function, but rather handle the promise's resolution. Note that this only applies to Electron >= version 6. If you however run an older version of Electron, your code would be correct -- but then you should really update to a newer version (Electron v6 was released well over a year ago).
Adapting your code like below can be a starting point to solve your problem. However, since you do not state how you generate the hash (where does hash.createHash come from?; did you forget to declare/import hash?; did you forget to pass any message string?; are you using hash as an alias for NodeJS' crypto module?), it is (at this time) impossible to debug why you do not get any output from console.log (filename) (I assume you mean this by "in the code, the random filename will not be created"). Once you provide more details on this problem, I'd be happy to update this answer accordingly.
As for the default filename: As per the Electron documentation, you can pass a file path into dialog.showSaveDialog () to provide the user with a default filename.
The file type extension you're using should also actually be passed with the file extension into the save dialog. Also passing this file extension as a filter into the dialog will prevent users from selecting any other file type, which is ultimately what you're also currently doing by appending it to the filename.
Also, you could utilise CryptoJS for the filename generation: Given some arbitrary string, which could really be random bytes, you could do: filename = CryptoJS.MD5 ('some text here') + '.mfs'; However, remember to choose the input string wisely. MD5 has been broken and should thus no longer be used to store secrets -- using any known information which is crucial for the encryption of the files you're storing (such as data.password) is inherently insecure. There are some good examples on how to create random strings in JavaScript around the internet, along with this answer here on SO.
Taking all these issues into account, one might end up with the following code:
const downloadPath = app.getPath('downloads'),
path = require('path');
ipcMain.on('encryptFiles', (event, data) => {
let output = [];
const password = data.password;
data.files.forEach((file) => {
const buffer = fs.readFileSync(file.path);
const dataURI = dauria.getBase64DataURI(buffer, file.type);
const encrypted = CryptoJS.AES.encrypt(dataURI, password).toString();
output.push(encrypted);
})
// not working:
// const filename = hash.createHash('md5').toString('hex') + '.mfs';
// alternative requiring more research on your end
const filename = CryptoJS.MD5('replace me with some random bytes') + '.mfs';
console.log(filename);
const response = output.join(' :: ');
dialog.showSaveDialog(
{
title: 'Save encrypted file',
defaultPath: path.format ({ dir: downloadPath, base: filename }), // construct a proper path
filters: [{ name: 'Encrypted File (*.mfs)', extensions: ['mfs'] }] // filter the possible files
}
).then ((result) => {
if (result.canceled) return; // discard the result altogether; user has clicked "cancel"
else {
var filePath = result.filePath;
if (!filePath.endsWith('.mfs')) {
// This is an additional safety check which should not actually trigger.
// However, generally appending a file extension to a filename is not a
// good idea, as they would be (possibly) doubled without this check.
filePath += '.mfs';
}
fs.writeFile(filePath, response, (err) => console.log(err) )
}
}).catch ((err) => {
console.log (err);
});
})
I have a function that monitors a node in a Realtime database and once a new child is written to the node the function simply needs to create a html document in a Google Cloud bucket. The HTML document will have a unique name and will contain some data from the node. It's all fairly straightforward, however I can't actually create and write to the document. I've tried 3 methods so far (outlined in the code below), none of these methods work.
const {Storage} = require('#google-cloud/storage');
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const fs = require('fs');
const {StringStream} = require('#rauschma/stringio')
const instanceId = 'my-project-12345';
const bucketName = 'my-bucket';
exports.processCertification = functions.database.instance(instanceId).ref('/t/{userId}/{testId}')
.onCreate((snapshot, context) => {
const dataJ = snapshot.toJSON();
var testResult = "Invalid";
if(dataJ.r == 1) {testResult = "Positive";}
else if(dataJ.r == 2) {testResult = "Negative";}
console.log('Processing certificate:', context.params.testId, testResult);
var storage = new Storage({projectId: instanceId});
const fileName = context.params.testId + '.html';
const fileContents = "<html><head></head><body>Result: " + testResult + "</body></html>"
const options = {resumable:false, metadata:{contentType:'text/html'}};
const bucket = storage.bucket(bucketName);
const file = bucket.file(fileName);
console.log('Saving to:' + bucketName + '/' + fileName);
if(false) {
// Test 1. the file.save method
// Errors with:
// (node:2) MetadataLookupWarning: received unexpected error = URL is not defined code = UNKNOWN
file.save(fileContents, options, function(err) {
if (!err) {console.log("Save created object at " + bucketName + "/" + fileName);}
else {console.log("Save Failed " + err);}
});
} else if(true) {
// Test 2. the readStream.pipe method
// No errors, doesn't output error message, doesn't output finish message, no file created
fs.createReadStream(fileContents)
.pipe(file.createWriteStream(options))
.on('error', function(err) {console.log('WriteStream Error');})
.on('finish', function() {console.log('WriteStream Written');});
} else {
// Test 3. the StringStream with readStream.pipe method
// Errors with:
// (node:2) MetadataLookupWarning: received unexpected error = URL is not defined code = UNKNOWN
const writeStream = storage.bucket(bucketName).file(fileName).createWriteStream(options);
writeStream.on('finish', function(){console.log('WriteStream Written');}).on('error', function(err){console.log('WriteStream Error');});
const readStream = new StringStream(fileContents);
readStream.pipe(writeStream);
}
console.log('Function Finished');
return 0;
});
In all cases the "Processing certificate" and "Saving to" outputs appear, I also get the "Function Finished" message every time. The errors (or in one case no response) is written against each of the tests in the code.
My next step will be to create the file locally and then use upload() method, however each of these methods seem like they should work, plus the only error message I have is talking about URL errors so I suspect trying to use upload() method would run into the same problems as well.
I'm using Node.JS v8.17.0 and the following packages
"dependencies": {
"#google-cloud/storage": "^5.0.0",
"#rauschma/stringio": "^1.4.0",
"firebase-admin": "^8.10.0",
"firebase-functions": "^3.6.1"
}
Any advice is most welcome
In each case, you are not working with promises correctly. For database triggers (and all other background triggers), you must return a promise that resolves when all of the asynchronous work is complete in a function. Right now, you're not doing anything at all with promises, while each of the APIs you're calling are all asynchronous. Your function is just returning 0 immediately without waiting for the upload to complete, and Cloud Functions is simply terminating and cleaning up before anything can happen.
I suggest choosing one of the methods that returns a promise with the upload is complete (probably file.save()), then return that promise from the function.
Asynchronous function definitions on MongoDB (Atlas) Stitch display warnings on the GUI editor. Including the example code provided on the reference for Triggers.
The code found here can be was copied over directly to the Stitch Function editor and produces warnings because of the async keyword.
Example code from the docs.
exports = async function (changeEvent) {
// Destructure out fields from the change stream event object
const { updateDescription, fullDocument } = changeEvent;
// Check if the shippingLocation field was updated
const updatedFields = Object.keys(updateDescription.updatedFields);
const isNewLocation = updatedFields.some(field =>
field.match(/shippingLocation/)
);
// If the location changed, text the customer the updated location.
if (isNewLocation) {
const { customerId, shippingLocation } = fullDocument;
const twilio = context.services.get("myTwilioService");
const mongodb = context.services.get("mongodb-atlas");
const customers = mongodb.db("store").collection("customers");
const { location } = shippingLocation.pop();
const customer = await customers.findOne({ _id: customer_id })
twilio.send({
to: customer.phoneNumber,
from: context.values.get("ourPhoneNumber"),
body: `Your order has moved! The new location is ${location}.`
});
}
};
I want to know if Stitch supports the async/await paradigm and if I should be concerned about the warnings shown.
After some testing I found that at this time the async/await keywords cause the linter to throw errors and warnings. This means that for async callbacks it is best to define them separately as it will improve the linting. IE. [].map(async () => {}) will prompt errors that can be worked around.
The runtime execution returns the results as expected from standard asynchronous operations.
Here is my nodejs code:
const cp = require('child_process');
describe('cloud function test suites', () => {
describe('deleteCampaign test suites', () => {
const cloudFunctionName = 'deleteCampaign';
it('should print campaign data', () => {
const campaign = { id: '1' };
const encodedCampaign = Buffer.from(JSON.stringify(campaign)).toString(
'base64',
);
const data = JSON.stringify({ data: encodedCampaign });
const executeResultOutput = cp
.execSync(
`gcloud beta functions call ${cloudFunctionName} --data '${data}'`,
)
.toString();
const executionId = executeResultOutput.split(': ')[1];
const logs = cp
.execSync(
`gcloud beta functions logs read ${cloudFunctionName} --execution-id ${executionId}`,
)
.toString();
console.log(logs);
expect(logs).toContain('campaign: {"id":"1"}');
});
});
});
I want to print the logs to stdout, but logs is empty string.
But when I read logs using gcloud command line, it's ok. The stdout is correct:
gcloud beta functions logs read deleteCampaign --execution-id ee5owvtzlekc
LEVEL NAME EXECUTION_ID TIME_UTC LOG
D deleteCampaign ee5owvtzlekc 2018-09-13 12:46:17.734 Function execution started
I deleteCampaign ee5owvtzlekc 2018-09-13 12:46:17.738 campaign: {"id":"1"}
D deleteCampaign ee5owvtzlekc 2018-09-13 12:46:17.742 Function execution took 9 ms, finished with status: 'ok'
I use jest and nodejs write some tests for my cloud functions. Why the logs is empty string?
The string you are trying to get is empty, because the logs take a bit more time to generate. Even though the Google Cloud Function has finished executing, you'll have to wait a few seconds for the logs to be ready.
Reading your code, you are not letting this happen, hence you are getting an empty string.
The first thing that I noticed reading your code was this part:
const executionId = executeResultOutput.split(': ')[1];
I understand that you want to extract the Google Cloud Function's Execution ID. I had problems here because the string was not limited to the execution ID, it also included a new line character and the word "result". I made sure to just extract the necessary Execution ID with the next code:
const executionId = executeResultOutput.split(':')[1]; //We get the GCP ID.
const executionId2 = executionId.split("\n")[0].toString(); //removing the right part of the string.
If you have found the way to get the execution ID without problems then ignore my code.
Below you can find the code that has worked for me implementing functions.
let cloudFunctionLog ='';
function getLogs(){
console.log('Trying to get logs...');
const logs = cp
.execSync(`gcloud beta functions logs read ${cloudFunctionName} --execution-id ${executionId2}`);
return logs;
}
do{
cloudFunctionLog=getLogs();
if(!cloudFunctionLog){
console.log('Logs are not ready yet...');
}else{
console.log(`${cloudFunctionLog}`);
}
}while(!cloudFunctionLog);//Do it while the string comes empty.
When the logs are no longer empty, they'll show up in your console.
This question already has an answer here:
Cloud Functions for Firebase HTTP timeout
(1 answer)
Closed 5 years ago.
I can see a new entry is added as soon as I get the URL, but the request hangs and times out. Why?
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.add = functions.https.onRequest((request, response)=>
{
var ref = admin.database().ref("jobs");
var childRef = ref.push();
childRef.set
(
{
title: "test",
pay: 100
}
);
})
The code is based on the following example. https://firebase.google.com/docs/database/admin/save-data
Result
{"error":{"code":500,"status":"INTERNAL","message":"function execution attempt timed out"}}
Cloud Functions triggered by HTTP requests need to be terminated by ending them with a send(), redirect(), or end(), otherwise they will continue running and reach the timeout.
From the terminate HTTP functions section of the documentation on HTTP triggers:
Always end an HTTP function with send(), redirect(), or end(). Otherwise, your function might to continue to run and be forcibly terminated by the system. See also Sync, Async and Promises.
Therefore, in your example, you can end the request by sending a response:
exports.add = functions.https.onRequest((request, response)=>
{
var ref = admin.database().ref("jobs");
var childRef = ref.push();
childRef.set
(
{
title: "test",
pay: 100
}
);
response.status(200).send("OK!");
})