We Keep Coding

What is context.done doing in AWS Lambda? [duplicate]

I was following the guide here for setting up a presignup trigger.
However, when I used callback(null, event) my lambda function would never actually return and I would end up getting an error
{ code: 'UnexpectedLambdaException',
name: 'UnexpectedLambdaException',
message: 'arn:aws:lambda:us-east-2:642684845958:function:proj-dev-confirm-1OP5DB3KK5WTA failed with error Socket timeout while invoking Lambda function.' }
I found a similar link here that says to use context.done().
After switching it works perfectly fine.
What's the difference?
exports.confirm = (event, context, callback) => {
event.response.autoConfirmUser = true;
context.done(null, event);
//callback(null, event); does not work
}

Back in the original Lambda runtime environment for Node.js 0.10, Lambda provided helper functions in the context object: context.done(err, res) context.succeed(res) and context.fail(err).
This was formerly documented, but has been removed.
Using the Earlier Node.js Runtime v0.10.42 is an archived copy of a page that no longer exists in the Lambda documentation, that explains how these methods were used.
When the Node.js 4.3 runtime for Lambda was launched, these remained for backwards compatibility (and remain available but undocumented), and callback(err, res) was introduced.
Here's the nature of your problem, and why the two solutions you found actually seem to solve it.
Context.succeed, context.done, and context.fail however, are more than just bookkeeping – they cause the request to return after the current task completes and freeze the process immediately, even if other tasks remain in the Node.js event loop. Generally that’s not what you want if those tasks represent incomplete callbacks.
https://aws.amazon.com/blogs/compute/node-js-4-3-2-runtime-now-available-on-lambda/
So with callback, Lambda functions now behave in a more paradigmatically correct way, but this is a problem if you intend for certain objects to remain on the event loop during the freeze that occurs between invocations -- unlike the old (deprecated) done fail succeed methods, using the callback doesn't suspend things immediately. Instead, it waits for the event loop to be empty.
context.callbackWaitsForEmptyEventLoop -- default true -- was introduced so that you can set it to false for those cases where you want the Lambda function to return immediately after you call the callback, regardless of what's happening in the event loop. The default is true because false can mask bugs in your function and can cause very erratic/unexpected behavior if you fail to consider the implications of container reuse -- so you shouldn't set this to false unless and until you understand why it is needed.
A common reason false is needed would be a database connection made by your function. If you create a database connection object in a global variable, it will have an open socket, and potentially other things like timers, sitting on the event loop. This prevents the callback from causing Lambda to return a response, until these operations are also finished or the invocation timeout timer fires.
Identify why you need to set this to false, and if it's a valid reason, then it is correct to use it.
Otherwise, your code may have a bug that you need to understand and fix, such as leaving requests in flight or other work unfinished, when calling the callback.
So, how do we parse the Cognito error? At first, it seemed pretty unusual, but now it's clear that it is not.
When executing a function, Lambda will throw an error that the tasked timed out after the configured number of seconds. You should find this to be what happens when you test your function in the Lambda console.
Unfortunately, Cognito appears to have taken an internal design shortcut when invoking a Lambda function, and instead of waiting for Lambda to timeout the invocarion (which could tie up resources inside Cognito) or imposing its own explicit timer on the maximum duration Cognito will wait for a Lambda response, it's relying on a lower layer socket timer to constrain this wait... thus an "unexpected" error is thrown while invoking the timeout.
Further complicating interpreting the error message, there are missing quotes in the error, where the lower layer exception is interpolated.
To me, the problem would be much more clear if the error read like this:
'arn:aws:lambda:...' failed with error 'Socket timeout' while invoking Lambda function
This format would more clearly indicate that while Cognito was invoking the function, it threw an internal Socket timeout error (as opposed to Lambda encountering an unexpected internal error, which was my original -- and incorrect -- assumption).
It's quite reasonable for Cognito to impose some kind of response time limit on the Lambda function, but I don't see this documented. I suspect a short timeout on your Lambda function itself (making it fail more promptly) would cause Cognito to throw a somewhat more useful error, but in my mind, Cognito should have been designed to include logic to make this an expected, defined error, rather than categorizing it as "unexpected."

As an update the Runtime Node.js 10.x handler supports an async function that makes use of return and throw statements to return success or error responses, respectively. Additionally, if your function performs asynchronous tasks then you can return a Promise where you would then use resolve or reject to return a success or error, respectively. Either approach simplifies things by not requiring context or callback to signal completion to the invoker, so your lambda function could look something like this:
exports.handler = async (event) => {
// perform tasking...
const data = doStuffWith(event)
// later encounter an error situation
throw new Error('tell invoker you encountered an error')
// finished tasking with no errors
return { data }
}
Of course you can still use context but its not required to signal completion.

Firebase function timeout regardless of code

Edit: 2021 feb this problem was fixed.
I have been dealing with a timeout error using node js firebase function emulator. My function was working, but now the code will wait for a timeout regardless of the code. I tried copying the example on the quick start page, and the same error is occurring.
I can put console statements in the code, and I will see nothing output. I have another function that works properly when a document is created. The response errors out, but the function will continue executing for the duration of the timeout.
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
exports.addMessage = functions.https.onRequest(async (req, res) => {
// Grab the text parameter.
const original = req.query.text;
// Push the new message into Firestore using the Firebase Admin SDK.
const writeResult = await admin.firestore().collection('messages').add({original: original});
// Send back a message that we've successfully written the message
res.json({result: `Message with ID: ${writeResult.id} added.`});
});
Error: Function timed out.
at Timeout._onTimeout (/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:640:19)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
i functions: Beginning execution of "createTokenForEvents"
⚠ functions: Your function timed out after ~60s. To configure this timeout, see
https://firebase.google.com/docs/functions/manage-functions#set_timeout_and_memory_allocation.
/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:640
throw new Error("Function timed out.");
^
Edit: I have come to the conclusion that this function does work, but it just always error out. I was writing another function when I discover this error and even when I switch to this simple case, the error was present. However, in this case, the function does create a document, but whatever is keeping it on for the entire duration may also be hiding log statements too. My question has change; why is the function executing for the entire duration even after the code completes.

https://firebase.google.com/docs/functions/terminate-functions
As stated in the documentation, one of the principles of writing a good functions is
Terminate HTTP functions with res.redirect(), res.send(), or
res.end().
Since my other functions where synchronous, writing a return statement was good enough. However, I solve the error by explicitly adding terminating response statement res.end(). I also experiment with returning a promise, but it did not solve the problem either.
Edit: This problem continues to occur in different scenarios. I literally cannot get the code to run when using a query https://github.com/firebase/firebase-functions/issues/847. As of Jan 2021, the firebase-functions repo seems to be unmaintained as far as fixing community issues. A user got a response from the firebase team saying they have been trying to fix the logging issues, but they would not commit to a date on the problem would be resolved. I would just not even bother using the firebase functions emulator. Deploying the function works fine.

Due to the query parameter, add the slash symbol:
http://localhost:5001/.../us-central1/helloWorld/?foo=bar&test=string
http://localhost:5001/.../us-central1/helloWorld?foo=bar&test=string (no slash before ?)
Refer from https://github.com/firebase/firebase-tools/issues/1314

Why does my async AWS Lambda require a callback in order to recursively call itself?

Setup: Because of the time limit on lambdas, I am using a lambda to process some work, then recursively call itself to continue the work, again and again until it is complete. It's just a chron job that I split up to make sure it won't timeout if the data is too large sometimes.
It looks roughly like this:
exports.handler = async (event, context, callback) => {
let response;
let err;
try {
const data = await getSomeAsyncDataToUpdate(); // this won't get previously updated data, so each recursion gets the first X data until there's none left
response = await updateSomeDataAsync(data);
if (!doneConditionIsMet(response)) {
callLambda(); // we do NOT await because the top level lambda would eventually timeout
}
} catch(e) {
err = e;
}
callback(err, response);
};
This is being called from a Cloudwatch Event (chron).
The Problem:
According to the docs, Cloudwatch Events are asynchronous calls and therefore, do not use the callback. https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html
For async handlers, you return a response, error, or promise to the runtime instead of using callback.
Here's the rub: if I omit the callback, the recursive lambda call doesn't happen. I can see that the function gets called, but it does not result in another lambda getting triggered and running.
Why?
I am out of ideas. The only thing I could come up with is that the lambda is exiting before the call actually executes, which would be wild because I assume the actual execution is synchronous. Also, if that were the case, I do not see how calling the callback would keep it alive long enough for that to finish.
If it is working by some fluke, is there a correct way to do this?
Notes on things I've already checked/tried:
the callback does not keep the lambda alive for a long time. It exits immediately as expected
I tried every version I could think of with removing the callback and returning something
As noted in the code snippet, I do NOT want to await the recursive call because that would mean the first lambda has to wait for them all to complete, which defeats the entire purpose of this

mdg:validated-method _execute asynchronous issues

I'm running into problems with the validated method package in my app tests. I'm calling my methods through the _execute function in order to be able to pass a userId to simulate a logged-in user while testing. The problem is that my asserts right underneath that _execute are called before the method has a chance of completing. I know my test works though because it only happens sometimes, mostly because mongo isn't always returning results quite as fast.
I looked around and found a todos app that uses the _execute function in its tests. I can't get those tests to fail no matter how many times I rerun them, though.
This is an example of my test code.
describe('clients.add', function() {
it('should add an empty (draft) client', function() {
const res = clients_add._execute({ userId: 'CURRENTUSERID' }, { company_id: c1._id });
assert.instanceOf(res, Mongo.ObjectID, 'method returns the newly created clients ID');
const db_client = Clients.findOne(res);
assert.isTrue(db_client.draft, 'client is drafted');
assert.isDefined(db_client.created, 'there\'s a created date');
});
});
clients_add does quite a few permission checks and can therefor take a little while before completing. Rerunning this test 20 times will fail about 5 times and pass the other 15.
Shouldn't the _execute function be synchronous? How do I make it? What am I missing?

In server code, if you provide a callback to database modification functions like insert, it returns the created ID instantaneously, and runs the callback only once the database has acknowledged the write. If you don't provide a callback, the insert call is synchronous and throws an error if the operation fails. See more about this in Meteor docs.
It seems that you have provided an error-handling callback to the insert-function in your method code. This causes the inconsistent behavior, since the database might not actually have had time to do the write before you call findOne in your test. Also, this is redundant since if an error occurs in the insert, the method has already returned and the error is never shown to the user. It's better to simply omit the error-handling callback altogether:
return Clients.insert(new_client);

node.js async request with timeout?

Is it possible, in node.js, to make an asynchronous call that times out if it takes too long (or doesn't complete) and triggers a default callback?
The details:
I have a node.js server that receives a request and then makes multiple requests asynchronously behind the scenes, before responding. The basic issue is covered by an existing question, but some of these calls are considered 'nice to have'. What I mean is that if we get the response back, then it enhances the response to the client, but if they take too long to respond it is better to respond to the client in a timely manner than with those responses.
At the same time this approach would allow to protect against services that simply aren't completing or failing, while allowing the main thread of operation to respond.
You can think of this in the same way as a Google search that has one core set of results, but provides extra responses based on other behind the scenes queries.

If its simple just use setTimout
app.get('/', function (req, res) {
var result = {};
// populate object
http.get('http://www.google.com/index.html', (res) => {
result.property = response;
return res.send(result);
});
// if we havent returned within a second, return without data
setTimeout(function(){
return res.send(result);
}, 1000);
});
Edit: as mentioned by peteb i forgot to check to see if we already sent. This can be accomplished by using res.headerSent or by maintaining a 'sent' value yourself. I also noticed res variable was being reassigned
app.get('/', function (req, res) {
var result = {};
// populate object
http.get('http://www.google.com/index.html', (httpResponse) => {
result.property = httpResponse;
if(!res.headersSent){
res.send(result);
}
});
// if we havent returned within a second, return without data
setTimeout(function(){
if(!res.headersSent){
res.send(result);
}
}, 1000);
});

Check this example of timeout callback https://github.com/jakubknejzlik/node-timeout-callback/blob/master/index.js
You could modify it to do action if time's out or just simply catch error.

You can try using a timeout. For example using the setTimeout() method:
Setup a timeout handler: var timeOutX = setTimeout(function…
Set that variable to null: timeOutX = NULL (to indicate that the timeout has been fired)
Then execute your callback function with one argument (error handling): callback({error:'The async request timed out'});
You add the time for your timeout function, for example 3 seconds
Something like this:
var timeoutX = setTimeout(function() {
timeOutX = null;
yourCallbackFunction({error:'The async request timed out'});
}, 3000);
With that set, you can then call your async function and you put a timeout check to make sure that your timeout handler didn’t fire yet.
Finally, before you run your callback function, you must clear that scheduled timeout handler using the clearTimeout() method.
Something like this:
yourAsyncFunction(yourArguments, function() {
if (timeOutX) {
clearTimeout(timeOutX);
yourCallbackFunction();
}
});