I have an azure function using service bus topic trigger and I want to handle error messages gracefully, I want to be able to do an abandon of the message and pass the exception to it so I can see it in a property when I read the dead letters queue.
this is my code:
const serviceBusTopicTrigger: AzureFunction = async function(context: Context, mySbMsg: any): Promise<void> {
// do something messy that may fail.
};
When my function fails with an exception the message goes to the DLQ as expected, but the problem is, that it doesn't save the exception thrown, it only tells you that it tried to execute the method 10 times and it couldn't.
What I want its to be able to catch the exception and add it to the message properties so, when I process the DLQ queue I will be able to know the reason for the error. Even more, as the code is failing with an exception I would like it to abandon from the first time it runs the message, so it doesn't have to retry 10 times.
I'm thinking something like this:
const serviceBusTopicTrigger: AzureFunction = async function(context: Context, mySbMsg: any): Promise<void> {
try{
// do something messy and that may fail
}
catch(error){
context.bindingData.userProperties['DeadLetterReason'] = 'Internal server error';
context.bindingData.userProperties['DeadLetterErrorDescription'] = JSON.stringify(error);
context.bindingData.abandonMsg();
}
};
I haven't been able to find any documentation about something like this, so is it possible? or can I force the message to the DLQ? with something like this:
const serviceBusTopicTrigger: AzureFunction = async function(context: Context, mySbMsg: any): Promise<void> {
try{
// do something messy and that may fail
}
catch(error){
context.bindings.deadLetterQueue.userProperties['DeadLetterReason'] = 'Internal server error';
context.bindings.deadLetterQueue.userProperties['DeadLetterErrorDescription'] = JSON.stringify(error);
context.bindings.deadLetterQueue = mySbMsg;
}
};
Or finally and sadly do I have to manage my error directly from the method and maybe send it directly from there to an azure storage table or queue to notify errors?, I wouldn't like that because then I would be handling, both errors from the dead letter queue, and from my functions in different places. Is this the only way?
Any more ideas?
Thanks.
First of all, I think nodejs can not do this. nodejs lost the type information in C#, it should have been sent like this:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.brokeredmessage.deadletterasync?view=azure-dotnet
When my function fails with an exception the message goes to the DLQ
as expected, but the problem is, that it doesn't save the exception
thrown, it only tells you that it tried to execute the method 10 times
and it couldn't.
Max Delivery Count is set to 10 by default when you create the subscription of the service bus topic. The default value is 10, and the minimum can be set to 1.
When you turn on the service bus information automatic completion, just like below:
host.json
{
"version": "2.0",
"extensions": {
"serviceBus": {
"messageHandlerOptions": {
"autoComplete": true
}
}
},
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 3.1.0)"
}
}
At that time, when an error occurs, the function run failed, call the abandon method, Delivery Count +1, and send the message back at the same time.
Then once you use try-catch, you will not retry 10 times at all. The function is regarded as a successful operation, function run success and the message disappears without sending it back.(That is, once try-catch you will only execute it once.)
I haven't been able to find any documentation about something like
this, so is it possible? or can I force the message to the DLQ?
No, it can't be done. The related type is missing and cannot be manually sent to dead-letter via JavaScript.
Or finally and sadly do I have to manage my error directly from the
method and maybe send it directly from there to an azure storage table
or queue to notify errors?, I wouldn't like that because then I would
be handling, both errors from the dead letter queue, and from my
functions in different places. Is this the only way?
Why do you say that the error still appears in the dead-letter? Using try-catch in the case of automatic completion should not be sent to dead-letter.
Related
(new to js)
I am making a discord to minecraft bot which uses discord.js on one file and mineflayer for the minecraft bot. When a command is run in discord, it makes the bot run it and runs a function which detects the right line in chat, problem is even after it detects the message, it constantly stays on and reads every line, forever. Meaning if someone else types what its detecting it interferes and thinks thats the right chat line..
Im not sure how to make it stop either, after running the event once, or after finding the line in chat, I would probably need it to stop after running it once incase the line doesnt show up for some reason.
function balOutput() {
bot.on('message', jsonMsg => {
if (Array.isArray(jsonMsg.json.extra)) {
var message = '';
jsonMsg.json.extra.forEach(function(element) {
message = message + element.text;
});
if (message.toLowerCase().includes('\'s balance')) {
var msg = message;
fs.writeFile('output.txt', msg, err => {
if (err) throw err;
});
} else if (message.toLowerCase().includes('is not locally online')) {
var msg = 'That player is not online!';
fs.writeFile('output.txt', msg, err => {
if (err) throw err;
});
}
}
continue;
});
}
That function is called once the discord command goes through and it runs it in minecraft. It does work for the one time but then other messages ruin it. The problem is I cant just use regex and make it detect that message of a player saying it because, some other commands need to run with the exact same output but be placed in different spots, so I need it not to trigger the bot.on('message') in the baloutput function... Pretty much I need to close that event after running it once.
I’m not entirely sure what jsonMsg represents, as Discord.js’s message event passes a Message object as its first argument, and that doesn’t have a .json property, while you are expecting jsonMsg.json.extra to be an array, and that seems to be the case sometimes.
However, whenever you call your balOutput function, it seems you are adding yet another listener callback function to bot’s message event. That could be causing the behaviour you’re faced with, where things only work the first time. You’ll want to add just one callback.
You may also want to look into using a normal for loop, rather than a .forEach(), from which you can break out of with break; the moment your if condition is first met.
I'm creating a web application for monitoring by smartphones using WebRTC, and for the signalling server I'm using socket.io.
When I'm sending a stream, I create an RTCPeerConnection object on the "watch" page that receives this stream. Streams I send on separate pages. The user can attach up to four streams from a smartphone, so on the "watch" page there are up to four RTCPeerConnection objects.
Streams are received automatically as soon as the offer from the "transmit" page appears, then the RTCPeerConnection object is created on the "watch" page and connected to the standard WebRTC schema.
"transmit" page:
function onCreateOfferSuccess(sdp){
//console.log(sdp);
pc.setLocalDescription(new RTCSessionDescription(sdp));
console.log('ask');
socket.emit('ask', {"sdp":JSON.stringify(pc.localDescription),
"user": loggedUserID,
"fromSocket": ownSocket});
}
"watch" page:
socket.on('ask', function(offer){
if (offer.user === loggedUserID){
TempDescriptions = JSON.parse(offer.sdp);
console.log(TempDescriptions)
currTransmiterSocket = offer.fromSocket;
console.log(currTransmiterSocket);
getStream();
}
function getStream(){
try {
setTimeout(function(){
console.log(time, 'getStream()');
connection = getPeerConnection();
connection.setRemoteDescription(
new RTCSessionDescription(TempDescriptions),
function() {
connection.createAnswer(gotDescription, function(error){
console.log(error)
});
}, function(error){
console.log(error)
});
}, getStreamDelay*3000)
getStreamDelay++
}
catch(err){
console.log(err);
}
};
My web application requires functionality in which when we exit from the "watch" page and return to it again, all previously included streams must be displayed.
To implement this functionality, I use the oniceconnectionstatechange method. If the stream is disconnected, the iceRestart function is executed which creates the offer with the option {iceRestart: true}
"transmit" page:
var options_with_restart = {offerToReceiveAudio: false,
offerToReceiveVideo: true,
iceRestart: true};
function iceRestart(event){
try{
setTimeout(function(){
pc.createOffer(options_with_restart).then(onCreateOfferSuccess, onCreateOfferError);
},1000);
} catch(error) {
console.log(error);
The problem is that when I restart the "watch" page, all pages "transmit" send to ask at once. Only one object is connected, although four RTCPeerConnection objects are created at once (let's assume that the user sends four streams).
I have been struggling with this problem for several days. I tried to set an increasing time delay on subsequent calls to the getStream() function as seen in the above code, I tried to check the signallingState connections before executing the getStream() function, I tried several other methods but none of them worked.
If you need some part of my code to help, please write.
edit:
gotDescription() method in "watch" page.
function gotDescription(sdp) {
try{
connection.setLocalDescription(sdp,
function() {
registerIceCandidate();
socket.emit('response', {"sdp": sdp,
"user": loggedUserID,
"fromSocket": ownSocket,
"toSocket": currTransmiterSocket});
}, function(error){
console.log(error)
});
} catch(err){
console.log(err);
}
}
I add console.log with RTCPeerConnection object
console output:
https://i.stack.imgur.com/dQXkE.png1
log shows that the signalingState of connection is "stable" but when I develop the object, signalingState is equal to "have-remote-offer"
like here
Remove the TempDescriptions global variable, and pass the sdp to getStream(offer.sdp) directly.
Otherwise, you've socket.on('ask', function(offer){ called 4 times, overwriting TempDescriptions. Then 3+ seconds later your 4 setTimeouts roll around, all accessing the final value of TempDescriptions only.
That's probably why only one RTCPeerConnection re-connects.
In general, using time delay to separate connections seems like a bad idea, as it slows down re-connection. Instead, why not send an id? E.g.
socket.emit('ask', {id: connectionNumber++,
sdp: JSON.stringify(pc.localDescription),
user: loggedUserID,
fromSocket: ownSocket});
Update: Stop adding global variables to window
Whenever you assign to an undeclared variable like this:
connection = getPeerConnection();
...it creates a global on window, e.g. window.connection, and you have the same problem. You have 4 connection, but you're storing them in one variable.
Type "use strict"; at the head of your source file to catch this:
ReferenceError: assignment to undeclared variable connection
Scoping: The general problem
You're dealing with 4 connections here, but you lack an approach for scoping each instance.
Most other languages would tell you to create a class and make object instances, and put everything including connection on this. That's one good approach. In JS you can use closures instead. But at minimum you still need 4 variables holding the 4 connections, or an array of connections. Then you look up—e.g. from the id I mentioned—which connection to deal with.
Also, your try/catches aren't going to catch asynchronous errors. Instead of defining all these callbacks, I strongly recommend using promises, or even async/await when dealing with the highly asynchronous WebRTC API. This makes scoping trivial. E.g.
const connections = [];
socket.on('ask', async ({user, id, sdp, fromSocket}) => {
try {
if (user != loggedUserID) return;
if (!connections[id]) {
connections[id] = getPeerConnection();
registerIceCandidate(connections[id]);
}
const connection = connections[id];
await connection.setRemoteDescription(JSON.parse(sdp));
await connection.setLocalDescription(await connection.createAnswer());
socket.emit('response', {sdp,
user: loggedUserID,
fromSocket: ownSocket,
toSocket: fromSocket});
} catch (err) {
console.log(err);
}
};
This way the error handling is solid.
I have created a public Web App with access to my private spreadsheet data. I can catch and log exceptions intry..catch, but:
is it possible to catch all unhandled exceptions, like browsers window.onerror?
can I view logs of unhandled exceptions somewhere?
by exceptions like "Service invoked too many times" my app is even not getting run, so here I definitely can`t handle the exceptions. Is there logs with such kind of exceptions?
These are so simple questions, so that I'm bit confused to ask them, but after hours of research I could not find the answers.
Thank you in advance.
These are issues that are being addressed currently. Right now in the Apps Script Early Access Program are two new additions that handle these cases. The first is native integration with stackdriver logging and the addition of google.script.run.withLogger().
First off for now you need to apply for the EAP:
https://developers.google.com/apps-script/guides/apps-script-eap
Stackdriver Logging:
To log to stackdriver the console object has been added to the server side.
code.gs
console.log('This will log to stackdriver')
Check out the docs for all the methods of console.
https://developers.google.com/apps-script/guides/logging#stackdriver_logging
Example from the docs:
function measuringExecutionTime() {
// A simple INFO log message, using sprintf() formatting.
console.info('Timing the %s function (%d arguments)', 'myFunction', 1);
// Log a JSON object at a DEBUG level. The log is labeled
// with the message string in the log viewer, and the JSON content
// is displayed in the expanded log structure under "structPayload".
var parameters = {
isValid: true,
content: 'some string',
timestamp: new Date()
};
console.log({message: 'Function Input', initialData: parameters});
var label = 'myFunction() time'; // Labels the timing log entry.
console.time(label); // Starts the timer.
try {
myFunction(parameters); // Function to time.
} catch (e) {
// Logs an ERROR message.
console.error('myFunction() yielded an error: ' + e);
}
console.timeEnd(label);
}
In addition you can also check Log Exceptions in the scripts properties. This will generate a stackdriver entry every time any error occurs in your script.
Error recovery in a web app
To recover in a web app from a failure you have access to the withFailureHandler() method found in the google.script.run object. With this you can register a callback in the event your script hits an exception.
Full documentation can be found at:
https://developers.google.com/apps-script/guides/html/reference/run
If you are doing server side checks with try...catch you may be getting an exception but gracefully handling it. In this case withFailureHandler() will not execute and onSuccessHandler() propably isnt the best place to handle errors. In the EAP there is now a withLogger method to google.script.run. For now there no documentation for google.script.run.withLogger(). I found it by digging through devtools. withLogger() allows you to register a function as a callback when ever a stackdriver entry is created. This is particularly helpful when you have log exceptions checked in your script properties. In this sense it is a bit like withFailureHandler() but it can be triggered by any stackdriver entry you add though the server-side console object.
index.html
<script>
google.script.run
.withSuccessHandler(function(){console.log('OK')})
.withFailureHandler(function(e){console.error(e)})
.withLogger(function(e){console.warn("The following log was generated:"+e)})
.serverFunctionCall();
</script>
code.gs
function serverFunctionCall(){
console.log("This log will generate a callback");
return true;
}
Try/catch at global scope will work, however any let/const container will not be globally exposed.
To fix this, you can use var within the try/catch at global scope
Consider the following:
function useCredits(userId, amount){
var userRef = firebase.database().ref().child('users').child(userId);
userRef.transaction(function(user) {
if (!user){
return user;
}
user.credits -= amount;
return user;
}, NOOP, false);
}
function notifyUser(userId, message){
var notificationId = Math.random();
var userNotificationRef = firebase.database().ref().child('users').child(userId).child('notifications').child(notificationId);
userNotificationRef.transaction(function(notification) {
return message;
}, NOOP, false);
}
These are called from the same node js process.
A user looks like this:
{
"name": 'Alex',
"age": 22,
"credits": 100,
"notifications": {
"1": "notification 1",
"2": "notification 2"
}
}
When I run my stress tests I notice that sometimes the user object passed to the userRef transaction update function is not the full user it is only the following:
{
"notifications": {
"1": "notification 1",
"2": "notification 2"
}
}
This obviously causes an Error because user.credits does not exist.
It is suspicious that the user object passed to update function of the userRef transaction is the same as the data returned by the userNotificationRef transaction's update function.
Why is this the case? This problem goes away if I run both transactions on the user parent location, but this is a less optimal solution as I am then effectively locking on and reading the whole user object, which is redundant when adding a write once notification.
In my experience, you can't rely on the initial value passed into a transaction update function. Even if the data is populated in the datastore, the function might be called with null, a partial value, or a stale old value (in case of a local update in flight). This is not usually a problem as long as you take a defensive approach when writing the function (and you should!), since the bogus update will be refused and the transaction retried.
But beware: if you abort the transaction (by returning undefined) because the data doesn't make sense, then it's not checked against the server and won't get retried. For this reason, I recommend never aborting transactions. I built a monkey patch to apply this fix (and others) transparently; it's browser-only but could be adapted to Node trivially.
Another thing you can do to help a bit is to insert an on('value') call on the same ref just before the transaction and keep it alive until the transaction completes. This will usually cause the transaction to run on the correct data on the first try, doesn't affect bandwidth too much (since the current value would need to be transmitted anyway), and increases local latency a little if you have applyLocally set or defaulting to true. I do this in my NodeFire library, among many other optimizations and tweaks.
On top of all the above, as of this writing there's still a bug in the SDK where very rarely the wrong base value will get "stuck" and the transaction retry continuously (failing with maxretry every so often) until you restart the process.
Good luck! I still use transactions in my server, where failures can be retried easily and I have multiple processes running, but have given up on using them on the client -- they're just too unreliable. In my opinion it's often better to redesign your data structures so that transactions aren't needed.
Does socket.io ignore\drop them?
The reason why Im asking this is the following.
There is a client with several states. Each state has its own set of socket handlers. At different moments server notifies the client of state change and after that sends several state dependent messages.
But! It takes some time for the client to change state and to set new handlers. In this case client can miss some msgs... because there are no handlers at the moment.
If I understand correctly unhandled msgs are lost for client.
May be I miss the concept or do smth wrong... How to hanle this issues?
Unhandled messages are just ignored. It's just like when an event occurs and there are no event listeners for that event. The socket receives the msg and doesn't find a handler for it so nothing happens with it.
You could avoid missing messages by always having the handlers installed and then deciding in the handlers (based on other state) whether to do anything with the message or not.
jfriend00's answer is a good one, and you are probably fine just leaving the handlers in place and using logic in the callback to ignore events as needed. If you really want to manage the unhandled packets though, read on...
You can get the list of callbacks from the socket internals, and use it to compare to the incoming message header. This client-side code will do just that.
// Save a copy of the onevent function
socket._onevent = socket.onevent;
// Replace the onevent function with a handler that captures all messages
socket.onevent = function (packet) {
// Compare the list of callbacks to the incoming event name
if( !Object.keys(socket._callbacks).map(x => x.substr(1)).includes(packet.data[0]) ) {
console.log(`WARNING: Unhandled Event: ${packet.data}`);
}
socket._onevent.apply(socket, Array.prototype.slice.call(arguments));
};
The object socket._callbacks contains the callbacks and the keys are the names. They have a $ prepended to them, so you can trim that off the entire list by mapping substring(1) onto it. That results in a nice clean list of event names.
IMPORTANT NOTE: Normally you should not attempt to externally modify any object member starting with an underscore. Also, expect that any data in it is unstable. The underscore indicates it is for internal use in that object, class or function. Though this object is not stable, it should be up to date enough for us to use it, and we aren't modifying it directly.
The event name is stored in the first entry under packet.data. Just check to see if it is in the list, and raise the alarm if it is not. Now when you send an event from the server the client does not know it will note it in the browser console.
Now you need to save the unhandled messages in a buffer, to play back once the handlers are available again. So to expand on our client-side code from before...
// Save a copy of the onevent function
socket._onevent = socket.onevent;
// Make buffer and configure buffer timings
socket._packetBuffer = [];
socket._packetBufferWaitTime = 1000; // in milliseconds
socket._packetBufferPopDelay = 50; // in milliseconds
function isPacketUnhandled(packet) {
return !Object.keys(socket._callbacks).map(x => x.substr(1)).includes(packet.data[0]);
}
// Define the function that will process the buffer
socket._packetBufferHandler = function(packet) {
if( isPacketUnhandled(packet) ) {
// packet can't be processed yet, restart wait cycle
socket._packetBuffer.push(packet);
console.log(`packet handling not completed, retrying`)
setTimeout(socket._packetBufferHandler, socket._packetBufferWaitTime, socket._packetBuffer.pop());
}
else {
// packet can be processed now, start going through buffer
socket._onevent.apply(socket, Array.prototype.slice.call(arguments));
if(socket._packetBuffer.length > 0) {
setTimeout(socket._packetBufferHandler,socket._packetBufferPopDelay(), socket._packetBuffer.pop());
}
else {
console.log(`all packets in buffer processed`)
socket._packetsWaiting = false;
}
}
}
// Replace the onevent function with a handler that captures all messages
socket.onevent = function (packet) {
// Compare the list of callbacks to the incoming event name
if( isPacketUnhandled(packet) ) {
console.log(`WARNING: Unhandled Event: ${packet.data}`);
socket._packetBuffer.push(packet);
if(!socket._packetsWaiting) {
socket._packetsWaiting = true;
setTimeout(socket._packetBufferHandler, socket._packetBufferWaitTime, socket._packetBuffer.pop());
}
}
socket._onevent.apply(socket, Array.prototype.slice.call(arguments));
};
Here the unhandled packets get pushed into the buffer and a timer is set running. Once the given amount of time has passed, if starts checking to see if the handlers for each item are ready. Each one is handled until all are exhausted or a handler is missing, which trigger another wait.
This can and will stack up unhandled calls until you blow out the client's allotted memory, so make sure that those handlers DO get loaded in a reasonable time span. And take care not to send it anything that will never get handled, because it will keep trying forever.
I tested it with really long strings and it was able to push them through, so what they are calling 'packet' is probably not a standard packet.
Tested with SocketIO version 2.2.0 on Chrome.