I use RPC endpoints and in one of them I have the following problem: I do not receive a message so the callback function is not executed on channel.consume().
At that endpoint, I send a message, a process that takes time is running on the server side and it responds to me with a message about whether the process was executed correctly. At other endpoints where the message is sent immediately by the server there is no problem.
I think there is a problem with the timeout. I tried to place the object {timeout: 3600000} after amqpOptions but again the problem was not solved. Specifically, the connection and channel objects have the same parameters regardless of the object I added. How could I change the timeout correctly?
const amqp = require('amqplib/callback_api');
const amqpOptions = {
protocol: 'amqp',
hostname: process.env.RABBITMQ_HOST,
port: process.env.RABBITMQ_PORT,
username: process.env.RABBITMQ_USER,
password: process.env.RABBITMQ_PASS,
vhost: '/',
};
const message = Buffer.from(JSON.stringify({}));
amqp.connect(amqpOptions, (error0, connection) => {
if (error0) { throw error0; }
connection.createChannel((error1, channel) => {
if (error1) { throw error1; }
const correlationId = generateUuid();
channel.consume(replyQueue, (msg) => {
if (JSON.parse(msg.content).error) {
console.log(JSON.parse(msg.content));
const error = JSON.parse(msg.content.toString());
return next(error);
}
console.log(JSON.parse(msg.content));
console.log('msg:',msg);
const {tunnel_info} = JSON.parse(msg.content.toString());
}, {noAck: true});
channel.sendToQueue(`${brokerUri}`,
message, {correlationId, contentType: 'application/json', contentEncoding: 'utf8', replyTo: replyQueue});
});
});
Because channel is unidirectional. You should use two different channels for Publish and Consume.
AMQP specification says:
Channels are unidirectional, and thus at each connection endpoint the incoming and outgoing channels are completely distinct.
Related
I am working on Azure service bus topic. Following the documentation, created a sender and reciever code.
This is the sender code i am having,
const { ServiceBusClient } = require("#azure/service-bus");
const connectionString = "<SERVICE BUS NAMESPACE CONNECTION STRING>"
const topicName = "<TOPIC NAME>";
const messages = [
{ body: "Albert Einstein" },
{ body: "Werner Heisenberg" },
{ body: "Marie Curie" },
{ body: "Steven Hawking" },
{ body: "Isaac Newton" },
{ body: "Niels Bohr" },
{ body: "Michael Faraday" },
{ body: "Galileo Galilei" },
{ body: "Johannes Kepler" },
{ body: "Nikolaus Kopernikus" }
];
async function main() {
// create a Service Bus client using the connection string to the Service Bus namespace
const sbClient = new ServiceBusClient(connectionString);
// createSender() can also be used to create a sender for a queue.
const sender = sbClient.createSender(topicName);
try {
// Tries to send all messages in a single batch.
// Will fail if the messages cannot fit in a batch.
// await sender.sendMessages(messages);
// create a batch object
let batch = await sender.createMessageBatch();
for (let i = 0; i < messages.length; i++) {
// for each message in the arry
// try to add the message to the batch
if (!batch.tryAddMessage(messages[i])) {
// if it fails to add the message to the current batch
// send the current batch as it is full
await sender.sendMessages(batch);
// then, create a new batch
batch = await sender.createMessageBatch();
// now, add the message failed to be added to the previous batch to this batch
if (!batch.tryAddMessage(messages[i])) {
// if it still can't be added to the batch, the message is probably too big to fit in a batch
throw new Error("Message too big to fit in a batch");
}
}
}
// Send the last created batch of messages to the topic
await sender.sendMessages(batch);
console.log(`Sent a batch of messages to the topic: ${topicName}`);
// Close the sender
await sender.close();
} finally {
await sbClient.close();
}
}
// call the main function
main().catch((err) => {
console.log("Error occurred: ", err);
process.exit(1);
});
This code is working fine, but instead of sending a batch of dummy data to the service bus topic i want to implement my use case here.
My use case is I will be using this sender code in a react front end application, where there is a node API call happening at the end of a form submission. So at the end of form submission, i will send that unique form ID to the topic and i need to somehow trigger the api call for that form id.
I am unable to connect the dots. How to do this?
Added reciever side code.
const { delay, ServiceBusClient, ServiceBusMessage } = require("#azure/service-bus");
const axios = require("axios").default;
const connectionString = "<ConnectionString>"
const topicName = "<TopicName>";
const subscriptionName = "<Subscription>";
async function main() {
// create a Service Bus client using the connection string to the Service Bus namespace
const sbClient = new ServiceBusClient(connectionString);
// createReceiver() can also be used to create a receiver for a queue.
const receiver = sbClient.createReceiver(topicName, subscriptionName);
// function to handle messages
const myMessageHandler = async (messageReceived) => {
console.log(`Received message: ${messageReceived.body}`);
const response = axios({
method: 'post',
url: 'http://localhost:8080/gitWrite?userprojectid=63874e2e3981e40a6f4e04a7',
});
console.log(response);
};
// function to handle any errors
const myErrorHandler = async (error) => {
console.log(error);
};
// subscribe and specify the message and error handlers
receiver.subscribe({
processMessage: myMessageHandler,
processError: myErrorHandler
});
// Waiting long enough before closing the sender to send messages
await delay(5000);
await receiver.close();
await sbClient.close();
}
// call the main function
main().catch((err) => {
console.log("Error occurred: ", err);
process.exit(1);
});
While messages are published to a topic, they are recieved by subscriptions under the topic. You'll need to define one or more subscriptions to receive the messages. That's on the broker. For your code, you'll need a receiving code on the server-side/backend. Could be something like a node.js service or Azure Function. But a code that would receive from the subscription(s).
I would review the idea of publishing messages from the client side directly to Azure Service Bus. If the code is a React front end application, make sure the connection string is not embedded in resources or can be revealed.
we are using Amazon Connect and we are running into an issue.
On the service-desk, we have a "standby shift", this means the user that is in that queue/shift is the only on-call. However, if he/she misses said call. They are stuck in the "Missed" state within the Connect CCP. I already tried the custom CCP method, however this does not fit our requirements yet.
Our second option is making a lambda that gets executed on a missed call (Right before the disconnect of a user) to remove the "Missed" state and set it on "Available". This is what I tried to do so:
var AWS = require('aws-sdk');
exports.handler = async (event) => {
var connect = new AWS.Connect();
let agentARN = event['Details']['Parameters']['agent'];
var params = {
AgentStatusId: 'Available',
InstanceId: 'arn:aws:connect:eu-central-1:ID:ARN',
UserId: agentARN
};
let errors;
connect.putUserStatus(params, function(err, data) {
if (err) errors = err.stack; // an error occurred
else errors = data; // successful response
});
const response = {
statusCode: 200,
body: errors
};
return response;
};
Any ideas?
I have running an app in heroku, separate in back and front
back: node + express + mysql
front: vue
the app works fine, but I have an error random: sometimes i have duplicates records inserted from frontend. (I guess the error comes from the front)
from the frontend I use fetch to add the records
const requestOptions = {
method: "POST",
headers: {
"Accept": "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify({
data: data,
...: ...,
}),
};
const response = await fetch(`url_backend_heroku/api/add`, requestOptions);
the records are inserted correctly , but sometimes are inserted duplicated with correct next id
Could it be that fetch is sending 2 requests in some cases?
before deploying in heroku on my local machine I never duplicate records
I've been going around for days and I can't find why this happens
Yeah it is possible you are sending 2 requests somewhere. Put logs in heroku BACK on the specific endpoint to see whats exactly happening.
Also while requesting from Front end check your network tab in developer tools to see if you are actually firing the request twice. Because as you said duplicate records but they have correct ID's , could only mean what you said.
Also, this might or might not be true, but heroku server sleeps on inactivity, so it is possible that might cause an issue but I am not entirely sure on that, will have to check the code and environment for that.
looking at the heroku (back) log it looks like 2 records were inserted normally
looking at the network tab in the browser, only 1 request appears: OPTIONS (204) and POST (200)
the table has an id that is the primary key nothing complicated
on the other hand I am on a Dynos Hobby plan that does not have sleep times (if the free)
put here part or my backend
database.js
const mysql = require('mysql')
const { promisify } = require('util')
const config = { database keys }
const pool = mysql.createPool(config);
pool.getConnection((err: any, connection: any) => {
if (err) {
if (err.code === 'PROTOCOL_CONNECTION_LOST') {
console.error('DATABASE CONNECTION WAS CLOSED')
}
if (err.code === 'ER_CON_COUNT_ERROR') {
console.error('DATABASE HAS TO MANY CONNECTIONS')
}
if (err.code === 'ECONNREFUSED') {
console.error('DATABASE CONNECTION WAS REFUSED')
}
}
if (connection) connection.release()
console.log('DB is Connected')
return
})
pool.query = promisify(pool.query)
export default pool
controller.js
import pool from '../database';
public async insert(req: Request, res: Response) {
let body = req.body
try {
const response = await pool.query('SELECT MAX(id) + 1 as idNew FROM table')
let data = {Id: response[0].idNew, ...body}
//before it had an auto_increment id and it also failed
const result = await pool.query('INSERT INTO table set ?', [data])
res.json({
insertId: response[0].idNew,
message: "Saved OK"
})
} catch (error) {
console.log("error", error)
return res.status(500).send("error")
}
}
can it be a fetch problem? don't try yet to use axios for example
I am trying to send images captured from a canvas to my NodeJS backend server using the WebRTC data channel. That is I am trying to make my server a peer. But for some reason, I am unable to establish a connection.
Client Side
async function initChannel()
{
const offer = await peer.createOffer();
await peer.setLocalDescription(offer);
const response = await fetch("/connect", {
headers: {
'Content-Type': 'application/json',
},
method: 'post',
body: JSON.stringify({ sdp: offer, id: Math.random() })
}).then((res) => res.json());
peer.setRemoteDescription(response.sdp);
const imageChannel = peer.createDataChannel("imageChannel", { ordered: false, maxPacketLifeTime: 100 });
peer.addEventListener("icecandidate", console.log);
peer.addEventListener("icegatheringstatechange",console.log);
// drawCanvas function draws images got from the server.
imageChannel.addEventListener("message", message => drawCanvas(remoteCanvasCtx, message.data, imageChannel));
// captureImage function captures and sends image to server using imageChannel.send()
imageChannel.addEventListener("open", () => captureImage(recordCanvasCtx, recordCanvas, imageChannel));
}
const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
initChannel();
Here both captureImage and drawCanvas are not being invoked.
Server Side
import webrtc from "wrtc"; // The wrtc module ( npm i wrtc )
function handleChannel(channel)
{
console.log(channel.label); // This function is not being called.
}
app.use(express.static(resolve(__dirname, "public")))
.use(bodyParser.json())
.use(bodyParser.urlencoded({ extended: true }));
app.post("/connect", async ({ body }, res) =>
{
console.log("Connecting to client...");
let answer, id = body.id;
const peer = new webrtc.RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
await peer.setLocalDescription(answer = await peer.createAnswer());
peer.addEventListener("datachannel",handleChannel)
return res.json({ sdp: answer });
});
app.listen(process.env.PORT || 2000);
Here the post request is handled fine but handleChannel is never called.
When I run this I don't get any errors but when I check the connection status it shows "new" forever. I console logged remote and local description and they seem to be all set.
What am I doing wrong here?
I am pretty new to WebRTC and I am not even sure if this is the correct approach to continuously send images (frames of user's webcam feed) to and back from the server, if anyone can tell me a better way please do.
And one more thing, how can I send image blobs ( got from canvas.toBlob() ) via the data channel with low latency.
I finally figured this out with the help of a friend of mine. The problem was that I have to create DataChannel before calling peer.createOffer(). peer.onnegotiationneeded callback is only called once the a channel is created. Usually this happens when you create a media channel ( either audio or video ) by passing a stream to WebRTC, but here since I am not using them I have to to this this way.
Client Side
const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
const imageChannel = peer.createDataChannel("imageChannel");
imageChannel.onmessage = ({ data }) =>
{
// Do something with received data.
};
imageChannel.onopen = () => imageChannel.send(imageData);// Data channel opened, start sending data.
peer.onnegotiationneeded = initChannel
async function initChannel()
{
const offer = await peer.createOffer();
await peer.setLocalDescription(offer);
// Send offer and fetch answer from the server
const { sdp } = await fetch("/connect", {
headers: {
"Content-Type": "application/json",
},
method: "post",
body: JSON.stringify({ sdp: peer.localDescription }),
})
.then(res => res.json());
peer.setRemoteDescription(new RTCSessionDescription(sdp));
}
Server
Receive offer from client sent via post request. Create an answer for it and send as response.
app.post('/connect', async ({ body }, res) =>
{
const peer = new webrtc.RTCPeerConnection({
iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
});
console.log('Connecting to client...');
peer.ondatachannel = handleChannel;
await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
await peer.setLocalDescription(await peer.createAnswer());
return res.json({ sdp: peer.localDescription });
});
The function to handle data channel.
/**
* This function is called once a data channel is ready.
*
* #param {{ type: 'datachannel', channel: RTCDataChannel }} event
*/
function handleChannel({ channel })
{
channel.addEventListener("message", {data} =>
{
// Do something with data received from client.
});
// Can use the channel to send data to client.
channel.send("Hi from server");
}
So here is what happens :
Client creates a Data-Channel.
Once data channel is created onnegotiationneeded callback is called.
The client creates an offer and sends it to the server (as post request).
Server receives the offer and creates an answer.
Server sends the answer back to the client (as post response).
Client completes the initialization using the received answer.
ondatachannel callback gets called on the server and the client.
I have used post request here to exchange offer and answer but it should be fairly easy to do the same using Web Socket if that is what you prefer.
I'm using Serverless Framework to host my WebSocket, which has the typical $connect, $disconnect, $default, etc methods that updates my connections db:
case '$connect':
await dynamoDb.put({
TableName: process.env.CONNECTIONS_TABLE,
Item: {
connectionId,
// Expire the connection an hour later. This is optional, but recommended.
// You will have to decide how often to time out and/or refresh the ttl.
ttl: parseInt((Date.now() / 1000) + 3600)
}
}).promise();
My WebSocket setup is:
WebSocket URL: wss://1111111111.execute-api.ap-southeast-2.amazonaws.com/dev/
Connection URL: https://1111111111.execute-api.ap-southeast-2.amazonaws.com/dev/#connections
My HTTP setup is:
Invoke API at: https://222222222.execute-api.ap-southeast-2.amazonaws.com/dev/
I have a broadcast function which I am using to send data to the connections, which I am invoking with:
sls invoke --function broadcast --data '{ \"body\": \"Hello from server\" }'
The source sends a message to each connection, as provided in the params of the request:
async function sendMessage(connectionId, body) {
try {
await apig.postToConnection({
ConnectionId: connectionId,
Data: body
}).promise();
} catch (err) {
// Ignore if connection no longer exists
if(err.statusCode !== 400 && err.statusCode !== 410) {
throw err;
}
}
}
async function getAllConnections(ExclusiveStartKey) {
const { Items, LastEvaluatedKey } = await dynamoDb.scan({
TableName: process.env.CONNECTIONS_TABLE,
AttributesToGet: [ 'connectionId' ],
ExclusiveStartKey
}).promise();
const connections = Items.map(({ connectionId }) => connectionId);
if(LastEvaluatedKey) {
connections.push(...await getAllConnections(LastEvaluatedKey));
}
return connections;
}
module.exports.handler = async function(event, context) {
const { body } = event;
const connections = await getAllConnections();
await Promise.all(
connections.map(connectionId => sendMessage(connectionId, body))
);
}
A connection can be established (I can connect to the WebSocket), however when I try to invoke this function I am receiving the error:
No method found matching route #connections/ZE4SDfSJSwMCJ4g%3D for http method POST.
The ZE4SDfSJSwMCJ4g is my connectionId, which exists in my database. I am unsure if this routing issue has to do with my HTTP API and my WebSocket API being pointed at different API Gateway URL?
I appreciate the help!
Make sure that the endpoint in your API Gateway management configuration is the same as your ws endpoint.
const agma = new AWS.ApiGatewayManagementApi({
apiVersion: AGMA_VERSION,
endpoint: WS_ENDPOINT // 1111111111.execute-api.ap-southeast-2.amazonaws.com/dev
})
I too had this issue with websockets and google cloud functions. I think it's because the web socket server is put to sleep and doesn't actively listen because it's serverless.
Functions are configured to wake up on http requests (ports 80/443) and go back to sleep, so they need to be specially configured to wake up to requests on web socket port.
This seems possible using a serverless websockets plugin https://github.com/serverless/serverless-websockets-plugin there is a how to article at https://www.serverless.com/blog/api-gateway-websockets-example