This is my first question.
I am trying to build a websocket server that can handle 50-100 concurrent users and send 50 messages per second.
On my dev machine it works for one user but when uploaded on the server it slows down more and more depending on the users.
Thanks for your time.
superwebsocket code sample:
public class SuperWs
{
WebSocketServer server;
WebSocketSession[] array = new WebSocketSession[100];
public SuperWs()
{
server = new WebSocketServer();
server.Setup(new ServerConfig
{
Port = 23023,
Name = "super web socket",
LogAllSocketException = true,
LogBasicSessionActivity = true,
});
server.NewSessionConnected += NewSessionConnected;
server.NewMessageReceived += NewMessageReceived;
server.SessionClosed += SessionClosed;
server.Start();
Task.Run(() =>
{
while (true)
{
Thread.Sleep(1000);
Stopwatch sw = new Stopwatch();
sw.Start();
for (int i = 0; i < 50; i++)
{
for (int j = 0; j < array.Length; j++)
{
if (array[j] != null)
array[j].Send(string.Format("Hello! ms: {0}, mes: {1}", sw.ElapsedMilliseconds, i));
}
}
sw.Stop();
}
});
}
private void SessionClosed(WebSocketSession session, CloseReason value)
{
Console.WriteLine("{0} {1}", session.SessionID, value);
}
void NewMessageReceived(WebSocketSession session, string value)
{
Console.WriteLine("{0} {1}", session.SessionID, value);
}
private int count;
private void NewSessionConnected(WebSocketSession session)
{
Console.WriteLine(session);
array[count++] = session;
}
}
First, when client and server are not in the same machine (like in your DEV environment), network will cause some delay. If you have 'n' clients connected, it will take 'n * delay' to send a message to all nodes. You should create a different loop per session, so basically on NewSessionConnected create a new loop that sends 50 messages each second and ends when the session is disconnected (rather than while(true)).
If you are going to have that parallel work, you should use async programming and use await Task.Delay(1000) rather than Thread.Sleep(1000), since the latest will block the thread, where the previous will allow the thread to go and do something else in the mean time. Unfortunately SuperWebSocket does not support async programming.
Pseudo-code:
private async void NewSessionConnected(WebSocketSession session)
{
while (session.IsConnected)
{
await Task.Delay(1000 / 50);
session.Send("Hello");
}
}
Also, you have to start thinking what are you going to do when nodes disconnect :)
As a side note, I develop a WebSocket framework that supports async programming, that amount of traffic should be no problem.
Related
Context
I am getting high memory usage with the vertx circuit breaker. I am just using httpbin.org to get all success responses. For individual requests, it works fine. While running a load test the JVM old gen utilization is spiking up.
Reproducer
I have the main verticle code, pasting it here itself:
public class CleanServer extends AbstractVerticle {
Logger logger = Logger.getLogger(CleanServer.class.getName());
#Override
public void start(Promise<Void> startPromise) throws Exception {
Router router = Router.router(vertx);
CircuitBreakerCache cbc = new CircuitBreakerCache(vertx);
router.route(HttpMethod.GET, "/get").handler(context -> {
List<String> domains = context.queryParam("user");
String domain = domains.get(0);
CircuitBreaker cb = cbc.getCircuitBreaker(domain + context.request().path());
HttpServerResponse serverResponse =
context.response().setChunked(true);
cb.executeWithFallback(promise -> {
WebClientOptions options = new WebClientOptions().setTryUseCompression(true).setTcpNoDelay(true).setTcpCork(true).setReceiveBufferSize(128).setConnectTimeout(400);
WebClient client = WebClient.create(vertx, options);
client.get(80, "httpbin.org", "/status/200")
.timeout(2000)
.send(ar -> {
if (ar.succeeded()) {
HttpResponse<Buffer> response = ar.result();
int statusCode = response.statusCode();
if (statusCode != 200) {
promise.fail(response.statusMessage());
} else {
serverResponse.end("Hello!!");
promise.complete();
}
} else {
promise.fail(ar.cause().getMessage());
}
});
}, v -> {
// Executed when the circuit is opened
logger.log(Level.INFO, domain + " Failed " + cb.state().toString() + " Error: Circuit open");
serverResponse.setStatusCode(200).setStatusMessage("Circuit Open").end("Circuit Open");
return context;
});
});
// Create the HTTP server
vertx.createHttpServer(new HttpServerOptions().setMaxInitialLineLength(10000))
// Handle every request using the router
.requestHandler(router)
// Start listening
.listen(8080)
// Print the port
.onSuccess(server ->
System.out.println(
"HTTP server started on port " + server.actualPort()
)
);
}
}
Circuit breaker options:
CircuitBreakerOptions()
.setMaxFailures(50)
.setTimeout(5000)
.setFallbackOnFailure(true)
.setResetTimeout(10000)));
Steps to reproduce
API used: http://localhost:8080/get?user=abc
When I hit the above API at 50 QPS for 30 minutes. The java heap is getting filled up.
Extra
<vertx.version>4.2.6</vertx.version>
JVM params used:
-XX:+UseG1GC -Xms4g -Xmx4g -XX:InitiatingHeapOccupancyPercent=70 -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5
JVM memory with the load test.
Error:
WARNING: Thread Thread[vert.x-eventloop-thread-3,5,main] has been blocked for 3050 ms, time limit is 2000 ms
I think I am blocking the thread somewhere but not sure where exactly as the code seems pretty simple as given in the documentation.
my company choose "Mercure" (https://mercure.rocks/docs/getting-started) to manage Server-Sent Events.
We install "Mercure HUB" on a server and now, in C# .NET 5.0, I must implement the server-side (publisher, that I already implemented) and the client-side (subscriber).
The subscriber must be done with a WPF
From the "getting-started" page I can see a Javascript example that I need to transform into C#
I don't know how to manage a "EventSource" in C#
Any ideas ?
// The subscriber subscribes to updates for the https://example.com/users/dunglas topic
// and to any topic matching https://example.com/books/{id}
const url = new URL('https://localhost/.well-known/mercure');
url.searchParams.append('topic', 'https://example.com/books/{id}');
url.searchParams.append('topic', 'https://example.com/users/dunglas');
// The URL class is a convenient way to generate URLs such as https://localhost/.well-known/mercure?topic=https://example.com/books/{id}&topic=https://example.com/users/dunglas
const eventSource = new EventSource(url);
// The callback will be called every time an update is published
eventSource.onmessage = e => console.log(e); // do something with the payload
The code of this page works (https://makolyte.com/event-driven-dotnet-how-to-consume-an-sse-endpoint-with-httpclient/)
static async Task Main(string[] args)
{
HttpClient client = new HttpClient();
client.Timeout = TimeSpan.FromSeconds(5);
string stockSymbol = "VTSAX";
string url = $"http://localhost:9000/stockpriceupdates/{stockSymbol}";
while (true)
{
try
{
Console.WriteLine("Establishing connection");
using (var streamReader = new StreamReader(await client.GetStreamAsync(url)))
{
while (!streamReader.EndOfStream)
{
var message = await streamReader.ReadLineAsync();
Console.WriteLine($"Received price update: {message}");
}
}
}
catch(Exception ex)
{
//Here you can check for
//specific types of errors before continuing
//Since this is a simple example, i'm always going to retry
Console.WriteLine($"Error: {ex.Message}");
Console.WriteLine("Retrying in 5 seconds");
await Task.Delay(TimeSpan.FromSeconds(5));
}
}
}
I am currently trying to setup a server stream with the gRPC Node.js API. For that I want to achieve that when I write on server side to the stream that the client immediately receives the data event.
At the moment I don't receive anything on client side if I only call write on server side. However as soon as I call the end function on the server the client receives all data events.
To test this I used an endless while loop for writing messages on server side. Then the client does not receive messages (data events). If instead I use a for loop and call end afterwards the client receives all the messages (data events) when end is called.
My .proto file:
syntax = "proto3";
message ControlMessage {
enum Control {
Undefined = 0;
Start = 1;
Stop = 2;
}
Control control = 1;
}
message ImageMessage {
enum ImageType {
Raw = 0;
Mono8 = 1;
RGB8 = 2;
}
ImageType type = 1;
int32 width = 2;
int32 height = 3;
bytes image = 4;
}
service StartImageTransmission {
rpc Start(ControlMessage) returns (stream ImageMessage);
}
On the server side I implement the start function and try to endlessly write messages to the call:
function doStart(call) {
var imgMsg = {type: "Mono8", width: 600, height: 600, image: new ArrayBuffer(600*600)};
//for(var i = 0; i < 10; i++) {
while(true) {
call.write(imgMsg);
console.log("Message sent");
}
call.end();
}
I register the function as service in the server:
var server = new grpc.Server();
server.addService(protoDescriptor.StartImageTransmission.service, {Start: doStart});
On client side I generate an appropriate call and register the data and end event:
var call = client.Start({control: 0});
call.on('data', (imgMessage) => {
console.log('received image message');
});
call.read();
call.on('end', () => {console.log('end');});
I also tried to write the server side in python. In this case the node client instantly receives messages and not only after stream was ended on server side. So I guess this should be also possible for the server written with the Node API.
It seems that the problem was that the endless while loop is blocking all background tasks in node. A possible solution is to use setTimeout to create the loop. The following code worked for me:
First in the gRPC call store the call object in an array:
function doStart(call) {
calls.push(call);
}
For sending to all clients I use a setTimeout:
function sendToAllClients() {
calls.forEach((call) => {
call.write(imgMsg);
});
setTimeout(sendToAllClients, 10);
}
setTimeout(sendToAllClients, 10);
Helpful stackoverflow atricle: Why does a while loop block the event loop?
I was able to use uncork which comes from Node.js's Writable.
Here is an example. Pseudocode, but pulled from across a working implementation:
import * as grpc from '#grpc/grpc-js';
import * as proto from './src/proto/generated/organizations'; // via protoc w/ ts-proto
const OrganizationsGrpcServer: proto.OrganizationsServer = {
async getMany(call: ServerWritableStream<proto.Empty, proto.OrganizationCollection>) {
call.write(proto.OrganizationCollection.fromJSON({ value: [{}] }));
call.uncork();
// do some blocking stuff
call.write(proto.OrganizationCollection.fromJSON({ value: [{}] }));
call.uncork();
// call.end(), or client.close() below, at some point?
},
ping(call, callback) {
callback(null);
}
};
const client = new proto.OrganizationsClient('127.0.0.1:5000', grpc.credentials.createInsecure());
const stream = client.getMany(null);
stream.on('data', data => {
// this cb should run twice
});
export default OrganizationsGrpcServer;
//.proto
service Organizations {
rpc GetMany (google.protobuf.Empty) returns (stream OrganizationCollection) {}
}
message OrganizationCollection {
repeated Organization value = 1;
}
Versions:
#grpc/grpc-js 1.4.4
#grpc/proto-loader 0.6.7
ts-proto 1.92.1
npm 8.1.4
node 17
I have a client with written c# and a server with written java. I capture audio and send with socket to the server and server send with web socket to the browser and want to play with browser. But when i try browser says Uncaught (in promise) DOMException: Failed to load because no supported source was found.
Could you help me?
private static void Recordwav()
{
waveInEvent = new WaveInEvent();
int devicenum = 0;
for (int i = 0; i < WaveIn.DeviceCount; i++)
{
if (WaveIn.GetCapabilities(i).ProductName.Contains("icrophone"))
devicenum = i;
}
waveInEvent.DeviceNumber = devicenum;
waveInEvent.WaveFormat = new WaveFormat(44100, WaveIn.GetCapabilities(devicenum).Channels);
waveInEvent.DataAvailable += new EventHandler<WaveInEventArgs>(VoiceDataAvailable);
waveInEvent.StartRecording();
}
private static void VoiceDataAvailable(object sender, WaveInEventArgs e)
{
JObject jObject = new JObject();
jObject["voice"] = Convert.ToBase64String(e.Buffer);
byte[] messageByte = Encoding.ASCII.GetBytes(jObject.ToString().Replace("\r\n", "") + "\n");
socket.Send(messageByte);
}
$scope.socket.onmessage = function (response)
{
var data = JSON.parse(response.data);
if(data.id == $scope.id) {
if(data.voice) {
var voice = data.voice;
var sound = new Audio("data:audio/wav;base64," + voice);
sound.play();
}
}
};
you're just sending raw samples, not a properly formatted WAV file. You'd need to use WaveFileWriter to write to a MemoryStream (wrapped in an IgnoreDisposeStream) dispose the WaveFileWriter and then access the MemoryStream underlying byte array. Also you're not taking into account BytesRecorded.
Even if you get this working, I suspect you'll get very choppy audio, as each WAV file will be a few hundred ms, and they won't necessarily play perfectly one after the other.
I'm using RabbitMQ and web-stomp in the web browser according to this tutorial:
https://www.rabbitmq.com/web-stomp.htm
I succeded to connect and the get messages in the browser.
But,
the message I sent and consumed in the client is still in the queue and not being dequeing(I did manual ack and auto ack), it still exists.
when I subscribe to a queue I'm not getting all the messages in the queue, but only the last.. only when the websocket is open and then the server send the message i get the last message but not the old ones.
The server Code:
private static final String EXCHANGE_NAME = "amq.topic";
public static void AddToQueue(String RoutingKey, String message) throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
channel.basicPublish(EXCHANGE_NAME, RoutingKey, null, message.getBytes());
channel.close();
connection.close();
}
The client code:
var ws = new SockJS('http://' + window.location.hostname + ':15674/stomp');
$scope.client = Stomp.over(ws);
$scope.client.heartbeat.outgoing = 0;
$scope.client.heartbeat.incoming = 0;
var on_connect = function(x) {
$scope.client.subscribe("/topic/status", function(d) {
console.log(d.body);
});
};
var on_error = function() {
console.log('error');
};
$scope.client.connect('guest', 'guest', on_connect, on_error, '/');
Thanks.
Solved it, the exchange name needs to be "amq.topic"