Node.js and zmq - javascript

I have a strange issue with a basic pubsub application with node and zmq:
a client is publishing strings to a broker, the problem is that the broker only receives the first line. At network level I've noticed that only the first message is sent then the next calls to .send() function have no effect (no packets are sent) so I suppose the problem is in the client/publisher.
I used the example code provided in the official guide which works perfectly, the only difference in my code is that I use prototype to have a reusable structure.
(I didn't paste subscriber's code because is not relevant and took some other not relevant stuff out)
relevant part of the client/publisher:
Publisher = function(zmq, pport) {
this.logread = spawn('tail', ['-n0', '-f', '/var/log/auth.log']);
this.publisher = zmq.socket('req');
this.pport = pport;
};
Publisher.prototype.start = function() {
var self = this;
this.publisher.connect('tcp://127.0.0.1:' + this.pport);
this.logread.stdout.on('data', function(data){
self.publisher.send(data.toString());
console.log(data.toString());
});
};
relevant part of the broker:
Broker = function(zmq, bpport, bsport) {
this.server = zmq.socket('rep');
this.bpport = bpport;
this.bsport = bsport;
};
Broker.prototype.start = function() {
this.server.on('message', function(request) {
console.log(request.toString());
});
this.server.bind('tcp://127.0.0.1:' + this.bsport, function(err) {
if (err)
console.log(err);
});
};

You are talking about publish subscribe pattern, but in your code, you create a req socket, and in the broker a rep socket, which is for the request-reply pattern. The request-reply pattern is strictly need to send first, than receive, see the api docs docs, or read more from the guide
I suppose you should use pub socket on the client side, and a sub socket on the other side, but don't know what do you want to achieve, maybe a different pattern would fit your needs better.

so I'll answer my question:
the server must send a reply to the client, until then the client will not send more messages
server.send('OK');
I also suppose there is a different way to achieve this

Related

How to create sockets that particular to user and disposable on Socket.Io

I write a Node.Js app and I use Socket.Io as the data transfer system, so requests should be particular to per user. How can I make this?
My actual code;
node:
io.on('connection', (socket) => {
socket.on('loginP', data => {
console.log(data);
})
})
js:
var socket = io('',{forceNew : false});
$("#loginbutton").click(function() {
var sessionInfo = {
name : $("#login input[name='username']").val(),
pass : $("#login input[name='pass']").val()
}
socket.emit("loginP", sessionInfo)
})
It returns one more data for per request and this is a problem for me. Can I make this on Socket.Io or should I use another module, and If I should, which module?
If I understand your question correctly (It's possible I don't), you want to have just one connection from each user's browser to your nodejs program.
On the nodejs side, your io.on('connection'...) event fires with each new incoming user connection, and gives you the socket for that specific connection. So, keep track of your sockets; you'll have one socket per user.
On the browser side, you should build your code to ensure it only calls
var socket = io(path, ...);
once for each path (your path is ''). TheforceNew option is for situations where you have multiple paths from one program.

Is there some way of importing a variable(specifically map) from the server side onto the client side(Socket.io)?

I am trying to assign unique colors to each different client( by using socket.id ). In my map() I have paired (socket.id,randomcolor()), but this variable is on the server side. I've found out that the require() statement doesn't work on client side,
why is that and what is a solution to it? I want to be able to pass map() variable to the client side so that it uses the color assigned to that socket.id and displays the color accordingly.
Or is there some way to know the socket.id on the client side(I don't think it is but not sure), specifically a users computer has to know who sent the message i.e. what socket.id was used to send the message, Is it possible to know that?
Here's my server side:
var express = require('express');
var app = express();
app.use(express.static('public'))
var http = require('http').createServer(app);
var io = require('socket.io')(http);
const map = new Map();
io.on('connection', function(socket) {
console.log('connected by ' + socket.id);
map.set(socket.id, RandomColor())
socket.on('chat', function(data) {
//emitting to all sockets connected
io.emit('chat', data);
console.log(map.entries());
});
socket.on('typing', function(data) {
socket.broadcast.emit('typing', data);
})
});
http.listen(3000, function() {
console.log('listening on port 3000');
});
Here's client side :
// import '../index';
var socket = io.connect('http://localhost:3000')
var message = document.getElementById('Message');
var handle = document.getElementById('Handle');
var btn = document.getElementById('Send');
var output = document.getElementById('Output');
var feedback = document.getElementById('Feedback');
var ids = []
console.log(server);
//emit event
btn.addEventListener('click', function() {
socket.emit('chat', {
message: message.value,
handle: handle.value,
})
})
message.addEventListener('keypress', function() {
socket.emit('typing', handle.value)
})
messageArray = []
//listening for any message received
socket.on('chat', function(data) {
// console.log(data);
feedback.innerHTML = ""
var item = document.createElement('li')
item.innerHTML = "<span style=\"font-family:\"cursive\";\" ;><strong>" + data.handle + ": " + data.message + "</strong></span>";
document.getElementById('Output').appendChild(item)
})
//listening for any typing event listener
socket.on('typing', function(data) {
feedback.innerHTML = "<p><strong>" + data + " is typing a message </strong></p>";
})
PS: Also, I'm new to JS and Socket.io so please suggest some good practices for anything in the code.
First of all, JS has no built-in include/reference property.
So you can't just join another file into another file. But some libraries achieve this with their own written methods etc.
A JS executed on the client-side is not able to access local files. Although you may access an online file load into the document or to an object. So similar functionality can be achieved via 3rd party scripts.
Node.JS follows the CommonJS module system and uses the power of being able to access the local file system.
About the index: So you don't need a Map and Map is pretty similar to a standard object, main difference is might be the order of contents.
But since all you need is a dictionary object. Just create a simple object. Then you can emit the color index whenever you want.
const colorIndex = {}
colorIndex[socketID] = color
Each can set their color on client-side and send it to the server, on each update server has to update every other client about the color.
A client cannot know other clients otherwise wouldn't be secure and it doesn't work like that. It works more like you are calling someone and the server is a middle man that connecting you two.
So, create an object, store socket ids, nicknames, any other info you need. Keep it on serverside, on each message send all of them together with the message.
const users = {}
io.on('connection', function(socket) {
users[socket.id] = {//Add new user
color:RandomColor()
}
socket.on('chat', function(message) {
let u = users[socket.id];//Get user from index
let data = {//Create a message package
user:(u.username)?u.username:"Guest", //name of the user if set
color:u.color,//color of user
message
}
io.emit('chat', data );//Send
});
socket.on('setColor', function(color) {//User can update color
users[socket.id].color = color
});
socket.on('setname', function(name) {//User can update username
users[socket.id].username = name
});
});
So you probably get the idea. There are bunch of ways to achieve.
I don't think you could send that map as an argument, but you can't try creating an array of arrays and emit it to an event like io.emit(colors, array) and once you have it on the client side you can transform back to a map using something like map or reduce
RequireJS is responsible to handle dependencies and ensure that you have everything you need. It is a Javascript library which can work anywhere you use Javascript at, including your server and client-side. The reason it does not work on your client-side (which manifests in the error you see) is that it's not configured on your client-side.
You can read about configurating RequireJS as well.
However, if you set it up properly on your client-side, then there might still be issues, particularly if you try to use on your client-side something which is available on the server. Client-side is a browser, potentially very far from the server. Luckily there is a client API for Socket.IO.
EDIT
Server and client-side can share values in several ways:
WebSockets (a duplex protocol which should be chosen if available in most cases)
Push notifications
AJAX
Page load
Forever frame (that's a hack which should be avoided)

RabbitMQ amqp.node integration with nodejs express

The official RabbitMQ Javascript tutorials show usage of the amqp.node client library
amqp.connect('amqp://localhost', function(err, conn) {
conn.createChannel(function(err, ch) {
var q = 'hello';
ch.assertQueue(q, {durable: false});
// Note: on Node 6 Buffer.from(msg) should be used
ch.sendToQueue(q, new Buffer('Hello World!'));
console.log(" [x] Sent 'Hello World!'");
});
});
However, I find it's hard to reuse this code elsewhere. In particular, I don't know how to exports the channel object since it's in a callback. For example in my NodeJs/Express App:
app.post('/posts', (req, res) => {
-- Create a new Post
-- Publish a message saying that a new Post has been created
-- Another 'newsfeed' server consume that message and update the newsfeed table
// How do I reuse the channel 'ch' object from amqp.node here
});
Do you guys have any guidance on this one? Suggestion of other libraries is welcomed (Since I'm starting out, ease of use is what I considered the most important)
amqp.node is a low-level API set that does minimal translation from AMQP to Node.js. It's basically a driver that should be used from a more friendly API.
If you want a DIY solution, create an API that you can export from your module and manage the connection, channel and other objects from within that API file.
But I don't recommend doing it yourself. It's not easy to get things right.
I would suggest using a library like Rabbot (https://github.com/arobson/rabbot/) to handle this for you.
I've been using Rabbot for quite some time now, and I really like the way it works. It pushes the details of AMQP off to the side and lets me focus on the business value of my applications and the messaging patterns that I need, to build featurs.
As explained in the comments, you could use the module.exports to expose the newly created channel. Of course this will be overridden each time you create a new channel, unless you want to keep an array of channels or some other data structure.
Assuming this is in a script called channelCreator.js:
amqp.connect('amqp://localhost', function(err, conn) {
conn.createChannel(function(err, ch) {
var q = 'hello';
ch.assertQueue(q, {durable: false});
//this is where you can export the channel object
module.exports.channel = ch;
//moved the sending-code to some 'external script'
});
});
In the script where you may want to use the "exported" channel:
var channelCreator = require("<path>/channelCreator.js");
//this is where you can access the channel object:
if(channelCreator.channel){
channelCreator.channel.sendToQueue('QueueName', new Buffer('This is Some Message.'));
console.log(" [x] Sent 'Message'");
}
Hope this helps.

NodeJS Event Emitter Blocking Issue

I have a node application handling some ZeroMQ events coming from another application utilizing the Node-ZMQ bindings found here: https://github.com/JustinTulloss/zeromq.node
The issue I am running into is one of the operations from an event takes a long time to process and this appears to be blocking any other event from being processed during this time. Although the application is not currently clustered, doing so would only afford a few more threads and doesn't really solve the issue. I am wondering if there is a way of allowing for these async calls to not block other incoming requests while they process, and how I might go about implementing them.
Here is a highly condensed/contrived code example of what I am doing currently:
var zmq = require('zmq');
var zmqResponder = zmq.socket('rep');
var Client = require('node-rest-client').Client;
var client = new Client();
zmqResponder.on('message', function (msg, data) {
var parsed = JSON.parse(msg);
logging.info('ZMQ Request received: ' + parsed.event);
switch (parsed.event) {
case 'create':
//Typically short running process, not an issue
case 'update':
//Long running process this is the issue
serverRequest().then(function(response){
zmqResponder.send(JSON.stringify(response));
});
}
});
function serverRequest(){
var deferred = Q.defer();
client.get(function (data, response) {
if (response.statusCode !== 200) {
deferred.reject(data.data);
} else {
deferred.resolve(data.data);
}
});
return deferred.promise;
}
EDIT** Here's a gist of the code: https://gist.github.com/battlecow/cd0c2233e9f197ec0049
I think, through the comment thread, I've identified your issue. REQ/REP has a strict synchronous message order guarantee... You must receive-send-receive-send-etc. REQ must start with send and REP must start with receive. So, you're only processing one message at a time because the socket types you've chosen enforce that.
If you were using a different, non-event-driven language, you'd likely get an error telling you what you'd done wrong when you tried to send or receive twice in a row, but node lets you do it and just queues the subsequent messages until it's their turn in the message order.
You want to change REQ/REP to DEALER/ROUTER and it'll work the way you expect. You'll have to change your logic slightly for the ROUTER socket to get it to send appropriately, but everything else should work the same.
Rough example code, using the relevant portions of the posted gist:
var zmqResponder = zmq.socket('router');
zmqResponder.on('message', function (msg, data) {
var peer_id = msg[0];
var parsed = JSON.parse(msg[1]);
switch (parsed.event) {
case 'create':
// build parsedResponse, then...
zmqResponder.send([peer_id, JSON.stringify(parsedResponse)]);
break;
}
});
zmqResponder.bind('tcp://*:5668', function (err) {
if (err) {
logging.error(err);
} else {
logging.info("ZMQ awaiting orders on port 5668");
}
});
... you need to grab the peer_id (or whatever you want to call it, in ZMQ nomenclature it's the socket ID of the socket you're sending from, think of it as an "address" of sorts) from the first frame of the message you receive, and then use send it as the first frame of the message you send back.
By the way, I just noticed in your gist you are both connect()-ing and bind()-ing on the same socket (zmq.js lines 52 & 143, respectively). Don't do that. Inferring from other clues, you just want to bind() on this side of the process.

Using Node and Socket.io I am trying to pass a simple array and I'm getting nothing. What did I miss?

I followed the net tuts tutorial to build a simple chat application with socket and node, and now I'm trying to extend the app to allow people to play a game I've written, so I want to let people list the games available, but I can't seem to pass even a simple test array from server to client.
I'll let the code speak for itself:
Relevant Server Code:
var games = ["test"];
io.sockets.on('connection', function (socket) {
socket.emit('message', { message: 'welcome to the chat' });
socket.on('gameList', function (data) {
io.sockets.emit("games",games);
});
socket.on('send', function (data) {
io.sockets.emit('message', data);
});
});
ClientSide:
window.games = [];
$("#listGames").click(function(){
socket.emit("gameList", function(data){
window.games = data;
})
})
So I console.log games before and after the button click, and it's always just an empty array, when in my head, it should contain the string "test".
So where did I go wrong?
Note:
Added jQuery to the codebase even though the tutorial missed it.
You are using socket.emit() to attempt to receive data. That only sends data to the server. You need a games event handler on your client to handle the event accordingly.
window.games = [];
$("#listGames").click(function() {
socket.emit('gameList');
});
socket.on('games', function (games) {
window.games = games;
});
The event handler for games is what will fire when you execute io.sockets.emit('games', games); on the server side.
Also make sure to always pass an object as the response over Socket.IO. Change the server-side code from : var games=[]; to var games={'games':[]}; or something similar.

Categories