I'm having an issue with calling a function in a loop, with JavaScript. As I'm new to JavaScript, I thought perhaps my approach must be wrong. Can someone help me out with the following issue?
Basically, each time I learn a new language, I try and write a port scanner in it. In Python, I used a for loop to iterate over a range of numbers, passing them in as ports to a host. It worked fine and I attempted the same approach in JavaScript, with some socket connection code I found online:
const net = require('net');
function Scanner(host, port){
const s = new net.Socket();
s.setTimeout(2000, function() { s.destroy(); });
s.connect(port, host, function () {
console.log('Open: '+ port);
});
s.on('data', function(data){
console.log(port +': ' +data);
s.destroy();
});
s.on('error', function (e) {
s.destroy();
})
}
for(let p = 15000; p < 30000; p++){
let scan = new Scanner('localhost', p);
}
In the above example, I'm iterating over a port range of 15000 to 30000. It appears to run very fast, giving me two results: port 15292 and 15393 as being open on my test vm. However, it's not picking up several ports in the 20,000 range, like 27017.
If I narrow the range from 25000 to 30000 it picks those up just fine. The problem seems to be when I have a larger range, the code isn't discovering anything after a few hits.
In looking at some other JS implementations of port scanners, I noticed the same issue. It works great when the range is 5,000 ports or so, but scale it up to 20k or 30k ports and it only finds the first few open ones.
What am I doing wrong?
Related
i would like to understand DOS (denial-of-service) attacks better and I would like to know what my options are for learning about it with an example.
i have a basic express server.
app.get('/ping', (req, res) => {
res.send({ pong: 'pong', time: new Date().valueOf(), memory: process.memoryUsage()})
})
I will separately create some javascript code that with make multiple requests to the server. but I don't know to devise strategies to try and bring down the server (consider that this is all running on localhost)
I want to see what the upper limit of making requests is possible when locally testing this. i am experiencing what is described here: Sending thousands of fetch requests crashes the browser. Out of memory
... the suggestions on that thread are more along the lines of "browser running out of memory" and that I should "throttle requests".... but I am actively trying to max out the requests the browser can make without crashing. so far my observations are that the server does not have any difficulty. (so maybe I should also make requests from my phone and tablet?)
the code have run on the browser isn't much more than:
const makeRequestAndAlogTime = () => {
const startTime = new Date().valueOf();
fetch('http://localhost:4000/ping')
.then(async (response) => {
const { time, memory } = await response.json();
console.log({
startTime: 0,
processTime: time - startTime,
endTime: new Date().valueOf() - startTime,
serverMemory: memory,
browserMemory: performance['memory']
})
})
}
for(let x = 0; x < 100; x++) {
makeRequestAndAlogTime()
}
but depending on what value I put in for the number of times to go through the for loop, performance is slower and eventually crashes (as expected)... but I want to know if there is a way I could automate determining the upper limit of requests that I can make on my browsers?
I have a problem with codding a node.js program that forwards traffic from an port to another. The scenario goes like this. I forward all traffic from port 55555 to a sshtunnel that have a SOCKS5 opened on port 44444. The thing is that everything works smoothly, until i run the command htop -d 1 and i see high load when i am visiting 2-3 sites simoultaniously. If i go trough SOCKS5 SOCKS sshtunnel directly i see load at peek 1% of a core, but with node.js i se 22% 26% 60% 70% even 100% sometimes. What is happening, why is this? I mean think about when i open like 1000 of those what would happen!!
Here is my first try (proxy1.js) :
var net = require('net');
require('longjohn');
var regex = /^[\x09\x0A\x0D\x20-\x7E]+$/;
var regexIP = /^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$/;
// parse "80" and "localhost:80" or even "42mEANINg-life.com:80"
var addrRegex = /^(([a-zA-Z\-\.0-9]+):)?(\d+)$/;
var addr = {
from: addrRegex.exec(process.argv[2]),
to: addrRegex.exec(process.argv[3])
};
if (!addr.from || !addr.to) {s=
console.log('Usage: <from> <to>');
}
net.createServer(function(from) {
var to = net.createConnection({
host: addr.to[2],
port: addr.to[3]
});
// REQUESTS BEGIN
from.on('data', function(data){
});
from.on('end', function(end){
});
from.on('close', function(close){
});
// error handeling
from.on('error', function(error)
{
});
from.pipe(to);
// REQUESTS END
// RESPONSES BEGIN
to.on('data', function(data){
});
to.on('end', function(end){
});
to.on('close', function(close){
});
to.on('error', function(error)
{
});
to.pipe(from);
// RESPONSES END
}).listen(addr.from[3], addr.from[2]);
Here is my second try (proxy2.js) :
var net = require('net');
var sourceport = 55555;
var destport = 62240;
net.createServer(function(s)
{
var buff = "";
var connected = false;
var cli = net.createConnection(destport,"127.0.0.1");
s.on('data', function(d) {
if (connected)
{
cli.write(d);
} else {
buff += d.toString();
}
});
s.on('error', function() {
});
cli.on('connect', function() {
connected = true;
cli.write(buff);
});
cli.on('error', function() {
});
cli.pipe(s);
}).listen(sourceport);
I also tryed to run cpulimit -l 10 nodejs proxy.js 55555 44444 also makes load and it seems like it is oppening new forks, processes ...
cat /etc/issue
Ubuntu 14.04.3 LTS
nodejs --version
v0.10.25
processor
Intel(R) Xeon(R) CPU E3-1246 v3 # 3.50GHz with 8 cores
RAM
32 RAM (that stays free all the time)
Server config :
Why is the load so big?
How can i write the code to not make that load?
Why 'cpulimit -l 10 nodejs proxy.js 55555 44444' dosen't work as expected?
Why node.js is using CPU and not RAM ?
Thanks in advice.
Port is merely a segment in the memory and writing fast on ports may load the CPU because it may create too many async IO requests. However these requests are even though they are IO bound are indirectly CPU bound.
To avoid this problem you may have to limit too many connection requests by streaming data. Rather than sending 1000 small requests, make 100 large requests.
I'm not sure how to solve this or what exactly is happening. May be socket.io with streaming can help.
I am currently learning how to use NightmareJS. I found the performance is so slow while I was running the code below. It took up to 30 seconds to get the output. Did I do anything wrong?
Moreover, I have tried to use wait() with a selector but that does not help so much.
I am not sure whether this is related to my Internet connection, however, open the same site using Google Chrome and perform the same task is faster than using Nightmare.
Soruce Code
var Nightmare = require('nightmare');
var after;
var before = Date.now();
new Nightmare({
loadImages: false
}).goto('https://www.wikipedia.org/')
.type('#searchInput', process.argv[2])
.click('input[name="go"]')
.wait()
.url(function(url) {
after = Date.now();
console.log('>>> [' + (after - before) / 1000.0 + 's] ' + url);
})
.run(function(err, nightmare) {
if (err) console.log(err);
});
Output
node n02_extract_wiki_link.js "node.js"
>>> [31.227s] https://en.wikipedia.org/wiki/Node.js
My current environment is listed below.
Mac OS X 10.10.4
node v0.12.5
PhantomJS 2.0.0
nightmare#1.8.2
This worked for me:
https://github.com/segmentio/nightmare/issues/126#issuecomment-75944613
It's the socket connection between the phantomjs module and it's dependency, shoe.
You can manually edit shoe yourself. Go into node_modules/phantom/shoe/index.js and change line 8 to read
var server = sockjs.createServer({
heartbeat_delay : 200
});
I've been experimenting with node-serialport library to access devices connected to a USB hub and send/receive data to these devices. The code works fine on linux but on windows(windows 8.1 and windows 7) I get some odd behaviour. It doesn't seem to work for more than 2 devices, it just hangs when writing to the port. The callback for write method never gets called. I'm not sure how to go about debugging this issue. I'm not a windows person, if someone can give me some directions it would be great.
Below is the code I'm currently using to test.
/*
Sample code to debug node-serialport library on windows
*/
//var SerialPort = require("./build/Debug/serialport");
var s = require("./serialport-logger");
var parsers = require('./parsers');
var ee = require('events');
s.list(function(err, ports) {
console.log("Number of ports available: " + ports.length);
ports.forEach(function(port) {
var cName = port.comName,
sp;
//console.log(cName);
sp = new s.SerialPort(cName, {
parser: s.parsers.readline("\r\n")
}, false);
// sp.once('data', function(data) {
// if (data) {
// console.log("Retrieved data " + data);
// //console.log(data);
// }
// });
//console.log("Is port open " + sp.isOpen());
if(!sp.isOpen()) {
sp.open(function(err) {
if(err) {
console.log("Port cannot be opened manually");
} else {
console.log("Port is open " + cName);
sp.write("LED=2\r\n", function(err) {
if (err) {
console.log("Cannot write to port");
console.error(err);
} else {
console.log("Written to port " + cName);
}
});
}
});
}
//sp.close();
});
});
I'm sure you'd have noticed I'm not require'ing serialport library instead I'm using serialport-logger library it's just a way to use the serialport addons which are compiled with debug switch on windows box.
TLDR; For me it works by increasing the threadpool size for libuv.
$ UV_THREADPOOL_SIZE=20 && node server.js
I was fine with opening/closing port for each command for a while but a feature request I'm working on now needs to keep the port open and reuse the connection to run the commands. So I had to find an answer for this issue.
The number of devices I could support by opening a connection and holding on to it is 3. The issue happens to be the default threadpool size of 4. I already have another background worker occupying 1 thread so I have only 3 threads left. The EIO_WatchPort function in node-serialport runs as a background worker which results in blocking a thread. So when I use more than 3 devices the "open" method call is waiting in the queue to be pushed to the background worker but since they are all busy it blocks node. Then any subsequent requests cannot be handled by node. Finally increasing the thread pool size did the trick, it's working fine now. It might help someone. Also this thread definitely helped me.
As opensourcegeek pointed all u need to do is to set UV_THREADPOOL_SIZE variable above default 4 threads.
I had problems at my project with node.js and modbus-rtu or modbus-serial library when I tried to query more tan 3 RS-485 devices on USB ports. 3 devices, no problem, 4th or more and permanent timeouts. Those devices responded in 600 ms interval each, but when pool was busy they never get response back.
So on Windows simply put in your node.js environment command line:
set UV_THREADPOOL_SIZE=8
or whatever u like till 128. I had 6 USB ports queried so I used 8.
Having a hard time implementing a node.js/server.js setup
I'm a bit stuck right now, and hoping someone can shed some light. I'm relatively new to sockets in general, but have been programming in javascript on and off for several years, although only about as deep as is necessary to accomplish the task at hand. As a result, my understanding of some of the concepts surrounding the javascript stack heap, and sockets in general are somewhat limited.
Ok Here's the situation:
I've created an application intended to simply increment a counter, on several machines. Several users can click the "next" button and it will update instantly on all machines.
When you first connect, it retrieves the current number, and spits it out locally.
I've created the server here:
var io = require("socket.io");
var sockets = io.listen(8000);
var currentlyServing=0;
sockets.on("connection", function (socket)
{
console.log("client connected");
socket.emit("receive", currentlyServing);
socket.on("update", function(serving)
{
currentlyServing=serving;
if(currentlyServing>100)
currentlyServing=0;
if(currentlyServing<0)
currentlyServing=99;
socket.broadcast.emit("receive", currentlyServing);
console.log("update received: "+currentlyServing);
});
});
console.log("Server Started");
Here is the relevant (I hope) excerpt from the client side:
var socket = io.connect("http://www.sampledomain.com:8000");
//function to update the page when a new update is received
socket.on("receive", function(receivedServing)
{
document.getElementById('msgs').value=""+String("00" + receivedServing).slice(-2);
document.getElementById('nowServing').value=receivedServing;
});
//this is called in an onClick event in the HTML source
//sends the new number to all other stations except this one (handled by server side)
function nextServing()
{
var sendServing = parseInt(document.getElementById('nowServing').value)+1;
socket.emit("update", sendServing);
document.getElementById('nowServing').value=sendServing;
document.getElementById('msgs').value=""+String("00" + sendServing).slice(-2);
}
Ok so here's my problem. This runs absolutely fine in every system I've put it in, smoothly and beautifully - except for IE8. If left alone for more than 2-3 minutes (with no activity at all), I eventually receive a "stack overflow" error. The line number it appears on fluctuates (haven't determined the factors involved yet), but it always happens at that interval. On some workstations it takes longer, which I'm beginning to think has a direct correlation to the amount of phsyical RAM the machine has, or at least how much is being allocated to the web browser.
I found an online function to determine "max stack size", which I realize is not an exact science, however I did consistently get a number in the area of 3000. On my IE11 machine with considerable more resources, I found it to be in the area of 20,000. This may not be relevant, but I figured the more info the better :)
To avoid this problem for now so that the end users don't see this error message, I've take the entire client script, and put it into an iFrame which reloads itself every 60 seconds,essentially resetting the stack, which feels so dirty sitting so close to a web socket, but has bought me the time to post here. I've googled until I can't google any more, but when you search "node.js" or "socket.io" along with "stack overflow" on google, you just get a lot of posts about the two topics that are hosted on the stackoverflow dot com website. ARG lol
Anyone?
EDIT ON NOVEMBER 18TH 2014 AS PER COMMENTS BELOW:
the error message is most often claiming stack overflow at line 1056. IE Developer tools points towards the file socket.io.js. Line 1056 is:
return fn.apply(obj, args.concat(slice.call(arguments)));
which is insdie this section of the file:
var slice = [].slice;
/**
* Bind `obj` to `fn`.
*
* #param {Object} obj
* #param {Function|String} fn or string
* #return {Function}
* #api public
*/
module.exports = function(obj, fn){
if ('string' == typeof fn) fn = obj[fn];
if ('function' != typeof fn) throw new Error('bind() requires a function');
var args = slice.call(arguments, 2);
return function(){
return fn.apply(obj, args.concat(slice.call(arguments)));
}
};
From what I've read it seems that the problem on IE8 might be related to flash. It IE8 uses flashsocket as the default configuration. I suggest to try the following on the client side:
if(navigator.appName.indexOf("Internet Explorer")!=-1 && navigator.appVersion.indexOf("MSIE 8")==-1 ){
socket = io.connect("http://www.sampledomain.com:8000", {
transports: ['xhr-polling']
});
}
else
{
socket = io.connect("http://www.sampledomain.com:8000" );
}
This should make IE8 use long polling while all other machines use the best method they can.
On a side note: You might also want to consider incrementing the "serving" variable on the server.
Find existing issue Causes a "Stack Overflow" in IE8 when using xhr-polling #385.
This was fixed by disabling Flash.
Also find Safari over windows client use xhr-polling instead of websocket - performance are severely harm #1147. While this is Safari it may apply to IE8 because it is using similar mechanism.
I did a small test using your socket.io but in IE 10 and emulated IE8
so that I could debug well. Started capturing Network in the tab and
noticed the requests logging every few seconds.Left alone for few
minutes and I see a lot of requests logged in. You will not see this
in Chrome because it has true WebSockets. While IE8 does not support
WebSockets socket.io emulate that using plain HTTP GET/POST using some
mechanism. So my theory is that even if socket.io works with IE8 it
does not reliably emulate web sockets
My advice is to rule out IE 8 for long running client application. IE8 is no longer supported by Microsoft.
maybe try to replace
""+String("00" + receivedServing).slice(-2)
with
('00' + receivedServing).slice(-2)