I'm working on a small personal project (to learn node) and I'm wondering what the best way to send chunks of a video data from a client to the server is? Obviously, I don't want to use Node's http module, but I've never used anything other than http. I know there is a net module -- is this better?
Related
What is the best way to send a very big String to a nodeJs express server?
I have a Webpage with codemirror which is able to load files from a express server and load them into Codemirror.But what method is the best to send "the file"(actually it's a string,can be realy big)
back to the express api ?
The most common way is to stream the data instead of sending everything at the same time. I don't know how to do that on the client side, I actually don't think that to stream a request is even possible.
I'm not an expert and I don't know much about large strings, but why is that a problem? From a networking point of vue, isn't the TCP protocol supposed to packet everything?
So my application is running Sails.js, which is an extension of node.js. I'm very new to the JS backend framework scene, and I'm attempting to send a TCP message using a socket from the client side. However, in order to do that I need to require('net'). How can I do this? I don't even see the net module in my node_modules folder - does it not come with Sails.js?
How can I resolve this issue?
Also, just for extra clarification, I need to use require() on the browser side - apologies for not being more clear in my original question.
You can't use the node.js net module on the client side. Browsers don't allow you to access plain TCP sockets. That would enable you to circumvent many of their security features, so it simply is not going to happen.
You can implement realtime communication with your own server with web sockets or a wrapper suck as socket.io, but that obviously doesn't let you talk in arbitrary protocols.
I have a node server that will serve more than 10,000 users at the same time with big data, I already tried lz-string https://www.npmjs.org/package/lz-string but it's not good module because it's blocking the node thread.
Please answer these questions:
is it better to compress the data in server and then decompress in client instead of send plain/json data?
what is the best and fastest way to compress/decompress the data?
If you are sending large chunks of text data over the internet using HTTP protocol, then there are already some technologies in place to help you.
One is called HTTP Compression. HTTP protocol specifications allow few compression algorithms to perform on data being sent, but that requires the server and client to be properly configured for compression. Standard Node.js server will not compress the data without modifying code.
For bare Node.js without any frameworks and 3rd party modules there is zlib module made specially for HTTP compression, both server and client.
Using Express? Then there is a compression middleware.
It might also be worth looking using nginx as a proxy server to your node.js applications. Then you can easily flip the switch ON for compression in nginx, without needing to do anything in your Node.js application at all:
server {
gzip on;
...
}
It really depends on the stack you are using, but the idea is same: compress the HTTP stream itself, as it is supported by the protocol.
To use socket.io on the client side, usually we start a node.js server and go like this:
<script src="/socket.io/socket.io.js"></script>
or with specific port:
<script src="http://localhost:3700/socket.io/socket.io.js"></script>
Question is:
is it necessary to use node.js server to serve socket.io.js ?
...or is it possible to
make a local copy of socket.io.js instead of goes to server every single time we need socket.io?
like, we go to view source and copy everything we got from the source of script tag,
paste and save it as socket.io-local.js so that next time we use:
<script src="socket.io-local.js"></script>
will that work ?
Updates
Thanks for everyone's great response,
I'm asking this because in the case I'm involved, I don't actually have access to the server:
I am writing the client-side to connect to other developer's Socket Sever which is written in Java.
Therefore I'll have to think a way to work around the fact that I don't have a server there for me.
from what I've been testing,
this way seems to work but I really don't know what's happening behind the scene.
You obviously can host the socket.io client library anywhere and pull it in to a page. However, it will almost certainly not work with your Java-based server.
To understand why, you need to understand what socket.io is really doing behind the scenes; the client library is only a small part of it.
Socket.io actually defines and implements its own protocol for realtime communication between a browser and a server. It does so in a way that supports multiple transports: if—for example—a user's browser or proxy doesn't support WebSockets, it can fall back to long polling.
What the socket.io client actually does is:
Makes a XHR GET request for /socket.io/1. The server responds with a session ID, configured timeouts, and supported transports.
The client chooses the best transport that the user browser supports. In modern browsers, it will use WebSockets.
If WebSockets are supported, it creates a new WebSocket to initiate a WebSocket connection (HTTP GET with Upgrade: websocket header) to a special URL – /socket.io/1/websocket/<session id>.
If WebSockets aren't supported by the browser or fail to connect (there are lots of intermediaries in the wild like proxies, filters, network security devices, and so forth that don't support WebSocket requests), the library falls back to XHR long polling, and makes a XHR request to /socket.io/1/xhr-polling/<sesion id>. The server does not respond to the request until a new message is available or a timeout is reached, at which point the client repeats the XHR request.
Socket.io's server component handles the other end of that mess. It handles all the URLs under /socket.io/, setting up sessions, parsing WebSocket upgrades, actually sending messages, and a bunch of other bookkeeping.
Without all of the services provided by the socket.io server, the client library is pretty useless. It will just make a XHR request to a URL that doesn't exist on your server.
My guess is that your Java-based server just implements the WebSockets protocol. You can connect directly to it using the browser-provided WebSocket APIs.
It is possible that your server does implement the socket.io protocol – there are a few abandoned Java projects to do that – but that's unlikely. Talk with the developer of your server to find out exactly how he's implemented a "socket server."
A standalone build of socket.io-client is exposed automatically by the socket.io server as /socket.io/socket.io.js. Alternatively you can serve the file socket.io-client.js found at the root of this repository.
https://github.com/LearnBoost/socket.io-client
I have a module called shotgun-client that actually wraps socket.io. I needed to serve a custom client script as well as the socket.io client script, but I didn't want every user of my module to have to include multiple script references on their pages.
I found that, when installed, you can serve the generated client script from socket.io by reading the file /node_modules/socket.io/node_modules/socket.io-client/dist/socket.io.js. So my module adds a listener for its own URL and when it serves my custom client script it also serves the socket.io client script with it. Viola! Only a single script reference for the users of my module :)
While this is technically possible, I don't see why you'd need to do that. If you're concerned about reducing the data that goes over the wire, this change won't actually do much beyond the few characters saved in the shorter src tag. Simply changing the location of the JS file on the server won't actually improve performance - the JS has to be sent.
Proper caching (which Socket.IO has) will return a 304 Not Modified (and not re-send the JS file every time you load a page).
I have a client program and a server program. The server is on my localhost and it has my .mpeg video.
Using node JS I am supposed to stream a video from a server. The client requests messages, such as play/pause/resume/rewind etc. so I guess I have to use RTSP, to figure out what to send over the RTP. But I don't know from where to start.
All I have so far is the RegEx to filter the message, for example on the client there are buttons like play/pause/setup etc. so I can grab that text. And I have a switch.
But if I get setup what I should I do?
P.S I am not allowed to use RTSP modules or RTP modules. Gotta do it all from scratch.
When streaming mpeg file over the wire you will have to tackle RTSP and RTP separately. RTSP is used for signaling, session establishing and starting the underlying RTP stream.If you need to do this in node.js, I recommend loading a library that already implements RTSP/RTP(creating your own is quite a undertaking, but it is doable as well).
Some info on loading c++ libraries in node.js that: How can I use a C++ library from node.js?
So basically, from you mpeg file, you need to extract raw h264 stream. For this I recommend ffmpeg or some other libraries/code that understands mpeg file structure. Then you need to packetize the encoded frames inside of RTP packets; which you will then send back to the client from the server. The client will then depacketize the encoded frames into actual frame; and then decoded/display them on the screen .
I recommend reading http://www.ietf.org/rfc/rfc3984.txt for info on standard way to packetize H264 video.
This is all very general approach, but it gives you a general idea.
Hopefully this info helps, good luck.