PROBLEM:
WebRTC gives us peer-to-peer video/audio connections. It is perfect for p2p calls, hangouts. But what about broadcasting (one-to-many, for example, 1-to-10000)?
Lets say we have a broadcaster "B" and two attendees "A1", "A2". Of course it seems to be solvable: we just connect B with A1 and then B with A2. So B sends video/audio stream directly to A1 and another stream to A2. B sends streams twice.
Now lets imagine there are 10000 attendees: A1, A2, ..., A10000. It means B must send 10000 streams. Each stream is ~40KB/s which means B needs 400MB/s outgoing internet speed to maintain this broadcast. Unacceptable.
ORIGINAL QUESTION (OBSOLETE)
Is it possible somehow to solve this, so B sends only one stream on some server and attendees just pull this stream from this server? Yes, this means the outgoing speed on this server must be high, but I can maintain it.
Or maybe this means ruining WebRTC idea?
NOTES
Flash is not working for my needs as per poor UX for end customers.
SOLUTION (NOT REALLY)
26.05.2015 - There is no such a solution for scalable broadcasting for WebRTC at the moment, where you do not use media-servers at all. There are server-side solutions as well as hybrid (p2p + server-side depending on different conditions) on the market.
There are some promising techs though like https://github.com/muaz-khan/WebRTC-Scalable-Broadcast but they need to answer those possible issues: latency, overall network connection stability, scalability formula (they are not infinite-scalable probably).
SUGGESTIONS
Decrease CPU/Bandwidth by tweaking both audio and video codecs;
Get a media server.
As it was pretty much covered here, what you are trying to do here is not possible with plain, old-fashionned WebRTC (strictly peer-to-peer). Because as it was said earlier, WebRTC connections renegotiate encryption keys to encrypt data, for each session. So your broadcaster (B) will indeed need to upload its stream as many times as there are attendees.
However, there is a quite simple solution, which works very well: I have tested it, it is called a WebRTC gateway. Janus is a good example. It is completely open source (github repo here).
This works as follows: your broadcaster contacts the gateway (Janus) which speaks WebRTC. So there is a key negotiation: B transmits securely (encrypted streams) to Janus.
Now, when attendees connect, they connect to Janus, again: WebRTC negotiation, secured keys, etc. From now on, Janus will emit back the streams to each attendees.
This works well because the broadcaster (B) only uploads its stream once, to Janus. Now Janus decodes the data using its own key and have access to the raw data (that it, RTP packets) and can emit back those packets to each attendee (Janus takes care of encryption for you). And since you put Janus on a server, it has a great upload bandwidth, so you will be able to stream to many peer.
So yes, it does involve a server, but that server speaks WebRTC, and you "own" it: you implement the Janus part so you don't have to worry about data corruption or man in the middle. Well unless your server is compromised, of course. But there is so much you can do.
To show you how easy it is to use, in Janus, you have a function called incoming_rtp() (and incoming_rtcp()) that you can call, which gives you a pointer to the rt(c)p packets. You can then send it to each attendee (they are stored in sessions that Janus makes very easy to use). Look here for one implementation of the incoming_rtp() function, a couple of lines below you can see how to transmit the packets to all attendees and here you can see the actual function to relay an rtp packet.
It all works pretty well, the documentation is fairly easy to read and understand. I suggest you start with the "echotest" example, it is the simplest and you can understand the inner workings of Janus. I suggest you edit the echo test file to make your own, because there is a lot of redundant code to write, so you might as well start from a complete file.
Have fun! Hope I helped.
As #MuazKhan noted above:
https://github.com/muaz-khan/WebRTC-Scalable-Broadcast
works in chrome, and no audio-broadcast yet, but it seems to be a 1st Solution.
A Scalable WebRTC peer-to-peer broadcasting demo.
This module simply initializes socket.io and configures it in a way
that single broadcast can be relayed over unlimited users without any
bandwidth/CPU usage issues. Everything happens peer-to-peer!
This should definitely be possible to complete.
Others are also able to achieve this: http://www.streamroot.io/
AFAIK the only current implementation of this that is relevant and mature is Adobe Flash Player, which has supported p2p multicast for peer to peer video broadcasting since version 10.1.
http://tomkrcha.com/?p=1526.
"Scalable" broadcasting is not possible on the Internet, because the IP UDP multicasting is not allowed there. But in theory it's possible on a LAN. The problem with Websockets is that you don't have access to RAW UDP by design and it won't be allowed.
The problem with WebRTC is that it's data channels use a form of SRTP, where each session has own encryption key. So unless somebody "invents" or an API allows a way to share one session key between all clients, the multicast is useless.
There is the solution of peer-assisted delivery, meaning the approach is hybrid. Both server and peers help distribute the resource. That's the approach peer5.com and peercdn.com have taken.
If we're talking specifically about live broadcast it'll look something like this:
Broadcaster sends the live video to a server.
The server saves the video (usually also transcodes it to all the relevant formats).
A metadata about this live stream is being created, compatible with HLS or HDS or MPEG_DASH
Consumers browse to the relevant live stream there the player gets the metadata and knows which chunks of the video to get next.
At the same time the consumer is being connected to other consumers (via WebRTC)
Then the player downloads the relevant chunk either directly from the server or from peers.
Following such a model can save up to ~90% of the server's bandwidth depending on bitrate of the live stream and the collaborative uplink of the viewers.
disclaimer: the author is working at Peer5
My masters is focused on the development of a hybrid cdn/p2p live streaming protocol using WebRTC. I've published my first results at http://bem.tv
Everything is open source and I'm seeking for contributors! :-)
The answer from Angel Genchev seems to be correct, however, there is a theoretical architecture, that allows low-latency broadcasting via WebRTC. Imagine B (broadcaster) streams to A1 (attendee 1). Then A2 (attendee 2) connects. Instead of streaming from B to A2, A1 starts streaming video being received from B to A2. If A1 disconnects then A2 starts receiving from B.
This architecture could work if there are no latencies and connection timeouts. So theoretically it is right, but not practically.
At the moment I am using server side solution.
I'm developing WebRTC broadcasting system using the Kurento Media Server. Kurento Supports several kinds of streaming protocol such as RTSP, WebRTC, HLS. It works as well in term of real-time and scaling.
Hence, Kurento doesn't support RTMP which is used in Youtube or Twitch now. One of the problem with me is the number of user concurrent with this.
Hope it help.
You are describing using WebRTC with a one-to-many requirement. WebRTC is designed for peer-to-peer streaming, however there are configurations that will let you benefit from the low latency of WebRTC while delivering video to many viewers.
The trick is to not tax the streaming client with every viewer and, like you mentioned, have a "relay" media server. You can build this yourself but honestly the best solution is often to use something like Wowza's WebRTC Streaming product.
To stream efficiently from a phone you can use Wowza's GoCoder SDK but in my experience a more advanced SDK like StreamGears works best.
Related
Is it possible to implement a WebService over a WebRTC Data Channel ?
The idea is:
The client makes one https request to the server for signaling and session establishment
The client and the server start to communicate via a WebRTC DataChannel bidirectionally
Benefits?:
Performance ?
Requests goes over one connection and the standard allows for multiple datachannels over the same connection ( ports )
Flexible networking topologies
UDP
End to end encryption
The server can send events over the same connection
Load balancing could be implemented from a pool of servers client side without a load balancer , or all kinds of different solutions
Currently being debated the addition of DataChannels to Workers/Service Workers/ etc https://github.com/w3c/webrtc-extensions/issues/64
Drawbacks:
Application specific code for implementing request fragmentation and control over buffer limits
[EDIT 3] I don't know how much of a difference in terms of performance and cpu/memory usage will it be against HTTP/2 Stream
Ideas:
Clients could be read replicas of the data for sync, or any other applications that are suitable for orbit-db https://github.com/orbitdb/orbit-db in the public IPFS network, the benefit of using orbit-db is that only allows to the owner to make writes, then the server could additionally sign with his key all the data so that the clients could verify and trust it's from the server, that could offload the main server for reads, just an idea.
[EDIT]
I've found this repo: https://github.com/jsmouret/grpc-over-webrtc
amazing!
[EDIT2]
Changed Orbit-db idea and removed cluster IPFS after investigating a bit
[EDIT3]
After searching Fetch PROS for HTTP/2 i've found Fetch upload streaming with ReadableStreams, i don't know how much of a difference will it be to run GRPC (bidi) over a WebRTC DataChannel or a HTTP/2 Stream
https://www.chromestatus.com/feature/5274139738767360#:~:text=Fetch%20upload%20streaming%20lets%20web,things%20involved%20with%20network%20requests).
Very cool video explaining the feature: https://www.youtube.com/watch?v=G9PpImUEeUA
Lots of different points here, will try to address them all.
The idea is 100% feasible. Check out Pion WebRTC's data-channels example. All it takes a single request/response to establish a connection.
Performance
Data channels are a much better fit if you are doing latency sensitive work.
With data channels you can measure backpressure. You can tell how much data has been delivered, and how much has has been queued. If the queue is getting full you know you are sending too much data. Other APIs in the browser don't give you this. There are some future APIs (WebTransport) but they aren't available yet.
Data channels allow unordered/unreliable delivery. With TCP everything you send will be delivered and in order, this issue is known as head-of-line blocking. That means if you lose a packet all subsequent packets must be delayed. An example would be if you sent 0 1 2 3, if packet 1 hasn't arrived yet 2 and 3 can't be processed yet. Data channels can be configured to give you packets as soon as they arrive.
I can't give you specific numbers on the CPU/Memory costs of running DTLS+SCTP vs TLS+WebSocket server. It depends on hardware/network you have, what the workload is etc...
Multiplexing
You can serve multiple DataChannel streams over a single WebRTC Connection (PeerConnection). You can also serve multiple PeerConnections over a single port.
Network Transport
WebRTC can be run over UDP or TCP
Load Balancing
This is harder (but not intractable) moving DTLS and SCTP sessions between servers isn't easy with existing libraries. With pion/dtls it has the support to export/resume a session. I don't know support in other libraries however.
TLS/Websocket is much easier to load balance.
End to end encryption
WebRTC has mandatory encryption. This is nice over HTTP 1.1 which might accidentally fall back to non-TLS if configured incorrectly.
If you want to route a message through the server (and not have the server see it) I don't think what protocol you use matters.
Topologies
WebRTC can be run in many different topologies. You can do P2P or Client/Server, and lots of things in between. Depending on what you are building you could build a hybrid mesh. You could create a graph of connections, and deploy servers as needed. This flexibility lets you do some interesting things.
Hopefully addressed all your points! Happy to discuss further in the comments/will keep editing the question.
I was also wondering about this HTTP-over-WebRTC DataChannel idea a couple of years ago. The problem at hand was how to securely connect from a web app to an IoT device (raspberry pi) that sits behind a firewall.
Since there was no readily available solution, I ended up building a prototype. It did the job and has been in live deployment since 2019.
See this technical blog post that covers the design and implementation in more detail:
https://webrtchacks.com/private-home-surveillance-with-the-webrtc-datachannel/
High level architecture:
Simplified sequence diagram:
Recently began the process of extracting the code into a standalone repo.
https://github.com/ambianic/peerfetch
If your main use-case exchanges small content, you may have a look at CoAP RFC 7252. A peer may easily implement both roles, client and server, though the exchanged messages for request and response share the same fomat.
For some advanced usage of DTLS 1.2, DTLS Connection ID can do some magic for you.
If you don't stick to javascript and java is an option, you may check the open source project Eclipse/Californium. That's a CoAP/DTLS implementation, which comes with DTLS Connection ID and some prepared advanced examples as built-in-cid-load-balancer-support or DTLS-graceful-restart.
WebRTC signalling is driving me crazy. My use-case is quite simple: a bidirectional audio intercom between a kiosk and to a control room webapp. Both computers are on the same network. Neither has internet access, all machines have known static IPs.
Everything I read wants me to use STUN/TURN/ICE servers. The acronyms for this is endless, contributing to my migraine but if this were a standard application, I'd just open a port, tell the other client about it (I can do this via the webapp if I need to) and have the other connect.
Can I do this with WebRTC? Without running a dozen signalling servers?
For the sake of examples, how would you connect a browser running on 192.168.0.101 to one running on 192.168.0.102?
STUN/TURN is different from signaling.
STUN/TURN in WebRTC are used to gather ICE candidates. Signaling is used to transmit between these two PCs the session description (offer and answer).
You can use free STUN server (like stun.l.google.com or stun.services.mozilla.org). There are also free TURN servers, but not too many (these are resource expensive). One is numb.vigenie.ca.
Now there's no signaling server, because these are custom and can be done in many ways. Here's an article that I wrote. I ended up using Stomp now on client side and Spring on server side.
I guess you can tamper with SDP and inject the ICE candidates statically, but you'll still need to exchange SDP (and that's dinamycally generated each session) between these two PCs somehow. Even though, taking into account that the configuration will not change, I guess you can exchange it once (through the means of copy-paste :) ), stored it somewhere and use it every time.
If your end-points have static IPs then you can ignore STUN, TURN and ICE, which are just power-tools to drill holes in firewalls. Most people aren't that lucky.
Due to how WebRTC is structured, end-points do need a way to exchange call setup information (SDP) like media ports and key information ahead of time. How you get that information from A to B and back to A, is entirely up to you ("signaling server" is just a fancy word for this), but most people use something like a web socket server, the tic-tac-toe of client-initiated communication.
I think the simplest way to make this work on a private network without an internet connection is to install a basic web socket server on one of the machines.
As an example I recommend the very simple https://github.com/emannion/webrtc-web-socket which worked on my private network without an internet connection.
Follow the instructions to install the web socket server on e.g. 192.168.1.101, then have both end-points connect to 192.168.0.101:1337 with Chrome or Firefox. Share camera on both ends in the basic demo web UI, and hit Connect and you should be good to go.
If you need to do this entirely without any server, then this answer to a related question at least highlights the information you'd need to send across (in a cut'n'paste demo).
I'm new to WebRTC, actually just heard about it a few days ago and I've read a lot about it. However, I still have a few questions.
What do I need to explore the usage of WebRTC? E.g.: do I need a server, any libraries etc.? I'm aware that new version of Chrome and Firefox support WebRTC, but besides these two browsers, is there anything else that is necessary?
What is the main purpose of WebRTC when addressing practical usage? To video chat? Audio chat? What about text-chatting?
Does WebRTC need a server for any kind of browser-to-browser interaction? I've seen some libraries, such as PeerJS that don't explicitly mention any kind of server... so is it possible to connect two clients directly? There's also a PeerServer, which supposedly helps broker connections between PeerJS clients. Can I use WebRTC without such a server?
What are the most commonly used libraries for WebRTC?
What's a good starting point for someone who's totally new in WebRTC? I'd like to setup a basic google-talk kind of service, to chat with one person.
Thank you so much guys.
You can find many docs here E.g. this one, this one and this one!
You can find a few libraries here.
A simple multi-user WebRTC app needs following things:
Signalling server to exchange sdp/ice/etc. ---- e.g. socket.io/websockets/xmpp/sip/XHR/etc.
ICE server i.e. STUN and/or TURN; to make sure Firewalls doesn't block UDP/TCP ports
JavaScript app to access/invoke RTCWeb JavaScript API i.e. RTCPeerConnection.
It just takes a few minutes to setup WebRTC peer-to-peer connection. You can setup peer-to-server connections as well where media-servers can be used to transcode/record/merge streams; or to relay to PSTN networks.
WebRTC DataChannels can be used for gaming, webpage synchronizing; fetching static contents, peer-to-peer or peer-to-server data transmission, etc.
What do I need to explore the usage of WebRTC? E.g.: do I need a
server, any libraries etc.? I'm aware that new version of Chrome and
Firefox support WebRTC, but besides these two browsers, is there
anything else that is necessary?
WebRTC it is JavaScript API for web developers which can be used for audio and video streaming.
But there are 2 notices:
You need a signaling path.
For example, if your first user is Alice using Firefox and second user is Bob using Chrome,
they should negotiate used codecs and streams.
WebRTC does not offer the signalling implementation. So you need to implement the signaling yourself. It is quite simple. You need to send SDP(stream config) to participant and receive an SDP answer. You can use plain HTTP via apahe server or use Websockets or any other transport to negotiate SDP.
So, it seems you need an intermediary signaling server workning with websockets or HTTP/HTTPS.
Once you negotiated the streams you are sending your audio or video stream, but the distanation user might have a simmetric NAT. It means that you stream will not be delivered to the target user. In such situation you need a TURN server to traverse the NAT.
Finally you will need 2 server-side logic items:
1) Signaling server
2) TURN or proxy server
To start, take a look Web Call Server.
The server implements HTML5 Websocket signaling and SRTP proxying as a TURN server.
You can also learn the webrtc application open source code.
First steps:
1. Download the signaling and streaming server.
2. Download and unzip web client.
3. Start the web client and debug javascript code to learn more how webrtc works.
if I am in a room with other 7 users, I am wondering if WebRTC force every user to establish a connection to each one of other participants.
Obviously it would consume something like 7kb/s*7 download and even upload, and many connection cannot handle this if their connection is already busy.
Instead with some kind of media relay the bandwidth usage would be only 7kb/s but you would lose bandwidth adaptation between peers.
Do you know any media relay, or way to solve this problem? is TURN server ( like https://code.google.com/p/rfc5766-turn-server/ ) suitable for this kind of job ( multicast included )?
A TURN server works as a fallback relay server in order to enable connectivity when direct peer-to-peer connectivity is impossible because of firewalls or other network issues. (More information here: press P for speaker notes.) TURN servers are not designed for media distribution.
A Multipoint Control Unit could solve the problem you refer to: there's an example topology for this here. As stated in the notes for that slide:
This is a server that's made specifically to do distribution of media,
and can handle large numbers of participants; it can also do smart
things like selective stream forwarding, mixing of the audio or video,
or recording.
Have a look at https://datatracker.ietf.org/doc/html/draft-ietf-rtcweb-use-cases-and-requirements-06 for details about WebRTC use cases. The authors mention a multi-user conferencing solution that uses a central server. So the best solution of establishing multi-user A/V conferences using WebRTC is to have such a central server that does the audio mixing and A/V "broadcasting" to all peers.
This circumvents the bandwidth problems you mention in your question. Currently a whole bunch of start-ups and established service providers are working on WebRTC-based conferencing solutions, just let your favourite web search engine pick some examples.
A TURN server alone doesn't suffice since TURN is only used to relay data for hosts that can't be reached directly (possibly because of firewalls). TURN servers don't terminate WebRTC connections.
Yes, you would have to establish separate connections to each of your peers. In order to solve this you could use a media server like kurento.
With a media server every peer would connect to the media server, the server would then combine the video streams from your peers into one by placing them side by side and then send you the new stream. This saves peers the trouble of having to download streams from every other peer.
You are right that bandwidth adaption between peers is an issue.
A TURN server does not solve this issue since all it does is provide a stable endpoint, typically for people behind very restrictive NAT setups.
The solution to this issue lies in scalable video codecs. These video codecs are specifically designed to solve the multi-way video conferencing problem. H.264/SVC is one such scalable codec and it is currently used by Google+ Hangouts. VP8 also has temporal and spatial scalability and is used in WebRTC.
The scalable video codecs are designed so that parts of the stream, typically individual UDP packets, can be removed from the stream while preserving the ability to decode the video at a lower quality. At least three types of scalabilities are used:
Temporal, in which the frames-per-second is reduced.
Spatial, where the number of pixels is reduced.
Quality, where the color resolutios is reduced.
If you implement a video conferencing server, you can go into the VP8 stream at a lower level than the WebRTC-level, do the necessary changes to each video stream, and solve the bandwidth adaption issue.
If the question still stands , here is my suggestion :
Based on your SIP server install a RTP proxy software such as if you are using kamailio couple it with rtpengine.
I'm exploring the options for building a bleeding edge whiteboard application (much like Big Blue Button or Wimba) using webRTC, HTML5, socket.io and node.js. This would be for one-on-one communication ... no more than two users communicating with each other at a time.
I'd like use data channels to pass whiteboard data between clients, then have some sort of listener on the server end that can record the video/audio/whiteboard actions. The server listeners would in effect be wire-tapping the whiteboard/audio/video chat. In other words, if the server to client connection gets laggy, it would not affect the quality of the P2P (client-to-client) webRTC connection. This way the server can record the whiteboard interactions at a lower priority leaving the one-on-one chat interactions as high speed as possible.
Are there any libraries or javascript frameworks that would work well for a shared whiteboard that I should know about?
Disclaimer: this answer is outdated, see this answer on how something like this can be accomplished.
WebRTC is still very new, support is also still a bit shaky. Needless to say no libraries or frameworks that do this sort of thing for you exist.
What you probably want to do is use WebSockets (with socket.io which you are already using) to send some of the data from the client to the server, and simultaneously send the data over WebRTC.
You can check how fast the connection is (by a sample upload and a sample download) and decide which strategy you want to use:
Use WebRTC and record nothing, or record only on the client side and upload to the server when done
Use both WebRTC and WebSockets , only use WebSockets to transfer 'some' data to the server where 'some' depends on what the connection can hold.
Use only WebSockets and transfer all the data to the server, with no P2P involved.
I think the most viable options are the first and second.
This also greatly depends on the implementation of the White Board, if you're only passing vector data for what was drawn on the board, even AJAX can handle the load, however, if you are passing live video I think nothing but WebRTC will effectively handle the load.
Good luck! This sounds like an interesting idea. I hope I will be able to edit this answer in a few months with a long list of libraries and frameworks that would let you easily do this.