Sending video from ReactNative app to Python Server via socket - javascript

I am developing an application in RN which sends the video from the front camera.
I tried using WebSockets and SocketIO, but that only helps with text transfer.
Also, I know about WebSocketStream in Js but I just can't find any implementation in python.
Guide me please.

Websocket is not suitable for this type of applications. you can use technologies like WebRTC for this this purpose. take a look at these technology and frameworks to see which one fit your needs.
webRTC: for p2p connection
kurento: webRTC media server
hls and live streaming technologies
webRTC is a technology developed by google for real-time P2P communication. You can use webRTC to connect two or more end user together and send Data/Audio/Video. Read following resources for more information.
webRTC
React Native Implementation
But its only offer P2P connection for you. If you want to store/modify/broadcast your streams you must use something like Kurento media server.

Related

How can I have a server stream a video with WebRTC?

My current use case is that I'm trying to mock a system that uses WebRTC for live video streaming (for a robot). This way, I don't have to be connected to the robot to develop the client.
My issue as of now is that I have no idea how to stream a video using WebRTC to connected peers. I've seen many examples of how to do this from client to client using a signaling server, but other than directly sending the video buffer using socket.io, I haven't seen an example of server -> client WebRTC streaming.
I'm planning to use Node.JS for mocking the video stream as I've been using it for the rest of the robot's systems.
It really isn't that different though client to client or server to client. You want to stream/broadcast a video to all the connected peers. Think of your server will be a client in the setup.
You can also use a WebRTC solution like Janus Repo it is a simple gateway and completely open source. Refer to - WebRTC & Dev API's for more info.
If you find latency issues after peers have increased in number you can check - Mesh, Routing, Multi peer architecture for some solutions for it.
Hope it helps.

How can I mix incoming WebRTC audio streams into a single stream on a server?

I'm working on a project which involves building an audio-conferencing app for the web. Currently my working system uses a WebSocket server to negotiate connections between peers, which can then stream audio directly to one another. However, I wish to implement the server as its own client/peer, which will receive all incoming audio streams, "mix" them into a single source/stream, and then stream it to all peers individually. The goal is to avoid direct peer-to-peer connections between user connections.
Perhaps a more simple question would be how I can accomplish the concept of the given figure, the green squares being RTCPeerConnections, and the server "forwarding" the incoming streams to the recipient?
Figure
How can I accomplish this, and is the concept feasible in regards to system resources of the server?
Thanks.
You can use kurento. Its based on webRTC and its Media Server features include group communications, transcoding, recording, mixing, broadcasting and routing of audiovisual flows.
The concept you are looking for is called Multi Conferencing Unit (MCU). MCU is not part of standard WebRTC. WebRTC is peer-to-peer only.
There are several media server solution that offer the MCU functionality. kurento, as suggested by Milad, is one options. Others examples are Jitsi Videobridge or Janus.
A more recent approach you might want to consider is SFU (Selective Forwarding Unit).

Make complex apps using p2p connections

in the last days I seen a technology called WebRTC that permit directly in Javascript to make p2p connections. What I've said to myself is: can i make complex web apps, like forums or social networks, that actually need databases to work fine, with WebRTC and p2p connection? Can someone tell me if is it possible and how it can be do this.
Sure, you can do it, any project complexity can be implemented using WebRTC API
WebRTC is a kind of standard now for audio/video p2p communication at this moment, so almost any projects that have audio/video calls use WebRTC API - at browser side, and also at iOS/Android
Here is an open source example of 1-1 and group video chat app, which uses WebRTC API broadly

WebRTC - help me understand a few concepts

I'm new to WebRTC, actually just heard about it a few days ago and I've read a lot about it. However, I still have a few questions.
What do I need to explore the usage of WebRTC? E.g.: do I need a server, any libraries etc.? I'm aware that new version of Chrome and Firefox support WebRTC, but besides these two browsers, is there anything else that is necessary?
What is the main purpose of WebRTC when addressing practical usage? To video chat? Audio chat? What about text-chatting?
Does WebRTC need a server for any kind of browser-to-browser interaction? I've seen some libraries, such as PeerJS that don't explicitly mention any kind of server... so is it possible to connect two clients directly? There's also a PeerServer, which supposedly helps broker connections between PeerJS clients. Can I use WebRTC without such a server?
What are the most commonly used libraries for WebRTC?
What's a good starting point for someone who's totally new in WebRTC? I'd like to setup a basic google-talk kind of service, to chat with one person.
Thank you so much guys.
You can find many docs here E.g. this one, this one and this one!
You can find a few libraries here.
A simple multi-user WebRTC app needs following things:
Signalling server to exchange sdp/ice/etc. ---- e.g. socket.io/websockets/xmpp/sip/XHR/etc.
ICE server i.e. STUN and/or TURN; to make sure Firewalls doesn't block UDP/TCP ports
JavaScript app to access/invoke RTCWeb JavaScript API i.e. RTCPeerConnection.
It just takes a few minutes to setup WebRTC peer-to-peer connection. You can setup peer-to-server connections as well where media-servers can be used to transcode/record/merge streams; or to relay to PSTN networks.
WebRTC DataChannels can be used for gaming, webpage synchronizing; fetching static contents, peer-to-peer or peer-to-server data transmission, etc.
What do I need to explore the usage of WebRTC? E.g.: do I need a
server, any libraries etc.? I'm aware that new version of Chrome and
Firefox support WebRTC, but besides these two browsers, is there
anything else that is necessary?
WebRTC it is JavaScript API for web developers which can be used for audio and video streaming.
But there are 2 notices:
You need a signaling path.
For example, if your first user is Alice using Firefox and second user is Bob using Chrome,
they should negotiate used codecs and streams.
WebRTC does not offer the signalling implementation. So you need to implement the signaling yourself. It is quite simple. You need to send SDP(stream config) to participant and receive an SDP answer. You can use plain HTTP via apahe server or use Websockets or any other transport to negotiate SDP.
So, it seems you need an intermediary signaling server workning with websockets or HTTP/HTTPS.
Once you negotiated the streams you are sending your audio or video stream, but the distanation user might have a simmetric NAT. It means that you stream will not be delivered to the target user. In such situation you need a TURN server to traverse the NAT.
Finally you will need 2 server-side logic items:
1) Signaling server
2) TURN or proxy server
To start, take a look Web Call Server.
The server implements HTML5 Websocket signaling and SRTP proxying as a TURN server.
You can also learn the webrtc application open source code.
First steps:
1. Download the signaling and streaming server.
2. Download and unzip web client.
3. Start the web client and debug javascript code to learn more how webrtc works.

WebRTC and Websockets. Is there a difference

I'm assuming that WebRTC is an API that decodes/encodes audio and video, although the communication between the server and the clients is done via web sockets, or some other network protocol? I'm a bit confused. Does WebRTC have its own communications protocol?
There's two sides to WebRTC.
JavaScript APIs (getUserMedia) that allow an app to access camera and microphone hardware. You can use this access to simply display the stream locally (perhaps applying effects), or send the stream over the network. You could send the data to your server, or you could use...
PeerConnection, an API that allows browsers to establish direct peer-to-peer socket connections. You can establish a connection directly to someone else's browser and exchange data directly. This is very useful for high-bandwidth data like video, where you don't want your server to have to deal with relaying large amounts of data.
Take a look at the demos to see both parts of WebRTC in action.
So in a nutshell:
WebSockets allow full-duplex communication between a browser and a web server.
WebRTC's PeerConnection allows full-duplex communication between two browsers.
WebRTC uses RTP (a UDP based protocol) for the media transport, but requires an out-of-band signaling channel to setup the communication. One option for the signaling channel is WebSocket.
Instead of peerConnection you can also look at the WebRTC data channel draft: https://datatracker.ietf.org/doc/html/draft-jesup-rtcweb-data-protocol-00 which is basically bidirectional udp. Which can be a really valuable alternative to WebSockets as doesn't have the "negative" sides of a tcp connection.
No, Signaling is not defined by WebRTC.
Here is an post by the IETF which explains it pretty good why it is not:
http://www.ietf.org/mail-archive/web/rtcweb/current/msg01143.html
This means that you are free to choose how you exchange network information. I.e. you could use websockets, HTTP and even Email, but that would be a bit of a struggle :)

Categories