We Keep Coding

WebRTC Piping: Video Negotiation through Intermediate Server

I am developing a 3-part system for a remote vehicle to connect to a base station via WebRTC, which will pipe video from the remote vehicle to a remote station (also using an RTC Peer Connection). So, the architecture looks like this:
Vehicle -- PC --> Base Station -- PC --> Remote Station
Where PC is a peer connection. Video is sent from vehicle to Base Station using RTCPeerConnection's addTrack method. The video track must then be sent from Base Station (using addTrack method) to Remote Station via Remote Station's onTrack handler. The problem is, NodeJS (which Base Station is implemented in) does not natively support MediaStreams (Which are only available via a 3rd party library) and I am not able to find a way to trigger the onTrack handler in Remote Station. If Vehicle-Remote Station connect, the video comes through perfectly, but the Base Station is not triggering the appropriate event in Remote Station when piping. I added the package to support media streams, but this produces an error trying to locally store the stream and/or track (see tempStream/tempTrack variables),and does not seem to resolve the issue of the ontrack handler not being triggered in the Remote Station and displaying the video.
Here is the handler for the vehicle adding a track to pc, and trying to pipe that to rcspc, the RTCPeerConnection object for Remote Station. Any tips or insight is greatly appreciated.
//Event listener for new track
/**
* When pc receives a track (video), pipe it over to Remote Station
* using the addTrack method. (Currently NOT piping the video appropriately)
* #param {} event
*/
pc.ontrack = (event) =>{
console.log("--------------------RECEIVED TRACK--------------------");
if (event.streams && event.streams[0]) {
console.log('received track from vehicle');
//add these streams to a temporary holding place
tempStream.push(event.streams);
tempTrack = event.track;
//checking that rcspc peer connection has configured the remote description/finished
//signaling process
if(rcspc.remoteDescription){
//changing to tempStream/tempTrack does not resolve the issue here.
rcspc.addTrack(event.track, event.streams[0]);
}
} else {
if (!remoteStream) {
remoteStream = new MediaStream();
//videoElem.srcObject = remoteStream;
console.log('created new remote stream!');
}
remoteStream.addTrack(event.track);
}
};

To onTrack to fire on the other side of rcspc connection, you need offer/answer exchange after you've called addTrack. Otherwise the receiving side won't know what codec is used for the media you sending.
From mozilla docs:
Note: Adding a track to a connection triggers renegotiation by firing a negotiationneeded event. See Starting negotiation for details.
So I would try creating a new offer on rcspc after you add the track. Then wait for a new answer and set remote description. Then onTrack should fire on RS-side.
Or you can do offer-answer exchange in onnegotiationneeded callback, should make no difference.

Webrtc handling multiple streams, reduce bandwidth

Hey Folks
I am new on WebRTC and have a question about how to start and stop a WebRTC track to save bandwidth.
My backend provide multiple video streams so my SDK looks like this:
a=ssrc:3671328831 msid:user3339856498#host-602bca webrtctransceiver2
a=ssrc:3671328831 cname:user3339856498#host-602bca
a=mid:video0
a=ssrc:3671267278 msid:user3339856498#host-602bca webrtctransceiver3
a=ssrc:3671267278 cname:user3339856498#host-602bca
a=mid:video1
My goal is to let the user choose which stream they can look,
my problem is that, tracks they are added from RTCPeerConnection already send data.
when I set the track.enabled=false it also send data with full bandwith.
Is there a way to control start and stop they track so that no data is transmittet (RTCRtpReceiver)
thanks

OpenVidu - Adding stream for existing publisher - Javascript

I am using openvidu for one of my projects. Say there are 2 users User-A and User-B. Both are publishers and wish to subscribe to each others videos. User-A initiated conference session. Once User-B join streamCreated is triggered for User-B in javascript on User-A browser. But User-B is unaware that User-A exists. For this we pull exisiting publishers for get-session api call, but that does not give access to stream. How to get access to stream, all we have is stream id

User B will always receive (just after successfully calling Session.connect() method) one connectionCreated event for each user already connected to the session and one streamCreated event for each stream already being published into the session.

webRTC how to tell if there is audio

I am using WebRTC with Asterisk, and getting an error about 5% of the time where there is no audio due to an error with signaling. The simple fix is if there is no audio coming through, then stop the connection and try again. I know this is a bandaid while I fix the real issue. For now though, I will bandaid my code.
To get the audio I am doing the following:
var remoteStream = new MediaStream();
peerConnection.getReceivers().forEach((receiver) => {
remoteStream.addTrack(receiver.track);
});
callaudio.srcObject = remoteStream;
callaudio.play();
The problem here is that the remote stream always adds a track even when there is no audio coming out of the speakers.
If you inspect chrome://webrtc-internals you can see there is no audio being sent, but there is still a receiver. You can look at the media stream, and see there is indeed an audio track. There is everything supporting the fact that I should be hearing something, but I hear nothing 5% of the time.
My solution is to get the data from the receiver track and check if there is anything coming across there, but I have no Idea how to read that data. I have the web audio API working but it only works if there is some sound currently being played. Sometimes the person on the other end does not talk for 10 seconds. I need a way to read the raw data and see that something is going across that. I just want to know is there ANY data on a MediaStream!
If you do remoteStream.getAudioTracks() you get back an audio track because there is one, there is just no audio going across that track.

In the lastest API, receiver.track is present before a connection is made, even if it goes unused, so you shouldn't infer anything from its presence.
There are at least 5 ways to check when audio reception has been negotiated:
Retroactively: Check receiver.track.muted. Remote tracks are born muted, and receive an unmute event if/once data arrives:
audioReceiver.track.onunmute = () => console.log("Audio data arriving!");
Proactively: Use pc.ontrack. A track event is fired as a result of negotiation, but only for tracks that will receive data. An event for trackEvent.track.kind == "audio" means there will be audio.
Automatic: Use the remote stream provided in trackEvent.streams[0] instead of your own (assuming the other side added one in addTrack). RTCPeerConnection populates this one based on what's negotiated only (no audio track present unless audio is received).
Unified-plan: Check transceiver.direction: "sendrecv" or "recvonly" means you're receiving something; "sendonly" or "inactive" means you're not.
Beyond negotiation: Use getStats() to inspect .packetsReceived of the "inbound-rtp" stats for .kind == "audio" to see if packets are flowing.
The first four are deterministic checks of what's been negotiated. The fifth is only if everything else checks out, but you're still not receiving audio for some reason.
All these work regardless of whether the audio is silent or not, as you requested (your question is really is about what's been negotiated, not about what's audible).
For more on this, check out my blog with working examples in Chrome and Firefox.

Simple hack is:
On first second play back to server 1hz tone.
If server got it on first second, server play back 2hz, if no, play back 1hz.
If client not got 2hz back from server, it restart.
Please note, you should be muted while do that.

WebRTC: Unable to successfully complete signalling process using DataChannel

I've been having trouble establishing a WebRTC session and am trying to simplify the issue as much as possible. So I've written up a simple copy & paste example, where you just paste the offer/answer into webforms and click submit.
The HTML+JS, all in one file, can be found here: http://pastebin.com/Ktmb3mVf
I'm on a local network, and am therefore removing the ICE server initialisation process to make this example as bare-bones as possible.
Here are the steps I'm carrying out in the example:
Page 1
Page 1 (loads page), enters a channel name (e.g. test) and clicks create.
A new Host object is created, new PeerConnection() and createDataChannel are called.
createOffer is called, and the resulting offerSDP is pasted into the offer textarea.
Page 2
Copy offerSDP from Page 1 and paste into offer textarea on Page 2, click join.
New Guest object is created, PeerConnection and an ondatachannel handler is set.
setRemoteDescription is called for the Guest object, with the offerSDP data.
createAnswer is called and the result is pasted into the answer textarea box.
Page 1
The answerSDP is copied from Page 2 and pasted into the answer textarea of Page 1, submit answer is clicked.
Host.setRemoteDescription is called with the answerSDP data. This creates a SessionDescription, then peer.setRemoteDescription is called with the resulting data.
Those are the steps carried out in the example, but it seems I'm missing something critical. After the offerer's remoteDescription is set with the answerSDP, I try to send a test message on the dataChannel:
Chrome 40
"-- complete"
> host.dataChannel.send('hello world');
VM1387:2 Uncaught DOMException: Failed to execute 'send' on 'RTCDataChannel': RTCDataChannel.readyState is not 'open'
Firefox 35
"-- complete"
ICE failed, see about:webrtc for more details
> host.dataChannel.send('hello world');
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
I also had a more complicated demo operating, with a WebSocket signalling server, and ICE candidates listed, but was getting the same error. So I hope this simplification can help to track down the issue.
Again, the single-file code link: http://pastebin.com/Ktmb3mVf

To enable webRTC clients to connect to each other, you need ICE. While STUN and TURN, which you don't need for such a test, are part of that, even without these helpers you still need to use ICE to tell the other end which IP/port/protocol to connect to.
There are two ways to do this: Google's "trickle ice", where the SDP (answer/offer) is passed on without any ICE candidates. These are then transported over a separate signaling layer and added as they are discovered. This speeds up the connection process, as ICE takes time and some late ICE candidates might not be needed.
The classic method is to wait until all ICE candidates have been gathered, and then generate the SDP with these already included.
I have modified your latest version to do that: http://pastebin.com/g2YVvrRd
You also need to wait for the datachannel/connection to become available before being able to use it, so I've moved the sending of the message to the channels onopen event.
The significant changes to the original code:
The interface callbacks were removed from Host.prototype.createOffer and Guest.prototype.createAnswer, instead we attach the provided callback function to the respective objects for later use.
self.cb = cb;
Both Host and Guest have an added ICE handler for the PeerConnection:
var self = this;
this.peer.onicecandidate = function (event) {
// This event is called for every discovered ICE candidate.
// If this was trickle ICE, you'd pass them on here.
// An event without an actual candidate signals the end of the
// ICE collection process, which is what we need for classic ICE.
if (!event.candidate) {
// We fetch the up to date description from the PeerConnection
// It now contains lines with the available ICE candidates
self.offer = self.peer.localDescription;
// Now we move on to the deferred callback function
self.cb(self.offer);
}
}
For the guest self.offer becomes self.answer
The interface handler $("#submitAnswer").click() does not send the message anymore, instead it is send when the datachannel is ready in the onopen event defined in setChannelEvents().
channel.onopen = function () {
console.log('** channel.onopen');
channel.send('hello world!');
};