We Keep Coding

Webrtc handling multiple streams, reduce bandwidth

Hey Folks
I am new on WebRTC and have a question about how to start and stop a WebRTC track to save bandwidth.
My backend provide multiple video streams so my SDK looks like this:
a=ssrc:3671328831 msid:user3339856498#host-602bca webrtctransceiver2
a=ssrc:3671328831 cname:user3339856498#host-602bca
a=mid:video0
a=ssrc:3671267278 msid:user3339856498#host-602bca webrtctransceiver3
a=ssrc:3671267278 cname:user3339856498#host-602bca
a=mid:video1
My goal is to let the user choose which stream they can look,
my problem is that, tracks they are added from RTCPeerConnection already send data.
when I set the track.enabled=false it also send data with full bandwith.
Is there a way to control start and stop they track so that no data is transmittet (RTCRtpReceiver)
thanks

Streaming Real Time Gyroscope of Phones(Android/IOS) to a Website Application on laptop

Hi im making a website and want to stream the gyroscope sensors that are in phones(android or iOS) to my website in real time. I have looked at this site https://community.wolfram.com/groups/-/m/t/1386358 however I am not sure on how to get this method to work as I've never used wolfram language. Does anyone have any ideas?

I think you might be better off with a codesandbox while you try to figure things out. I used one while trying to figure out some ThreeJS stuff, tapping in to the device gyroscope with the Device Orientation API.
You can user webrtc to create a peer to peer connection if you want to read the data on your laptop.
Here is another post on the topic:
How to access accelerometer/gyroscope data from Javascript?
Here is some sample code from that post:
window.addEventListener("deviceorientation", handleOrientation, true);
The event handler function can look something like this:
function handleOrientation(event) {
var absolute = event.absolute;
var alpha = event.alpha;
var beta = event.beta;
var gamma = event.gamma;
// Do stuff with the new orientation data
}

Media object doesn't respond to cookie containing object

Sorry if the question is very stupid
I am trying to show the stream from a person to another user using js
I have tried putting it in a cookie but it doesnt work even then.Even when the object in video is the same as the other
File 1
var video = document.querySelector("#videoElement");
var x=document.cookie
console.log(x)
video.srcObject = x;
File 2
var video = document.querySelector("#videoElement");
if (navigator.mediaDevices.getUserMedia) {
navigator.mediaDevices.getUserMedia({ video: true })
.then(function (stream) {
video.srcObject = stream;
document.cookie=video.srcObject
console.log(stream,video.srcObject)
})
.catch(function (err0r) {
console.log("Something went wrong!");
});
console.log(stream,video.srcObject)
}
I would like to just for now show it on two pages but for future what language should i use to store the video if you know you can share it

A cookie is not a centralized universal camera access repository on the web. Thank goodness.
A MediaStream is a local resource object representing active camera use, not a shareable URL.
This object lives solely in your local JS page, and isn't addressable on the web.
Since it doesn't live on any server, transporting the graphical bits from your camera to a friend's system, requires quite a bit of heavy lifting. This includes establishing an RTCPeerConnection which is the domain of WebRTC:
navigator.mediaDevices.getUserMedia({ video: true })
.then(function (stream) {
const iceServers = [{urls: "stun:stun.l.google.com:19302"}];
const pc = new RTCPeerConnection({iceServers});
for (const track of stream.getTracks())
pc.addTrack(track, stream);
/* lots of other WebRTC negotiation code */
You'll also typically need a server of some kind, both to solve discovery, i.e. point of contact, as well as a web socket server to exchange critcal offer/answer session descriptions that are necessary for connection establishment, between the two peers.
Perhaps the simplest proof of concept is this cut'n'paste demo, which let you and a friend exchange the required WebRTC offer/answer session descriptions manually, letting you establish a connection without any server, to see and talk to each other.
That has about a 70% chance of working. If you're both behind symmetric NATs (most mobile networks), then it gets harder still (you'll need a TURN server, which costs money).

webRTC how to tell if there is audio

I am using WebRTC with Asterisk, and getting an error about 5% of the time where there is no audio due to an error with signaling. The simple fix is if there is no audio coming through, then stop the connection and try again. I know this is a bandaid while I fix the real issue. For now though, I will bandaid my code.
To get the audio I am doing the following:
var remoteStream = new MediaStream();
peerConnection.getReceivers().forEach((receiver) => {
remoteStream.addTrack(receiver.track);
});
callaudio.srcObject = remoteStream;
callaudio.play();
The problem here is that the remote stream always adds a track even when there is no audio coming out of the speakers.
If you inspect chrome://webrtc-internals you can see there is no audio being sent, but there is still a receiver. You can look at the media stream, and see there is indeed an audio track. There is everything supporting the fact that I should be hearing something, but I hear nothing 5% of the time.
My solution is to get the data from the receiver track and check if there is anything coming across there, but I have no Idea how to read that data. I have the web audio API working but it only works if there is some sound currently being played. Sometimes the person on the other end does not talk for 10 seconds. I need a way to read the raw data and see that something is going across that. I just want to know is there ANY data on a MediaStream!
If you do remoteStream.getAudioTracks() you get back an audio track because there is one, there is just no audio going across that track.

In the lastest API, receiver.track is present before a connection is made, even if it goes unused, so you shouldn't infer anything from its presence.
There are at least 5 ways to check when audio reception has been negotiated:
Retroactively: Check receiver.track.muted. Remote tracks are born muted, and receive an unmute event if/once data arrives:
audioReceiver.track.onunmute = () => console.log("Audio data arriving!");
Proactively: Use pc.ontrack. A track event is fired as a result of negotiation, but only for tracks that will receive data. An event for trackEvent.track.kind == "audio" means there will be audio.
Automatic: Use the remote stream provided in trackEvent.streams[0] instead of your own (assuming the other side added one in addTrack). RTCPeerConnection populates this one based on what's negotiated only (no audio track present unless audio is received).
Unified-plan: Check transceiver.direction: "sendrecv" or "recvonly" means you're receiving something; "sendonly" or "inactive" means you're not.
Beyond negotiation: Use getStats() to inspect .packetsReceived of the "inbound-rtp" stats for .kind == "audio" to see if packets are flowing.
The first four are deterministic checks of what's been negotiated. The fifth is only if everything else checks out, but you're still not receiving audio for some reason.
All these work regardless of whether the audio is silent or not, as you requested (your question is really is about what's been negotiated, not about what's audible).
For more on this, check out my blog with working examples in Chrome and Firefox.

Simple hack is:
On first second play back to server 1hz tone.
If server got it on first second, server play back 2hz, if no, play back 1hz.
If client not got 2hz back from server, it restart.
Please note, you should be muted while do that.

WebRTC: Can't set local description after creating an answer

I'm currently attempting to create a simple video chat service using WebRTC with Ajax for the signalling method.
As per the recommendation of another Stack Overflow user, in order to make sure I was understanding the flow of a standard WebRTC app properly, I first created a simple WebRTC video chat service in which I printed the created offer or answer and ICE candidates out to the screen, and manually copied and pasted that info into a text area in the other client window to process everything. Upon doing that, I was able to successfully get both videos to pop up.
After getting that to work properly, I decided to try and use Ajax as the signalling method. However, I can't seem to get it to work now.
In my current implementation, every time offer/answer or ICE candidate info is created, I instantly create a new Ajax object, which is used to add that info (after the JSON.stringify method has been executed on it) to a DB table. Both clients are constantly polling that DB table, searching for new info from the other client.
I've been echoing a lot of information out to the console, and as far as I can tell, a valid offer is always sent from one client to another, but upon receiving that offer, successfully setting it as the remote description, and creating an answer, any attempts I make to set the local description of the "answerer" fails.
Is there any particular reason why this might happen? Here's a snippet of my code:
var i,
len;
for (i = 0, len = responseData.length; i < len; i += 1) {
message = JSON.parse(responseData[i]);
if (message.type === 'offer') {
makeAnswer(message);
}
// Code omitted,
}
...
makeAnswer = function (offer) {
pc.setRemoteDescription(new RTCSessionDescription(offer), function () {
pc.createAnswer(function (desc) {
// An answer is always properly generated here.
pc.setLocalDescription(desc, function () {
// This success callback function is never executed.
setPayload(JSON.stringify(pc.localDescription));
}, function () {
// I always end up here.
});
});
});
};
In essence, I loop through any data retrieved from the DB (sometimes there's both an offer and lots of candidate info that's gathered all at once), and if the type property of a message is 'offer', I call the makeAnswer function, and from there, I set the remote description to the received offer, create an answer, and try to set the answer to the local description, but it always fails at that last step.
If anyone can offer any advice as to why this might be happening, I would be very appreciative.
Thank you very much.

Well, I figured out the problem. It turns out that I wasn't encoding the SDP and ICE info before sending it to a PHP script via Ajax. As a result, any plus signs (+) in the SDP/ICE info were being turned into spaces, thus causing the strings to differ between the local and remote clients and not work.
I've always used encodeURIComponent on GET requests with Ajax, but I never knew you had to use that function with POST requests as well. That's good to know.
Anyway, after I started using the encodeURIComponent function with the posted data, and then fixed my logic up a bit so that ICE candidates are never set until after both local and remote descriptions are set, it started working like a charm every time.
That's the good news. The bad news is that everything was working fine on my local host, but as soon as I ported the exact same code over to my web-hosted server, even though the console was reporting that the offer/answer and ICE info were all properly being received and set, the remote video isn't popping up.
Sigh. One more hurdle to cross before I can be done with this.
Anyway, just to let everyone know, the key is to use encodeURIComponent before sending the SDP/ICE info to a server-side script, so that the string received on the other end is exactly the same.