I'm programming a very simple WebRTC application to stream real-time video from a RaspberryPi Zero camera. I'm using Linux Project's UV4L driver to setup the server and JavaScript to connect and play the video stream. My JavaScript code is based on UV4L's demo, which essentially uses RTC web socket methods to perform negotiation.
Their code works beautifully in Chrome, but doesn't seem to work under Firefox or Safari.
RTCPeerConnection = window.webkitRTCPeerConnection;
RTCSessionDescription = window.RTCSessionDescription;
RTCIceCandidate = window.RTCIceCandidate;
var ws;
function signal(url, onStream, onError, onClose, onMessage) {
if("WebSocket" in window) {
var pc;
ws = new WebSocket(url);
ws.onopen = function () {
var config = {"iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]};
pc = new RTCPeerConnection(config); // <---- ERROR here.
pc.onicecandidate = function (event) {
// ... ICE negotiation.
};
if('ontrack' in pc) {
pc.ontrack = function(event) {
// ... set stream object and play
};
} else { // onaddstream() is deprecated
pc.onaddstream = function (event) {
// ... set stream object and play
};
}
// ... other event listeners.
ws.send(...); // Signals the remote peer to initiate a call
};
}
}
In particular, I get an error When I try to connect, the following error is thrown in Firefox v60.0.1 (and a very similar in Safari):
TypeError: RTCPeerConnection is not a constructor
According to MDN docs, Firefox has support for this constructor since v22. What could be the issue?
My error turned out to be a silly typo. The declaration of RTCPeerConnection at the beginning of the code, was wrong. It should be:
RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection;
Related
I have tried a couple of solutions already, but nothing works for me.
I want to stream audio from my PC to another computer with almost zero latency. Things are working fine so far in a sense of lagging and everything, sound is clear and not choppy at all, but there is something like a delay between the moment when audio starts playing on my PC and remote PC. For example when I click on Youtube 'play' button audio starts playing only after 3-4 seconds on the remote machine. The same when I click 'Pause', the sound on the remote PC stops after a couple of seconds.
I've tried to use websockets\plain audio tag, but no luck so far.
For example this is my solution by using websockets and pipes:
def create_pipe():
return win32pipe.CreateNamedPipe(r'\\.\pipe\__audio_ffmpeg', win32pipe.PIPE_ACCESS_INBOUND,
win32pipe.PIPE_TYPE_MESSAGE |
win32pipe.PIPE_READMODE_MESSAGE |
win32pipe.PIPE_WAIT, 1, 1024 * 8, 1024 * 8, 0, None)
async def echo(websocket):
pipe = create_pipe()
win32pipe.ConnectNamedPipe(pipe, None)
while True:
data = win32file.ReadFile(pipe, 1024 * 2)
await websocket.send(data[1])
async def main():
async with websockets.serve(echo, "0.0.0.0", 7777):
await asyncio.Future() # run forever
if __name__ == '__main__':
asyncio.run(main())
The way I start ffmpeg
.\ffmpeg.exe -f dshow -i audio="Stereo Mix (Realtek High Definition Audio)" -acodec libmp3lame -ab 320k -f mp3 -probesize 32 -muxdelay 0.01 -y \\.\pipe\__audio_ffmpeg
On the JS side the code is a little bit long, but essentially I am just reading a web socket and appending to buffer
this.buffer = this.mediaSource.addSourceBuffer('audio/mpeg')
Also as you see I tried to use -probesize 32 -muxdelay 0.01 flags, but no luck as well
I tried to use plain tag as well, but still - this couple-of-seconds delay exists
What can I do? Am I missing something? Maybe I have to disable buffering somewhere?
I have some code, but all I learned was from https://webrtc.github.io/samples/ website and some from MDN. It's pretty simple.
The idea is to connect 2 peers using a negotiating server just for the initial connection. Afterwards they can share streams (audio, video, data). When I say peers I mean client computers like browsers.
So here's an example for connecting, and broadcasting and of course receiving.
Now for some of my code.
a sketch of the process
note: the same code is used for connecting to and connecting from. this is how my app works bcz it's kind of like a chat. ClientOutgoingMessages and ClientIncomingMessages are just my wrapper around sending messages to server (I use websockets, but it's possible also ajax).
Start: peer initiates RTCPeerConnection and sends an offer via server. also setup events for receiving. The other peer is notified of the offer by the server, then sends answer the same way (should he choose to) and finally the original peer accepts the answer and starts streaming. Among this there is another event about candidate I didn't even bothered to know what it is. It works without knowing it.
function create_pc(peer_id) {
var pc = new RTCPeerConnection(configuration);
var sender
var localStream = MyStreamer.get_dummy_stream();
for (var track of localStream.getTracks()) {
sender = pc.addTrack(track, localStream);
}
// when a remote user adds stream to the peer connection, we display it
pc.ontrack = function (e) {
console.log("got a remote stream")
remoteVideo.style.visibility = 'visible'
remoteVideo.srcObject = e.streams[0]
};
// Setup ice handling
pc.onicecandidate = function (ev) {
if (ev.candidate) {
ClientOutgoingMessages.candidate(peer_id, ev.candidate);
}
};
// status
pc.oniceconnectionstatechange = function (ev) {
var state = pc.iceConnectionState;
console.log("oniceconnectionstatechange: " + state)
};
MyRTC.set_pc(peer_id, {
pc: pc,
sender: sender
});
return pc;
}
function offer_someone(peer_id, peer_name) {
var pc = MyRTC.create_pc(peer_id)
pc.createOffer().then(function (offer) {
ClientOutgoingMessages.offer(peer_id, offer);
pc.setLocalDescription(offer);
});
}
function answer_offer(peer_id) {
var pc = MyRTC.create_pc(peer_id)
var offer = MyOpponents.get_offer(peer_id)
pc.setRemoteDescription(new RTCSessionDescription(offer));
pc.createAnswer().then(function (answer) {
pc.setLocalDescription(answer);
ClientOutgoingMessages.answer(peer_id, answer);
// alert ("rtc established!")
MyStreamer.stream_current();
});
}
handling messages from server
offer: function offer(data) {
if (MyRTC.get_pc(data.connectedUser)) {
// alert("Not accepting offers already have a conn to " + data.connectedUser)
// return;
}
MyOpponents.set_offer(data.connectedUser, data.offer)
},
answer: function answer(data) {
var opc = MyRTC.get_pc(data.connectedUser)
opc && opc.pc.setRemoteDescription(new RTCSessionDescription(data.answer)).catch(function (err) {
console.error(err)
// alert (err)
});
// alert ("rtc established!")
MyStreamer.stream_current();
},
candidate: function candidate(data) {
var opc = MyRTC.get_pc(data.connectedUser)
opc && opc.pc.addIceCandidate(new RTCIceCandidate(data.candidate));
},
leave: function leave(data) {
MyRTC.close_pc(data.connectedUser);
},
I have a working WebRTC connection in Chrome. It uses 1 data channel as part of a chat application.
I want to also support Firefox thus I need to change some not supported events:
For the RTCPeerConnection as well as for the DataChannel.
The changes to the data channel worked as expected:
//chrome implenetation
dc.onopen = this.conncectionStats.bind(this);
dc.onmessage = onMessage;
// chrome and firefox
dc.addEventListener('open', (event) => {
this.conncectionStats.bind(this)
});
dc.addEventListener('message', (event) => {
onMessage(event)
});
However, the problem arises when changing the PeerConnection:
// chrome implenetation
pc.onconnectionstatechange = this.onConnectionStateChange.bind(this);
// chrome and firefox
pc.addEventListener('onconnectionstatechange', (event) => {
console.log("onconnectionstatechange fired")
this.onConnectionStateChange.bind(this);
})
The event is never occuring. Any ideas why this is the case?
The event should be correct, but on the other hand, the documentation is missing on MDN Web Docs.
You should use WebRTC adapter so that unsupported events will be shimmed for you:
https://github.com/webrtc/adapter
I am using it on my webpages, and onconnectionstatechange fires fine in Firefox:
...
pc.onconnectionstatechange = onConnStateChange;
...
function onConnStateChange(event) {
if (pc.connectionState === "failed") {
Terminate();
alert("Connection failed; playback stopped");
}
}
I am trying to build video chat but unable to understand some piece of code:
I have Found that part of Code From WEBRTC Sample
link:-https://github.com/webrtc/samples/commit/ecca1124803688bf512874188624f6d4538f69d0
var servers = null;
pc1 = new RTCPeerConnection(servers);
trace('Created local peer connection object pc1');
pc1.onicecandidate = function(e) {
onIceCandidate(pc1, e);
};
pc2 = new RTCPeerConnection(servers);
trace('Created remote peer connection object pc2');
pc2.onicecandidate = function(e) {
onIceCandidate(pc2, e);
};
pc1.oniceconnectionstatechange = function(e) {
onIceStateChange(pc1, e);
};
pc2.oniceconnectionstatechange = function(e) {
onIceStateChange(pc2, e);
};
pc2.ontrack = gotRemoteStream;
What is happening when I am passing null value to RTCPeerConnection()?
The constructor RTCPeerConnection accepts a configuration object. Among other things, one of the configurations required by most apps would be iceServers.
iceServers is a list of STUN or TURN servers. For example, your constructor could look like:
var configuration = {
"iceServers": [{ "urls": ["stun:stun.1.google.com:19302"] }]
};
myConnection = new RTCPeerConnection(configuration);
In simple words these STUN and TURN servers help the two peers discover a direct path to each other. However when you leave out the servers, it means the app will be able to discover the peer only within the intranet, or essentially within the set of devices that your computer can interact directly with. [Citation Needed]
In any production app, you would most likely have to add the STUN and TURN servers. But since you are referring to the code from a tutorial, where connection across internet is not required, you can skip the servers.
I've been working on browser extensions that interact with a local application running a WebSocket server.
Safari and Chrome Extensions were very easy to implement, and after some headache getting a feel for FF development, I thought I would be able to implement WebSockets as I had in the other browsers. However I have had some issues.
I understand that I can't directly create a WebSocket in the "main" js file, and so attempted to use workarounds I found on the internet:
https://github.com/canuckistani/Jetpack-Websocket-Example uses a page-worker as a sort of proxy between main and the WebSocket code. When I implement this code, my WebSocket connection immediately errors w/ {"isTrusted":true} as the only information.
I also tried to use a hiddenframe as it appears this is how 1Password deals with websocket communication in their FF Addon, but this also results in the same immediate error.
When I simply open a websocket connection to my server in my normal FF instance, it connects perfectly, but so far, I haven't gotten anything to work from addon.
making pageWorker with:
var pw = pageWorker.Page({
contentUrl: self.data.url('com.html'),
contentScriptFile: self.data.url('com.js')
})
com.html:
<!DOCTYPE HTML>
<html>
<body>
</body>
</html>
com.js:
document.onready = launchCom();
// Could this need to be on ready?
function launchCom() {
console.log("[com.js] launchCom Called");
var wsAvailable = false;
if ("WebSocket" in window) {
console.log("[com.js] Detected Websocket in Window, attempting to open...");
// WebSocket is supported.
ws = new WebSocket('ws://localhost:9001');
wsAvailable = true;
} else {
console.log("[com.js] Websocket is not supported, upgrade your browser!");
}
}
ws.onmessage = function(event) {
console.log(event.data);
}
ws.onopen = function(evt) {
console.log("[com.js] ws opened. evt: " + evt);
}
ws.onerror = function(evt) {
console.log("[com.js] ws error: " + JSON.stringify(evt));
}
Running this results in:
console.log: xxx: [com.js] launchCom Called
console.log: xxx: [com.js] Detected Websocket in Window, attempting to open...
console.log: xxx: [com.js] ws error: {"isTrusted":true}
console.log: xxx: [com.js] ws closed. evt: {"isTrusted":true}
Any help would be greatly appreciated!
I've solved the problem:
I'm using https://github.com/zwopple/PocketSocket in my OS X application as my server, and there appears to be an issue with PocketSocket and FF.
After changing PocketSocket's PSWebSocketDriver.m line 87 code from
[[headers[#"Connection"] lowercaseString] isEqualToString:#"upgrade"]
to
[[headers[#"Connection"] lowercaseString] containsString:#"upgrade"]
per https://github.com/zwopple/PocketSocket/issues/34,
I was able to open a WebSocket connection from FF addon using the original code, but the server errored on messages.
Setting network.websocket.extensions.permessage-deflate to false in about:config allowed messages to be sent so I added
require("sdk/preferences/service").set("network.websocket.extensions.permessage-deflate", false);
to my main.js and everthing is working!
The tiny change to PocketSocket's code hasn't had any effects on the server interacting with other WebSocket clients.
I also got stuck in similar situation as websocket can't be implemented directly in main.js. I also did the same as you did , may be server is refusing connection. Snippet from my code look like below :
main.js
var wsWorker = require('sdk/page-worker').Page({
contentURL: "./firefoxScript/webSocket.html",
contentScriptFile : ["./firefoxScript/webSocket.js"]
});
webSocket.html
<html>
<head>
<title></title>
</head>
<body>
</body>
</html>
webSocket.js
var ws = new WebSocket('ws://localhost:9451');
ws.onopen = function() {
console.log('Connection open...');
};
ws.onclose = function() {
console.log('Connection closed...');
};
ws.onmessage = function(event) {
console.log('Message recieved...');
};
ws.onerror = function(event) {
console.log('Connection Error...');
};
It's perfectly working fine for me.
I'm experimenting with WebRTC between two browsers using RTCPeerConnection and my own long-polling implementation. I've created demo application, which successfully works with Mozilla Nightly (22), however in Chrome (25), I can't get no remote video and only "empty black video" appears. Is there something wrong in my JS code?
Function sendMessage(message) sends message to server via long-polling and on the other side, it is accepted using onMessage()
var peerConnection;
var peerConnection_config = {"iceServers": [{"url": "stun:23.21.150.121"}]};
// when message from server is received
function onMessage(evt) {
if (!peerConnection)
call(false);
var signal = JSON.parse(evt);
if (signal.sdp) {
peerConnection.setRemoteDescription(new RTCSessionDescription(signal.sdp));
} else {
peerConnection.addIceCandidate(new RTCIceCandidate(signal.candidate));
}
}
function call(isCaller) {
peerConnection = new RTCPeerConnection(peerConnection_config);
// send any ice candidates to the other peer
peerConnection.onicecandidate = function(evt) {
sendMessage(JSON.stringify({"candidate": evt.candidate}));
};
// once remote stream arrives, show it in the remote video element
peerConnection.onaddstream = function(evt) {
// attach media stream to local video - WebRTC Wrapper
attachMediaStream($("#remote-video").get("0"), evt.stream);
};
// get the local stream, show it in the local video element and send it
getUserMedia({"audio": true, "video": true}, function(stream) {
// attach media stream to local video - WebRTC Wrapper
attachMediaStream($("#local-video").get("0"), stream);
$("#local-video").get(0).muted = true;
peerConnection.addStream(stream);
if (isCaller)
peerConnection.createOffer(gotDescription);
else {
peerConnection.createAnswer(gotDescription);
}
function gotDescription(desc) {
sendMessage(JSON.stringify({"sdp": desc}));
peerConnection.setLocalDescription(desc);
}
}, function() {
});
}
My best guess is that there is a problem with your STUN server configuration. To determine if this is the issue, try using google's public stun server stun:stun.l.google.com:19302 (which won't work in Firefox, but should definitely work in Chrome) or test on a local network with no STUN server configured.
Also, verify that your ice candidates are being delivered properly. Firefox doesn't actually generate 'icecandidate' events (it includes the candidates in the offer/answer), so an issue with delivering candidate messages could also explain the discrepancy.
Make sure your video tag attribute autoplay is set to 'autoplay'.