I'm trying to build a Javascript bot by using Puppeteer to open a https URL where I can listen for the microphone and output a transcript from the SpeechRecogniton API built in a browser, the below code seems to log something in normal Chrome, but on Chromium I get nothing despite this feature apparently being supported according to Modernizr. I've allowed microphone permissions but I get a dead console.log
window.SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition
const recognition = new SpeechRecognition()
recognition.interimResults = true
recognition.addEventListener('result', e => {
const transcript = Array.from(e.results)
.map(result => result[0])
.map(result => result.transcript)
.join('')
// I get nothing logged here in Chromium
console.log(transcript)
})
recognition.addEventListener('end', recognition.start)
recognition.start()
UPDATE
After adding the following...
recognition.addEventListener('error', function(event) {
console.log('Speech recognition error detected: ' + event.error);
});
I'm getting a Network error... and don't know what to do about this in Chromium?
Chromium no longer supports SpeechRecognition. I believe it's got something to do with Google wanting people to use their cloud services instead.
Related
Hello can you help me regarding speech synthesis in normal use if browser like chrome it's working, but when i use kiosk mode app in android and running google chrome as browser there's an error SpeechSynthesisUtterance is not defined. any encounter regarding this matter?
Please help.
Thank you.
const speak = () => {
const speech = new SpeechSynthesisUtterance(data.description);
speech.voice = voices?.find((x: any) => x.name === 'Google US English');
speech.rate = rate;
//window.speechSynthesis
ws.speak(speech);
};
i am currently implementing speech input on a website and using the web speech API for it.
voice recognition works as expected in Chrome on desktop and android. Firefox does not have support for the API so it is not being used there.
the problem is with chrome on iOS where the service throws a "service-not-allowed" error.
this error seems to be distinctly different from the "not-allowed" error that is being thrown when the browser does not have permission to use the microphone.
in my case chrome has all permissions it would need (microphone permission, pop-ups are not blocked).
at first i thought the problem was, that for some reason chrome on iOS does not show me the permission pop-up, specifically for microphone usage, directly on the web page, but now i am not so sure anymore.
does anyone have experience with this problem or have a solution for this?
here is the working code for android/desktop, the function gets triggered by a button click:
function startDictation() {
try {
var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition;
var recognition = new SpeechRecognition();
} catch (e) {
console.log(e);
}
if (recognition) {
recognition.continuous = false;
recognition.interimResults = true;
recognition.lang = $('html').attr('lang');
recognition.start();
recognition.onresult = function(e) {
$('#searchInput').val(e.results[0][0].transcript);
console.log(e.results[0][0].transcript);
};
recognition.onerror = (e) => {
console.error('Speech recognition error detected: ' + e.error);
recognition.stop();
};
recognition.onend = () => {
console.log('Speech recognition service disconnected');
}
}
}
a few helpful links
web speech api error values
web speech api demo from google, that also doesn't work on iOS for me
i have tried various end devices at this point, two different iPads and an iPhone and the same error gets thrown everywhere.
any help is appreciated, thanks!
The Broadcast Channel API seems like an alternative to postMessage or the Channel Messaging API (aka MessageChannel.)
I've used both successfully in recent versions of Google Chrome to send Shared Array Buffers; however, I am having trouble sending a Shared Array Buffer using the Broadcast Channel API.
Mozilla's doc at https://developer.mozilla.org/en-US/docs/Web/API/BroadcastChannel references the spec at https://html.spec.whatwg.org/multipage/web-messaging.html#broadcastchannel, which says:
For each destination in destinations...
Let data be StructuredDeserialize(serialized, targetRealm).
If this throws an exception, catch it, fire an event named messageerror at destination, using MessageEvent, with the origin attribute initialized to the serialization of sourceOrigin, and then abort these steps.
StructuredDeserialize is defined at https://html.spec.whatwg.org/multipage/structured-data.html#structureddeserialize and seems to suggest it covers SharedArrayBuffers:
Otherwise, if serialized.[[Type]] is "SharedArrayBuffer", then:
If targetRealm's corresponding agent cluster is not serialized.[[AgentCluster]], then then throw a "DataCloneError" DOMException.
Otherwise, set value to a new SharedArrayBuffer object in targetRealm whose [[ArrayBufferData]] internal slot value is serialized.[[ArrayBufferData]] and whose [[ArrayBufferByteLength]] internal slot value is serialized.[[ArrayBufferByteLength]].
Reading this, it seems to me that this should work, but I'm getting a message event where the data is simply null.
If this were a security issue, I would expect to get a messageerror event instead of a message event.
Here's my minimal test case:
broadcast-test.html (must be served from an http server - won't work via file://)
<!DOCTYPE html>
<html>
<head><title></title></head>
<body>
<script src="broadcast-test.js"></script>
</body>
</html>
broadcast-test.js
const isThisTheWorker = this.document === undefined
const broadcastChannel = new BroadcastChannel('foo')
if (!isThisTheWorker) {
broadcastChannel.addEventListener('message', (event) => {
console.log('main received', event.data)
const sab = new SharedArrayBuffer(100)
broadcastChannel.postMessage({ hello: 'from main', sab })
})
var myWorker = new Worker('broadcast-test.js')
}
else {
broadcastChannel.addEventListener('message', (event) => {
console.log('worker received', event.data)
})
broadcastChannel.postMessage({ hello: 'from worker' })
}
Observed console output: (Chrome 84.0.4147.135 on Windows 10)
main received {hello: "from worker"}
worker received null
Is Google Chrome's implementation incorrect, or am I misunderstanding the spec?
I have a working WebRTC connection in Chrome. It uses 1 data channel as part of a chat application.
I want to also support Firefox thus I need to change some not supported events:
For the RTCPeerConnection as well as for the DataChannel.
The changes to the data channel worked as expected:
//chrome implenetation
dc.onopen = this.conncectionStats.bind(this);
dc.onmessage = onMessage;
// chrome and firefox
dc.addEventListener('open', (event) => {
this.conncectionStats.bind(this)
});
dc.addEventListener('message', (event) => {
onMessage(event)
});
However, the problem arises when changing the PeerConnection:
// chrome implenetation
pc.onconnectionstatechange = this.onConnectionStateChange.bind(this);
// chrome and firefox
pc.addEventListener('onconnectionstatechange', (event) => {
console.log("onconnectionstatechange fired")
this.onConnectionStateChange.bind(this);
})
The event is never occuring. Any ideas why this is the case?
The event should be correct, but on the other hand, the documentation is missing on MDN Web Docs.
You should use WebRTC adapter so that unsupported events will be shimmed for you:
https://github.com/webrtc/adapter
I am using it on my webpages, and onconnectionstatechange fires fine in Firefox:
...
pc.onconnectionstatechange = onConnStateChange;
...
function onConnStateChange(event) {
if (pc.connectionState === "failed") {
Terminate();
alert("Connection failed; playback stopped");
}
}
My goal is to get data stored in device.
Like device which is measuring temp or whatever and store it to its memory. i need to query all this data device has via Record Access Control Point (RACP).
First thought, to achieve it, was
get characteristic
start notifications
write code to descriptor
get all data via eventListener
result: throws error on starting notifications
examples used:
https://googlechrome.github.io/samples/web-bluetooth/notifications-async-await.html
https://bugs.chromium.org/p/chromium/issues/detail?id=664863
Next thought was to not starting notification since characteristic is
INDICATE, WRITE type.
So was thinking about add listener and write to descriptor code from device docs which states:
OP Code:
1 – Report stored records
even with deleted startNotifications line is throwing error
so my code example is:
const mainService = 'my correct service uuid';
const characteristicUUID1 = 'my correct char uuid';
const characteristicUUID2 = 'my correct char uuid';
const descriptorUUID = '00002902-0000-1000-8000-00805f9b34fb';
let deviceCache = null;
let serverCache = null;
let serviceCache = null;
let characteristicCacheA = null;
let characteristicCacheB = null;
let descriptorCache = null;
try {
deviceCache = await navigator.bluetooth.requestDevice({ filters: [{ name: 'my device' }] });
console.log('Connecting to GATT Server...');
serverCache = await deviceCache.gatt.connect();
console.log('Getting Services...');
serviceCache = await serverCache.getPrimaryService(mainService);
console.log('Getting Characteristics A...');
characteristicCacheA = await serviceCache.getCharacteristic(characteristicUUID1);
console.log('Start Notifications A...');
await characteristicCacheA.startNotifications();
console.log('Getting Characteristics B...');
characteristicCacheB = await serviceCache.getCharacteristic(characteristicUUID2);
console.log('Start Notifications B...');
await characteristicCacheB.startNotifications();
console.log('Add event listener...');
characteristicCacheA.addEventListener('characteristicvaluechanged', this.handleNotifications);
console.log('Getting Descriptor...');
descriptorCache = await characteristicCacheA.getDescriptor(descriptorUUID);
console.log('Write value to descr...');
await descriptorCache.writeValue(new Uint8Array([1]));
} catch (error) {
console.log(error.message, 'error');
}
Error with notifications is(with experimental chrome features it doesn't throw error):
error: GATT operation failed for unknown reason.
Error with descriptor is:
writeValue() called on blocklisted object marked exclude-writes.
Also my device is asking for pin but web is connecting without prompting anything. And so maybe it says that writing to descriptor is blocked.
How to handle pin input - no clue(once i got prompt to enter pin after enabling chrome experimental features not sure if its related).
Is my logic correct? - dont think so.
Any suggestions?
What i have investigated so far?
https://googlechrome.github.io/samples/web-bluetooth/
https://www.oreilly.com/library/view/getting-started-with/9781491900550/ch04.html
https://webbluetoothcg.github.io/web-bluetooth/
Edit: After reading this article - https://medium.com/#devdevcharlie/experimenting-with-web-bluetooth-1f1176047ddd
I think correct logic should be, write to command characteristic commands you need(like get all data). After that find correct characteristic responsible for this data from device docs, and start notifications and add eventListener and receive data.
The call to writeValue() is failing because access to the CCCD is on the blocklist. The call to startNotifications() will write to the descriptor as necessary to enable notifications.
We need to investigate this "unknown reason" for startNotifications() failing. What operating system are you using? Please follow the instructions for reporting Web Bluetooth bugs and file an issue in the Chromium project's issue tracker.
As for now, chrome is not able or got issues with communicating with protected characteristics on windows 10, on macOS it is all working perfectly. I have posted issue on chromium bug tracker if someone is in to watching it.
https://bugs.chromium.org/p/chromium/issues/detail?id=960258#c6