I am using the Web Speech API in a web page. However, I am finding that it takes 3-5 seconds to bring back a result, which is a lot of lag time in today's web world. Has anyone else had this problem? Has anyone found a solution?
Here's the barebones of what I have so far -- it works, per se, but I need it to be faster.
var recognition = new webkitSpeechRecognition();
recognition.lang = 'en-US';
recognition.onresult = function(evt) {
console.log(evt);
};
recognition.start();
Assuming you don't a network speed issue, using
recognition.continuous = false;
will get a faster result (with some caveats)
see my detailed answer here
Related
So I have an Electron app that uses the web speech API (SpeechRecognition) to take the user's voice, however, it's not working. The code:
if ("webkitSpeechRecognition" in window) {
let SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
let recognition = new SpeechRecognition();
recognition.onstart = () => {
console.log("We are listening. Try speaking into the microphone.");
};
recognition.onspeechend = () => {
recognition.stop();
};
recognition.onresult = (event) => {
let transcript = event.results[0][0].transcript;
console.log(transcript);
};
recognition.start();
} else {
alert("Browser not supported.")
}
It says We are listening... in the console, but no matter what you say, it doesn't give an output. On the other hand, running the exact same thing in Google Chrome works and whatever I say gets console logged out with the console.log(transcript); part. I did some more research and it turns out that Google has recently stopped support for the Web Speech API in shell-based Chromium windows (Tmk, everything that is not Google Chrome or MS Edge), so that seems to be the reason it is not working on my Electron app.
See: electron-speech library's end Artyom.js issue another stackOverflow question regarding this
So is there any way I can get it to work in Electron?
I ended up doing an implementation that uses the media devices API to get the user's speech through their microphone and then sends it to a Python server using WebSockets which uses the audio stream with the SpeechRecognition pip package and returns the transcribed text to the client (Electron app).
This is what I implemented, it is way too long for a thing as simple as this, but if someone has a better suggestion, please do let me know by writing an answer.
I'm trying to use the SpeechRecognition API to create a speech-to-text interface.
Here's my configuration:
const recognition = new SpeechRecognition();
recognition.continuous = true;
recognition.lang = 'en-US';
recognition.onresult = (event) => {...}
recognition.start();
It works great on desktop, but it seems that most mobile devices don't support continuous listening, and when trying to record from them the microphone turns on and off constantly.
I tried to work around it by immediately starting it again after the onend event:
recognition.onend= (event) => {recognition.start();}
But the problem is that onend fires after the microphone has been turned off, which makes constant beeping sounds on some devices, and problems fully recognizing the speech.
Is there any way to intervene before the microphone is turned off?
I'm building a small HTML based software for an embedded system that has a client and server.
The software works on a single tab that is active all the time.
The client and the server exchange messages all the time.
After ~50 hours, the client is crushing.
After a brief investigation, It seems that the client keeps consuming memory until there is none left and a call to allocate new memory fails.
Then i get "Aw, snap" on the web page.
I tried debugging using chromium's "developer tools".
It seems that when i press the "Collect garbage" button, most of the memory clears up.
Here's the code for the client after reducing it as much as i can:
const socket = new WebSocket('ws://localhost:8070');
socket.binaryType = "blob";
socket.onopen = handleOpen;
socket.onmessage = handleSocketMessage;
function handleOpen() {
console.log('socket is ready');
}
function handleSocketMessage(message) {
// console.log(message);
}
The code runs on Variscite's imx6 SOM.
The OS is made with the Yocto project version Sumo.
Chromium version is 67.0.3396.99
Browsing online, chromium/chrome doesn't seem to allow manually calling the garbage collector.
Is there a way to go around this?
Can you think of a different possible solution?
What else could cause the client to consume all the OS's memory and then crush?
I'm trying to get sound working on my iPhone game using the Web Audio API. The problem is that this app is entirely client side. I want to store my mp3s in a local folder (and without being user input driven) so I can't use XMLHttpRequest to read the data. I was looking into using FileSystem but Safari doesn't support it.
Is there any alternative?
Edit: Thanks for the below responses. Unfortunately the Audio API is horribly slow for games. I had this working and the latency just makes the user experience unacceptable. To clarify, what I need is sounething like -
var request = new XMLHttpRequest();
request.open('GET', 'file:///./../sounds/beep-1.mp3', true);
request.responseType = 'arraybuffer';
request.onload = function() {
context.decodeAudioData(request.response, function(buffer) {
dogBarkingBuffer = buffer;
}, onError);
}
request.send();
But this gives me the errors -
XMLHttpRequest cannot load file:///sounds/beep-1.mp3. Cross origin requests are only supported for HTTP.
Uncaught Error: NETWORK_ERR: XMLHttpRequest Exception 101
I understand the security risks with reading local files but surely within your own domain should be ok?
I had the same problem and I found this very simple solution.
audio_file.onchange = function(){
var files = this.files;
var file = URL.createObjectURL(files[0]);
audio_player.src = file;
audio_player.play();
};
<input id="audio_file" type="file" accept="audio/*" />
<audio id="audio_player" />
You can test here:
http://jsfiddle.net/Tv8Cm/
Ok, it's taken me two days of prototyping different solutions and I've finally figured out how I can do this without storing my resources on a server. There's a few blogs that detail this but I couldn't find the full solution in one place so I'm adding it here. This may be considered a bit hacky by seasoned programmers but it's the only way I can see this working, so if anyone has a more elegent solution I'd love to hear it.
The solution was to store my sound files as a Base64 encoded string. The sound files are relatively small (less than 30kb) so I'm hoping performance won't be too much of an issue. Note that I put 'xxx' in front of some of the hyperlinks as my n00b status means I can't post more than two links.
Step 1: create Base 64 sound font
First I need to convert my mp3 to a Base64 encoded string and store it as JSON. I found a website that does this conversion for me here - xxxhttp://www.mobilefish.com/services/base64/base64.php
You may need to remove return characters using a text editor but for anyone that needs an example I found some piano tones here - xxxhttps://raw.github.com/mudcube/MIDI.js/master/soundfont/acoustic_grand_piano-mp3.js
Note that in order to work with my example you're need to remove the header part data:audio/mpeg;base64,
Step 2: decode sound font to ArrayBuffer
You could implement this yourself but I found an API that does this perfectly (why re-invent the wheel, right?) - https://github.com/danguer/blog-examples/blob/master/js/base64-binary.js
Resource taken from - here
Step 3: Adding the rest of the code
Fairly straightforward
var cNote = acoustic_grand_piano.C2;
var byteArray = Base64Binary.decodeArrayBuffer(cNote);
var context = new webkitAudioContext();
context.decodeAudioData(byteArray, function(buffer) {
var source = context.createBufferSource(); // creates a sound source
source.buffer = buffer;
source.connect(context.destination); // connect the source to the context's destination (the speakers)
source.noteOn(0);
}, function(err) { console.log("err(decodeAudioData): "+err); });
And that's it! I have this working through my desktop version of Chrome and also running on mobile Safari (iOS 6 only of course as Web Audio is not supported in older versions). It takes a couple of seconds to load on mobile Safari (Vs less than 1 second on desktop Chrome) but this might be due to the fact that it spends time downloading the sound fonts. It might also be the fact that iOS prevents any sound playing until a user interaction event has occured. I need to do more work looking at how it performs.
Hope this saves someone else the grief I went through.
Because ios apps are sandboxed, the web view (basically safari wrapped in phonegap) allows you to store your mp3 file locally. I.e, there is no "cross domain" security issue.
This is as of ios6 as previous ios versions didn't support web audio api
Use HTML5 Audio tag for playing audio file in browser.
Ajax request works with http protocol so when you try to get audio file using file://, browser mark this request as cross domain request. Set following code in request header -
header('Access-Control-Allow-Origin: *');
I have a python/flask application that sends mjpeg video to a element.
It works fine regarding the streaming, but I have a problem aborting the video. To abort the video at my phone (android) I have to click a link to another page or else the stream continues.
Currently I am controlling the stream with javascript. Setting the "src" to either a url for a static image from the cam, or an url to the video stream.
But between the src-change I first change it to "#".
A problem, using flask, is that when 1 client is receiving the stream (using generator & yield) no other cant communicate with the server. This might be the source to the problem?!
So, With javascript I control the stream with the following code:
if (streaming==false){
document.getElementById(img_id).src=C_vidsource;
streaming = true;
} else {
var currDate = new Date();
document.getElementById(img_id).src="#";
document.getElementById(img_id).src=C_statimage + "?" + currDate.getTime();
streaming = false;
}
I control this using a simple
I Think that androids web browser differ from the one I am using on the computer. It seems like it tries to download content before changing anything on the page. So it lets the videostream continue until the new image is loaded. But the new image will not be loaded until the stream has stopped.
is there a way to solve this?
Thanks!
Directly after i posted the question I found the solution.
I added an delay between the two src-changes.
after:
document.getElementById(img_id).src="";
I added
sleep(1000);
And sleep is a function I created (a very dirty):
function sleep(ms){
stoptime = Date.now() + ms;
while(Date.now() < stoptime){ }
return;
}
I guess that for a longer sleep this is not a good solutoion, but it solves my problem, or at least gives me a hint about what to search for.