record audio from browser and visualize in real time - javascript

I am looking for a package which will take input of audio from browser through microphone then visualize it in the real time.Any javascript package available to implement.I check most of the audio visualizer but they need audio file .I want the package should directly get it from microphone in browser and then show visualization.I am implementing it in the nodejs so need something in java script

Google search of "web audio api visualizer with microphone" should have obviously pulled some decent results for you :)
local microphone stream + Web Audio api + canvas are the main requirements here!
Start with this volume meter example that gets input from microphone using webrtc api and show some visuals in GUI
Explore more examples from https://webaudiodemos.appspot.com/

Related

Can I save or record the audio I used in Web Speech API, also can I import audio or video? Javascript

I will make a transcript web app. I don't know if that is possible using javascript. Im not sure if I can combine Web Speech API and MediaStream Recording API. Please help me 😊
I want to find ways I can make a transcript web app with Javascript.

Web Speech API filter audio input

I am using Web Speech API for a chrome extension. Link
What I want to do is filter the audio being sent to the recognition instance. Does anyone know of a way to do this? The API really only gives you controls to the output of the recognition.

Web Audio Api integration with Web Speech Api - stream speaker/soundcard output to voice recognition api

Problem:
Ideally I would acquire the streaming output from the soundcard (generated by an mp4 file being played) and send it to both the microphone and speakers. I know I can use "getUserMedia" and "createChannelSplitter" (in the Web Audio Api) to acquire & split (based on Audacity analysis the original signal is in stereo) the user media into 2 outputs which leaves me with 2 problems.
getUserMedia can only get streaming input from the microphone
not from the soundcard (from what I have read)
streaming output can only be recorded/sent to a buffer and not sent
to the microphone directly (from what I have read)
Is this correct?
Possible workaround - stalled:
The user will most likely have a headset microphone on but one workaround I have thought of is to switch to the inbuilt microphone on the device and capture what comes out of the speakers and then switch back to the headset for user input. However, I haven't found a way to switch between the inbuilt microphone and the headset microphone without asking the user every time.
Is there a way to do this that I haven't found?
What other solutions would you suggest?
Project Explanation:
I am creating a Spanish language practice program/website written in html & javascript. An mp4 will play and the speech recognition api will display what it says on the screen (as it is spoken in Spanish) and it will be translated into english so the user hears, sees, and understands what is being said by the person speaking in the mp4. Then the user will use the headset microphone to answer the mp4 person (often the inbuilt microphone doesn't give good enough quality for voice recognition - depending on the device - thus the use of the headset).
flow chart of my workaround using inbuilt microphone
mp4->soundcard-> Web Audio Api -> channel 1 -> user's ears
channel 2 -> microphone input-> Web Speech Api-> html->text onscreen
flow chart of ideal situation skipping microphone input
mp4->soundcard-> Web Audio Api -> channel 1 -> user's ears
channel 2-> Web Speech Api-> html->text onscreen -> user's eyes
Another potential work around:
I would like to avoid having to manually strip an mp3 from each mp4 and then have to try and sync them so the voice recognition happens as the mp4 person speaks. I have read that I can run an mp3 through the voice recognition api.
The short answer is that there is not currently (12/19) a way to accomplish this on this platform with the tools and budget I have. I have opted for the laborious way to do this which is setting up individual divs with text blocks to be revealed as the person is speaking on a timer. I will still use the speech api to capture what the user says so the program can run the correct video in response.
Switching between speaker and user headset is a definite no go.
Speech recognition software usually requires clean and well captured audio. So, if the sound is coming from speakers, the users microphone is not likely to pick it up very well. And if the user is using headphones, then there is no way for the microphone to capture the audio at all.
As far as I know, you cannot send audio files Web Speech Api directly (I may be wrong here)
Web Speech Api Is not supported by all browsers so that is a downside to consider too: https://caniuse.com/#feat=speech-recognition
What I would recommend is checking out Google's Speech to text API: https://cloud.google.com/speech-to-text/
With this service you can send them directly the audio file and they will send back the transcription.
It does support streaming so you could have the audio transcribed at the same time it is playing. The timing wouldn't be perfect though.

Reading output audio data from Spotify Web Playback stream

I am currently playing around with audio visualization and I am trying to work with Spotify's Web Playback SDK to stream and analyze songs directly on my site.
However, I am unsure what the limitations are when it comes to actually reading the streamed data. I've noticed that an iframe is generated for the Spotify player, and I've read that spotify uses the encrypted media extensions to stream the audio on chrome.
Is it even possible to read the music data from the Spotify api? Maybe, I can read the output audio from the browser?
According to the web API documentation, you aren't able to play back full songs and get the audio data like you desire (for obvious reasons). Although, 30 second "song previews" are allowed through URL streaming, as well as full song playback on desktop browsers (excluding safari at the time of this post), with the Web Playback SDK.
However on the mobile API it is now possible to get the raw PCM data (Android or iOS). This will require you registering for a developers account and setting up the access tokens if you haven't already done so.
For quick reference, on Android it involves using the AudioController class.
EDIT : Thanks to #Leme for the Web Playback SDK link.

Access/process system audio with Javascript/Web Audio API

Is it possible to access system audio using the Web Audio API, in order to visualize or apply an equalizer to it? It looks like it's possible to hook up system audio to an input device that Web Audio API can access (i.e. Web Audio API, get the output from the soundcard); however ideally I would like to be able to process all sound output without making any local configuration changes.
No, this isn't possible. The closest you could get is installing a loopback audio device on the user's systems, like Soundflower on OSX, and use that as the default audio output device. It's not possible without local config changes.

Categories