audio frequency detection in javascript - javascript

is there anyway to determine the frequencies in audio? I would like to create a waveform to react to frequencies, but I am not sure where to start, I would like to create my own without having to use other pre-created one.
I know that mozilla has something similar to this, but I want something that I can use across all browsers, so is there anyway I can detect frequencies in audio through javascript code.

Related

javascript audio API -- analyze audio file to detect exact sounds, for lip syncing

I've seen things like waveform.js which uses the Web Audio API to display waveform data, and there are many other tools out there which are able to analyze the exact sound points of an audio file in JavaScript.
If so, it should be possible to use this power of analyzation to use for real-time lip syncing using JavaScript, i.e., to get an animated character to speak at the same time the user is speaking, by simply using an audio context, and reading the data-points some how to find the right sounds.
So the question becomes, more specifically:
How exactly do I analyze audio data to extract what exact sounds are made at specific timestamps?
I want to get the end result of something like Rhubarb Lip Sync, except with JavaScript, and in real time. It doesn't have to be exact, but as close as possible.
There is no algorithm that allows you to detect phonemes correctly 100% of the time.
You didn't say whether this was for real-time use or for offline use, but that would strongly affect which algorithm you'd use.
An algorithm based on mel frequency cepstral coefficients would be expected to give you about 80% accuracy, which would be good enough for video games or the like.
Deep learning systems based on covolutional neural nets would give you excellent recognition, but they are not real time systems (yet).
You could maybe start with Meyda, for example, and compare the audio features of the signal you're listening to, with a human-cataloged library of audio features for each phoneme.

How to take video snapshot with VLC Web Plugin

I currently need to extract snapshot(s) from an IP Camera using RTSP on a web page.
VLC Web Plugin works well with playing stream, but before I get my hands dirty on playing with its Javascript API, can some one tell me whether the API can help me to take the snapshot(s) of the stream, like the way it done with VLC Media Player, cuz it does not present on the above page.
If the answer is 'No', please give me some other way to do this.
Thanks in advance.
Dang Loi.
The VLC plugin only provides metadata properties accessible from JavaScript.
For this reason there is no way to access the bitmap/video itself as plugins runs sand-boxed in the browser. The only way to obtain such data would be if the plugin itself provided a mechanism for it.
The only way to grab a frame is therefor to use a generic screen snagger (such as SnagIt), of course, without the ability to control it from JavaScript.
You could, as an option, look into the HTML5 Video element to see if you can use your video source with that. In that case you could grab frames, draw them to canvas and from there save it as an image.
Another option in case the original stream format isn't supported, is to transcode it on the fly to a format supported by the browser. Here is one such transcoder.

Recording audio from multiple microphones simultaneously with getUserMedia()

it is possible to access different microphones at the same time using getUserMedia()?
This whould be useful to
filter out background noise;
create some sort of stereoscopic effect;
make available multiple audio tracks for an international
streaming conference.
Apparently, it is quite tricky for video source:
Capture video from several webcams with getUserMedia
I was wondering if, for the audio source, the problem was different.
You should be able to do this but I imagine the browser support will let you down somewhere along the line.
You should be able to create several media sources by specifying a microphone ID when using getUserMedia. You can find the IDs of all connected media devices using MediaDevices.enumerateDevices()
Once you have two separate microphone inputs you should be able to get the data using an AudioContext.
Then it's a case of doing what ever you're doing to the bit data before it's output to the browser.
This is all very high level and the details of the actual implementation would probably take quite a long time to figure out but as for your question: Yes, it should be possible if browser support is there.

how to generate sound of exact frequecy in javascript

I want to create a project in JavaScript in which I want to generate sounds of all frequency from 20 to 20K. is there any way to do ?
I have used http://codebase.es/riffwave/ this library also but if I select frequency of sound greater than 20K db, I am still hearing that. so this js library is not accurate according to my need.
can you suggest some way in achieving my task.
The Web Audio API can be used to generate sounds.
http://chimera.labs.oreilly.com/books/1234000001552/ch04.html#s04_5

Sounds for my javascript game with pure js?

Is it possible to play a sound in my javascript game without using anything like flash or a HTML workaround?
When I searched google it gave me various plugins or ways of doing it with flash, but I'd like to do it with pure js, while being able to play many sounds parallel.
The HTML5 <audio> element has fairly decent support, and there's also an audio API (which allows advanced audio manipulation) lagging somewhat behind.
There's even a tutorial on using it in a JS game.

Categories