JavaScript and microphone audio stream - javascript

Since HTML5 isn't ready yet, and getUserMedia doesn't work in browsers i have tested, I am asking that is there any "wrappers" so I could grap audio stream from microphone and send it to server with Javascript. Similar wrapper has been made with web cameras: (google for a "jquery-webcam-plugin". Two link restriction), but I haven't found any similar things for microphone audio streaming.
Audio can be in any commonly known format.
Flash is not my daily basis, flash tips might not help.
Since Google gears has been deprecated, I think it is not wise to use AudioApi. This is not a good news either, so I am guessing that only choices are Java and Flash.
I will appreciate every hint and tip I get.

I'm not sure what you mean by "Flash is not my daily basis". Your other option is Java, but that's less user-friendly. If you're going to require a plug-in (and you'll have to until getUserMedia() is implemented), you might as well make it Flash.
Here is a project that wraps a simple Flash app with a Javascript API that streams audio via an HTTP post to a web-server of your choice:
https://code.google.com/p/wami-recorder/
It's not ideal, but it works and does not require a clunky Flash Media Server.

Related

Can HTML5 Web Audio be as high quality as a DAW?

This question is mainly geared towards audio engineers.
I was looking at a few videos here and here concerning the Web Audio API in javascript and how people are using it to make music. I was wondering can this API be used to create sound quality on the level of existing Digital Audio Workstations (DAWs) like Ableton, FL Studio, Reason, Logic, etc.
Or is there some kind of inherent limitation to the browser? I think VSTs cannot be used by Web Audio API, but regardless of VSTs is there something that an audio engine used by DAWs can do that Web Audio cannot do in terms of quality? Or can the sound be just as good. I believe that samples can be played just as effectively (?), but maybe the synthesis would be limited?
Can HTML5 Web Audio be as high quality as a DAW?
Yes! Browsers use a high quality audio engine, with floating point samples all the way through.
Additionally, the API itself is one of the most straightforward to use and is quite extensible. It also supports very specific timing of control.
I think VSTs cannot be used by Web Audio API
That is correct.
but maybe the synthesis would be limited?
There are no real fundamental limitations. If you can code it, you can do it.
Now, there are a couple things you didn't ask about but still should know about. The first is an issue of latency. Browsers don't use ASIO or anything like that, so if you're using the Web Audio API to process and return low latency audio, you're going to have a bad time. In Chrome, you can improve this by letting Chrome open the audio device in exclusive mode.
Another issue is input/output channel count. This typically maxes out at 8 channels. Internally though, you can do pretty much whatever you want. The limitation is from the browser, not the Web Audio API itself.
There are some high quality audio applications that could run in the browser, but you can only do realtime audio in Chromium-based browsers, and you can't get realtime input even then.
Only Chromium supports high-priority audio threads in Audio Worklets, which you need for stable, realtime audio.
The WebMIDI API only runs on the main thread, which makes it useless for realtime input.
The WebUSB API is threadable, but does not permit communication with audio-class devices (which includes MIDI controllers (and soundcards)).
Keyboard, touch and mouse events must (currently) be handled by the main thread.
So, in short, you have no (practical) way to control realtime audio in Chromium, and cannot even do realtime audio (reliably) on any other platform.
Note: Adding support for WebMIDI workers was requested nine years ago, and the issue is still open (just for adding it to the spec).
Note: Electron will allow you to access protected USB device classes (Audio/MIDI), but Electron WebUSB support is still in progress. NWJS has WebUSB support, but no way to access protected device classes (though I opened an issue recently). WebUSB will work on these platforms eventually, but you would still need to implement web drivers for your controllers (even class-compliant ones), so this will not be an ideal solution, even once it it works.

Viability of HTML5 audio for a music web app?

To preface this, all of my experience has been with developing Ruby on Rails web apps serving, essentially, text data - I don't have any experience yet with rich media.
I'm looking into developing a music notation app (for practice and because I'm dissatisfied with many of the current options), and am trying to determine whether or not to build a native app or a web app. Since all of my programming experience lies in web development, I would prefer to make this a web app if I can, because if I chose to go native I would need to find a decent cross-platform solution in addition to the usual pitfalls of developing natively.
I suspect that even with HTML5 I wouldn't have access to enough processing power to make a browser-based audio engine feasible. It would need to have the ability to play and sequence not a single large .wav file of a song, but many short .wav files concurrently and in sequence. There would be no need for recording - only playback.
But since I've never done this before I really don't know if this is possible. So, I wanted to pose this to more experienced HTML5 media developers:
For modern browsers (I'm not too concerned with support for older browsers), would such an audio engine be feasible? (As in, is the current landscape of JS libraries and the HTML5 audio API sufficient for building something like this?) Or would something like this be far too CPU / memory hungry in a browser?
Thanks!
In my experience Chrome and Firefox for desktop both work great with multiple, sequenced, .wav files embedded in audio tags. You can trigger them with JavaScript and it all seems to work great.
Unfortunately, Mobile Safari seemed to be unhappy with more than one audio tag playing at once, and Chrome for Android had similar limitations. This was a year ago, so things might be different now.
Since we were targeting iPads, we wound up using Phonegap to mix in some native sound libraries into our HTML & JavaScript app. One gotcha that tripped us up: if you want looping audio samples, stick to .wav files; .mp3 files have some silence at the start and end that's really tricky to get rid of.
I've been investigating Web Audio Api to build synths, and it's still very primitive and difficult to use. The infrastructure seems to have had a boost when Web Audio was introduced, but not much development since. Mobile audio on the web is rarely mentioned, too clunky, or unsupported.
At the same time, JS has become significantly faster with the V8 engine, JS loading can happen as needed, asynchronously, and non-blocking, which may solve many of the CPU/memory concerns you have. Look into NodeJS and various module loaders such as webpack, ES6, for more info.
I think there will be attention to web-based music apps because the web can support multiple users. That could be a good reason to keep pushing through these murky waters.. especially if you're talking concurrent connections.
See Molgav tracker. It works on desktop and mobile, it uses hundred of sampled instruments, it has UI adapted for small mobile touchscreens. See how examples sounds.

Is HTML5 developed and documented enough?

I own a site which offers a Flash based music shopping cart which steams music using RTMP. Mobile devices are getting more and more popular and the exclusion of Flash means they don't support my Flash application. I was wondering if it's possible to create the same kind of Flash widget I have on my site in HTML5 or maybe even JavaScript alone? Is HTML5 developed enough for this kind of system? If possible, would users be able to embed an HTML5 version of our players? Can you stream music via RTMP using anything but Flash?
Thanks for your help
To answer your question YES HTML5 is documented and developed enough but the question that you should be asking is "Is HTML5 supported enough?"
HTML5 is markup AND JavaScript and its not supported in every browser. This is mitigated though by many great JavaScript libraries that will gracefully degrade depending on what browser/device is viewing it.
If you just want to learn more and make a decision for yourself I would suggest reading Dive into HTML5 to figure out what HTML5 is all about and what it offers.
No, you cannot stream music using RTMP in HTML5. However, you can offer music in mp3 and ogg formats. (You'll need to offer both to cover all browsers)
After looking at your site, I would recommend you continue to use flash because HTML5 does not support streaming audio, only audio available to download. While the shopping cart and everything else can mostly be done in HTML5, your users will still need Flash unless you want to allow users to download the music easily.

Detecting audio in third party online advertisements

Is is possible to detect when an online advertisement is playing audio? In an ideal world, I could do this through a browser in real-time and send a notification that would include the DOM information needed to remove the ad. In this type of model, I could theoretically then use all clients (web visitors) as test subjects. I believe that this might be far fetched, and thus will need to rely on a smaller subset of users. I'm looking for any way to do this. I have even thought about using speakers and a microphone to detect when a sound is playing, and then some how capture a screenshot and the DOM information. Please help. Thank you.
In a previous question, it was stated that this is not possible.
Edit:
No. The best you could do is maybe be able to detect that the browser is or is not playing sound. Even if you could do that, you would have difficulty detecting which tab or page was creating the sound. And if you could, you would have difficulty detecting the offending HTML. Could be flash, javascript, or HTML 5's tag. And even if you could do all of this, IE would surely (being the pain that it is) not support any of your attempts to complete this objective.
Bottom Line: You simply can't do what you are wanting to do. You would need to be able to query the browser for playing audio to be confident that you could handle all supported methods of playing sound in a browser. AFAIK, no browsers support this, so you are out of luck.

Get default media player using Javascript?

Whether is it possible to get default media player using Javascript, customize play controls and control play functions (Play, Stop, Seek, Pause..etc)
Thanks in advance,
Sri
No. The operating system's idea of a default media player (if it even has such a concept) is not accessible to the web browser, and most media players can't be controlled from a web page's JavaScript code anyway.
(With good reason: exposing a highly complex program like a media player to the Internet is a dangerous proposition. If just one bit of code in the handling of any format the player can read has an exploitable bug, then now every web page you visit could embed a media player and use the security hole to take over your machine. Indeed, this has happened many times in the past with Windows Media Player, QuickTime and RealPlayer, even though they were designed to be Internet-facing.)
You can embed a particular media player that was designed for it, and fall back to another media player if not available. So you could try to embed WMP, then fall back to QT if that failed, and fall back to Real if that failed.
But really, the days when it made any sense to do this are long gone. Today's it's all about the HTML5 and Flash video. The embedding of media players is dead, and good riddance to it.
I'm not sure what you mean but if you're talking about Windows Media player specifically I'm sure their embed version will have some skin Params, etc., you can play with.
Although I would advise you to take a look at HTML5 Media Player (http://html5demos.com/video) which is customizable and controllable via JavaScript
Here's a certified library that would be helpful in pushing html5 to the limit when it comes to video embedding: http://www.kaltura.org/project/

Categories