JS pitch shift with timbre control - javascript

i need a good pitch shift solution for my project to change the voice. a lot of pitch shift js libraries around - tried them all but they don't provide the desired result. main thing is no control on the result voice timbre and i get Mickey mouse or hell zombie sounding stuff but not real voices with it. while here the result is just outstanding if to test with vega's voice: http://www.sonicapi.com/docs/live-task-demo?task=process-elastiqueTune#demo_form
unfortunately i'm total zero with audio processing and wanna know at least how it's done, what a type of shifting algorythm is used here and how we can achieve timbre/formant control over the process. any hints highly appreciated. thanks ;)

This question touches a very broad subject. Here's a few pointers.
Pitch can be, in general, shifted by offsetting the frequencies that form the voice material. An easy version of this is resampling in temporal domain, where in essential the recording is played back in a different speed. This naturally leads to a tempo change as well which is often not desirable.
In order to preserve the tempo, you need to "explode" the material into its components, in other words, make a domain change from temporal domain to frequency domain. This is what Fourier Transform is for. Once done, you have an estimate of set of frequencies (and respective phases if properly done in complex space) per sample.
The perceived timbre of the voice depends on the relative amplitudes of the frequency set called overtones. Overtones are formed in the speaker's vocal tract and to the listener, heard together with the fundamental frequency. You can control the timbre using different filters in either time domain, spectral
(frequency) domain or cepstral domain. This kind of signal processing is a subject for a library section full of books.
You can move from back from the spectral (frequency) domain to the temporal (time) domain using inverse Fourier transform.
To sum up, the naive approach to shift the pitch you need to transform the samples from temporal to spectral domain, resample along the time axis, and then do the inverse Fourier transform to get back to the time domain.
Besides Fourier transform, you could use wavelets. I hope this gets you started.

Related

How to make boids pursue a movable character in a 2d game

I was wondering what would be the simplest way to make my group of boids pursue my character in a top down 2d game with a path finding algorithm?
I'm thinking I'd have to first find a path to the player with the path finding, get the desired vector and then subtract that from the velocity to get the steering value.
I found "Pursuit and Evasion steering behaviors" on Craig Reynolds' website but I'm not sure if I'm heading in the right direction.
Note that there is a little more detail in a 1999 GDC paper. There are also many open source implementations, such as this (in retrospect, surprisingly complicated) one in OpenSteer.
An agent’s seek behavior simply assumes a desired velocity toward a static target, and defines a “steering force” which is the error/difference between that and its current velocity.
So a pursue behavior requires only to estimate a current target location. A rough approximation is usually good enough, since it is discarded after this animation frame / simulation step. Typically steering behaviors use constant velocity predictions, simply multiplying the current velocity by some time interval. A helpful estimate for that interval is the distance to the quarry agent, divided by our current speed.
How this interacts with (a) the pursuers being a flock, and (b) a simulation based on path finding, would depend on specific details of your game. Probably you want to blend a small pursuit “force” with the flocking behavior so it does not interfere with the nature of the flock.

Vibration (shaking) elements in Box2D after using the Apply Force

Sorry for my bad English.
I do a little game in JavaScript using Box2D (although I think the language is not important, because the engine is the same, as is well known, and has been exported to JavaScript).
Connecting several bodies via Distance Joint. When creating a joint pointing frequencyHz = 11 and dampingRatio = 0 (as I understand it, the less dampingRatio, the faster the item returns to its original position, that is, the spring will be tougher that I need). For each body to use force:
body.ApplyForce(new b2Vec2(0, 5800), body.GetWorldCenter())
That I needed to have these bodies was greater gravity than others.
After creating several elements, they begin to twitch, to vibrate as if they were something haunted, stop and engage in a certain position.
I tried to "play" with parameters frequencyHz and dampingRatio have joint, and sought to design not twitch, but then the construction seems less harsh and smoother.
There is some sort of solution?
Thanks.
I think that the structure is vibrating due to the fact that the weight of several elements, and the power applicable to them, the object is pulled down by increasing the length of the joint, the joint is aimed szhimatsya back to its original position, due to this vibration is obtained, because the element moves quickly in different directions

How can I detect the direction/distance of movement on iOS with javascript? [duplicate]

I was looking into implementing an Inertial Navigation System for an Android phone, which I realise is hard given the accelerometer accuracy, and constant fluctuation of readings.
To start with, I set the phone on a flat surface and sampled 1000 accelerometer readings in the X and Y directions (parallel to the table, so no gravity acting in these directions). I then averaged these readings and used this value to calibrate the phone (subtracting this value from each subsequent reading).
I then tested the system by again placing it on the table and sampling 5000 accelerometer readings in the X and Y directions. I would expect, given the calibration, that these accelerations should add up to 0 (roughly) in each direction. However, this is not the case, and the total acceleration over 5000 iterations is nowhere near 0 (averaging around 10 on each axis).
I realise without seeing my code this might be difficult to answer but in a more general sense...
Is this simply an example of how inaccurate the accelerometer readings are on a mobile phone (HTC Desire S), or is it more likely that I've made some errors in my coding?
You get position by integrating the linear acceleration twice but the error is horrible. It is useless in practice.
Here is an explanation why (Google Tech Talk) at 23:20. I highly recommend this video.
It is not the accelerometer noise that causes the problem but the gyro white noise, see subsection 6.2.3 Propagation of Errors. (By the way, you will need the gyroscopes too.)
As for indoor positioning, I have found these useful:
RSSI-Based Indoor Localization and Tracking Using Sigma-Point Kalman Smoothers
Pedestrian Tracking with Shoe-Mounted Inertial Sensors
Enhancing the Performance of Pedometers Using a Single Accelerometer
I have no idea how these methods would perform in real-life applications or how to turn them into a nice Android app.
A similar question is this.
UPDATE:
Apparently there is a newer version than the above Oliver J. Woodman, "An introduction to inertial navigation", his PhD thesis:
Pedestrian Localisation for Indoor Environments
I am just thinking out loud, and I haven't played with an android accelerometer API yet, so bear with me.
First of all, traditionally, to get navigation from accelerometers you would need a 6-axis accelerometer. You need accelerations in X, Y, and Z, but also rotations Xr, Yr, and Zr. Without the rotation data, you don't have enough data to establish a vector unless you assume the device never changes it's attitude, which would be pretty limiting. No one reads the TOS anyway.
Oh, and you know that INS drifts with the rotation of the earth, right? So there's that too. One hour later and you're mysteriously climbing on a 15° slope into space. That's assuming you had an INS capable of maintaining location that long, which a phone can't do yet.
A better way to utilize accelerometers -even with a 3-axis accelerometer- for navigation would be to tie into GPS to calibrate the INS whenever possible. Where GPS falls short, INS compliments nicely. GPS can suddenly shoot you off 3 blocks away because you got too close to a tree. INS isn't great, but at least it knows you weren't hit by a meteor.
What you could do is log the phones accelerometer data, and a lot of it. Like weeks worth. Compare it with good (I mean really good) GPS data and use datamining to establish correlation of trends between accelerometer data and known GPS data. (Pro tip: You'll want to check the GPS almanac for days with good geometry and a lot of satellites. Some days you may only have 4 satellites and that's not enough) What you might be able to do is find that when a person is walking with their phone in their pocket, the accelerometer data logs a very specific pattern. Based on the datamining, you establish a profile for that device, with that user, and what sort of velocity that pattern represents when it had GPS data to go along with it. You should be able to detect turns, climbing stairs, sitting down (calibration to 0 velocity time!) and various other tasks. How the phone is being held would need to be treated as separate data inputs entirely. I smell a neural network being used to do the data mining. Something blind to what the inputs mean, in other words. The algorithm would only look for trends in the patterns, and not really paying attention to the actual measurements of the INS. All it would know is historically, when this pattern occurs, the device is traveling and 2.72 m/s X, 0.17m/s Y, 0.01m/s Z, so the device must be doing that now. And it would move the piece forward accordingly. It's important that it's completely blind, because just putting a phone in your pocket might be oriented in one of 4 different orientations, and 8 if you switch pockets. And there's many ways to hold your phone, as well. We're talking a lot of data here.
You'll obviously still have a lot of drift, but I think you'd have better luck this way because the device will know when you stopped walking, and the positional drift will not be a perpetuating. It knows that you're standing still based on historical data. Traditional INS systems don't have this feature. The drift perpetuates to all future measurements and compounds exponentially. Ungodly accuracy, or having a secondary navigation to check with at regular intervals, is absolutely vital with traditional INS.
Each device, and each person would have to have their own profile. It's a lot of data and a lot of calculations. Everyone walks different speeds, with different steps, and puts their phones in different pockets, etc. Surely to implement this in the real world would require number-crunching to be handled server-side.
If you did use GPS for the initial baseline, part of the problem there is GPS tends to have it's own migrations over time, but they are non-perpetuating errors. Sit a receiver in one location and log the data. If there's no WAAS corrections, you can easily get location fixes drifting in random directions 100 feet around you. With WAAS, maybe down to 6 feet. You might actually have better luck with a sub-meter RTK system on a backpack to at least get the ANN's algorithm down.
You will still have angular drift with the INS using my method. This is a problem. But, if you went so far to build an ANN to pour over weeks worth of GPS and INS data among n users, and actually got it working to this point, you obviously don't mind big data so far. Keep going down that path and use more data to help resolve the angular drift: People are creatures of habit. We pretty much do the same things like walk on sidewalks, through doors, up stairs, and don't do crazy things like walk across freeways, through walls, or off balconies.
So let's say you are taking a page from Big Brother and start storing data on where people are going. You can start mapping where people would be expected to walk. It's a pretty sure bet that if the user starts walking up stairs, she's at the same base of stairs that the person before her walked up. After 1000 iterations and some least-squares adjustments, your database pretty much knows where those stairs are with great accuracy. Now you can correct angular drift and location as the person starts walking. When she hits those stairs, or turns down that hall, or travels down a sidewalk, any drift can be corrected. Your database would contain sectors that are weighted by the likelihood that a person would walk there, or that this user has walked there in the past. Spatial databases are optimized for this using divide and conquer to only allocate sectors that are meaningful. It would be sort of like those MIT projects where the laser-equipped robot starts off with a black image, and paints the maze in memory by taking every turn, illuminating where all the walls are.
Areas of high traffic would get higher weights, and areas where no one has ever been get 0 weight. Higher traffic areas are have higher resolution. You would essentially end up with a map of everywhere anyone has been and use it as a prediction model.
I wouldn't be surprised if you could determine what seat a person took in a theater using this method. Given enough users going to the theater, and enough resolution, you would have data mapping each row of the theater, and how wide each row is. The more people visit a location, the higher fidelity with which you could predict that that person is located.
Also, I highly recommend you get a (free) subscription to GPS World magazine if you're interested in the current research into this sort of stuff. Every month I geek out with it.
I'm not sure how great your offset is, because you forgot to include units. ("Around 10 on each axis" doesn't say much. :P) That said, it's still likely due to inaccuracy in the hardware.
The accelerometer is fine for things like determining the phone's orientation relative to gravity, or detecting gestures (shaking or bumping the phone, etc.)
However, trying to do dead reckoning using the accelerometer is going to subject you to a lot of compound error. The accelerometer would need to be insanely accurate otherwise, and this isn't a common use case, so I doubt hardware manufacturers are optimizing for it.
Android accelerometer is digital, it samples acceleration using the same number of "buckets", lets say there are 256 buckets and the accelerometer is capable of sensing from -2g to +2g. This means that your output would be quantized in terms of these "buckets" and would be jumping around some set of values.
To calibrate an android accelerometer, you need to sample a lot more than 1000 points and find the "mode" around which the accelerometer is fluctuating. Then find the number of digital points by how much the output fluctuates and use that for your filtering.
I recommend Kalman filtering once you get the mode and +/- fluctuation.
I realise this is quite old, but the issue at hand is not addressed in ANY of the answers given.
What you are seeing is the linear acceleration of the device including the effect of gravity. If you lay the phone on a flat surface the sensor will report the acceleration due to gravity which is approximately 9.80665 m/s2, hence giving the 10 you are seeing. The sensors are inaccurate, but they are not THAT inaccurate! See here for some useful links and information about the sensor you may be after.
You are making the assumption that the accelerometer readings in the X and Y directions, which in this case is entirely hardware noise, would form a normal distribution around your average. Apparently that is not the case.
One thing you can try is to plot these values on a graph and see whether any pattern emerges. If not then the noise is statistically random and cannot be calibrated against--at least for your particular phone hardware.

Pitch detection on array of floating point numbers

I'm doing voice recording in javascript, and storing the recording as an array of signed floats. What would I need to determine (and ultimately, adjust) pitch on the array? I've seen various algorithms for C++, but they don't seem to be very helpful in my situation. I even downloaded and tried this one to see if I could convert parts of it to javascript:
http://voicerecorder.codeplex.com/SourceControl/latest
But all that actually did was make the recording louder, regardless of the settings I chose.
I'm not going to try to provide an exhaustive answer here, but rather describe my own findings discovered on my journey of wrestling with similar issues in audio programming.
Pitch Detection
If you're sound is monophonic (as it sounds that is is based on your comment to jeff), I've implemented pitch detection using auto-correlation techniques, mostly because it's relatively simple compared to other pitch detection algorithms.
The idea, if you're unfamiliar, is as follows:
Slide a sample over itself (with a predetermined window size; in 1-sample increments)
At each step, calculate the absolute difference between the original wave and the slid window (hard to explain verbally).
as you slide the window, keep a record of the score calculated in (2)
when the wave correlates with itself, the score will hit a minimum, and the time-location of this minimum specifies the signals periodicity.
In my implementation, this was the only algorithm that worked well (when fed samples of my voice; I didn't try a variety of samples however).
That was a crude explanation of how autocorrelation works, and this article provides a very nice comparison of different pitch-detection algorithms:
https://ccrma.stanford.edu/~pdelac/154/m154paper.htm
Pitch Shifting
Of course, you could get really cheap pitch shifting by just resampling, but that sounds similar to a record being played too fast, which is not acceptable in many circumstances.
As far as pitch shifting goes, I haven't gotten that far yet in my implementation, but last I left off, I was looking at phase vocoders as a possible solution. What's hard is finding a decent explanation of how these algorithms work that provides some intuition on the reason why they work the way they do instead of just providing soley abstract mathmatical equations.

Getting Started With Traffic Simulation in JavaScript

I am going to be asking a lot of questions in the upcoming months. For my ninth grade science fair project I would like to create a traffic simulator in order to test whether or not interconnected communicating traffic lights can increase traffic flow. I have a couple of generic questions that I need help with...
How would I represent roads?
How would I make car follow a road?
How would I make a car switch lanes or roads?
I am not looking for specific code, just good pointers and resources to help me get started. Any help is appreciated, C.Ruhl.
PS I am only in high school so no advanced math notations please :)
One possible approach which is taken quite often is to use a discrete model for roads and cars' positions.
Each position on the road can either be occupied by a car (blue dot) or be empty. Cars move at discrete time steps by exactly one position (if the target position is empty) along the given arrows. Thus a car can even switch lanes if it would otherwise have to slow down or stop.
You can further improve it by using separate timesteps per car (simulating faster/slower cars) or in many other ways.
After you've defined your roads (i.e. the positions and their follow-up positions) by an appropriate data structure this model is relatively easy to simulate but already shows interesting effects.
Forget about the UI.
Represent each object in its base form --only put object properties in it. Example, a car will have a size and ability to move. But it won't have the logic to make it move. Similarly a traffic light will have states such as green, amber and red. But it won't have the logic to switch between these states. Similar classes for roads, lanes etc.
Build a different class for the driver. This class will contain all methods such as lane shifting, stopping, turning, moving forward etc. More technically, this will be your "actor" and will act on the veichle. A similar actor would be for traffic light control which will act on a network of traffic lights. Make it an interface and have two implementations to it --one that takes advantage of interconnectedness and other that operates on static times.
Optional add a UI on top of this object model. Don't go fancy, have simple dots to begin with. Once you get all simple stuff working, adding more fancy features should be easy and impact free (relatively).
This will be a very challenging project.
But if your objective is a proof of concept, I have a simpler suggestion. You can go user generated here and get all the complexity of simulation out and all the accuracy in. Start with 15-20 remote controlled cars, a cardboard model of a fictional town, some bulbs to simulate traffic lights and some volunteers who know how to drive. Have a preprogrammed sequence of on and offs written on paper and assign some of the volunteers to control those lights. Have another set of volunteers control the cars. If you have hands on experience in basic electronics you can build a timer controlled circuit to control the lights.
All the very best!
You could try the SIM.JS discrete event simulation library in Javascript. They have a very simple example for traffic at road intersection simulation here.
Ooh, Conner, you've found an interesting question indeed -- and one which is the subject of research even today. Here's a suggestion: before you fret about how to do it in JavaScript, spend some time thinking just how to do it at all.
Here's a suggestion: think about the objects invovled first. You have Cars, and they travel along Roads. Start with a square grid of roads, so your cars go from intersection to intersection.
Pick a fixed speed for the cars, so it takes a constant time to travel from intersection to intersection.
Each intersection has a traffic light, which can be red or green. If it's red, of course cars can't go through; they have to wait.
Now, your basic program will look like
time = 0
while time < end-time:
for each car:
update the car's location
add time consumed to time
when you update the cars location, what happens? (Hint: the car moves; can it go through an intersection or not?)
That will give you a start.
For my Bachelor's degree exam I developed a traffic control web-app that tracked the vehicles in my town in real-time and I used google maps api.
I suggest you use a map service such as maps.google.com , yahoo.maps.com...
They have an api for everything... you can use markers to represent anything on the map (cars,street lights,even pedestrians :)) ) and you can use their api to calculate distances and paths.
It may seem a bit more complex then the average div-implementation, but, trust me, it's a big plus to use a service with a well-organised api.
+it would have a more professional look ;).

Categories