How to get "Web audio spatialization" to work - javascript

I am trying to move the position of a media source when it gets a position update to allow for proximity voice chat.
right now all the voices are at a constant volume no matter where the panner or listner are set in the world
https://gist.github.com/DrMeepso/127ee76ea62581fcf517464e58c58f59
At case 21 of the websocket player postions are stored as vector6
X, Y, Z: position and A, B, C: rotation
I've tryed all diffrent types of syntax and i just cant seem to get it to work. any help would be apresheated

Related

Rotation around a point causes weird glitches in THREE.js

Every element from my scene is made of a chain of 3 Object3Ds. The order parent-to-child is cellPivot -> modifier -> setup
setup's purpose is to permanently align a loaded object by resizing / giving some padding that must always be there. It is not supposed to be changed once set
modifier's purpose is to actually perform the real transformation on the object
cellPivot's purpose is to allow me to drag modifier into a cell grid
An example why all this is needed: let's say I have a vertical door in an orthographic perspective that I wanna fit in a 1x1 space, so I give some padding on the x-axis to align the door in the center, similar to the picture below where the orange block is the door
Since I want to move this in any cell in the map, I use cellPivot's position to decide where. I can't use right away modifier since sometimes I wanna rotate the model inside the cell, which requires to modify both position and rotation (since my models are not built around (0, 0, 0), but along +X and +Z)
I have succesfully managed to rotate these doors by rotating modifier around the center of the model (which acts as a pivot). Here's the functions that does the rotation:
three.Object3D.prototype.pivot = function(pivot, f) {
pivot = lib.VecToVector3(three, pivot); // just a conversion between libs
this.position.sub(pivot);
f(this);
this.position.add(pivot);
return this;
};
three.Object3D.prototype.pivotRotate = function(pivot, axis, theta, rotational = false, abs = false) {
if(abs)
theta -= this.rotation.y; /// not good, handles only y
this.pivot(pivot, () => this.position.applyAxisAngle(axis, theta));
if(rotational)
this.rotateOnAxis(axis, theta);
return this;
};
The line that rotates the door and works:
this.o3d.userData.modifier.pivotRotate(this.o3d.userData.center, new three.Vector3(0, 1, 0), this.rot, true);
I'm now trying to do something similar with the player too. I record what keys are pressed, I calculate the normal of the vector of desired direction (if I press W and D I'll get (1, 1), if I press just W I'll get (0, 1)), after which I use the following line to detect the angle at which the user wanna move:
Math.atan2(-normal[1], normal[0]);
I have already tested that the angle is correct. On top of that, the codebase before "rotating around a pivot" used the same code and it worked fine
Everytime there's actually a direction the user wanna go, I'll run the following line:
this.o3d.userData.modifier.pivotRotate(this.o3d.userData.center, new three.Vector3(0, 1, 0), Math.atan2(-normal[1], normal[0]), true, true);
If the user just keeps a key pressed, then abs will make sure that no visible rotation is made (since theta will be 0)
Here's the problem: everytime I press A, be it in combination with W or S or not, the character will rotate like insane. I put after the line from above the following code to see what's happening:
com.log(new three.Euler().setFromQuaternion(this.o3d.userData.setup.getWorldQuaternion(new three.Quaternion())));
I'm getting this:
As you can see, x and z are reaching -pi, and y bouces back and forth. This does not happen for any other combination that does not contain key A
Update after 2 days:
I have rewrote my function like this:
I got these in console while trying to move in the problematic positions:
As it can be seen in the first log, my target is at rotation 0 and is going for -2.35..., but rotAfterRot is showing weird results..: -pi and -.78...
This is the result of running this.rotateOnAxis(axis, theta). I have changed this exact line with this.rotation.y += theta. Now everything is working as it should be: no weird -pi and rotAfterRot.y is actually theta
My guess is that rotateOnAxis is also counting other features of the object, like position, but still can't figure how it spits that -pi

webapi 3D spatialization: how to apply distance delay

Goal
I'm trying to build a site with spatialized audio.
Measure
I'm using WebAudioAPI's PannerNode.
My Audiopipeline looks like: audioTrack → PannerNode → audioContext.destination
Expected
I thought a PannerNode would do everything to create 3D audio experience.
Problem
I feel like there is no sound delay between the left and right speaker. The kind of sound delay that occurs because both ears have a spatial distance to each other.
Question
Is my feeling wrong, and there is actually a delay between left and right speaker? (maybe my settings are wrong/ not optimal.)
Is there a convenient way of implementing the sound delay or do I need to go through track -> PannerNode -> SplitAudioInChannels -> ApplySoundDelayPerChannel -> Output manually?
Code
I'm basically using the code from mozilla's tutorial here.
PannerNode is setup as follows:
var panner = new PannerNode(audioCtx, {
panningModel: 'HRTF',
distanceModel: 'linear',
positionX: audioSource.x,
positionY: audioSource.y,
positionZ: 0,
orientationX: 0.0,
orientationY: 0.0,
orientationZ: 0.0, // -1 = face out of the screen
refDistance: 1,
maxDistance: 400,
rolloffFactor: 10,
coneInnerAngle: 360, //60,
coneOuterAngle: 360, //90,
coneOuterGain: 0.3
})
To hear the spatialization effect, you need to change your orientation and/or position. To see if your panner is doing something, I'd look at Canopy to see the left and right channels from the panner. Adjusting the position and/or orientation should show there is a delay difference between the left and right channels.
Here's a short example for use in Canopy to show that the left and right channels have different delays. Paste this into the code window and press the triangle on the left to render. You'll see waveforms at the top, one for each channel. If you zoom in, you can see the first channel doesn't start at the same time as the second.
// #channels 2
// #duration 1.0
// #sampleRate 44100
let osc = new OscillatorNode(context, {type: "square");
let p = new PannerNode(context);
p.panningModel = "HRTF";
p.positionX.value = 10;
p.positionY.value = 10;
p.positionZ.value = 10;
osc.connect(p).connect(context.destination);
osc.start();

Web Audio Spatialization based on view direction

I'm trying to modulate the volume of some sound based on the view direction between the camera and the sound. So if you are fully looking at the sound source the volume is 100%, if you turn away it is turned down.
Setting the built-in directionalCone, which links to the Panner Audio API is not what i want. This defines if audio is enabled while the player is positioned inside the cone, i'd like to to work based on the view direction.
I have something working in Aframe, by doing a dot between the camera view direction and direction between the player and audio clip. However this (for some reason) is quite expensive, i'm wondering if there is some built in functionality that i am overlooking.
tick: function() {
if(!this.sound.isPlaying) return; //todo: this is true even outside the spatial distance!
var camFwd = this.camFwd;
this.camera.object3D.getWorldPosition(camFwd);
var dir = this.dir;
this.el.object3D.getWorldPosition(dir);
dir.subVectors(
camFwd, //camera pos
dir //element pos
).normalize();
this.camera.object3D.getWorldDirection(camFwd);
var dot = THREE.Math.clamp(camFwd.dot(dir), 0, 1);
//float dot = Mathf.dot(transform.forward, (camTrans.position-transform.position).normalized);
this.setVolume(THREE.Math.lerp(
this.data.minVolume,
this.data.maxVolume,
dot));
},
This gives the intended effect, but it shows up in the performance profiler as quite expensive. Especialy the getWorldDirection for some reason is costly, eventhough the hierarchy itself is simple.
Especialy the getWorldDirection for some reason is costly
Object3D.getWorldPosition() and Object3D.getWorldDirection() always force a recomputation of the object's world matrix. Depending on the time when tick is executed, it is sufficient to do this:
camFwd.setFromMatrixPosition( this.camera.object3D.matrixWorld );
dir.setFromMatrixPosition( this.el.object3D.matrixWorld );
The code just extracts the position from the world matrix without updating it. You can use a similar approach for the direction vector although the code is a bit more complex:
var e = this.camera.object3D.matrixWorld.elements;
camFwd.set( e[ 8 ], e[ 9 ], e[ 10 ] ).normalize();

Force camera.lookAt to maintain current Z rotation

So, I'm putting together a 3D project that will eventually make it's way onto a kiosk in the office of a client, basically shows all of their branches as points on a globe. One of the features requested is that end users be able to swipe on the screen to orbit around the globe, in addition, they should be able to rotate the camera on it's Z axis via a rotation gesture. The problem is I'm using camera.lookAt in the animation loop, which relies on the up vector being updated correctly whenever I rotate the camera in order to not "snap" back to place along the previous up vector when a user swipes, and for the life of me, I cannot get it to cooperate.
Currently, what I'm doing to update the up vector (based off of another stackoverflow thread with a similar issue) is this:
//Current full 360 degree angle of rotation, calculated earlier
let radian = THREE.Math.degToRad(full);
//Create new vector at radian angle to camera's current position
let v1 = new THREE.Vector3(_this.object.position.x + Math.cos(radian), _this.object.position.y + Math.sin(radian), _this.object.position.z).sub( _this.object.position ).normalize();
//_this.target = 0,0,0
let v2 = _this.target.clone().sub( _this.object.position ).normalize();
//Cross vectors to get the proper up
let v3 = new THREE.Vector3().crossVectors( v1, v2 ).normalize();
_this.object.up.copy( v3 );
And this works... up to the point where the camera seemingly inverts once I head near the side of the globe opposite the camera's starting position (0,0,1.75) and then negates my rotations (as far as I can tell) which causes the same "snap" to a different rotation like before.
Once I rotate the camera, I want it to maintain the rotation when using lookAt, regardless of the lookAt inverting everything.

Javascript spring physics: Apply direction vector to spring

I'm implementing spring physics in Javascript, inspired by this blog post. My system uses Hooke's Law formula to calculate spring forces:
F = -k(|x|-d)(x/|x|) - bv
I made a CodePen that shows the implementation of a spring between two points. The CodePen has two points connected by a spring, and every 2 seconds the point positions are randomized. You can see the points bounce on the spring towards each other.
If you look at the source, you can see I've defined a direction vector for my spring:
var spring = {
length: 100,
direction: {
x: 1, y: 1
}
};
I'm trying to make it so that the spring always "resolves" in this direction. Put another way, I'd like the spring to always be "pointing" up and to the right. If this were implemented in the CodePen, it means the resting orientation of the points would always be the green point on the bottom left, and the blue point on the top right. No matter where the points start, they should end up in the following orientation (which matches the direction vector):
I've tried multiplying the normals by the spring vector:
norm1 = multiplyVectors( normalize( sub1 ), spring.direction ),
However this is a noop because the vector is (1,1). I've been hacking on this system for a few days now and I'm stuck. How can I constrain my 2d spring to a specific direction?
Spring forces are central just like gravity, which means that the total angular momentum of the system is conserved. Since you start with zero initial velocities, the angular momentum of the system is initially zero. The spring interaction keeps it zero, therefore the final orientation of the spring equals its initial orientation - the weights only move along the line connecting them.
To have the system rotate into the desired final position, you should also apply torque. The easiest way is to give the blue weight a positive charge and the green weight a negative one and then apply a constant external field in direction (1,1). That way the two charges will form a dipole and the interaction with the external field will generate the desired torque.
I don't get along with JavaScript, but I tried to write something based on your initial code here. The force that an external field with intensity E exerts on charge q is F = q * E, with both F and E being vectors. By adjusting q and E you can control how quickly the dipole will orient in the direction of the external field.
The force now becomes F = -k(|x|-d)(x/|x|) + qE - bv.
This has the probably undesired side effect that the final length of the spring will be slightly longer by delta, where delta = 2 * |q||E| / k. You can always adjust for that by reducing the length of the spring. Also, there is a little problem with that approach. Namely, there are two equilibrium states: one with the dipole facing the direction of the field (stable equilibrium) and one with the dipole facing the opposite direction (unstable equilibrium). A bit of random noise in the initial steps of the simulation will prevent the dipole from being trapped into the latter state.

Categories