I would like to build an AR experience (without web xr). The idea is to populate your surrounding with 3D cubes and be able to walk around them. I've created a github/codesandbox with a minimal demo. If you want to test the code run it from your favorite mobile phone. (it's just an experiment with some code sorry if it's not working on all phones tested with ios13+ and android OxygenOS 10.0 only for now)
What I did so far:
Populate your surrounding with 3D objects ✅
Create a gyro camera to see around you all the objects ✅
Update the camera position using the accelerometer from you phone to walk around the 3D scene and objects
Details:
In my scene I have X number of cubes placed around me given a distance and an angle as follow:
public async randomPosition(object: THREE.Object3D) {
const normalizedDistance = new THREE.Vector3()
const randomPosition = new THREE.Vector3()
const distance = getRandomNumber(1, 10)
const angleDeg = getRandomNumber(0, 360)
normalizedDistance.copy(new THREE.Vector3(0, 0, this.distance))
const angleRad = THREE.MathUtils.degToRad(this.angleDeg)
randomPosition.copy(normalizedDistance.applyAxisAngle(new THREE.Vector3(0, 1, 0), angleRad))
object.position.copy(this.randomPosition)
}
}
Then I used a gyro camera and now when I move around with my phone I'm able to see all the cubes around me. For now this code is based on the DeviceOrientationControls.
Finally I would like to use the devicemotion event to get the acceleration x/y/z of my device and update the camera's position with those values every frame to be able to walk around my cubes. At the moment I'm doing this:
const normalizedDistance = new THREE.Vector3()
const currentPosition = new THREE.Vector3()
const accX = this.findDistance(this.accelerationX, 0.9) // m/s^2 convert to distance <=> 0.5 * acceleration * speed ** 2
const accY = this.findDistance(this.accelerationY, 0.9)
const accZ = this.findDistance(this.accelerationZ, 0.9)
normalizedDistance.copy(new THREE.Vector3(accX, accY, accZ))
const angleRad = THREE.MathUtils.degToRad(this.rotationGamma)
currentPosition.copy(
normalizedDistance.applyAxisAngle(new THREE.Vector3(0, 1, 0), angleRad)
)
this.camera.position.copy(currentPosition)
The result I'm getting is the camera jiggling a lot when using a value of ~1s for the findDistance (see code above: findDistance(this.accelerationX, speed) // m/s^2 convert to distance <=> 0.5 * acceleration * speed ** 2 ) and the camera is not moving when using the clock delta time for the speed.
Also when I'm moving my phone the whole scene is moving with it, at the same time, so it doesn't give the effect of a camera moving throughout the 3D environment around you, if it makes sense.
I'm looking for advices, maybe someone has already been through this and would know if I'm on the good way or not. Thank you.
Related
With Three.js I want to create the effect of an object swinging from a cable or rope. It doesn't require real physics as the "swinging" object simply follows a fixed animation. The easiest solution is using the THREE.Line, however the problem is that THREE.Line can only be 1px thick and looks kinda awful.
In the three.js examples there is a "fat lines" example :
https://threejs.org/examples/?q=lines#webgl_lines_fat
however the problem is that once I have created the line using LineGeometry() I cannot figure out how to animate it.
The only solution I have found so far is to delete then create a new line every single frame, which works but seems like a really uneconomical, poorly optimized way to do it.
Does anyone know of a better way to either animate Line Geometry without having to delete and replace each frame? Or is there another method within three.js which would allow me to create thicker animated lines?
Thanks!!
I actually have a small project where I animate a bulb swinging along some rope. You can access it here, the functions I'm talking about below are in helperFns.js.
Actually, what I basically do is create my attached object separately :
let geometry = new THREE.SphereGeometry( 1, 32, 32 );
var material = new THREE.MeshStandardMaterial({color:0x000000,emissive:0xffffff,emissiveIntensity:lightIntensity});
bulb = new THREE.Mesh( geometry, material );
light = new THREE.PointLight(0xF5DCAF,lightIntensity,Infinity,2)
light.power = lightIntensity*20000
light.position.set(0,length*Math.sin(theta),z0-length*Math.cos(theta))
light.add(bulb)
light.castShadow = true;
hemiLight = new THREE.HemisphereLight( 0xddeeff, 0x0f0e0d, 0.1 );
scene.add(hemiLight)
scene.add(light)
I then add a spline connected to it :
// Create the wire linking the bulb to the roof
var curveObject = drawSpline(light.position,{x:0,y:0,z:z0},0xffffff);
scene.add(curveObject)
Where drawSpline is the following function :
// Build a spline representing the wire between the roof and the bulb. The new middle point is computed as the middle point shifted orthogonally from the lign by shiftRatio
function drawSpline(beginning,end,clr){
// Compute y sign to know which way to bend the wire
let ySign = Math.sign((end.y+beginning.y)/2)
// Compute the bending strength and multiply per Math.abs(beginning.y) to ensure it decreases as the bulb gets closer to the theta = 0 position, and also to ensure
// that the shift is null if thete is null (no discontinuity in the wire movement)
let appliedRatio = -shiftRatio*Math.abs(beginning.y)
// Compute middle line position vector and the direction vector from the roof to the bulb
let midVector = new THREE.Vector3( 0, (end.y+beginning.y)/2, (end.z+beginning.z)/2 )
let positionVector = new THREE.Vector3(0,end.y-beginning.y,end.z-beginning.z)
// Compute the orthogonal vector to the direction vector (opposite sense to the bending shift)
let orthogVector = new THREE.Vector3(0,positionVector.z,-positionVector.y).normalize()
// Compute the curve passing by the three points
var curve = new THREE.CatmullRomCurve3( [
new THREE.Vector3( beginning.x, beginning.y, beginning.z ),
midVector.clone().addScaledVector(orthogVector,ySign*appliedRatio),
new THREE.Vector3( end.x, end.y, end.z ),
]);
// Build the curve line object
var points = curve.getPoints( 20 );
var geometry = new THREE.BufferGeometry().setFromPoints( points );
var material = new THREE.LineBasicMaterial( { color : clr } );
// Create the final object to add to the scene
var curveObject = new THREE.Line( geometry, material );
return curveObject;
}
It creates the CatmullRomCurve3 interpolating the 3 points (one fix end at (0, 0, 0), one middle point to apply the bend, and the bulb position. You can actually start with a straight line, and then try to compute some curve.
To do so, you want to get the vector orthogonal to the line and shift the line (on the good side) along this vector.
And finally, at each animate() call, I redraw the spline for the new position of the bulb :
scene.children[2] = drawSpline(light.position,{x:0,y:0,z:z0},0xffffff)
Tell me if there is a point you do not get, but it should help for your problem.
Just wanted to post a more detailed version of West Langleys great reply. To animate a THREE Line2 you need to use the commands :
line.geometry.attributes.instanceStart.setXYZ( index, x, y, z );
line.geometry.attributes.instanceEnd.setXYZ( index, x, y, z );
What confused me was the index value - rather than thinking about a Line2 as being vertex points (the method used for creating the line) you need to think of a Line2 as being made of separate individual lines between 2 sets of points... so each line has a Start point and and an End point.
A "W" is therefore NOT defined as 5 vertices but by 4 lines. So you can "split" a Line2 by setting a different Start point to the previous lines End point. The index is the number of lines that make up your object. In my case I have two lines forming a V shape... so I set my index to 1 to affect the end of line 0 and the start of line 1, as in West's example :
var index = 1;
line.geometry.attributes.instanceEnd.setXYZ( index - 1, x, y, z );
line.geometry.attributes.instanceStart.setXYZ( index, x, y, z );
And then you just need to update the line using :
line.geometry.attributes.instanceStart.data.needsUpdate = true;
Thanks again to West for this really useful answer. I'd never have guessed this as you cannot see these variables when you look at the Line2 object properties. Very useful info. I hope it helps someone else at some point.
I'm currently working on a VR app based on three.js
The problem I need to solve is that the equirectangular images, that are loaded into the panoramic viewer often have yaw/pitch/roll - which has a huge impact on the rest of the application.
Now I want the user to correct the rotation in an easy, interactive manner (setting yaw/pitch/roll manually as input values drives one mad) and I came up with the following solution:
The idea is to prompt the user to select three points: Two on the floor and one on the ceiling (a point that is above of one of the points on the floor). In the background I get the intersections with the panoramic sphere and calculate yaw/pitch/roll.
Now I got so far to get the intersections (vectors) of the mouse-pano-sphere interaction. However I just don't get the rest working. Tried it with three's lookAt() on the user-selected vectors and to modify the rotatePlane() method of https://codepen.io/maurizzzio/pen/GjAPYj to fit my use case:
rotatePlane() {
const vec1 = new Vector3().copy(this.floorPointMain.position).normalize();
const vec2 = new Vector3().copy(this.floorPointOne.position).normalize();
const vec3 = new Vector3().copy(this.ceilingPoint.position).normalize();
const xy = new THREE.Vector3().subVectors(vec2, vec1);
const xz = new THREE.Vector3().subVectors(vec3, vec1);
const normal = new THREE.Vector3().crossVectors(xy, xz).normalize();
const Z = new THREE.Vector3(0, 0, 1);
const axis = new THREE.Vector3().crossVectors(Z, normal).normalize();
const angle = Math.acos(Z.dot(normal));
const q = new THREE.Quaternion().setFromAxisAngle(axis, angle);
this.orientationGrid.rotation.setFromQuaternion(q);
this.floorPointOne.rotation.setFromQuaternion(q);
this.floorPointMain.rotation.setFromQuaternion(q);
this.ceilingPoint.rotation.setFromQuaternion(q);
const distanceToPlane = vec1.dot(normal);
}
see my current result of above:
However it just won't work and after two days of searching the web and trying a lot, I'm starting to get mad and be very desperate for help.
Please help - with three.js formulas or abstraction help - because I'm not even sure anymore if the very basic idea of solving the rotation problem is right...
Thanks a lot!
I'm creating a script that rotates a THREE.js camera arround based on a mobile phones gyroscope input. It's currently working pretty well, except that every time I rotate my phone over a quadrant, the camera will turn 180 degrees instead of continuing as intended. This is the code that I currently use:
private onDeviceOrientation = ( event ) => {
if( event.alpha !== null && event.beta !== null && event.gamma !== null ) {
let rotation = [
event.beta,
event.alpha,
event.gamma
],
this.orientation = new THREE.Vector3(rotation[0], rotation[1], rotation[2]);
this.viewer.navigation.setTarget(this.calcPosition());
}
};
private calcPosition = () => {
const camPosition = this.viewer.navigation.getPosition(),
radians = Math.PI / 180,
aAngle = radians * - this.orientation.y,
bAngle = radians * + this.orientation.z,
distance = this.calcDistance();
let medianX = Math.cos(bAngle) * Math.sin(aAngle);
let medianY = Math.cos(bAngle) * Math.cos(aAngle);
let nX = camPosition.x + (medianX * distance),
nY = camPosition.y + (medianY * distance),
nZ = camPosition.z + Math.sin(bAngle) * distance;
return new THREE.Vector3(nX, nY, nZ);
};
window.addEventListener('deviceorientation', this.onDeviceOrientation, false);
Soafter doing some research I found that I need to use a Quaternion prevent the switchen when going into a new quadrant. I have no experience with Quaternions, so I was wondering what the best way would be to combine the two Vector3's in the code above into a singel Quaternion.
[Edit]
I calculate the distance using this method:
private calcDistance = (): number => {
const camPosition = this.viewer.navigation.getPosition();
const curTarget = this.viewer.navigation.getTarget();
let nX = camPosition.x - curTarget.x,
nY = camPosition.y - curTarget.y,
nZ = camPosition.z - curTarget.z;
return Math.sqrt((nX * nX) + (nY * nY) + (nZ * nZ));from squared averages
};
And I follow the MDN conventions when working with the gyroscope.
[Edit #2]
Turns out I had my angle all wrong, I managed to fix it by calculating the final position like this:
let nX = camPosition.x - (Math.cos(zAngle) * Math.sin(yAngle)) * distance,
nY = camPosition.y + (Math.cos(zAngle) * Math.cos(yAngle)) * distance,
nZ = camPosition.z - (Math.cos(xAngle) * Math.sin(zAngle)) * distance;
Here is the closest I can give you to an answer:
First of all, you don't need a quaternion. (If you really find yourself needing to convert between Euler angles and quaternions, it is possible as long as you have all the axis conventions down pat.) The Euler angle orientation information you obtain from the device is sufficient to represent any rotation without ambiguity; if you were calculating angular velocities, I'd agree that you want to avoid Euler angles since there are some orientations in which the rates of change of the Euler angles go to infinity. But you're not, so you don't need it.
I'm going to try to summarize the underlying problem you're trying to solve, and then tell you why it might not be solvable. 🙁
You are given the full orientation of the device with a camera, as yaw, pitch, and roll. Assuming yaw is like panning the camera horizontally, and pitch is like tilting the camera vertically, then roll is a degree of freedom that doesn't change affect direction the camera is pointing, but it does affect the orientation of the images the camera sees. So you are given three coordinates, where two have to do with the direction the camera is pointing, and one does not.
You are trying to output this information to the camera controller but you are only allowed to specify the target location, which is the point in space that the camera is looking. This is to be specified via three Cartesian coordinates, which you can calculate from the direction the camera is pointing (2 degrees of freedom) and the distance to the target object (one degree of freedom).
So you have three inputs and three outputs, but only two of those have anything to do with each other. The target location has no way to represent the roll direction of the camera, and the orientation of the camera has no way to represent the distance to some target object.
Since you don't have a real target object, you can just pick an arbitrary fixed distance (1, for example) and use it. You certainly don't have anything from which to calculate it... if I follow your code, you are defining distance in terms of the target location, which is itself defined in terms of the distance from the previous step. This is extra work for no benefit at best (the distance drifts around some initial value), and numerically unstable at worst (the distance drifts toward zero and you lose precision or get infinities). Just use a fixed value for distance and make it simple.
So now you probably have a system that points a camera in a direction, but you cannot tell it what the roll angle is. That means your camera controller is apparently just going to choose it for you based on the yaw and pitch angles. Let's say it always picks zero degrees (that would be the least crazy thing it could do). This will cause discontinuities when the roll angle and yaw angle line up (when the pitch is at ±90°): Imagine pointing a physical camera at the northern horizon and yawing around westward, past the western horizon, and settling on the southern horizon. The whole time, the roll angle of the camera is 0°, so there's no problem. But now imagine pointing it at the northern horizon, and pitching upward, past the zenith, and continuing to pitch backward until you are facing the southern horizon. Now the camera is upside down; the roll angle is 180°. But if the camera controller doesn't change the roll angle from 0°, then it will do a nonphysical "flip" right when you pass the zenith. The problem is that there really is no way to synthesize a roll angle based purely on position and not have this happen. We've just demonstrated that there are two ways to move your camera from pointing north to pointing south, where the roll angle is completely different at the end.
So you're stuck, I guess. Well, maybe not. Can you rotate the image from the camera based on the roll angle of the device orientation? That is, add the roll back into the displayed image? If so, you may have a solution. Let's say the roll angle of the camera controller is always at zero. Then you just rotate the image by the desired roll angle (something derived from beta I guess?) and you're done. If the camera controller has some other convention for choosing the roll angle, you will need to figure that out, undo it, and add the roll angle back on.
Without the actual system in front of me I probably can't help you debug your way to a solution. So I think this is where my journey with this question must end. Good luck!
Summary:
You don't need a quaternion
Pick a fixed distance to your simulated target
Add the roll angle by rotating the image before displaying it
Good luck!
I'm working with Three.js, version 68. I'm using the same method for collision detection as this guy is using here, which is great most of the time (A big "thank you" goes out to the author!): http://stemkoski.github.io/Three.js/Collision-Detection.html
Here is a link to the source if you want to download it from github. Just look for Collision-Detection.html: https://github.com/stemkoski/stemkoski.github.com
Here is the code that is important to the collision detection:
var MovingCube;
var collidableMeshList = [];
var wall = new THREE.Mesh(wallGeometry, wallMaterial);
wall.position.set(100, 50, -100);
scene.add(wall);
collidableMeshList.push(wall);
var wall = new THREE.Mesh(wallGeometry, wireMaterial);
wall.position.set(100, 50, -100);
scene.add(wall);
var wall2 = new THREE.Mesh(wallGeometry, wallMaterial);
wall2.position.set(-150, 50, 0);
wall2.rotation.y = 3.14159 / 2;
scene.add(wall2);
collidableMeshList.push(wall2);
var wall2 = new THREE.Mesh(wallGeometry, wireMaterial);
wall2.position.set(-150, 50, 0);
wall2.rotation.y = 3.14159 / 2;
scene.add(wall2);
var cubeGeometry = new THREE.CubeGeometry(50,50,50,1,1,1);
var wireMaterial = new THREE.MeshBasicMaterial( { color: 0xff0000, wireframe:true } );
MovingCube = new THREE.Mesh( cubeGeometry, wireMaterial );
MovingCube.position.set(0, 25.1, 0);
// collision detection:
// determines if any of the rays from the cube's origin to each vertex
// intersects any face of a mesh in the array of target meshes
// for increased collision accuracy, add more vertices to the cube;
// for example, new THREE.CubeGeometry( 64, 64, 64, 8, 8, 8, wireMaterial )
// HOWEVER: when the origin of the ray is within the target mesh, collisions do not occur
var originPoint = MovingCube.position.clone();
for (var vertexIndex = 0; vertexIndex < MovingCube.geometry.vertices.length; vertexIndex++)
{
var localVertex = MovingCube.geometry.vertices[vertexIndex].clone();
var globalVertex = localVertex.applyMatrix4( MovingCube.matrix );
var directionVector = globalVertex.sub( MovingCube.position );
var ray = new THREE.Raycaster( originPoint, directionVector.clone().normalize() );
var collisionResults = ray.intersectObjects( collidableMeshList );
if ( collisionResults.length > 0 && collisionResults[0].distance < directionVector.length() )
appendText(" Hit ");
}
This works great most of the time, but there are times when I can move the cube partially into the wall, and it won't register a collision. For example, look at this image:
It should say "Hit" in the top-left corner where there are just a bunch of dots, and it's not.
NOTE: I also tried his suggestion and did the following, but it didn't seem to help much:
THREE.BoxGeometry( 64, 64, 64, 8, 8, 8, wireMaterial ) // BoxGeometry is used in version 68 instead of CubeGeometry
Does anyone know how this method could be more accurate? Another question: Does anyone know what the following if statement is for, i.e. why does the object's distance have to be less than the length of the direction vector?:
if ( collisionResults.length > 0 && collisionResults[0].distance < directionVector.length() )
To answer your last question first: that line detects whether the collision happened inside your MovingCube. Your raycasting code casts a ray from the MovingCube's position towards each of its vertices. Anything that the ray intersects with is returned, along with the distance from the MovingCube's position at which the intersected object was found (collisionResults[0].distance). That distance is compared with the distance from the MovingCube's position to the relevant vertex. If the distance to the collision is less than the distance to the vertex, the collision happened inside the cube.
Raycasting is a poor method of collision detection because it only detects collisions in the exact directions rays are cast. It also has some additional edge cases. For example, if the ray is cast from inside another object, the other object might not be considered to be colliding. As another example, raycasting in Three.js uses bounding spheres (or, if unavailable, bounding boxes) to calculate ray intersection, so rays can "intersect" with objects even if they wouldn't hit them visually.
If you're only dealing with spheres or upright cuboids, it's straightforward math to check collision. (That's why Three.js uses bounding spheres and bounding boxes - and most applications that need to do collision checking use secondary collision-only geometries that are less complicated than the rendered ones.) Spheres are colliding if the distance between their centers is less than the sum of their radii. Boxes are colliding if the edges overlap (e.g. if the left edge of box 1 is to the left of the right edge of box 2, and the boxes are within a vertical distance the sum of their half-heights and a horizontal distance the sum of their half-lengths).
For certain applications you can also use voxels, e.g. divide the world into cubical units, do box math, and say that two objects are colliding if they overlap with the same cube-unit.
For more complex applications, you'll probably want to use a library like Ammo.js, Cannon.js, or Physi.js.
The reason raycasting is appealing is because it's workable with more complex geometries without using a library. As you've discovered, however, it's less than perfect. :-)
I wrote a book called Game Development with Three.js which goes into this topic in some depth. (I won't link to it here because I'm not here to promote it, but you can Google it if you're interested.) The book comes with sample code that shows how to do basic collision detection, including full code for a 3D capture-the-flag game.
In Three.js (which uses JavaScript/ WebGL), how would one create a camera which flies around a sphere at fixed height, fixed forward speed, and fixed orientation in relation to the sphere, with the user only being able to steer left and right?
Imagine an airplane on an invisible string to the center of a globe, flying near ground and always seeing part of the sphere:
(I currently have code which rotates the sphere so to the camera it looks like it's flying -- left and right steering not implemented yet -- but I figure before I go further it might be cleaner to move the camera/ airplane, not the sphere group.)
Thanks!
You mean like in my Ludum Dare 23 game? I found this to be a bit more complicated than I expected. It's not difficult, though.
Here I'm assuming that you know the latitude and longitude of the camera and its distance from the center of the sphere (called radius), and want to create a transformation matrix for the camera.
Create the following objects only once to avoid creating new objects in the game loop:
var rotationY = new Matrix4();
var rotationX = new Matrix4();
var translation = new Matrix4();
var matrix = new Matrix4();
Then every time the camera moves, create the matrix as follows:
rotationY.setRotationY(longitude);
rotationX.setRotationX(-latitude);
translation.setTranslation(0, 0, radius);
matrix.multiply(rotationY, rotationX).multiplySelf(translation);
After this just set the camera matrix (assuming camera is your camera object):
// Clear the camera matrix.
// Strangely, Object3D doesn't have a way to just SET the matrix(?)
camera.matrix.identity();
camera.applyMatrix(matrix);
Thanks for Martin's answer! I've now got it running fine in another approach as follows (Martin's approach may be perfect too; also many thanks to Lmg!):
Set the camera to be a straight line atop the sphere in the beginning (i.e. a high y value, a bit beyond the radius, which was 200 in my case); make it look a bit lower:
camera.position.set(0, 210, 0);
camera.lookAt( new THREE.Vector3(0, 190, -50) );
Create an empty group (an Object3D) and put the camera in:
camGroup = new THREE.Object3D();
camGroup.add(camera);
scene.add(camGroup);
Track the mouse position in percent in relation to the screen half:
var halfWidth = window.innerWidth / 2, halfHeight = window.innerHeight / 2;
app.mouseX = event.pageX - halfWidth;
app.mouseY = event.pageY - halfHeight;
app.mouseXPercent = Math.ceil( (app.mouseX / halfWidth) * 100 );
app.mouseYPercent = Math.ceil( (app.mouseY / halfHeight) * 100 );
In the animation loop, apply this percent to a rotation, while automoving forward:
camGroup.matrix.rotateY(-app.mouseXPercent * .00025);
camGroup.matrix.rotateX(-.0025);
camGroup.rotation.getRotationFromMatrix(camGroup.matrix);
requestAnimationFrame(animate);
renderer.render(scene, camera);