We Keep Coding

Mapping a stereographic projection to the inside of a sphere in ThreeJS

When it comes to 3D animation, there are a lot of terms and concepts that I'm not familiar with (maybe a secondary question to append to this one: what are some good books to get familiar with the concepts?). I don't know what a "UV" is (in the context of 3D rendering) and I'm not familiar with what tools exist for mapping pixels on an image to points on a mesh.
I have the following image being produced by a 360-degree camera (it's actually the output of an HTML video element):
I want the center of this image to be the "top" of the sphere, and any radius of the circle in this image to be an arc along the sphere from top to bottom.
Here's my starting point (copying lines of code directly from the Three.JS documentation):
var video = document.getElementById( "texture-video" );
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1, 1000 );
var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
document.body.appendChild( renderer.domElement );
var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.LinearFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;
var material = new THREE.MeshBasicMaterial( { map: texture } );
var geometry = new THREE.SphereGeometry(0.5, 100, 100);
var mesh = new THREE.Mesh( geometry, material );
scene.add( mesh );
camera.position.z = 1
function animate()
{
mesh.rotation.y += 0.01;
requestAnimationFrame( animate );
renderer.render( scene, camera );
}
animate();
This produces the following:
There are a few problems:
The texture is rotated 90 degrees
The ground is distorted, although this may be fixed if the rotation is fixed?
Update: Upon further investigation of the sphere being produced, it's not actually rotated 90 degrees. Instead, the top of the image is the top of the sphere and the bottom of the image is the bottom of the sphere. This causes the left and right edges of the image to become the distorted "sideways ground" I saw
This is on the outside of the sphere. I want to project this to the inside of the sphere (and place the camera inside the sphere)
Currently if I place the camera inside the sphere I get solid black. I don't think it's a lighting issue because the Three.JS docs said that a MeshBasicMaterial didn't need lighting. I think the issue may be that the normals of all of the sphere faces point outward and I need to reverse them. I'm not sure how one would do this - but I'm pretty sure it's possible since I think this is how skyboxes work.
Doing some research I'm pretty sure I need to modify the "UV"s to fix this, I just don't know how or really what that even means...

Working Example
I forked #manthrax's CodeSandbox.io solution and updated it with my own:
https://codesandbox.io/s/4w1njkrv9
The Solution
So after spending a day researching UV mapping to understand what it meant and how it worked, I was able to sit down and scratch out some trig to map points on a sphere to points on my stereographic image. It basically came down to the following:
Use arccosine of the Y coordinate to determine the magnitude of a polar coordinate on the stereographic image
Use the arctangent of the X and Z coordinates to determine the angle of the polar coordinate on the stereographic image
Use x = Rcos(theta), y = Rsin(theta) to compute the rectangular coordinates on the stereographic image
If time permits I may draw a quick image in Illustrator or something to explain the math, but it's standard trigonometry
I went a step further after this, because the camera I was using only has a 240 degree vertical viewing angle - which caused the image to get slightly distorted (especially near the ground). By subtracting the vertical viewing angle from 360 and dividing by two, you get an angle from the vertical within which no mapping should occur. Because the sphere is oriented along the Y axis, this angle maps to a particular Y coordinate - above which there's data, and below which there isn't.
Calculate this "minimum Y value"
For all points on the sphere:
If the point is above the minimum Y value, scale it linearly so that the first such value is counted as "0" and the top of the sphere is still counted as "1" for mapping purposes
If the point is below the minimum Y value, return nothing
Weird Caveats
For some reason the code I wrote flipped the image upside down. I don't know if I messed up on my trigonometry or if I messed up on my understanding of UV maps. Whatever the case, this was trivially fixed by flipping the sphere 180 degrees after mapping
As well, I don't know how to "return nothing" in the UV map, so instead I mapped all points below the minimum Y value to the corner of the image (which was black)
With a 240-degree viewing angle the space at the bottom of the sphere with no image data was sufficiently large (on my monitor) that I could see the black circle when looking directly ahead. I didn't like the visual appearance of this, so I plugged in 270 for the vertical FOV. this leads to minor distortion around the ground, but not as bad as when using 360.
The Code
Here's the code I wrote for updating the UV maps:
// Enter the vertical FOV for the camera here
var vFov = 270; // = 240;
var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );
var geometry = new THREE.SphereGeometry(0.5, 200, 200);
function updateUVs()
{
var maxY = Math.cos(Math.PI * (360 - vFov) / 180 / 2);
var faceVertexUvs = geometry.faceVertexUvs[0];
// The sphere consists of many FACES
for ( var i = 0; i < faceVertexUvs.length; i++ )
{
// For each face...
var uvs = faceVertexUvs[i];
var face = geometry.faces[i];
// A face is a triangle (three vertices)
for ( var j = 0; j < 3; j ++ )
{
// For each vertex...
// x, y, and z refer to the point on the sphere in 3d space where this vertex resides
var x = face.vertexNormals[j].x;
var y = face.vertexNormals[j].y;
var z = face.vertexNormals[j].z;
// Because our stereograph goes from 0 to 1 but our vertical field of view cuts off our Y early
var scaledY = (((y + 1) / (maxY + 1)) * 2) - 1;
// uvs[j].x, uvs[j].y refer to a point on the 2d texture
if (y < maxY)
{
var radius = Math.acos(1 - ((scaledY / 2) + 0.5)) / Math.PI;
var angle = Math.atan2(x, z);
uvs[j].x = (radius * Math.cos(angle)) + 0.5;
uvs[j].y = (radius * Math.sin(angle)) + 0.5;
} else {
uvs[j].x = 0;
uvs[j].y = 0;
}
}
}
// For whatever reason my UV mapping turned everything upside down
// Rather than fix my math, I just replaced "minY" with "maxY" and
// rotated the sphere 180 degrees
geometry.rotateZ(Math.PI);
geometry.uvsNeedUpdate = true;
}
updateUVs();
var mesh = new THREE.Mesh( geometry, material );
The Results
Now if you add this mesh to a scene everything looks perfect:
One Thing I Still Don't Understand
Right around the "hole" at the bottom of the sphere there's a multi-colored ring. It almost looks like a mirror of the sky. I don't know why this exists or how it got there. Could anyone shed light on this in the comments?

Here is as close as I could get it in about 10 minutes of fiddling with a polar unwrapping of the uv's.
You can modify the polarUnwrap function to try and get a better mapping....
https://codesandbox.io/s/8nx75lkn28
You can replace the TextureLoader().loadTexture() with
//assuming you have created a HTML video element with id="video"
var video = document.getElementById( 'video' );
var texture = new THREE.VideoTexture( video );
texture.minFilter = THREE.LinearFilter;
texture.magFilter = THREE.LinearFilter;
texture.format = THREE.RGBFormat;
to get your video fed in there...
More info here:
https://threejs.org/docs/#api/textures/VideoTexture
Also this may be useful to you:
https://community.theta360.guide/t/displaying-thetas-dual-fisheye-video-with-three-js/1160

I think, it's would be quite difficult to modify the UVs, so that the stereographic projected image will fit. The UVs of a sphere are set to fit textures with equirectangular projection.
To transform the image from stereographic to equirectangular, you might want to use Panorama tools like PTGui or Hugin. Or you can use Photoshop (apply Filter > Distort > Polar Coordinates > polar to rectangular).
Equirectangular projection of the image (with Photoshop), resized to 2:1 aspect ratio (not necessary for texture)
If you want the texture to be inside the sphere (or normals flipped), you are able to set the material to THREE.BackSide.
var material = new THREE.MeshBasicMaterial( { map: texture, side: THREE.BackSide } );
Maybe, you have to flip the texture horizontally then: How to flip a Three.js texture horizontally

Click to place camera near an object in Three js

My I am trying to do a click to zoom feature with Three.js, I have a canvas and an object loaded in the canvas.On click I am trying to place the camera near the point of intersection(Actually like zooming that point).
Here is what I have done, but doesn't work as I wanted, on click camera positions changes but kind of works partially sometimes camera is placed near the point of intersection, some times not.
onmousedown = function (event) {
var raycaster = new THREE.Raycaster();
var mouse = new THREE.Vector2();
event.preventDefault();
mouse.x = (event.clientX / self.renderer.domElement.clientWidth) * 2 - 1;
mouse.y = -(event.clientY / self.renderer.domElement.clientHeight) * 2 + 1;
raycaster.setFromCamera(mouse, self.camera);
var objects = [];
for (var i = 0; i < self.scene.children.length; i++) {
if (self.scene.children[i] instanceof THREE.Group) {
objects.push(self.scene.children[i]);
}
}
console.log(objects);
var intersects = raycaster.intersectObjects( objects,true );
console.log(intersects.length);
if (intersects.length > 0) {
self.camera.up = new THREE.Vector3(0, 0, 1);
self.camera.lookAt(new THREE.Vector3(0, 0, 0));
self.camera.position.z = intersects[0].point.z * .9;
self.camera.position.x = intersects[0].point.x * .9;
self.camera.position.y = intersects[0].point.y * .9;
}
};
Here self is a global viewer object which holds camera, canvas, different objects etc.
0.9 is just a number used to place camera just near the point of intersection.
camera used is PerspectiveCamera and controls is TrackballControls
new THREE.PerspectiveCamera(90, this.width / this.height, 1, 1000);
The objects loaded are from .obj or .dae files ,I expect this to work like click on any point on the object and place the camera near that point. But camera is moving but sometimes not near the point I clicked.
Does intersects[0] gives the nearest intersection point? or nearest in the direction of camera ?
What is my mistake here ?
I am new to three js , just started learning it.If something or some logic is wrong help me with that.

The position is a bit complicated to calculate; you have to find the segment between camera and intersection and than place the camera at specific distance from intersection along the segment looking to the intersection point.
try this:
var length=[the desiderated distance camera-intersection]
var dir = camera.position.clone().sub(intersects[0].point).normalize().multiplyScalar(length);
camera.position = intersects[0].point.clone().add(dir);
camera.lookAt(intersects[0].point);

I have created a fiddle: http://jsfiddle.net/h5my29aL/
It's not so difficult. Think of your object as a planet, and your camera as a satellite. You need to position the camera somewhere in an orbit near your object. Three contains a distanceTo function that makes it simple. The example uses a sphere, but it will work with an arbitrary mesh. It measures the distance from the center point to the desired vector3. In your case the vector3 is likely the face position returned by a picker ray. But anyhow, the lookAt is set to the mesh, and then a distance from the vertex is calculated so that the camera is always the same altitude regardless of a vertex's or face's distance from the object center.
var point = THREE.GeometryUtils.randomPointsInGeometry( geometry, 1 );
var altitude = 100;
var rad = mesh.position.distanceTo( point[0] );
var coeff = 1+ altitude/rad;
camera.position.x = point[0].x * coeff;
camera.position.y = point[0].y * coeff;
camera.position.z = point[0].z * coeff;
camera.lookAt(mesh.position);

I've came somewhat close to what I want with an example from Three js.
Three JS webgl_decals
this is what I have done.
function zoomCam(event) {
var point_mouse = new THREE.Vector2(),
var point_x = null;
var point_y = null;
if (event.changedTouches) {
point_x = event.changedTouches[ 0 ].pageX;
point_y = event.changedTouches[ 0 ].pageY;
} else {
point_x = event.clientX;
point_y = event.clientY;
}
point_mouse.x = (point_x / window.innerWidth) * 2 - 1;
point_mouse.y = -(point_y / window.innerHeight) * 2 + 1;
if (sceneObjects.length > 0) {
var raycaster = new THREE.Raycaster();
raycaster.setFromCamera(point_mouse, camera);
var intersects = raycaster.intersectObjects(sceneObjects, true);
if (intersects.length > 0) {
var p = intersects[ 0 ].point;
var n = intersects[ 0 ].face.normal.clone();
n.multiplyScalar(10);
n.add(intersects[ 0 ].point);
camera.position.copy(n);
camera.lookAt(p);
}
}
There might be some minor issues as I formatted/changed the code for answering here. Check the code before implementing.

Three JS Camera Orientation returns value in 180 degrees instead of 360 degrees

I'm trying to build a compass inset in a Three.JS scene with fly controls. I've placed a north point far away from my plane and am using it as a north reference. I want the compass to rotate based on the angle from the camera to the point in space. I've used the following to determine the angle but what it spits out goes from 1 to 180 and then to 179 back down as I rotate around instead of continuing from 181 to 360. Any idea why this is?
var vector = new THREE.Vector3(0, 0, -1);
vector.applyQuaternion(camera.quaternion);
angle = vector.angleTo(north.position);
var degree = angle * (180 / 3.14159);
console.log(degree);
compass2.rotation.y = (angle);
I'll link to the project so you can see the issue in action.
www.googledrive.com/host/0B_XkORRtWN2OcGstUjE5R0RRRWc/PDSFlythrough.html

Solved! I opted for a different system in which I place the secondary camera inside of a cylinder with the end caps turned off and set to display the backside of the mesh. Then I simply added code to match the rotation angles from camera1 to camera1. Instant compass!
//Load Compass Model
var compassGeometry = new THREE.CylinderGeometry(30, 30, 30, 20, 1, 1);
var compassMaterial = new THREE.MeshPhongMaterial({
map: THREE.ImageUtils.loadTexture('assets/CompassTexture.svg')
});
compass2 = new THREE.Mesh(compassGeometry, compassMaterial);
compass2.material.side = THREE.BackSide;
compass2.rotation.x = 1.57;
compass2.rotation.y = 0.10;
scene2.add(compass2)
//Animate
function animate() {
requestAnimFrame(animate);
controls.update(1);
//match camera2 rotation to camera1
camera2.rotation.x = camera.rotation.x;
camera2.rotation.y = camera.rotation.y;
camera2.rotation.z = camera.rotation.z;
render();
}
Working Version

Three.js Javascript object transformation issue

I'm trying to scale height(y) of a cube as well as raising its y position simultaneously so that its base remains on the same x-z plane even after scaling. The rendering works fine but it is not working as expected. The object position increases rapidly in Y and then it starts disappearing. Can anyone help me out?
I've added an eventListener: mouseMove and its listener function to do the above transformations:
var fHeight = 5, yShift;
function onDocumentMouseMove(event) {
var mouseVector = new THREE.Vector3(2*(event.clientX/window.innerWidth) - 1, 1 - 2*(event.clientY/window.innerHeight));
var projector = new THREE.Projector();
var raycaster = projector.pickingRay(mouseVector.clone(), camera);
var intersects = raycaster.intersectObjects( Object.children );
if(intersects.length > 0) {
intersects[0].object.scale.y += 0.1;
fHeight = fHeight*intersects[0].object.scale.y;
yShift = fHeight/2 - intersects[0].object.position.y - 2.5;
intersects[0].object.position.y = intersects[0].object.position.y + yShift;
}
}

You can place the vertices of your cube so its base sticks to its y=0 plane.
Then object.scale.y will directly modify the cube height only, without displacing the base, and you will not have to reposition the object.
Hope it helps

How to rotate a 3D object on axis three.js?

I have a great problem about the rotation in three.js
I want to rotate my 3D cube in one of my game.
//init
geometry = new THREE.CubeGeometry grid, grid, grid
material = new THREE.MeshLambertMaterial {color:0xFFFFFF * Math.random(), shading:THREE.FlatShading, overdraw:true, transparent: true, opacity:0.8}
for i in [1...#shape.length]
othergeo = new THREE.Mesh new THREE.CubeGeometry(grid, grid, grid)
othergeo.position.x = grid * #shape[i][0]
othergeo.position.y = grid * #shape[i][1]
THREE.GeometryUtils.merge geometry, othergeo
#mesh = new THREE.Mesh geometry, material
//rotate
#mesh.rotation.y += y * Math.PI / 180
#mesh.rotation.x += x * Math.PI / 180
#mesh.rotation.z += z * Math.PI / 180
and (x, y, z) may be (1, 0, 0)
then the cube can rotate, but the problem is the cube rotate on its own axis,so after it has rotated, it can't rotate as expected.
I find the page How to rotate a Three.js Vector3 around an axis?, but it just let a Vector3 point rotate around the world axis?
and I have tried to use matrixRotationWorld as
#mesh.matrixRotationWorld.x += x * Math.PI / 180
#mesh.matrixRotationWorld.y += y * Math.PI / 180
#mesh.matrixRotationWorld.z += z * Math.PI / 180
but it doesn't work, I don't whether I used it in a wrong way or there are other ways..
so, how to let the 3D cube rotate around the world's axis???

Since release r59, three.js provides those three functions to rotate a object around object axis.
object.rotateX(angle);
object.rotateY(angle);
object.rotateZ(angle);

Here are the two functions I use. They are based on matrix rotations. and can rotate around arbitrary axes. To rotate using the world's axes you would want to use the second function rotateAroundWorldAxis().
// Rotate an object around an arbitrary axis in object space
var rotObjectMatrix;
function rotateAroundObjectAxis(object, axis, radians) {
rotObjectMatrix = new THREE.Matrix4();
rotObjectMatrix.makeRotationAxis(axis.normalize(), radians);
// old code for Three.JS pre r54:
// object.matrix.multiplySelf(rotObjectMatrix); // post-multiply
// new code for Three.JS r55+:
object.matrix.multiply(rotObjectMatrix);
// old code for Three.js pre r49:
// object.rotation.getRotationFromMatrix(object.matrix, object.scale);
// old code for Three.js r50-r58:
// object.rotation.setEulerFromRotationMatrix(object.matrix);
// new code for Three.js r59+:
object.rotation.setFromRotationMatrix(object.matrix);
}
var rotWorldMatrix;
// Rotate an object around an arbitrary axis in world space
function rotateAroundWorldAxis(object, axis, radians) {
rotWorldMatrix = new THREE.Matrix4();
rotWorldMatrix.makeRotationAxis(axis.normalize(), radians);
// old code for Three.JS pre r54:
// rotWorldMatrix.multiply(object.matrix);
// new code for Three.JS r55+:
rotWorldMatrix.multiply(object.matrix); // pre-multiply
object.matrix = rotWorldMatrix;
// old code for Three.js pre r49:
// object.rotation.getRotationFromMatrix(object.matrix, object.scale);
// old code for Three.js pre r59:
// object.rotation.setEulerFromRotationMatrix(object.matrix);
// code for r59+:
object.rotation.setFromRotationMatrix(object.matrix);
}
So you should call these functions within your anim function (requestAnimFrame callback), resulting in a rotation of 90 degrees on the x-axis:
var xAxis = new THREE.Vector3(1,0,0);
rotateAroundWorldAxis(mesh, xAxis, Math.PI / 180);

I needed the rotateAroundWorldAxis function but the above code doesn't work with the newest release (r52). It looks like getRotationFromMatrix() was replaced by setEulerFromRotationMatrix()
function rotateAroundWorldAxis( object, axis, radians ) {
var rotationMatrix = new THREE.Matrix4();
rotationMatrix.makeRotationAxis( axis.normalize(), radians );
rotationMatrix.multiplySelf( object.matrix ); // pre-multiply
object.matrix = rotationMatrix;
object.rotation.setEulerFromRotationMatrix( object.matrix );
}

with r55 you have to change
rotationMatrix.multiplySelf( object.matrix );
to
rotationMatrix.multiply( object.matrix );

In Three.js R59, object.rotation.setEulerFromRotationMatrix(object.matrix); has been changed to object.rotation.setFromRotationMatrix(object.matrix);
3js is changing so rapidly :D

Just in case...in r52 the method is called setEulerFromRotationMatrix instead of getRotationFromMatrix

Somewhere around r59 this gets easier (rotate around x):
bb.GraphicsEngine.prototype.calcRotation = function ( obj, rotationX)
{
var euler = new THREE.Euler( rotationX, 0, 0, 'XYZ' );
obj.position.applyEuler(euler);
}

In Three.js R66, this is what I use (CoffeeScript version):
THREE.Object3D.prototype.rotateAroundWorldAxis = (axis, radians) ->
rotWorldMatrix = new THREE.Matrix4()
rotWorldMatrix.makeRotationAxis axis.normalize(), radians
rotWorldMatrix.multiply this.matrix
this.matrix = rotWorldMatrix
this.rotation.setFromRotationMatrix this.matrix

I solved in this way:
I created the 'ObjectControls' module for ThreeJS that allows you to rotate a single OBJECT (or a Group), and not the SCENE.
Include the libary:
<script src="ObjectControls.js"></script>
Usage:
var controls = new ObjectControls(camera, renderer.domElement, yourMesh);
You can find here a live demo here: https://albertopiras.github.io/threeJS-object-controls/
Here is the repo: https://github.com/albertopiras/threeJS-object-controls.