Raycasting selection is working fine for my project on static meshes, however for animated meshes the ray selection doesn't seem to see the movement of the mesh and only responds to the mesh's non-animated (original) position.
My selection code is as follows:
element.addEventListener( 'mouseup', function ( event )
{
var vector = new THREE.Vector3(( event.clientX / window.innerWidth ) * 2 - 1, -( event.clientY / window.innerHeight ) * 2 + 1, 0.5);
vector = vector.unproject(camera);
var raycaster = new THREE.Raycaster(camera.position, vector.sub(camera.position).normalize());
var intersects = raycaster.intersectObjects(pickable_objects, true);
if (intersects.length > 0)
{
//I change the object's material colour so that I can see the selection
}
}
Where pickable_objects is an array of three.js mesh objects that are selectable, these do not carry animation information as far as I am aware, so the result I'm complaining about may not be surprising to some.
I haven't included the colour changing code or the JSON mesh reader code here as I don't believe it's pertinent.
So with static meshes the ray casting is very accurate, but for animated meshes I have to click in the very centre of the object or find its original non-animated position to make the selection work.
If you take a look at the source code in the github repo, particularly Raycaster.js, you'll see that the actual implementation is in the geometry object. For instance Mesh.js implements the raycast by iterating over its triangles and checking whether the ray intersects any of them but SkinnedMesh.js does not override this method.
Finally, a quick search in the issues page showed that this is actually a known limitation - https://github.com/mrdoob/three.js/issues/6440.
You seem to be left with 2 options - bump the issue in github and ask for an ETA or implement this yourself (and send a pull request afterwards hopefully).
Related
I'm trying to detect when an object in Three.js is partially and fully occluded (hidden behind) another object.
My current simple solution casts a single ray to the the center of the object:
function getScreenPos(object) {
var pos = object.position.clone();
camera.updateMatrixWorld();
pos.project(camera);
return new THREE.Vector2(pos.x, pos.y);
}
function isOccluded(object) {
raycaster.setFromCamera(getScreenPos(object), camera);
var intersects = raycaster.intersectObjects(scene.children);
if (intersects[0] && intersects[0].object === object) {
return false;
} else {
return true;
}
}
However it doesn't account for the object's dimensions (width, height, depth).
Not occluded (because center of object is not behind)
Occluded (because center of object is behind)
View working demo:
https://jsfiddle.net/kmturley/nb9f5gho/57/
Currently thinking I could calculate the object box size, and cast Rays for each corner of the box. But this might still be a little too simple:
var box = new THREE.Box3().setFromObject(object);
var size = box.getSize();
I would like to find a more robust approach which could give partially occluded and fully occluded booleans values or maybe even percentage occluded?
Search Stack Overflow and the Three.js examples for "GPU picking." The concept can be broken down into three basic steps:
Change the material of each shape to a unique flat (MeshBasicMaterial) color.
Render the scene with the unique materials.
Read the pixels of the rendered frame to collect color information.
Your scenario allows you a few caveats.
Give only the shape you're testing a unique color--everything else can be black.
You don't need to render the full scene to test one shape. You could adjust your viewport to render only the area surrounding the shape in question.
Because you only gave a color only to your test part, the rest of the data should be zeroes, making finding pixels matching your unique color much easier.
Now that you have the pixel data, you can determine the following:
If NO pixels matchthe unique color, then the shape is fully occluded.
If SOME pixels match the unique color, then the shape is at least partially visible.
The second bullet says that the shape is "at least partially" visible. This is because you can't test for full visibility with the information you currently have.
What I would do (and someone else might have a better solution) is render the same viewport a second time, but only have the test shape visible, which is the equivalent of the part being fully visible. With this information in hand, compare the pixels against the first render. If both have the same number (perhaps within a tolerance) of pixels of the unique color, then you can say the part is fully visible/not occluded.
I managed to get a working version for WebGL1 based on TheJim01's answer!
First create a second simpler scene to use for calculations:
pickingScene = new THREE.Scene();
pickingTextureOcclusion = new THREE.WebGLRenderTarget(window.innerWidth / 2, window.innerHeight / 2);
pickingMaterial = new THREE.MeshBasicMaterial({ vertexColors: THREE.VertexColors });
pickingScene.add(new THREE.Mesh(BufferGeometryUtils.mergeBufferGeometries([
createBuffer(geometry, mesh),
createBuffer(geometry2, mesh2)
]), pickingMaterial));
Recreate your objects as Buffer Geometry (faster for performance):
function createBuffer(geometry, mesh) {
var buffer = new THREE.SphereBufferGeometry(geometry.parameters.radius, geometry.parameters.widthSegments, geometry.parameters.heightSegments);
quaternion.setFromEuler(mesh.rotation);
matrix.compose(mesh.position, quaternion, mesh.scale);
buffer.applyMatrix4(matrix);
applyVertexColors(buffer, color.setHex(mesh.name));
return buffer;
}
Add a color based on the mesh.name e.g. an id 1, 2, 3, etc
function applyVertexColors(geometry, color) {
var position = geometry.attributes.position;
var colors = [];
for (var i = 0; i < position.count; i ++) {
colors.push(color.r, color.g, color.b);
}
geometry.setAttribute('color', new THREE.Float32BufferAttribute(colors, 3));
}
Then during the render loop check the second scene for that texture, and match pixel data to the mesh name:
function isOccludedBuffer(object) {
renderer.setRenderTarget(pickingTextureOcclusion);
renderer.render(pickingScene, camera);
var pixelBuffer = new Uint8Array(window.innerWidth * window.innerHeight);
renderer.readRenderTargetPixels(pickingTextureOcclusion, 0, 0, window.innerWidth / 2, window.innerHeight / 2, pixelBuffer);
renderer.setRenderTarget(null);
return !pixelBuffer.includes(object.name);
}
You can view the WebGL1 working demo here:
https://jsfiddle.net/kmturley/nb9f5gho/62/
One caveat to note with this approach is that your picking scene needs to stay up-to-date with changes in your main scene. So if your objects move position/rotation etc, they need to be updated in the picking scene too. In my example the camera is moving, not the objects so it doesn't need updating.
For WebGL2 we will have a better solution:
https://tsherif.github.io/webgl2examples/occlusion.html
But this is not supported in all browsers yet:
https://www.caniuse.com/#search=webgl
I am creating a sphere in Three.js which has to follow the mouse whenever it moves, as displayed in this example. The function that handles the mouse movement is the following:
function onMouseMove(event) {
// Update the mouse variable
event.preventDefault();
mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
mouse.y = - (event.clientY / window.innerHeight) * 2 + 1;
// Make the sphere follow the mouse
mouseMesh.position.set(event.clientX, event.clientY, 0);
};
I attach a JSFiddle with the complete code inside it, where you can see that according to the DOM, mouseMesh is undefined. Do you have an idea of what am I doing wrong?
Thanks in advance for your replies!
For sphere to follow mouse, you need to convert screen coordinates to threejs world position. Reference link.
Updated fiddle
var vector = new THREE.Vector3(mouse.x, mouse.y, 0.5);
vector.unproject( camera );
var dir = vector.sub( camera.position ).normalize();
var distance = - camera.position.z / dir.z;
var pos = camera.position.clone().add( dir.multiplyScalar( distance ) );
You should use a THREE.Raycaster for this. When you set a list of intersectObjects you will be able to get an array of objects that intersected with the ray. So you can get the position from the 'clicked' object from returned list
Basically, you need to project from the 3D world space and the 2D screen space.
Renderers use projectVector for translating 3D points to the 2D screen. unprojectVector is basically for doing the inverse, unprojecting 2D points into the 3D world. For both methods you pass the camera you're viewing the scene through.
So, in this code you're creating a normalised vector in 2D space.
I had implemented a globe similar to this one
But while trying to fetch the countries visible to the scene in Three.js when the globe is in a stationary position am stuck in a issue. I can retrieve the country color code when I mouse over / click on a specific country using readPixels and get the appropriate color code of the pixel as shown below
var gl = renderer.context;
var mx = ( mouseX + renderer.context.canvas.width/2 );//(mouseX + renderer.context.canvas.width/2) * 0.25;
var my = ( -mouseY + renderer.context.canvas.height/2 );//(-mouseY + renderer.context.canvas.height/2) * 0.25;
mx = Math.floor( mx );
my = Math.floor( my );
var buf = new Uint8Array( 4 );
gl.readPixels( mx, my, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, buf );
But when the globe stopped rotating then I need to get the list of user viewable countries from the scene, I tried to use Raycaster to get the mesh and scan through all the pixels and get the country code but seems to be a performance hit for us
Appreciate if someone can provide their views
You can render the scene to a texture (even with different shaders/uniforms). There are examples on the official site, like this: http://threejs.org/examples/webgl_rtt.html
For instance you could have a scene where the texture of the globe is country coded, and when the user clicks, you render that scene into a texture and read single pixel from that texture.
Be careful, when rendering to textures, you sometimes have to force update your textures by setting needsUpdate to true.
Hope I helped.
For each mesh (THREE.Object3D) Three.js provide a very handy properties - boundingSphere and boundingSphere that have intersectsSphere and isIntersectionBox methods.
With all this I thought I can use it for simple collision detection but when I try it appears that collision happens all the time because (I tried boundingSphere) boundingSphere.center is always in (0, 0, 0); So If I want to check collisions between 2 meshes I should for each object - clone boundingSphere object and then get it world coordinates and only then to use intersectsSphere.
something like this:
var bs = component.object.geometry.boundingSphere.clone();
bs.center.setFromMatrixPosition(component.object.matrixWorld);
...
if (_bs.intersectsSphere(bs)){
is this how it suppose to be used or am I missing something and there are more convenient way of doing collisions detection based on boundingBox/boundingSphere?
If you want to do collision detection with bounding boxes you need the boxes in the world coordinate system. The bounding volumes in the intersectsSphere and isIntersectionBox properties of the mesh are in the local coordinate system of the object.
You can do like you did: clone the volumes and move them to the correct position in the world coordinate system, that is a good solution.
Otherwise you can also set a new box from your meshes and do collision using those boxes. Let's say you have a THREE.Mesh called mesh then you can do:
sphere = new THREE.Sphere.setFromPoints( mesh.vertices );
box = new THREE.Box3.setFromObject( mesh );
A little tip. During development it can be nice to see the bounding boxes in your scene, for this you can use the THREE.BoundingBoxHelper:
var helper = new THREE.BoundingBoxHelper( mesh );
scene.add( helper );
I'm trying to check if an object (this.target) is looking at a specific position (newPosition). What I have right now is the following:
new THREE.Matrix4().lookAt( newPosition, this.target.position, this.target.up ) == this.target.matrix
But for some reason, this won't work. How are you meant to check if an Object3D is looking at a specific position?
That won't work because the matrix for your target is not in the same place, and can be scaled differently and all sorts of stuff.
A better technique is to raycast from your eye object, and see if it hits the target object.
http://threejs.org/examples/#webgl_geometry_terrain_raycast
var raycaster = new THREE.Raycaster();
raycaster.setFromCamera( mouse, camera );
// See if the ray from the camera into the world hits one of our meshes
var intersects = raycaster.intersectObject( mesh );