We Keep Coding

Three.js detect when object is partially and fully occluded

I'm trying to detect when an object in Three.js is partially and fully occluded (hidden behind) another object.
My current simple solution casts a single ray to the the center of the object:
function getScreenPos(object) {
var pos = object.position.clone();
camera.updateMatrixWorld();
pos.project(camera);
return new THREE.Vector2(pos.x, pos.y);
}
function isOccluded(object) {
raycaster.setFromCamera(getScreenPos(object), camera);
var intersects = raycaster.intersectObjects(scene.children);
if (intersects[0] && intersects[0].object === object) {
return false;
} else {
return true;
}
}
However it doesn't account for the object's dimensions (width, height, depth).
Not occluded (because center of object is not behind)
Occluded (because center of object is behind)
View working demo:
https://jsfiddle.net/kmturley/nb9f5gho/57/
Currently thinking I could calculate the object box size, and cast Rays for each corner of the box. But this might still be a little too simple:
var box = new THREE.Box3().setFromObject(object);
var size = box.getSize();
I would like to find a more robust approach which could give partially occluded and fully occluded booleans values or maybe even percentage occluded?

Search Stack Overflow and the Three.js examples for "GPU picking." The concept can be broken down into three basic steps:
Change the material of each shape to a unique flat (MeshBasicMaterial) color.
Render the scene with the unique materials.
Read the pixels of the rendered frame to collect color information.
Your scenario allows you a few caveats.
Give only the shape you're testing a unique color--everything else can be black.
You don't need to render the full scene to test one shape. You could adjust your viewport to render only the area surrounding the shape in question.
Because you only gave a color only to your test part, the rest of the data should be zeroes, making finding pixels matching your unique color much easier.
Now that you have the pixel data, you can determine the following:
If NO pixels matchthe unique color, then the shape is fully occluded.
If SOME pixels match the unique color, then the shape is at least partially visible.
The second bullet says that the shape is "at least partially" visible. This is because you can't test for full visibility with the information you currently have.
What I would do (and someone else might have a better solution) is render the same viewport a second time, but only have the test shape visible, which is the equivalent of the part being fully visible. With this information in hand, compare the pixels against the first render. If both have the same number (perhaps within a tolerance) of pixels of the unique color, then you can say the part is fully visible/not occluded.

I managed to get a working version for WebGL1 based on TheJim01's answer!
First create a second simpler scene to use for calculations:
pickingScene = new THREE.Scene();
pickingTextureOcclusion = new THREE.WebGLRenderTarget(window.innerWidth / 2, window.innerHeight / 2);
pickingMaterial = new THREE.MeshBasicMaterial({ vertexColors: THREE.VertexColors });
pickingScene.add(new THREE.Mesh(BufferGeometryUtils.mergeBufferGeometries([
createBuffer(geometry, mesh),
createBuffer(geometry2, mesh2)
]), pickingMaterial));
Recreate your objects as Buffer Geometry (faster for performance):
function createBuffer(geometry, mesh) {
var buffer = new THREE.SphereBufferGeometry(geometry.parameters.radius, geometry.parameters.widthSegments, geometry.parameters.heightSegments);
quaternion.setFromEuler(mesh.rotation);
matrix.compose(mesh.position, quaternion, mesh.scale);
buffer.applyMatrix4(matrix);
applyVertexColors(buffer, color.setHex(mesh.name));
return buffer;
}
Add a color based on the mesh.name e.g. an id 1, 2, 3, etc
function applyVertexColors(geometry, color) {
var position = geometry.attributes.position;
var colors = [];
for (var i = 0; i < position.count; i ++) {
colors.push(color.r, color.g, color.b);
}
geometry.setAttribute('color', new THREE.Float32BufferAttribute(colors, 3));
}
Then during the render loop check the second scene for that texture, and match pixel data to the mesh name:
function isOccludedBuffer(object) {
renderer.setRenderTarget(pickingTextureOcclusion);
renderer.render(pickingScene, camera);
var pixelBuffer = new Uint8Array(window.innerWidth * window.innerHeight);
renderer.readRenderTargetPixels(pickingTextureOcclusion, 0, 0, window.innerWidth / 2, window.innerHeight / 2, pixelBuffer);
renderer.setRenderTarget(null);
return !pixelBuffer.includes(object.name);
}
You can view the WebGL1 working demo here:
https://jsfiddle.net/kmturley/nb9f5gho/62/
One caveat to note with this approach is that your picking scene needs to stay up-to-date with changes in your main scene. So if your objects move position/rotation etc, they need to be updated in the picking scene too. In my example the camera is moving, not the objects so it doesn't need updating.
For WebGL2 we will have a better solution:
https://tsherif.github.io/webgl2examples/occlusion.html
But this is not supported in all browsers yet:
https://www.caniuse.com/#search=webgl

why shadow becomes blurred in Three.js

In a normal scenario, the shaow look like this
But after I modified the parameters ofdirectionalLight.shadow.camera
directionalLight.shadow.camera.left = -50
directionalLight.shadow.camera.right = 50
directionalLight.shadow.camera.top = -50
directionalLight.shadow.camera.bottom = 50
It became like this
How to resolve this problem?
https://jsfiddle.net/JesseLuo/z1m6uffu/12/

Shadows are simulated using a "camera" in the location of the light source, and that camera's frustum (or, its field of view) and resolution determine how precisely the shadow matches your object. You can't have perfectly detailed shadows covering large areas, so you need to tune the shadow camera to the part of the scene that's important. In this case, when you change the parameters to expand the camera's frustum, you've spread it over a larger area and lost precision.
To improve the result you can:
A. Increase the shadowMap's resolution. Higher values give better quality shadows at the cost of computation time. Values must be powers of 2.
light.shadow.mapSize.width = 1024;
light.shadow.mapSize.height = 1024;
B. Change the shadow type. PCFSoft may look better, but doesn't perform as well.
renderer.shadowMap.type = THREE.PCFSoftShadowMap;
C. Reduce the dimensions of the shadow camera frustum to just the area you need to cover. Use a CameraHelper to see where the shadows are covering, like in this example.
scene.add( new THREE.CameraHelper( light.shadow.camera ) );
See THREE.LightShadow docs for more information.

Apply three.js subdivision modifier without changing outer geometry?

I am trying to take any three.js geometry and subdivide its existing faces into smaller faces. This would essentially give the geometry a higher "resolution". There is a subdivision modifier tool in the examples of three.js that works great for what I'm trying to do, but it ends up changing and morphing the original shape of the geometry. I'd like to retain the original shape.
View the Subdivision Modifier Example
Example of how the current subdivision modifier behaves:
Rough example of how I'd like it to behave:
The subdivision modifier is applied like this:
let originalGeometry = new THREE.BoxGeometry(1, 1, 1);
let subdivisionModifier = new THREE.SubdivisionModifier(3);
let subdividedGeometry = originalGeometry.clone();
subdivisionModifier.modify(subdividedGeometry);
I attempted to dig around the source of the subdivision modifier, but I wasn't sure how to modify it to get the desired result.
Note: The subdivision should be able to be applied to any geometry. My example of the desired result might make it seem that a three.js PlaneGeometry with increased segments would work, but I need this to be applied to a variety of geometries.

Based on the suggestions in the comments by TheJim01, I was able to dig through the original source and modify the vertex weight, edge weight, and beta values to retain the original shape. My modifications should remove any averaging, and put all the weight toward the source shape.
There were three sections that had to be modified, so I went ahead and made it an option that can be passed into the constructor called retainShape, which defaults to false.
I made a gist with the modified code for SubdivisionGeometry.js.
View the modified SubdivisionGeometry.js Gist
Below is an example of a cube being subdivided with the option turned off, and turned on.
Left: new THREE.SubdivisionModifier(2, false);
Right: new THREE.SubdivisionModifier(2, true);
If anyone runs into any issues with this or has any questions, let me know!

The current version of three.js has optional parameters for PlaneGeometry that specify the number of segments for the width and height; both default to 1. In the example below I set both widthSegments and heightSegments to 128. This has a similar effect as using SubdivisionModifier. In fact, SubdivisionModifier distorts the shape, but specifying the segments does not distort the shape and works better for me.
var widthSegments = 128;
var heightSegments = 128;
var geometry = new THREE.PlaneGeometry(10, 10, widthSegments, heightSegments);
// var geometry = new THREE.PlaneGeoemtry(10,10); // segments default to 1
// var modifier = new THREE.SubdivisionModifier( 7 );
// geometry = modifier.modify(geometry);
https://threejs.org/docs/#api/en/geometries/PlaneGeometry

Many lights with shadows in three.js causes Fragment shader error

Assume to have a scene with a street with many streetlights (more 20), you move an object close by them and you expect a shadow.
The lights, simply
var light = new THREE.PointLight(0xffffff, 0.5, 6.0);
Only the street has .receiveShadow = true and only the car has .castShadow = true (besides later the lights)
In three.js adding .castShadow = true to all of the lights causes following error
THREE.WebGLProgram: shader error: 0 gl.VALIDATE_STATUS false
gl.getProgramInfoLog Fragment shader sampler count exceeds MAX_TEXTURE_IMAGE_UNITS (16).
Luckily in hour scene we only need a few (at max 4) of them to cast a shadow, as most of the lights are out of reach anyway.
I tried to use 2 approaches
Looping through all the lights and setting .castShadow = true or .castShadow = false dynamically.
Adding and removing the lights completely but setting them with no shadow or a shadow.
With both of them I got the same error.
What other approach would work?
Update
#neeh created a Fiddle for it here (to cause the error change var numLightRows = 8; to a higher number). Keep an eye on the error though, there will be another error with too many lights that isn't caused by the same problem
He also pointed out that we see here that a pointShadowMap is created even when not in use. This explains why there is no change with a "smarter" approach. This now is within the GLSL code.
So we are limited by the GPU, which in my case has 16 IMAGE_UNITS but that isn't the case for all GPUs (my CPU actually works fine with more). You can check on your system with renderer.capabilities.maxTextures. But as mentioned we really only need 4.
The problem remains.

The problem
Yes a new shadow map will be created for every light having castShadow = true (Actually, this is not the case, check this issue). A shadow map is a drawn on which a shadow is computed in order to be blended on a surface afterwards.
gl.getProgramInfoLog Fragment shader sampler count exceeds MAX_TEXTURE_IMAGE_UNITS (16).
It means that your device can send no more than 16 textures per draw call. Typically, the car (street?) on which you'd like to put shadows is 1 draw call.
To draw a object that receives shadows, all the shadow maps should be blended together with the diffuse map. So this requires to use N+1 texture units for one single draw call. (N being the number of lights that can cast shadow.)
If you dig into Three.js shaders, you'd find this :
#ifdef USE_SHADOWMAP
#if NUM_DIR_LIGHTS > 0
// Reserving NUM_DIR_LIGHTS texture units
uniform sampler2D directionalShadowMap[ NUM_DIR_LIGHTS ];
varying vec4 vDirectionalShadowCoord[ NUM_DIR_LIGHTS ];
#endif
...
#endif
Check this tool to see how much texture units your browser can handle (Fragment shader > Max Texture Image Units).
The solution ?
Dynamically creating and deleting lights is bad because it's memory-intensive (allocation of a shadow map...).
But, as gaitat said, you can enable shadows only for the nearest lights. Just do the following in your render loop :
Disable all shadows: light.castShadow = false;
Seek nearest lights
Enable shadow for N nearest lights: light.castShadow = true;
Improvement
This algorithm lonely is bad because it allocates one shadow map per light. In addition to be memory-consuming, the rendering would freeze for a bit every time you cross a new light that has no shadow map allocated...
Hence, the idea is to reuse the same shadows maps for the nearest lights. You can deal with shadow maps like this :
// create a new shadow map
var shadowMapCamera = new THREE.PerspectiveCamera(90, 1, 0.5, 500);
var shadow = new THREE.LightShadow(shadowMapCamera);
// use the shadow map on a light
light.shadow = shadow;
shadow.camera.position.copy(light.position);
light.castShadow = true;
You can get the maximum number of texture units with renderer.capabilities.maxTextures. So you can compute the number of shadow map to create based on it but remember to leave some for more regular maps like diffuseMap, normalMap...
Check out this fiddle for a full implementation (only 4 shadow maps are used).

Efficient way to light up/cast shadows on a voxel terrain

I'm using a BufferGeometry and some predefined data to create an object similar to a Minecraft chunk (made of voxels and containing cave-like structures). I'm having a problem lighting up this object efficently.
At the moment I'm using a MeshLambertMaterial and a DirectionalLight which enables me to cast shadows on voxels not in view of the light, however this isn't efficient to use for a large terrain because it requires a very large shadow map and will often cause glitchy shadow artifacts as a result.
Here's the code I'm using to add the indices and vertices to the BufferGeometry:
// Add indices to BufferGeometry
for ( var i = 0; i < section.indices.length; i ++ ) {
var j = i * 3;
var q = section.indices[i];
indices[ j ] = q[0] % chunkSize;
indices[ j + 1 ] = q[1] % chunkSize;
indices[ j + 2 ] = q[2] % chunkSize;
}
// Add vertices to BufferGeometry
for ( var i = 0; i < section.vertices.length; i ++ ) {
var q = section.vertices[i];
// There's 1 color for every 4 vertices (square)
var hexColor = section.colors[i / 4];
addVertex( i, q[0], q[1], q[2], hexColor );
}
And my 'chunk' example: http://jsfiddle.net/9sSyz/4/
A screenshot:
If I were to remove the shadows from my example, all voxels on the correct side would be lit up even if another voxel obstructed the light. I just need another scalable way to give the illusion of a shadow. Perhaps by changing vertex colors if not in view of the light? It doesn't have to be as accurate as the current shadow implementation so changing the vertex colors (to give a blocky vertex-bound shadow) would be enough.
Would appreciate any help or advice. Thanks.

Generally, if you have large terrains, the idea is to split the scene into more cascades and each cascade has its own shadow map. Technique is called CSM - cascaded shadow maps. Problem is, I haven't heard of an webGL example that implements this technique. CSMs are used on dynamic scenes. But I'm not sure how easy would be to implement this with Three.js.
Second option is adding ambient occlusion, as suggested by WestLagnley, but it's just an occlusion, not a shadow. Results are very different.
Third option, if your scene is mostly static - baked shadows. So, preprocessed textures that you simply apply to the terrain etc. To support dynamic objects, just render their shadow maps and apply those to some geometry that just mimics shadowed area (perhaps, a plane that hovers slightly above ground and receives the shadow etc).
Any combination of the techniques mentioned is also an option.
P.S. Could you also supply a screenshot, fiddles fail to load.