What's the best way to go about simulating a structured light camera?

[alexander-soare]

I want to simulate a particular aspect of structured light cameras. That is: only points in the FoV of both the image sensor and the structured light emitter have valid point cloud. To illustrate what I mean:

• Point A is illuminated by the light source but is obstructed with respect to the sensor. Therefore it would not appear on the rendered image. (no problem, this is how Blenderproc behaves)
• Points B and C are illuminated by the light source AND are visible to the sensor. They will appear on the rendered image, and should include valid depth map values. (no problem, this is how Blenderproc behaves)
• Point D is visible to the sensor, but is not illuminated. Therefore it should appear on the rendered image, but the depth map value should be NaN. (problem, how do I do this?)

So my specific ask is, how do I simulate the last bullet point: that is, point D has a NaN value on the depth map.

[cornerfarmer]

One thing you could try is:

• Project depth image received by sensor into pointcloud
• For each point do a ray cast starting from light source towards point
• if the raycast does not hit the point, but some obstacle in between then enter NaN into final depth map

[alexander-soare]

@cornerfarmer thanks for the tip. For the ray casting would you recommend anything other than Blender's ray_cast method? Not that I've got a particular issue with it. Just because I don't know what I don't know.