One of Psychopath's current limitations is that every surface has to have the same shader. This is obviously not workable if it's going to be used as a "real" renderer, so I need to design a shading system that allows different shaders to be assigned to different objects.
Eventually I want to use Open Shading Language for shading in Psychopath, but as a first pass I'm going to do something a little simpler to get my feet wet. I've never written a shading system, after all.
But even so, since Psychopath traces many rays at once there are actually some interesting choices to make here which will carry over to how Psychopath utilizes OSL. There are two broad approaches I can take:
Do the shading when a ray hits something. This would require storing the result of the shading in the intersection structure, which would be a BSDF closure.
Do the shading after all of the ray intersection tests have been completed. This would require storing all information necessary to do the shading in the intersection structure, which would be things such as a reference to the shader and any required differential geometry data.
The biggest benefit of the first approach is that it keeps the architecture a bit simpler. Displacement shaders will have to be evaluated during ray traversal anyway, so the first approach would keep all of the shading code in the same general part of Psychopath.
The biggest benefit of the second approach is that the shading evaluations can be reordered to maximize e.g. coherent texture access.
I've been going back-and-forth trying to decide which approach to take, but I think I've finally settled on the first approach. The first approach gives me more flexibility in how much data I store, and since Psychopath is using breadth first ray tracing there are already strong guarantees about accessing shaders and their associated data in a coherent way.
The main problem with the second approach is that the amount of data that needs to be stored is unbounded. If a mesh has e.g. multiple UV sets, then Psychopath has to store intersection data about all of those UV sets to be able to do the shading later. And I'd also like Psychopath to be able to support arbitrary geometry data for use in shaders, which would exacerbate the problem. It won't be possible to predict ahead of time how much data needs to be stored.
With the first approach, on the other hand, the amount of data can be bounded. There is still the possibility (even likelihood) of multiple mixed closures. But because they are closures, I have more flexibility. I can take a monte carlo approach and, after shading, select a subset of them to be stored along with the appropriate weighting information. The downside to that, of course, would be increased noise. But the max number of closures to store could be a render setting, which would allow the user to decide what to do with that memory/noise tradeoff.
The biggest problem with the first approach, however, is that the shader will need to be evaluated at every ray hit during traversal, rather than just the final hit. And especially for non-trivial shaders in a complex scene, that could have a non-negligible performance impact.
But in the end, I think it's a reasonable tradeoff. I would rather keep memory usage predictably bounded, as one of Psychopath's goals is to be able to render very large scenes.