Efficient Rendering of Layered Materials
using an Atomic Decomposition with Statistical Operators
Laurent Belcour
Thank you for coming to this talk.
Before I start, I would like to talk a little bit about how I came to work on such a topic.
Last Year ...
Rendered in Unity
Last year, I presented a technical paper on real-time rendering of iridescence produced by thin-film interference. Here is a snippet of the fast forward. This part was rendered in the engine Unity.
Last Year ...
Rendering thin-film iridescence
Using a clear-coat plugin in Mitsuba
But no clear-coat available in Unity 😭
I wanted to show the beetle live
One solution: code one in time!
Turn out we can do much more
Multiple rough interfaces
Energy conservation
...
Rendered in Mitsuba
With this car shot, I wanted to reproduce this Mitsuba rendering from last year's paper but in real-time. click This rendering used the clear-coat plugin on top of the iridescent one. The idea behind the clear-coat plugin is quite simple. The view ray is bent with respect to Snell's law and is used to sample the base BSDF. All those reflected rays are then bent to get out of the clear-coat. While this fails to account for multiple scattering in the clear-coat, it correctly follows light transport. click However, getting a physically based clear-coat in a real-time renderer is much more challenging. There, because IBL and area-lights are preintegrated so you cannot bent ray like mitsuba does. So people do a lot of hacks that are ... what they are. While they are efficient, they seem to take an different definition of physically based rendering. click
That didn't stop me from trying to make it. Only, I had to build it from the ground up and in time. click
But while doing it, I wanted to go further than a Mitsuba-like smooth clear-coat. I felt that an energy preserving rough coating was at reach. And fortunately, I could go even further than what I expected. And this model proved to work as well for offline rendering. So I guess that a nice first take home message would be that the constraints of real-time rendering can lead you to very interesting solutions, even for offline.
Layered Materials
Layered materials enable to build a richer set of appearances with relatively simple processes. It is not surprising that they are heavily used in *click* car paints, *click* furniture design, and *click* others applications for paints *click* and jewelry.
Layered Materials
$\boldsymbol{\omega}_i$
$\boldsymbol{\omega}_o$
In this talk, we restrict layered materials to be made of a stack of rough dieletric or conducting interfaces *click* that are parallel and planar at the macroscopic scale. *click* The number of interfaces is however arbitrary. *click*
We also assume that the material is viewed from sufficiently far away such that light enter and exits the layered structure approximately at the same position. However, for clarity, the illustrations will depict different outgoing position. Keep in mind that those are not accounted in the end. *click*
The challenge here is the following: Given an incident beam of light, we want to describe light transport in the layered structure *click* for any number of scattering events *click* . Note that those interaction can be fairly complicated *click* .
On issue is that *click* light can bounce many times between two interfaces or can even *click* bounces between many interfaces before reaching the outside.
Layered Materials
Brute-force rendering is not possible
At least not in real-time graphics
Precomputation is not statisfactory
Forbid to use textured assets
Memory limitation on GPUs
Our solution: tight approximation
Real-time constraints prevent us from brute-force rendering of such transport. There, we do not have the budget to sample enough path for a converged estimate. *click*
Because of the dimensionality of the transport, it is not possible to fully precompute its solution and keep and editable model. *click*
So instead, I will talk about an approximating method that tries to be as faithful as possible to the real light transport while being compatible with real-time contraints.
Our Idea: Sum of GGX Lobes
$\mathbf{\omega}_i$
$\mathbf{n}$
$$\rho(\mathbf{\omega}_i, \mathbf{\omega}_o) = \sum_{k} w_k \, \rho_k(\mathbf{\omega}_i, \mathbf{\omega}_o)$$
Our model works as follow: starting from an incident direction, we virtually group together paths that interact with a specific interface. For example, *click* we group together directly reflected paths, *click* or paths that interact with the second interface but not below, and finally in this case *click* all the paths that interact a least once with the bottom interface. *click*
However, we do not explicitly compute those paths and instead work with statistical summary of light transport. In the end, each group of paths is described by its zero, first and second order statistics. *click*
From each set of statistics, we instantiate a GGX microfacet model that approximate those statistics. In this talk, I will refer to those instantiated models as GGX lobes. *click* And our approximate BRDF *click* consists in the sum of those models.
Statistical Analysis: Mapping
We study BSDF statistics
In the orthographicaly projected disc
There, GGX is almost rotationaly symmetric
Orthographic projection GGX lobe with $\alpha = 0.01$
Lets have a look at how statistics can describe microfacet models. *click*
We used the hemisphere of directions orthographically projected onto the tangent plane to study the BRDF lobe statistics. *click* We do so because in this space, a GGX microfacet model has a shape that is close to being rotationally symmetric and this permits to restrict the number of statistics to study to only three values: the energy, the mean and the variance of the BRDF lobe.
Statistical Analysis: Mapping
We study BSDF statistics
In the orthographicaly projected disc
There, GGX is almost rotationaly symmetric
To find a mapping
From the three moments (energy, mean, variance )
To a BRDF lobe parameters (albedo, view, roughness )
Orthographic projection Equivalent Statistics
$\mathbf{\omega}_i$
$(e, \mathbf{\mu}, \sigma)$
We can do so because there is a bijective mapping between those statistical moments and the BRDF lobe parameters that are the directional albedo, the view direction and the roughness.
Statistical Analysis: Mapping
We study BSDF statistics
In the orthographicaly projected disc
There, GGX is almost rotationaly symmetric
To find a mapping
From the three moments (energy, mean, variance )
To a BRDF lobe parameters (albedo, view, roughness )
Can we find the statistics of layered materials?
So this is the setup of our model: given an incident direction, we need to find a set of statistics that approximate well the layered BRDF. Those statistics will instantiate multiple BRDF models that will render the correct appearance.
Statistical Analysis: Framework
Infer statistics atomically
Details in the paper
Update $e$, $\mu$, and $\sigma$
Example: refraction operator
Shift, scales and convolves the incident lobe
$$
e_t = \tilde{\mbox{F}} \, e_i
$$
$$
\mu_t = - \eta_{12} \, \mu_i
$$
$$
\sigma_t = \eta_{12} \, \color{blue}{\sigma_i} + \color{red}{s}
$$
Reflection
Refraction
Scattering
Now we have to answer the following question: what are the statistics of light transport in a layered structure? To answer this question, we can build a statistical framework that allows to infer the statistics of radiance after atomic operations such as surface reflection, surface refraction and medium scattering. Those operators are fully detailed in the paper. In this talk, it is sufficient to know that each operator update the statistics of the incident radiance. *click*
Let's take the example of light refracted by a dielectric surface. *click*
The refraction ... *click*
scales the energy due to the average Fresnel term in the incident directional footprint. *click*
shift the mean with respect to the index of refraction ratio. This is a simple translation of Snell's law to the directional statistics. *click*
Another impact is that the incident variance (here in blue) is scaled by the change of interface and increased by a term (here in red) that depends on the surface's roughness.
I encourage you to read the paper for more details on those operators.
Statistical Analysis: Validation
Interactively testing atomic operators
We validated the accuracy of our atomic operators with a set of unit test that you can find in our supplemental material. Here I will present the results for the refraction operator. *click*
Given an incident beam of light that you can see here highlighted in red. *click*
We can compute the ground truth transmitted radiance that you can see on the right. For a roughness of $0.1$ and an IOR of $1.5$.*click*
And now here is the approximation. As you can see the approximation is quite close for incidences close to the normal but starts to differ at grazing angles.
Statistical Analysis: Framework
Multiple layers: chain operators
Now that we can summarize light transport at every interface, we need to be able to express the transport for specific groups of light paths. *click*
Lets take the following example of a group of light paths refractive two times before being reflected by the bottom interface and leaving the layered structure after two more refractions. *click*
In our framework, we use the central path to estimate the statistics. *click* This central path is constructed by converting all interfaces as smooth ones. Thus it correspond to a purely specular path. *click*
We start at the first vertex of this path and set the statistics to match a dirac in the incident direction. *click*
Then, we apply the first operator which in this case is a refraction. *click* This will specify the statistics of the outgoing radiance. *click* We convert those statistics into incident statistics by mirroring the statistics. and proceed with the next interface. *click*
We continue this process for the remaining vertices of the path until the outgoing direction is reached. *click until end* Note that this example is here to give the intuition of how operators chain together, there exist an infinite number of specular paths and in practice we can not explicitly evaluate them all. Instead, we rely on three mechanisms for efficient evaluation: merging of statistics, analytic form for multiple scattering, and adding-doubling.
Statistical Analysis: Framework
$(e_1, \mu_1, \sigma_1)$
$(e, \mu, \sigma)$
$(e, \mu, \sigma) = \left(e_1+e_2, \mu, \dfrac{e_1}{e}\sigma_1+\dfrac{e_2}{e}\sigma_2\right)$
$(e_2, \mu_2, \sigma_2)$
$+$
Given two central paths carrying both statistics, *click* we need to be able to evaluate the averaged statistics carried by both group of paths. *click* Fortunately, central paths will always carry the same mean at the last vertex. *click* The statistics of this group of paths is given by the weighted average of those statistics.
Statistical Analysis: Framework
Merging statistics
Multiple scattering
Closed-form statistics
Arithmetico-Geometric series
We found out that, in a single layer, there is an closed-form form for the statistics of multiple scattering. In this case, we write the energy and variance as arithmetico-geometric series for which there is a closed-form solution.
Statistical Analysis: Framework
Merging statistics
Multiple scattering
Closed-form statistics
Arithmetico-Geometric series
Adding-Doubling
We associate this finding with a new adding-doubling method. The idea behind the adding-doubling method is to estimate the scattering statistics of a stack of layers by iteratively applying the multiple scattering formula on pairs of layers.
I will illustrate here how it works. Let say we want to evaluate the BRDF of the layered structure. Given the incident direction, *click* we start by computing the reflected and transmitted directional statistics due to the first interface and record the reflected statistics as our first BRDF lobe. *click* Next, we evaluate the multiple scattering between the first two interfaces and record the reflected statistics as our second BRDF lobe. *click* Next, we virtually merge these two interfaces of this layer into a single interface. This virtual interface response to an incident beam is defined by the reflected and transmitted statistics. *click* Finally, we evaluate the multiple scattering equation using this new layer and record the reflected statistics as the third lobe. Note that the adding-doubling will account for all the multiple scattering happening between the top layers and the current layer considered.
Offline Validation
Mitsuba renderer
Both opaque and transparent plugins
Varying number of textured layers
Multiple Importance Sampling with the lobes
First I will present the validation I made in the Mitsuba renderer. There I implemented two plugins both for opaque and semi-transparent surfaces. Those plugins can take an arbitrary number of layers with every parameter mapped to a texture. I importance sampled each lobe independently and used a multiple importance sampling weighting to aggregate them.
Offline Validation
Mitsuba renderer
Comparison with stochastic reference
Ours
Reference
Metal foil
Rough metal
Gold Coated
I validated that the predicted appearance was close to the reference. For that I used a stochastically evaluated ground truth. There, paths inside the layered structure are randomly sampled. *click* As shown in this comparison, the approximation stays close visualy from the reference.
Offline Validation
Mitsuba renderer
Comparison with stochastic reference
Layered method of Weidlich & Wilkie [2007 ]
Ours
Reference
[WW07]
Reference
I also validated those results regarding the model of Weidlich and Wilkie that is currently considered as the state of the art in industry. *click* In this example, the material consist of a rough dielectric on top of a smooth mirror. Because the model of Weidlich and Wilkie decouples more the evaluation of the different layers compared to ours, it differs significantly from the reference.
Offline Validation
Mitsuba renderer
Comparison with stochastic reference
Layered method of Weidlich & Wilkie [2007 ]
Multiple scattering
$R + TRT$
$R + TR^+T$
Ours
Finally, I validated that the model is energy preserving *click* and approximate the color saturation due to multiple scattering between different layers quite correctly. On the bottom are shown three rough gold sphere with a rough coating. On the left you can see the reference solution with where the light paths are restricted to only one reflection. This doesn't include multiple scattering between the two interfaces as shown in the middle. Notice how saturated the multiple scattering is.
Offline Results: Textures
Textured base $\alpha$
Textured top $\eta$
Textured top $\alpha$
Offline Results: Robot Bust
Two layer configuration
Multiple textured layers
Base and top Index of Refraction
Top roughness
Now lets have a look at more eye-candy results. In this scene, we have a robot *click* covered with a two-layers paint. *click* Both the base and top layer resp. real and complex IOR are textured as well as the top roughness. This validate that this model works well with textured data.
Offline Results: Robot Bust
Here we scale the texture from a constant to the texture value by interpolation. When we rotate the bust to see the angular effects.
Offline Results: Dragon
Gold metal dragon
Adding a medium layer
Simulate dust
Increase the haze
In this example, we illustrate the ability of the model to incorporate effets from participating medium between layers. We start from a gold metallic dragon and *click* add a layer of dust modeled using a forward participating media. This increases the haze of the appearance. *click* And here is the comparison when we remove the dust layer.
Real-Time Results in Unity
Last, lets look at the results obtained in the Unity engine. *click* This video showcase an editing session in the editor, recorded on a desktop PC with an Nvidia GTX 980. In the shader implementation, the number of layers is set three: first a dieletic coat, then a participating media and last a conducting base. Thanks to the efficiency of the model and because we do not precompute data per material, a user can edit the layer material in real-time and change the base color the top IOR and roughness... As you can see, verything is real-time.
Limitations: High Roughnesses
$\alpha = 0.3$
Ours
Reference
$\alpha = 0.6$
Ours
Reference
$\alpha = 0.9$
Ours
Reference
Before concluding, I would like to stress a limitation of the statistical analysis. We built our work on approximations of the scattering operators that are valid for low roughness. Thus, the statistical approximation becomes inaccurate for higher roughness values.
Summary
A novel BSDF model for layered structures
Accurate for low roughnesses
Accounts from multiple scattering
No parameter dependent precomputation
Compatible with real-time scenario
Our contributions
Statistical analysis of light transport in layers
New adding-doubling scheme for variance
To summarize, *click* I presented a novel BSDF model to handle layered materials efficiently. This model is very accurate for low roughnesses, preserves energy, doesn't not have parameter dependent precomputation allowing it to work with textured inputs. *click*
To achieve this, we made the following contribution: we developped a statistical analysis of light transport in layered structures and introduced an adding-doubling strategy for second order directional statistics.
Thank you for your attention
paper
supp. mat.
code
HDRP StackLit
available at belcour.github.io/blog
I encourage you to check out our paper and our supplementary pdf and code. Thank you for your attention.