Bindless Deferred Decals in The Surge 2

1. Bindless Deferred Decals in Philip Hammer (DECK13 Interactive GmbH) Digital Dragons 2019, Krakow

2. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow This talk is .. ● .. about ○ our own implementation of bindless deferred decals in D3D12 and Vulkan ○ overcoming problems of past approaches ○ practical bread & butter techniques ○ how to eﬃciently render many decals on the GPU ○ tackling common issues and glitches ● .. but unfortunately not ○ super-fancy new research ○ a comprehensive view on our new renderer ● .. and has ○ probably more too much source code 2 2

3. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Introduction ● Brief overview of the latest iteration of our inhouse FLEDGE engine ● 4th generation: re-designed renderer module from ground up: ○ Vulkan + D3D12 support only ○ Dropped legacy code (D3D11) ○ No more OOP code - all in for DOD / SoA ○ New possibilities for high-level renderer design (e.g. bindless texturing) ● Improved some existing features e.g. ○ Clustered Deferred + Clustered Forward Rendering ○ Physical-based Rendering ● Some fancy new rendering features, e.g. ○ Uniﬁed Volumetric Lighting, Irradiance Probe Grid, Simulated interactive grass, Temporal Antialiasing, etc. ○ Unfortunately not part of this talk :-) 3 3

4. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Motivation ● Decals = one key beneﬁt of deferred rendering ● Used in environment art + VFX ○ Static decals to detail the world ○ Dynamic decals spawned by the effect system ○ Gameplay-driven decals like footsteps ● High demands on decal count, usability and performance 8 8

5. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Decal usage & examples 99 9

6. Decal usage & examples 10

12. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Rasterization-based deferred decals ● Had deferred decals for quite a while [LotF14] ○ Be aware of decal count, overdraw, etc. ○ Limitations led to sparse usage ● “Classic” deferred decal rendering ○ analogous to deferred light volume rendering ● Improved that a lot over the years, but basic principle stayed the same 16 16

13. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Rasterization-based deferred decals ● Advantages ○ Easy to implement and possible with most (even older) graphics APIs ○ Near pixel-perfect culling and shader segmentation ● Disadvantages ○ CPU-Performance ■ one drawcall per decal ■ Lots of state setup per decal ○ GPU-Performance ■ Double work: fetching gbuffers, rasterizing fragments, etc. ■ Need to update the G-Buffer as intermediate blend buffer 17 17 Transformed box geometry rasterizing a decal

14. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Deferred Decals ● How about rendering all decals at once? 18 18

15. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Deferred Decals ● How about rendering all decals at once? ● Problem: Too many resource bindings ○ Each decal references a potentially unique set of textures ○ Not possible to bind arbitrary amount of texture resources 19 19

16. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Deferred Decals ● How about rendering all decals at once? ● Problem: Too many resource bindings ○ Each decal references a potentially unique set of textures ○ Not possible to bind arbitrary amount of texture resources ● Solution(s): ○ Texture atlases or arrays -> limited and inﬂexible ○ is around since 2013 [Everitt14] -> vendor speciﬁc and what about consoles? 20 20

17. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Texturing ● D3D12 + Vulkan to the rescue! ○ Built-in support for bindless texturing ○ API speciﬁc descriptor arrays ○ Create a descriptor table for all textures in the game ○ Descriptors are globally available and bound to any draw/dispatch in need 21 21

18. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Texturing ● D3D12 + Vulkan to the rescue! ○ Built-in support for bindless texturing ○ Create an (API speciﬁc) descriptor array for all textures in the game ○ Descriptors are globally available and bound to any draw/dispatch in need ● Idea of bindless texturing is around for a while! ○ e.g. [Reed14], [Pettineo16], [Sousa16], [Drobot17] 22 22

19. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Texturing D3D12 / HLSL Vulkan / GLSL 23 23

20. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Non-uniform resource indices ● Texture index must be uniform ○ In GCN terms: loaded into an SGPR ○ It’s not by default if fetched from a GPU buffer ○ Non-uniform indices lead to unexpected results and graphical corruptions ● Need to explicitly make the index uniform! 24 24

21. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Non-uniform resource indices ● D3D12 / HLSL: ● Vulkan / GLSL: ○ Check [GpuInfo] for availability ● Alternative: use wave intrinsics ○ D3D12: Shader Model 6 wave intrinsics ○ Vulkan: ballot extensions ● Heavily driver- and vendor-dependent ○ Availability of extensions and performance 25 25

22. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Clustered Deferred Decals ● We already have a clustered deferred renderer for lighting ○ Similar approach to [Persson], [Sousa16], etc. ○ Cull lights, specular cubemaps, etc. into subdivided view frustum bins ○ Render everything in one go (single compute dispatch / fullscreen quad) ○ Easy to query clustered data also for forward-rendered objects ○ Already a good coverage on the topic, so we won’t go much deeper here 26 26

23. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Clustered Deferred Decals ● We already have a clustered deferred renderer for lighting ○ Similar approach to [Persson], [Sousa16], etc. ○ Cull lights, specular cubemaps, etc. into subdivided view frustum bins ○ Render everything in one go (single compute dispatch / fullscreen quad) ○ Easy to query clustered data also for forward-rendered objects ○ Already a good coverage on the topic, so we won’t go much deeper here ● Good ﬁt also for decals! 27 27 Output scene color Fetch G-Buffer Compute decals Compute direct lighting Lighting shader Compute indirect specular Compute indirect diffuse

24. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Clustered Deferred Decals ● Decal blending only in registers, no double fetching or updating of G-Buffer ● Very eﬃcient, but be aware of register pressure / low CU occupancy 28 28

25. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Decals Optimization ● High-level art optimizations ○ Cull by distance with soft fade-out ○ Switch to simpler feature set over distance 29 29 Decal overlay icons for all decals

26. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Decals Optimization ● High-level art optimizations ○ Cull by distance with soft fade-out ○ Switch to simpler feature set over distance 30 30 Decal overlay icons for all decals surviving distance culling

27. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Decals Optimization ● Low-level optimizations ● Biggest bottleneck in shader: VGPRs / CU Occupation ● GCN Loop Scalarization works great with decals ○ Load decal and material data in SGPRs and loop over necessary lanes ○ Note: not necessary anymore! ● Splitting decal and material features into isolated blocks ○ Free up registers by grouping features ○ Register lifetime is not always obvious - measure and proﬁle ○ Reorganization of large shaders with a lot of resource access is always worth a try! 31 31

28. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow GCN loop scalarization ● Great in-depth coverage available [Drobot17] ● Make cluster index uniform / scalar ○ Load all related data (DecalData, MaterialData, etc.) into SGPRs ○ Index is actually not uniform, but most pixel in a wave likely hit the same index ○ Double work can happen but higher occupancy will cancel it out ○ Only doable with wave intrinsics 32 32

29. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Bindless Decals Limitations ● Downsides of blending decals in registers ● Decals not visible for other (deferred) renderpasses ○ normals + albedo won’t see decal modiﬁcations (e.g. SSAO, SSR, separate ambient pass) ○ Material-ID not updated (TAA Ghosting! [Xu16]) ● Could render decals before lighting and update G-Buffer ○ Additional costs for e.g. copying G-Buffer Normals 33 33

30. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues 34

31. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #1: Normal blending ● Linear blending is trivial and useful for e.g. snow accumulation ○ Perfect for thick covers like layers of sand or snow ○ Tends to “ﬂatten” normals in gradual blending: normal directions cancel out each other 35 35 Decal with linear normal blending

32. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #1: Normal blending ● Re-orientated normal mapping [Hill12] ○ Our default mode for blending, works best for most decals ○ Decal normal will always follow surface normal ○ Fits mostly better into the scene 36 36 Decal with re-oriented normal blending

33. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #2: screenspace UV gradients ● UV- and Z-gradients cannot be derived correctly in screenspace ○ No hardware gradient data (ddx/ddy) available in shader loops ○ No mip-mapping and wrong UV-gradients around object boundaries ● Solution: ○ Sample a ﬁxed Mip Level :-( ○ Compute gradients in shader ■ Slightly higher runtime cost ■ Need to compute 2 addition decal-space positions with 1-pixel-offset in X and Y direction ■ Be aware of decal & uv scaling! 37 37

34. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #2: screenspace UV gradients ● Computation of UV gradients in the shader ● Be aware that hardware gradients of position is not correct 38 38

35. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #3: screenspace Z gradients ● UV gradients = almost good ● But can get huge at depth discontinuities if gradient is computed in the shader ● Two possibilities ○ Store z-gradients in G-Buffer and compute with these ○ Or check for depth discontinuities in shader + ﬁx-up [Hammer18] 39 39

36. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #3: screenspace Z gradients ● Checkerboard is a prime example because of maximum contrast ○ lower mips converge to middle gray due to downscale ● Large UV gradients will fetch a very low mip ending up in a color discontinuity 40 40

37. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #3: screenspace Z gradients ● Find the edges causing the trouble ● Sample Lod0 if fragment is on edge ● Can be done directly in the decal fullscreen shader based on two depth quads 41 41

38. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #3: screenspace Z gradients ● Find the edges causing the trouble ● Sample Lod0 if fragment is on edge ● Can be done directly in the decal fullscreen shader based on two depth quads 42 42

39. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #3: screenspace Z gradients ● Color discontinuities are now gone ● Fixed edges can now suffer from Mip-Flickering ● In practice a rather minor issue ○ Barely noticeable during gameplay ○ Antialiasing - especially Temporal AA - migitates as well 43 43

40. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #4: Leaking and object separation ● Decals may leak into areas where they don’t belong 44 44

41. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Common decal issues #4: Leaking and object separation ● Decals may leak into areas where they don’t belong ● Solution: ○ Decals and materials both store a “DecalGroup” bitmask ○ Bitwise AND the masks and check if at least one bit matches ○ Up to artists how to assign bits (e.g. character, vegetation, environment, transparent, etc.) 45 45

42. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks 4646 46

43. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #1 - Smoothing Edges ● Modify decal normal to “smooth” corners and edges ● Interpolate surface normal towards (uniform) decal direction ● Simple but effective and artists love the hack ● Useful for covering uneven geometry e.g. with mud 47 47 Smoothed Edges off (smoothFactor = 0.0)

44. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #1 - Smoothing Edges ● Modify decal normal to “smooth” corners and edges ● Interpolate surface normal towards (uniform) decal direction ● Simple but effective and artists love the hack ● Useful for covering uneven geometry e.g. with mud ● Geometry normal is otherwise hard to override ● Be careful, it’s a hack 48 48 Smoothed Edges on (smoothFactor = 1.0)

45. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #2 - Water puddles ● Decalling shallow water puddles ○ Faking surface porosity by blending black decal albedo (only 10-20% opacity) ○ low roughness (100%) ○ water ripple normal (100%) ● No special code path, just special parametrization 49 49

46. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #3 - Texturing the world ● Base model (no decals) 50 50

47. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #3 - Texturing the world ● Base model + decal with UV-free texturing aka Triplanar mapping ○ Sample texture(s) 3x from different directions and blend based on decal normal ○ Triplanar contrast deﬁnes the “steepness” of the blending according to the normal 51 51

48. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Dirty decal tricks #3 - Texturing the world ● Base model + decal with triplanar + slope dependent blending ○ Generate a blend mask based on the slope of the underlying surface and the decal direction 52 52

49. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Future improvements 53

50. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Future improvements 54 54 ● Better tooling for bulk decalling ○ Brush decals with randomized parameters We already have that for small geometry instances like plants, debris, etc.! ○ Procedural placement based on world parameters (e.g. biomes, material types, indoor/outdoor) ○ Physically simulated placement (e.g. rigid body / particle simulation) ● Extend system to go full bindless ○ Bindless texturing also for scene rendering ○ Prototype exists but not yet ready for production

51. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Thank you for listening! @philiphammer0 phammer@deck13.com 55 Also a big thanks to the amazing DECK13 team which helped with contributions, reviews and suggestions!

52. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow We’re hiring! :) @deck13_de https://www.deck13.com/jobs/ 56

53. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow Q & A 57

54. Philip Hammer (Deck13 Interactive) Digital Dragons 2019, Krakow References [Persson] “Practical Clustered Shading”, Emil Persson, http://www.humus.name/Articles/PracticalClusteredShading.pdf [Drobot17] “Improved Culling for Tiled and Clustered Rendering”, Michal Drobot, Siggraph 2017 http://advances.realtimerendering.com/s2017/ [Hammer18] “Dissecting the Rendering of The Surge”, Philip Hammer, Quo Vadis Berlin 2018 https://de.slideshare.net/philiphammer/dissecting-the-rendering-of-the-surge [Xu16] "Temporal Antialiasing in Uncharted 4", Ke Xu, Siggraph 2016 http://advances.realtimerendering.com/s2016/ [Lagarde14] “Moving Frostbite to physical based rendering”, Sébastien Lagarde, Siggraph 2014 https://seblagarde.ﬁles.wordpress.com/2015/07/course_notes_moving_frostbite_to_pbr_v32.pdf [Wronski15] “Fixing screen-space deferred decals”, Bart Wronski, 2015 https://bartwronski.com/2015/03/12/ﬁxing-screen-space-deferred-decals/ [Sawicki15] "Watch out for non-uniform resource index!", Adam Sawicki, http://www.asawicki.info/news_1608_direct3d_12_-_watch_out_for_non-uniform_resource_index.html [Sousa16] "The Devil is in the Details: idTech 666", Tiago Sousa, Siggraph 2016 https://de.slideshare.net/TiagoAlexSousa/siggraph2016-the-devil-is-in-the-details-idtech-666 [Pettineo16] “Bindless Texturing for Deferred Rendering and Decals”, Matt Pettineo https://mynameismjp.wordpress.com/2016/03/25/bindless-texturing-for-deferred-rendering-and-decals/ [Reed14] “Deferred Texturing” http://www.reedbeta.com/blog/deferred-texturing/ [LotF14] “Building your own engine, part 3 – Visual effects for the next generation”, Benjamin Glatzel, David Reinig https://www.makinggames.biz/news/building-your-own-engine-part-3-visual-effects-for-the-next-generation,6330.html [Hill12] “Blending in Detail”, Stephen Hill, 2012 https://blog.selfshadow.com/publications/blending-in-detail/ [Everitt14] “Approaching zero driver overhead”, Cass Everitt et. al.,, Siggraph 2014 https://de.slideshare.net/CassEveritt/approaching-zero-driver-overhead [GpuInfo] “Vulkan Hardware Database”, Sascha Willems https://vulkan.gpuinfo.org/listextensions.php 58 58