+ All Categories
Home > Documents > today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via...

today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via...

Date post: 31-Aug-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
65
1
Transcript
Page 1: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

1

Page 2: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

2

Hello! On the behalf of AMD I would like to welcome you to today’s lecture on Authoring for Real-Time Tessellation and Displacement Mapping. My name is Abe Wiley and I am the Lead Artist in the Game Computing Applications Group at AMD. I have been with ATI/AMD for over 4 years now and have worked on projects such as the Toy Shop Demo, the Ruby: Whiteout Demo, and most recently the Froblins demo… and this last demo is what this presentation is all about. This lecture will discuss the most important and critical things I learned as a technical artist about real-time tessellation and displacement mapping as we developed the Froblins Demo. This demo was produced as a launch demo for the Radeon HD 4870 product, and while the tessellation technique that we use in the demo does utilize the native tessellation hardware on the card, I have strived to keep this presentation as vendor and platform agnostic as I possibly could. It is my goal to provide you with information that will be helpful to you, no matter what your target platform, as you begin to explore the possibility of adding real-time tessellation and displacement mapping technology to your next generation titles.

Page 3: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The motivation behind this talk is simple. We want characters in our games that look like this. We want characters that have an extremely high level of detail. We want characters that have smooth silhouettes and that exhibit no polygonal artifacts what-so-ever. We also want to be able to render these characters, even massive crowds of these characters, at interactive frame rates.

3

Page 4: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

We also refuse to accept any compromises. Not only do we want to render massive crowds of these highly detailed characters, but we want to render them while utilizing the most advanced lighting models available, such global illumination.

4

Page 5: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

We want to be able to render characters with meshes so dense… our wireframes look solid.

5

Page 6: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

We also want to place these characters in environments that are nearly as detailed as they are.

6

Page 7: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The purpose of this lecture is to provide you with everything you need to know in order to author art assets for use with this technique successfully, with minimal disruption to your existing art pipelines. This lecture will provide you with a high level understanding of the benefits of real-time tessellation. We will also take a look at the various methods in which this technique can be implemented and what implications these choices have on your art authoring pipelines. Next we will cover some specific high and low polygon modeling tips and techniques that will help you avoid common pitfalls and that will help you author the right way, the first time. Next we will provide an overview of the GPU Meshmapper tool. This tool is a freely available map generation tool that creates normal and displacement maps for use with this technique we chose to use in the Froblins Demo. Next we will discuss how it is possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action. After providing you with an overview of all the technology in the Froblins demo we will take a look at this incredible demonstration of real-time tessellation and displacement mapping applied to a crowd of 3000 characters.

7

Page 8: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

So, first, let us take a look at the obvious and not so obvious benefits of real-time tessellation.

8

Page 9: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The first and most obvious benefit of real-time tessellation and displacement mapping is the dramatic increase in visual quality. Tessellation in conjunction with displacement mapping eliminates one of the last hurdles towards achieving cinematic quality visuals in games. Film has been using this technique for years in order to provide animators with manageable meshes to work with during the animation portion of development while still providing the highest quality results at render time. This technique eliminates polygonal artifacts and provides highly detailed, smooth internal and external silhouettes.

9

Page 10: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Another slightly less obvious benefit of this technique is the effect that it has on your memory footprint. Essentially, you can think of real-time tessellation and displacement mapping as an effective form of geometry compression. This technique utilizes the same art assets as conventional rendering with the only additional storage requirement being the 16 bit displacement map. If we take the Froblin character as an example we see that the memory footprint for the low resolution mesh and 2k x 2k 16 bit displacement map requiring about 10 megabytes of video memory. This is compared to the 450 megabytes of video memory that would be required to render the high resolution Froblin model that weighs in at around 15 million triangles. So, for just a modest increase in memory footprint, we are able to dramatically increase the total polycount and visual detail of the render mesh when using this technique.

10

Page 11: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Another less obvious benefit of this technique is in animation quality. Transforming the low resolution mesh is faster than attempting to transform the high resolution equivalent. This means that we get better animation performance. Also, as we just saw in the previous slide, we are storing exponentially fewer vertices in video memory. This means that we are able to store more data per vertex. What this provides us, for example, is the ability to increase skinning quality by being able to store more influences per vertex. This would also allows us to store a much larger library of morph targets for better facial animation, etc.

11

Page 12: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Performance is yet another benefit of real-time tessellation and displacement mapping. You can see here that when implemented with multipass rendering and vertex compression we are able to render over 400 times as many polygons while only taking a 33 frames per second or 30 percent performance hit when compared to rendering only the corresponding low polygon mesh. That is a pretty good trade off.

12

Page 13: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

A robust level of detail system is both a benefit and an absolute necessity when using this technique. Despite having such strong performance characteristics attempting to tessellate all characters in a scene (such as this scene from the Froblins Demo) to the same level regardless of character count, distance from camera, etc. is both wasteful and will bring your frame rate to a crawl. This problem is compounded by dynamic interactive scenarios where the number of characters on the screen can change dramatically and unpredictably from frame to frame. In order to overcome these challenges we needed to design a LOD system that would allow us dynamically adjust per frame the tessellation level of the characters based on distance from camera and the number of froblins being tessellated. To do this we utilized DirectX 10.1 features to render the froblins as an army of instanced characters utilizing a dynamic tessellation level.

13

Page 14: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Lets take a look at how we calculate this tessellation level. In this equation Ti is the level to be used for characters in the highest LOD level… the level that is being tessellated. N is the number of character instances that are currently in the highest LOD, and Tmax is the maximum tessellation level allowed for a single character. This scheme effectively bounds the number of triangles created by the tessellation process, and ensures that the triangle count will never increase by more than the cost of M fully tessellated characters. If there are more than M such characters in the view frustum, this scheme will divide the tessellated triangles evenly among them. While this can and does lead to slight popping as the size of the crowd changes dramatically from one frame to the next, it is very difficult to perceive in a dynamic scene with animated characters.

14

Page 15: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

This powerful and dynamic tessellation system provides several benefits. The LOD system allows you to use dynamic or pre determined LOD levels, or tessellation factors, to deliver the highest quality visuals results while ensuring stable performance. So, for example, you could use dynamic tessellation LODs during game play, but switch to pre-determined LODs during your in engine cinematics in order to have more control over where you are using these additional polygons to maximize their impact on your visuals. This system also provides you with a very easy way to maximize performance across various graphics hardware. It is also equally useful if you have 5 or 500 characters on the screen, so it is applicable to nearly any game genre whether you are developing an FPS or an RTS.

15

Page 16: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Lets take a look at that scene again but this time with our LOD visualization mode turned on. You can see here that there are three LOD “levels”. The lowest LOD level, colorized in blue, is actually a pre-authored uber low polygon mesh that has about 600 triangles. The middle LOD level, colorized in green, is our base low polygon mesh which has 5,160 triangles. The highest LOD level, colorized in red, is the level that contains the tessellated froblins. These froblinsare rendered with a dynamically updated tessellation level that changes depending on the number of tessellated froblins in the view frustum.

It is important to note that when a froblin transitions from one LOD level to another there is technically a “pop” as one model is swapped out for another or as tessellation is turned on or off. However, this pop is not noticeable for a few reasons. Carefully chosen scene depths at which these transitions occur are used to minimize this pop. This means that the characters, when transitioning between LOD level are at a size on the screen that in and of itself makes this transition hard to notice. Additionally, the same normal map is used across all LOD levels to ensure unified lighting across the LODs. It also does not hurt that the characters are animated. With all of these factors combined the POP between LOD levels is essentially eliminated.

This LOD system is also combined with an occlusion culling system to further optimize the render load and help maximize the available polygon budget for the tessellated froblin characters.

16

Page 17: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Now that we have covered the benefits of real-timetessellation and displacement mapping, lets take a look at some of the core concepts that you need to understand before you begin authoring assets for use with this technique.

17

Page 18: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The first concept you should be aware of if you are thinking about implementing this technique is the concept of Continuous vs Adaptive tessellation. There are basically two general methods for tessellating triangle or quad primitives. With continuous tessellation, as you see in these diagrams, each edge is divided equally as the tessellation level or factor is increased. With continuous tessellation we have a smooth change in the level of polygonal detail as the tessellation factor is adjusted. This method exhibits less noticeable popping as new verticies are created and is much more conducive to ensuring that seamless displacement across UV borders. This is the most commonly used method for skinned and animated characters.

18

Page 19: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Adaptive tessellation allows for each edge to be tessellated todifferent tessellation factors. This method allows you to control which areas of an object get tessellated and how much tessellation is applied. This allows us to do things like only add detail where it is needed, or tessellate only on silhouettes, tessellate based on the size of the triangle on screen, etc. This method requires special consideration in order to avoid displacement seams and artifacts across a single mesh and it this is particularly challenging with skinned and animated meshes. It is for this reason that this method is most commonly used with static meshes, such as terrain, and in fact we used this method on the terrain in the Froblins Demo.

19

Page 20: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The next concept our choice you will have to make when deciding what real-time tessellation and displacement mapping technique you are going to implement is the concept of Surface Type. There are two main surface types. The first type of surface is a Higher Order Surface. Catmull-Clark subdivision surfaces is one example of a higher order surface that is widely used in offline rendering. These surface types are implemented in real-time as an approximation of the offline technique with largely similar results. The evaluation pre-pass repositions the verticies, essentially “smooths” the base mesh or control cage, and the displacement is then applied as an offset from these new “smoothed” vertex positions.

So… the implementation pipeline for this type of surface would be Transform Verts, Evaluated Surface, Tessellate, Displace, and then Render

This technique has several pros and cons…

Pros…

Same method as used in film pipelines

Tessellation creates new triangles that flow with original control cage… more elegant

Can be guaranteed to be “watertight” if properly implemented

Cons…

Has the additional complexity of the surface evaluation step when compared to Interpolative Planar Surfaces, which we will discuss next

Wide range of evaluation methods… all of which are approximations of the offline methods

Forces your character modelers to become proficient with sub-d modeling tools… unclear how many art departments would really want to do this

Requires you to use corresponding control cage which might not be optimal for animation or for rendering without tessellation and displacement

20

Page 21: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The second surface type can be described as Interpolative Planar Surfaces. Interpolative Planar surfaces do not require an evaluation step and are not “smoothed” prior to displacement. Prior to displacement, even when highly tessellated the topology of the mesh looks identical to the original low polygon mesh. This means that the displacement is applied to the mesh as an offset from the new verticies that are planar to the original triangle face. The point here is that displacement maps created to work with Higher Order surfaces will not work with Interpolative Planar Surfaces, and vice versa.

Interpolative Planar Surfaces also have several pros and cons…

Pros…

You can model your high and low resolution mesh any way you want

High and low resolution meshes are independent

Better backwards compatibility as low resolution mesh can be customized and optimized and lighting comes from tangent space normal map

Due to the lack of the surface evaluation step, this method should be less expensive than Higher Order Surfaces

Cons…

Tessellation creates triangles that do not flow with original mesh… not elegant

This creates a slightly irregular surface even at highest tessellation factors

Difficult if not impossible to eliminate displacement seams in all cases… aka not “watertight”

This is the surface type we chose to use in our implementation for the FroblinsDemo.

21

Page 22: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Lets take a look at some examples to better illustrate the differences between these two types of surfaces. For this example lets isolate the right hand of our froblin character. In this image we see the our low polygon base mesh.

I do want to point out that for illustrative purposes I hardened all normals on the mesh so that you can get a better idea of the actual mesh topology. The actual mesh that was used in the demo and for displacement map generation does have smooth normals.

22

Page 23: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

A higher order or subdivision surface implementation of this technique would first require an evaluation step where the “control cage” is converted to a lower order surface. This process actually alters the topology of the original mesh… it smooths and rounds off the hard edges of the original mesh. The mesh would then be displaced as an offset from this new “smoothed” mesh and your displacement map would need to be captured accordingly.

Again, for illustrative purposes this example is not an actual conversion of the original mesh to a lower order surface. I simply performed a smooth function on the original low polygon mesh to illustrate how the topology changes during this step of a subdivison surface implementation of this technique.

23

Page 24: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

And here we see that in Interpolative Planar Tessellation the actual topology of the mesh does not change when the mesh is tessellated. The mesh is then displaced from this newly tessellated mesh that actually has the same exact surface topology as the original low polygon base mesh.

Again, this is just meant as an illustration so ignore the triangle tessellation pattern you see here. The important knowledge to take away from these examples is that the displacement maps for each of these methods is unique and is not interchangeable. Also, for both methods custom export or map generation tools need to be created to generated displacement maps for use with each of these methods. For our demo we chose to use Interpolative Planar Tessellation and the tool that we created to generate our maps, the GPU MeshMapper will be covered in detail later in the presentation.

24

Page 25: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Now that we have covered some of the core concepts involved with real-time tessellation and displacement mapping, lets take a look at some specific tips and strategies that will help you author your low and high resolution models the right way the first time.

25

Page 26: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

As I mentioned earlier one of the main benefits of utilizing Interpolative Planar Surfaces is that your low and high resolution meshes can be authored completely independent of each other. This means that you can author then any way you want and with any tool you want. This means there is minimal disruption to your existing art pipelines as you can continue to use MudBox, Z-Brush or whatever tools and techniques your artists are already using. Any tool which allows you to export to OBJ will work with this method, and I will explain more about that a little bit later in the presentation.

26

Page 27: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The same thing applies to authoring your low resolution models. You can author them any way you want, with any tools you want as the mesh is completely independent from you high resolution model. This allows you to fine tune your model in several ways. For example you could adjust the poly count for performance reasons, add more edge loops around joints for better rigging, or place verticies and edges explicitly for better results with tessellation and displacement mapping.

27

Page 28: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

However… this flexibility in authoring your low and high polygon models, the complete lack of dependency between the two meshes, can lead to the temptation to do things that you just simply can not do. Essentially what you need to understand is that all major features need to be represented in some way in the low resolution mesh. Trying to capture and displace major features without enough representative geometry in the low resolution mesh will not provide satisfactory results. You will end up with shearing, texture stretching, and a jagged mesh surface and silhouette if you attempt to attempt to do this. Lets look at an example…

28

Page 29: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

So, to illustrate this point lets take a look at the rope that is used to hold the backpack onto the Froblin character. This object has a lot of very intricate detail and was modeled as a separate sub-tool in Z-Brush.

29

Page 30: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

In order to keep the polygon count as low as possible in your low resolution mesh you may be tempted to omit an approximation of the shape all together and attempt to simply displace the rope out from the surface of the body mesh itself. This is a bad idea and just simply will not provide good results. The displacement maps will not capture adequate data and the displacement of the rope out of the body will result in very ugly mesh and texture shearing in the areas of the most dramatic displacement.

30

Page 31: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The best practice is to create a low resolution mesh that accurately approximates the volume of the high resolution rope model. This will capture much more accurate displacement maps and will provide the highest quality results.

31

Page 32: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

This is what the final low polygon model for froblin and rope object looks like.

32

Page 33: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

When using real-time tessellation and displacement mapping your UV layout is of the utmost importance. All verticiesabsolutely must have unique UV coordinates and they can not share texture space. You also need to take great care to ensure that your texel density is as uniform as possible, particularly across UV borders as this is an important part of minimizing and eliminating displacement seams. Also, be sure to leave adequate space between UV chunks for texel border expansion, as again, this is an important part of eliminating artifacts. You must understand that tessellation and displacement mapping is not forgiving. Unlike what you may be used to with traditional normal mapping techniques, errors that originate in your UV layout will be obvious and are difficult if not impossible to paint out.

33

Page 34: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Next, lets take a look at how to generate maps for use with this technique.

34

Page 35: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

As we discussed earlier, in the Froblins demo we made the decision to use interpolative planar surfaces in our implementation of real-time tessellation and displacement mapping. The only problem with that decision was that there were not any off the shelf tools that would generate the displacement data that would work with this implementation. So, the great people on our tools team created a tool for us called the GPU Meshmapper. Lets take a look at the key features in this freely available powerful map generation tool.

35

Page 36: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The GPU MeshMapper can generate both tangent and world space normal maps as well as displacement maps that work with Interpolative Planar Tessellation. The displacement map generation process also has a post processing step that filters across UV borders in order to reduce and in some cases eliminate displacement seams. The tool can also generate ambient occlusion maps and that process can be GPU accelerated on ATI hardware.

The GPU MeshMapper also has a very useful map composer feature which allows you to automate the compositing of map chunks and has an additional feature that will normalize displacement scale and bias values during the compositing of multiple displacement maps.

The tool also has a very powerful and extremely useful asset preview feature. Lets take a look at some of these features in a little more detail.

36

Page 37: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

In order to make the GPU MeshMapper tool as universally accessible and useful as possible we decided to include and rely upon OBJ file format support. The obvious advantage here is that this file format is universal and allows developers to use basically any tools they want to generate their low and high resolution meshes. This choice was driven by our goal to make adoption of this technique as minimally disruptive as possible to existing art pipelines. The downside to this is the OBJ file format has a fairly bloated file structure. This is further complicated by the inconsistency in OBJ exporters across various applications. File sizes can vary dramatically when the same mesh is exported into an OBJ from various different applications and this makes it difficult to gauge or predict actual memory consumption when the data is loaded into the GPU MeshMapper tool.

37

Page 38: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The only drawback to this approach of using the OBJ file format is that depending on the type of system you are using (32 vs 64 bit) and the amount of memory you have in the system, you will more then likely have to break you high resolution mesh into a few chunks in order to be able to generate the maps without running out of memory. The number of chunks you will need is also dependent upon the overall polygonal density of the high resolution mesh itself. Your low polygon model will also need to be cut up into corresponding pieces in order for the GPU MeshMapper to automatically generate the corresponding mask textures for use with the Map composer. While your high resolution chunks should have some overlap to ensure consistent values across chunk borders, your low resolution chunks should not have any overlap so that the mask textures will work properly with the Map Composer. Another thing to keep in mind is that if you run the GPU MeshMapper from a script it actually loads one less copy of the mesh into memory which means that your pieces can be bigger and you can break the model into fewer chunks. As I mentioned before, due to the lack of consistency between OBJ exporters it is difficult to provide you with file size recommendations, but with a little trial and error you will quickly be able to determine guidelines for what works best in you pipeline.

38

Page 39: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

A very important concept to understand when generating and working with displacement maps for use with real-time tessellation and displacement mapping is the concept of Scale and Bias. The scale value describes the amount in which a vertex is offset from its original position and is applied as a multiplier to the corresponding value in the displacement map texture. The bias value determines the “zero value” of the displacement texture, or what greyscale value corresponds to zero displacement. I have included a screenshot of the displacement map rollout in the GPU MeshMapper tool. This is where the user controls the method in which the scale and bias values are determined. If you leave this set to “Auto” the GPU MeshMapper will normalize the displacement values in the texture to ensure maximum fidelity. This means that the scale and bias values can vary widely from map to map, and hence the need for the Map Composer feature. If you already have a good idea of what scale and bias values work best for your given models, you can pre-specify those values here and then you would be able to manually composite your displacement maps together in Photoshop if you decided you wanted to do that for whatever reason.

39

Page 40: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Lighting from Normal Maps… so, why do we still need a Normal Map when we now have this highly tessellated and displaced mesh?First of all, even at the highest level of tessellation the surface of the mesh can still have irregularities which will manifest as tiny irregular facets if you attempted to rely solely on the geometry normals for rendering. Using a normal map eliminates these artifacts. Also, as we discussed earlier when talking about the LOD system used in the Froblins demo, using a normal map helps unify lighting across all the different LODs or tessellation factors which eliminates any popping in the lighting as a character transitions between LODs. You will also still need this normal map for backwards compatibility with hardware that does not support tessellation.

40

Page 41: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

It is also important for you to understand that the displacement map and normal map must be calculated using the same tangent space, because as we displace the tessellated mesh we are actually altering the displayed normal. What this means from a map authoring or map generation perspective is that you must use both the normal and displacement map out of the GPU MeshMapper tool when using this technique. Trying to use a normal map that was generated in another application will not work unless you know for certain that it was calculated using the same exact tangent space.

41

Page 42: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Here are a few additional important bits of information about Tangent Space. I think that there is a lot of misunderstanding or lack of understanding out there about the nature of tangent space normal maps, particularly with the popularity of tools such as Crazy Bump. It is important to understand that tessellation and displacement mapping requires physically accurate normal maps generated from actual mesh data. It is also important to understand that not all tangent spaces are the same and that the tangent space you use must be the same throughout your entire pipeline. This means that the tangent space you use to generate your maps must be the same as the tangent space that you render with in your game engine. Also, in order to ensure continuous tangent space across UV borders you must use an orthonormal tangent frame matrix. This will eliminate lighting discontinuities across UV borders.

42

Page 43: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Next lets talk a little bit more about the Map Composer feature built into the GPU MeshMapper tool… a screenshot of which you see here.

43

Page 44: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

As I mentioned earlier, when the GPU MeshMapper generates a normal, displacement, or ambient occlusion map it automatically generates a corresponding mask texture. These mask textures tell the Map Composer which texels contain valid texture data and which are empty or simply contain texels created by the border expansion algorithm. When you load a texture into the Map Composer these mask textures are automatically loaded in as well. For displacement maps the GPU MeshMapper also outputs the scale and bias values associated with that particular displacement map into a text file, which again is automatically loaded in when you load a displacement map texture into the Map Composer. The most important feature of the Map Composer is the ability for it to normalize the scale and bias values across multiple displacement maps.

44

Page 45: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

One of the biggest challenges when working with real-time tessellation and displacement mapping is how to minimize or eliminate displacement seams and cracks across UV borders. These artifacts are caused by the unfortunate fact that UV parameterization is rarely if ever one to one across UV borders. This means that when the texture fetch looks up the displacement value for a vertex along a UV border it may sample from several different texels and then average them together to get that value. The texels it samples for the corresponding UV on the other side of the border most likely will be a different combination of texels and values. This can introduce a crack in the tessellated and displaced mesh. They way we decided to solve this problem is by applying a combination of Nearest Neighbor and Bilinear filtering across UV border texels in order to drastically reduce and in some cases completely eliminate displacement cracks along UV borders.

There are a few advantages to this solution. First, this was a relatively fast and easy technique to implement and provides very good results. It also does not require any change to your existing UV layout, not does it require your UVs to be charted or re-parameterized in any way. This technique also does not require you to store any additional per vertex data. Lastly this method does not require any additional computation at run-time.

So, while this real-time tessellation and displacement mapping technique is not technically watertight, with a combination of a clean UV layout and displacement map post processing, any displacement seams that remain are extremely difficult to notice.

45

Page 46: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

So, now we get to the shortest yet probably the most important section of this presentation. How do you integratethis new technology into your existing art pipeline without completely blowing your schedules and missing deadlines. The answer is simple…

46

Page 47: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Scripting. Scripting will save your pipeline. After you have determined the optimal settings for generating your maps the entire process can be automated, including the final map compositing phase if you had to break your model up into chunks. You also don’t have to be an accomplished programmer to utilize the scripting feature. The GPU MeshMapper documentation contains example scripts, but even better than that is the ability to save out an existing project file into script form directly out of the tool itself. Before you know it you will be using scripting instead of the GUI. I also want to mention that the GPU MeshMapper documentation is excellent. You and I both know that often the documentation that comes with some of the tools we use in our profession can be awful. I am happy to say that this is not the case with the GPU MeshMapper help docs and I highly recommend you take a look through them if you are interested in using the tool in your art pipeline.

47

Page 48: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

So, now I would like to give you a brief demonstration of the GPU MeshMapper in action and show you the powerful asset preview functionality that is built into this great map generation tool.

48

Page 49: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Now we get to the fun stuff…

49

Page 50: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

As I mentioned at the beginning of this presentation, the Froblins Demo was created for the launch of the Radeon HD 4870 product. When conceptualizing the demo we wanted to utilize the real-time tessellation and displacement mapping technique in a situation that would demonstrate the gamut of benefits that this technique can provide from increased visual quality to the amazing performance. We decided that the most challenging and powerful demonstration of this technique would be to use it in and RTS style environment with thousands of characters. But, the Froblins Demo is not just a demonstration of real-time tessellation and displacement mapping. We also wanted to prove that this is a viable technique in a real game. We wanted to prove that you could use this technique while still rendering the characters and environment with techniques used in some of the latest high end FPS style games. We also wanted to show off the sheer power and flexibility of the 4870 by utilizing the GPU for general purpose tasks that are more traditionally handled on the CPU. So, before we launch the demo and I show you the real-time tessellation and displacement mapping in action I wanted to quickly highlight some of the other features in the demo.

50

Page 51: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The heart of the Froblins Demo is an interactive, dynamic AI simulation with 3000 agents. This simulation controls the global pathfinding as well as the local avoidance system for the froblins as the wander about their world trying to achieve their various goals.

51

Page 52: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The cool part about this is that this traditionally CPU based task is entirely being evaluated on the GPU while simultaneously rendering the entire scene.

52

Page 53: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The simulation is both dynamic and interactive. The froblinsbasically have four goals in there lives. The mine gold, then drop it off at the pagoda style buildings, they eat mushrooms when they get hungry, and go back their camps and sleep when they decide they are tired.

53

Page 54: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Users can interact with the simulation in a variety of ways using a very familiar RTS style interface. For example the users can summon scary ghost froblins or place poisons clouds to force the froblins to take alternate routes to their goals.

54

Page 55: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

As I mentioned earlier in the presentation we used Adaptive Tesselation on our terrain to improve performance by maximizing the level of polygonal detail in the areas that need it most and reducing the polygon density in areas where it would be wasteful. The terrain shader also utilizes two layers of displacement textures for large and small scale terrain features.

55

Page 56: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

The global illumination lighting system in the demo is implemented via spherical harmonic light maps and uses a cascaded shadow mapping technique to integrate the dynamic character shadows with the static pre-calculated shadows being cast by the terrain.

56

Page 57: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

And of course the demo utilizes real-time tessellation and displacement mapping and a dynamic LOD system on the 3000 froblin characters wandering throughout the detail rich world.

57

Page 58: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

So now… lets take a look at the demo.

58

Page 59: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

And this brings us to the conclusion.

59

Page 60: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

60

Page 61: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

61

Page 62: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

I would like to thank all of the members of the demo team who worked on this project. This project would not have been possible without all the hard work and dedication of each and every member of the team. I would also like to give a special thanks to Budi Purnomo who put so much work into developing the GPU MeshMapper tool.

62

Page 63: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

We would also like to thank all of our Art Content partners including Allegorithmic who authored our beautiful terrain textures using their procedural texture authoring tools as well as ChainGun Studios who provided early conceptual work on the project. I would specifically like to thank Exigent, whose amazing team put together some of the most gorgeous and technically proficient art assets, including the high resolution Froblin model, under an extremely challenging schedule. These guys really came through for us and I can’t say enough good things about them.

63

Page 64: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Here is a list of resources related to this presentation. If you are interested in using the GPU MeshMapper tool, want to install the Froblins Demo, or just want to learn more about the technologies and techniques used in this demo these links will direct you to all the resources we have made available to developers. This includes various other presentations and whitepapers that include a much deeper look into the technical side of these topics as well as the publicly available DX9 Tessellation Library. Additionally several detailed articles about the tessellation techniques used in this demo written by Natalya Tatarchuck have just been published in ShaderX 7.

64

Page 65: today’s lecture on Authoring for Real · 2013. 10. 25. · possible to save your art pipline via automation and scripting. Finally, we will take a look at this technique in action.

Questions?

65


Recommended