Post on 29-Sep-2020
transcript
GTC 2016
WetBrush: GPU-based 3D Painting Simulation at the Bristle Level
Zhili Chen Byungmoon Kim
Adobe Research
GTC 2016 2
Oil painting with brush
Complex interactions
• Bristle-Bristle
• Bristle-Paint
• Paint-Paint
Motivation:
• Exhaustive simulation?
GTC 2016 3
Oil Painting Simulation
Brush Sim RendererUser Input
Brush head
movement
Two-way
Bristle-Paint Liquid
Interaction
Paint Liquid
GTC 2016 4
Brush Model
Previous works on brush model
2D Stamps 2D Surface wrapped
around skeleton
Individual bristles
Chu et al., 2002 Expresii DiVerdi et al., 2010 D-Brush in PhotoshopNatural brushes in Photoshop
• No variations from
brush dynamics
• No details from individual
bristles with large
deformation
• Brush shape not realistic without
inter-bristle interactions
GTC 2016 5
Brush Model
Our model
Bristle vertices
Bristle samples
50-100 bristles
~10 vertices per bristle
𝐩𝑖
𝐩𝑖+1
𝑙0
𝐩𝑖𝐩𝑖+1
𝐩𝑖−1
𝜃
Bending
Stretching
GTC 2016 6
Brush Model
Inter-bristle interactions
• Collision
• Friction
GTC 2016 7
Paint Liquid Model
Height field Volumetric grid Particles
Chu, et al. 2010 Microsoft FreshPaint Overhang
Multi-layer color mixing
?• Track volume and mass better
• Less blurring from grid-sampling
• Model paint carried by brush
GTC 2016 8
Hybrid Fluid Representation
Brush
Fluid
particles
Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity
GTC 2016 9
Hybrid Fluid Representation
Brush
Fluid
particles
Particles
Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity
Close to the brush
OR Fast Moving
Cover smaller region
GTC 2016 10
Hybrid Fluid Representation
Brush
Fluid
particles
Particles
Density
Grid
Adaptive Hybrid Fluid Representation based on
• Distance to brush
• Velocity
Close to the brush
OR Fast Moving
Cover smaller region
Further away from the brush
AND Slow moving
Cover larger region
GTC 2016 11
Fluid Conversions
Grid & Particles
Visualized
Only Particles
Visualized
FLIP• Particles sample from velocity grid
• Avoid expensive neighbor searching
Grid Fluid
Particle Fluid
• Sample new
particles
• Convert and delete
stationary particles
GTC 2016 12
Hybrid Fluid
Grid Only Grid+Particle
• Less volume loss
• Better surface tracking
• Sharper surface detail
• Sharper color mixing
GTC 2016 13
Brush-Paint Interaction
Brush Sim
Grid Fluid
Particle Fluid
• Brush emit new pigment particles
• Brush carries and pushes existing fluid particles
• Brush pick up color from paint on canvas
• Thick paint affect brush motion and shape
GTC 2016 14
Implementation
CUDA implementation
• Both brush and paint liquid in CUDA
• CUDA+OpenGL interop. for rendering
Rendering
• Ray casting on paint density grid and pigment grid
• Screen space particle rendering
GTC 2016 15
Implementation
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSorted
FindBrushBoundingBox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush
k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
Forward
Brush
Inter-Bristle
Collision
Bristle
Stretching
and Bending
Update Brush
Attributes for
Rendering
Update Brush
Paint Nodes
that holds
pigment
Voxelize
Brush
Create Brush
Distance Field
Find Brush
Bounding Box
Forward FLIP
Particles
Rasterize
Particles
onto Velocity
Grid
Diffusion and
Dissapation
Mix with
Brush
Velocity Field
Grid Fluid
Simulation
Brush Pickup Pigment
Sample New Particles
from Density Grid
Brush Vertices emit
new Paint Particles
Resample FLIP Particle
Velocities
Convert inactive
particles to density
grid
Save current
window back to full-
canvas texture
GTC 2016 16
Performance
GTX Titan X
Benchmark sequence: 46 fps, 92K Particles (~2M at maximum)
5%
30%
24%
24%
8%
9%
Grid-based liquid simulation(6.5ms)
Particle-based
liquid simulation (5.2ms)
Grid-particle transfer(5.2ms)
Brush simulation(1.1ms)
Rendering(2.0ms)
Bristle-particle transfer(1.6ms)
GTC 2016 17
Performance
Faster stroke requires higher framerate• Faster stroke => more stroke samples per unit time
• Maximum distance between stroke sample ~= 3-5 pixels
Stroke Sample
distance 2px
Stroke Sample
distance 30px
GTC 2016 18
Optimization
GTC 2016 19
Optimization
Adaptive simulation window (vs fixed simulation window)
Axis-aligned bounding
box for brush
Active simulation
window
thrust::minmax_element()
For grid fluid simulation46 fps-> 75 fps (63%)
GTC 2016 20
Optimization
Particle insertion
Num_Particles
Particle Array
AtomicInc(Num_Particles)
GTC 2016 21
Optimization
New Particle Array
• Pre-allocate maximum #
particle space for each cell• No more than 27 particles within one grid cell
• No more than one particle in one subcell
Marker Array
• 1 => a new particle is
added for the element
Cell 𝑖
Cell 𝑖 + 1
Optimized Particle insertion
Cell 𝑖 − 11
0
1
0
00
00
00
01
0
1
1
10
GTC 2016 22
Optimization
New Particle Array• Pre-allocate maximum #
particle space for each cell
Marker Array• 1 => a new particle is
added for the element
Cell 𝑖
Cell 𝑖 + 1
Optimized Particle insertion
Cell 𝑖 − 11
0
1
0
00
00
00
01
0
1
1
10
Compact New
Particle Array
thrust::remove_if()
Shrink List
Append
Particle Array
75 fps-> 92 fps (22%)
GTC 2016 23
Kernel Pipeline
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSorted
FindBrushBoundingBox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush
k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
GTC 2016 24
Kernel Pipeline
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSorted
FindBrushBoundingBox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush
k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
Low occupancy, GPU almost idle
GTC 2016 25
Dependencies
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSortedFindBrushBoundingB
ox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush k_FilterBrushVoxelization
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
k_BrushDistanceMap
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
GTC 2016 26
Dependencies
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSortedFindBrushBoundingB
ox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
• Brush # Vertices << Grid Dimension, Particle #
• Low occupancy
• First half of fluid simulation do not depend on brush
k_BrushDistanceMap
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
GTC 2016 27
Dependencies
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSortedFindBrushBoundingB
ox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
• Only for rendering
• Can take place anytime
in a frame
k_BrushPickupPigment
GTC 2016 28
Dependencies
k_ForwardPosition
Hash
SolveSelfCollision
k_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSortedFindBrushBoundingB
ox
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush k_FilterBrushVoxelization
k_BrushDistanceMap
k_Advection_Particles
k_RasterizeParticleTrilinear
k_NormalizeRasterization
k_MixBrushVelocity
Diffusion
PostProcessVelocity
k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PIC
k_FindConvertParticlesToPaintDensity
k_ExtendPigment
SaveActiveDomain
• More parallelization
GTC 2016 29
Optimization
k_ForwardPosition
Hash SolveSelfCollisionk_ProjectConstraints
k_UpdateFinalSim
k_UpdateRendering
k_UpdateSorted
k_InterpolatePaintNodes
k_UpdatePaintNodesRendering
k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap
k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity k_AdvectPaintDensityGrid
SpawnNewPaticles
k_BrushPickupPigment
k_BrushPaintNodesDistanceToPaintSurface
k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint
k_PICk_FindConvertParticle
sToPaintDensityk_ExtendPigment SaveActiveDomain
0
1
2
3
4
Sync
0->1,3,4Sync
0->3Sync
0->2
Sync
2->0
cudaEventRecord()
cudaStreamWaitEvent()
FindBrushBoundingBox
PostProcessVelocityDiffusion
Using CUDA Streams
92 fps-> 117 fps (27%)
GTC 2016 30
Optimization
46 fps -> 117 fps (154%)
Shared memory, Loop unrolling, Adjusting block size
100~200fps on GTX Titan X
30~50 fps on Surface Book
GTC 2016 31
Results
GTC 2016 32
Results
Better 3D shape
Finer surface details
More pigment variations along strokes
Thick “Impasto” Style
A lot of overhang
Thinner
painting
style
Thank you!