Physically based LIDAR

OptiX, a real time sensor (LIDAR) simulation powered by GPUs

Jakub Kolesik – October 2017

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 2

VIRES At A Glance

Founded 10/1996

Address Headquarter: Branch office:Grassinger Straße 8 Wiener Straße 1283043 Bad Aibling 2700 Wiener NeustadtGermany Austriap +49.8061.939093-0f +49.8061.939093-13e info@vires.com

Markets

www.vires.comwww.opencrg.org

www.opendrive.orgwww.openscenario.org

Certification ISO 9001:2015

Focus on key product VIRES Virtual Test Drive®

Software tool-chain supporting the development, test and validation of automated driving, driver assistance and active safety systems

Components• traffic simulation• scenario simulation• image generation• sensor simulation / stimulation• database design tools• SDKs• etc.

Partner in Standardization Initiatives

OpenSCENARIOVTD is Linux-only

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 3

● LIDAR Description and Usecases● Basic LIDAR Implementation● PBR – Physically Based Rendering● PBR vs Basic

● Back Scattering with Roughness● Metallic Combined Surfaces (rust)● Specular Reflectance f0● Physically Based Wetness

Approximation● Demo● Summary and Future Work● Q & A

Agenda

LIDAR Description and Use Cases

.

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 4

● Transmits optical laser light pulses

● Determines distance to objects

● 895-905nm lidar wavelength ● 380-750nm visible spectrum

Can we use algorithms derived from visual spectrum?

How accurate would such an approach be?

LIDAR Description and Usecases

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 5

● BK7 Glass 700–900nm = IOR 1.5131–1.5090 = (f0) 0.0416–0.0412 ~ 0.0004● Iron 700–900nm = IOR 2.8653–2.9565 = (f0) 0.2328–0.2445 ~ 0.0117● PVA 700–900nm = IOR 1.4736–1.4682 = (f0) 0.0366–0.0359 ~ 0.0007

LIDAR Description and Usecases

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 6

● IOR relatively stable from 300-1300nm for most materials

● Difference even less for 700 to 900nm

● Appealing for visual approaches of CG!

● PBR as the holy grail for visualisation

Basic LIDAR Implementation

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 7

● Poor detail of materials

● 1 reflectivity value for whole material or even objects

● Composed or dirty materials missing

● Surface normals taken from geometry

● No micro-surface considered

Basic LIDAR Implementation

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 8

Using synthetic noise function helps to hide missing details

PBR – Physically Based Rendering

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 9

● Based on multiple material properties

● BRDF - Bidirectional Reflectance Distribution Function

● Disney BRDF and BRDF viewer

PBR – BRDF Material Properties

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 10

● Disney material chart

● Metallic- should be 0 or 1- interpolation possible

● Specular- reflectivity at direct viewing angle

● Roughness- micro surface- rough vs smooth

PBR – Texturing

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 11

● Quixel Materials - real world scanned materials

● Disney BRDF calibrated

● Quixel Studio for advanced texturing

● Quixel Megascans – huge library

PBR – Texturing

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 12

Roughness

PBR – Physically Based Rendering

Back scattering with roughness

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 13

PBR – Physically Based Rendering

Fresnel with roughness

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 14

PBR – Texturing

Metallic and combined surfaces (rust)

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 15

PBR – Texturing

Specular reflectance f0

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 16

PBR – Texturing

Surface Normals

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 17

PBR vs Basic

refer to protection notice DIN ISO 16016

Slide 18

PBRBasic

PBR vs Basic

October 2017, Rev. A Slide 19

Basic PBR

refer to protection notice DIN ISO 16016

PBR vs Basic

October 2017, Rev. A Slide 20

Basic PBR

refer to protection notice DIN ISO 16016

PBR – Physically based wetness approximation

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 21

PBR – Physically based wetness approximation

October 2017, Rev. A Slide 22

Basic 100% wetness PBR 100% wetness

refer to protection notice DIN ISO 16016

PBR – Physically based wetness approximation

October 2017, Rev. A Slide 23

PBR wetness 0% PBR wetness 100%

refer to protection notice DIN ISO 16016

VTD Demo

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 24

May 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 25

External Tools:● Adams – MSC Software● NVIDIA PX2, DriveWorks, OptiX

Open Standards:● OpenSCENARIO● OpenDRIVE● OpenCRG

Sample Use Cases: General

Future work

October 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 26

● Full spectral LIDAR with F0 for all wavelengths

● Nvidia MDL support

● Daylight and other lights affecting sensor data

● Realistic cloth and vegetation

● Headless operation in the cloud

● Nvidia PX2

May 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 27

Thank you for your attention!

May 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 28

LinksNVIDIA http://www.nvidia.com/NVIDIA MDL http://www.nvidia.com/object/material-definition-language.htmlNVIDIA OptiX https://developer.nvidia.com/optixNVIDIA Cuda https://developer.nvidia.com/cuda-zoneNVIDIA PX2 http://www.nvidia.com/object/drive-px.htmlNVIDIA DriveWorks https://developer.nvidia.com/driveworks

Index of refraction library https://refractiveindex.info

Disney Brdf Explorer (Viewer) https://www.disneyanimation.com/technology/brdf.htmlDisney PBR https://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf

Quixel https://quixel.se/Quixel Megascans https://megascans.se/

VIRES https://vires.com/VIRES Virtual Test Drive (VTD) https://vires.com/vtd-vires-virtual-test-drive/OpenDRIVE http://opendrive.org/OpenCRG http://www.opencrg.org/OpenSCENARIO http://www.openscenario.org/MSC Adams http://www.mscsoftware.com/de/product/adams

Trademark InformationNVIDIA, the NVIDIA logo are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and/or other countries.

Links And Trademark Information

May 2017, Rev. A refer to protection notice DIN ISO 16016 Slide 29

Picture Sources

Pictures under Creative Commons 2.0 ( https://creativecommons.org/licenses/by-sa/2.0/ ):[Picture_IOR_Silver] https://refractiveindex.info/?shelf=main&book=Ag&page=Johnson Picture has been cropped and resized.[Picture_IOR_PVA] https://refractiveindex.info/?shelf=organic&book=polyvinyl_alcohol&page=Schnepf Picture has been cropped and resized.[Picture_IOR_Iron] https://refractiveindex.info/?shelf=main&book=Fe&page=Johnson Picture has been cropped and resized.[Picture_IOR_BK7] https://refractiveindex.info/?shelf=glass&book=BK7&page=SCHOTT Picture has been cropped and resized.

[Picture_nvidia_mdl] http://www.nvidia.com/object/material-definition-language.html Picture has been cropped and resized.

[Picture_quixel_tree] https://megascans.se/ Picture has been cropped and resized.[Picture_quixel_stone] https://megascans.se/ Picture has been cropped and resized.

[Picture_vires_demo] https://vires.com/wp-content/uploads/2017/03/showcase06-1024x576.png Picture has been resized.

[Picture_brdf] http://www.codinglabs.net/article_physically_based_rendering_cook_torrance.aspx Picture has been cropped and resized.[Picture_disney_material_chart] https://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf Picture has been cropped and resized.

Pictures under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported ( https://creativecommons.org/licenses/by-nc-sa/3.0/ ):

Pictures under Creative Commons Attribution 4.0 International ( https://creativecommons.org/licenses/by/4.0/ ):

Sources