Watch Dogs 2 - PC Adaptation Success Story with...

Jaakko Haapasalo, Producer, GeForce Titles (NVIDIA)

Marius Tudorache, Eastern European PC Director (Ubisoft)

Farid Rzaev, Developer Technology Engineer (NVIDIA)

Oleh Kuznetsov, Senior Programmer (Ubisoft)

Watch Dogs 2 - PC Adaptation Success Story with NVIDIA

2developer.nvidia.com

The Watch Dogs 2 PC port is great

[...] it runs well, has a ton of graphical options, and comes with a complete set of quality-of-life adjustments for mouse and keyboard players.

[T]here is a wide range of options that will allow you to cater the experience to your style, which is just what a PC port should offer.

The PC version is superb,

and the game is one of the

bright spots in this fall’s slate

of games.

November 2016…

gameworks.nvidia.com


14 out of 14 PC-specific mentions in critic reviews on Metacritic are positive

“Very Positive” - 82% on Steam



Success

PC Platform



Agenda

Game Ready Quality Program

Partnering with Watch Dogs 2 PC – Through The Numbers

Ubisoft PC Quality, and working with NVIDIA

Technical Case Study: Eliminating Stutter

Learnings



Game Ready Quality Program

Jaakko Haapasalo (NVIDIA)



Program Overview

2. Stability, Performance & Stutter Analysis 3. Minimum &

Recommended Specs

1. PC Technical Requirements

Checklist



(see also)

Game Ready Drivers

GTM

Optimal Performance Settings



Title Engagement

T – 12mo T – 3moT – 6mo T – 2mo T – 1mo Launch Live

Testing & Optimization

Review

GTM



1. TRC (Technical Requirements Checklist)

PC platform requirements to validate your game against

Focused on technical quality and readiness, such as resolution support, UI scaling, frame rate, smoothness, settings, and correctness of various effects.

Review involves several play-throughs at different settings, significant investment in QA time

3 review milestones

16 titles tested

in 2016 pilot

40 requirements



Requirement Categories

•Required (P0)

•Core

•Recommended (P1)

•Quality of Life

•Advised (P2)

•Forward-looking



Requirements and Rating

•Description

•Additional information (context, intent and failure modes)

•Criteria for Full, Partial and Failed compliance

•Overall TRC rating as a weighted average (0..100)



Timeline


Title Engagement

Review

GTM

TRCPreview

TRCUpdate

TRCFinal

Results



2. Stability, Performance and Stutter

Get Build

Test

Analyze

Improve



Timeline


Title Engagement

Review

GTM

TRCPreview

TRCUpdate

TRCFinal

Results

Performance & Stutter

Analysis



3. Minimum & Recommended Specs

High: IQ > AnyOther Platform

MinRec Search

GPU x CPU x SYSMEMBenchmark Search

(Playthrough!)

60FPS

Target

1K, 2K,

4K

Low / High

Presets

“Challenging”

BenchmarkRecSpec

MinSpec



Timeline


Title Engagement

Review

GTM

TRCPreview

TRCUpdate

TRCFinal

Results

Performance & Stutter

Analysis

Min/Rec SpecResults

Min/Rec SpecPrelims

Min/Rec Check

Min/RecCheck



WD2 Game Ready: The Numbers

Jaakko Haapasalo (NVIDIA)



PC Testing

TWIMTBP Labs:

•7 months

•26 builds

•10 Stutter & performance tests

•5 Min & Rec Spec tests

•3 TRC reviews



TRC Progression

0

10

20

30

40

50

60

70

80

90

100

1-Jan 3-Jan

TRC Rating



Min & Recommended Specs: 1K & 2K

40

45

50

55

60

65

1-Jan 2-Jan 3-Jan 4-Jan 5-Jan

Series1

Series2

Series3



Partnering with NVIDIA

Marius Tudorache (Ubisoft)



New NVIDIA-Ubisoft collaboration process

•Transparent communication

•Smoke-tested builds

•Delivery of builds and performance reports

•TRC simulations

•Post-Mortem and learnings across projects

•Dedicated JIRA for each project - tracking bugs outside of e-mails

! Think co-dev instead of tech support !



PC Testing – UBI PC Requirements

•13 months (1st check - October 2015)

•320+ builds (WD2-GAME-PC2->PC15x + RAW)

•~28 performance/TRC tests

•~28 Min & Rec Spec tests

•7 TRC official reviews (Alpha/Beta/Master/D1P)



Ubisoft - Winning PC gamers

After 20 years of PC releases:

•Knowledge & Information Sharing Management for PC

•Internal PC Summits

•Improving PC Technical Requirements

•Developing PC Communities - internally, externally, E-Sports

•Improving communication with PC gamers



Current UBI TRC Structure (WD2 proto.)

101 Tech. requirements

•Short title – for easy referencing

•Requirement description

•Terminology

•Remarks

•Intent

•Exemption

•TRC Rating per requirement –different for each milestone

•Overall Rating – work in progress



Using target TRCs as tech design

•Clarity

•Transparency

•Conformity

Major Challenge: Public Tech Requirements (open-sourced to

communities)

•Predictability

•Continuity



WD2 communicated Min & Reco

We wanted to express:

•Performance-based specs split per GPU series

•Coverage (old to new gen)

•Clarity for casual gamers





Detecting and Eliminating Stutter

Farid Rzaev (NVIDIA) and Oleh Kuznetsov (Ubisoft)


31www.gameworks.nvidia.com

Agenda

- Stutter overview

- Stutter detection

- Stutter analysis

- Fixing stutter

- Results



Stutter? Hitches?Everybody knows what it is….



Let’s DefineWhat stutter/hitches are?

- Inconsistency in performance between frames

0

10

20

30

40

50

60

70

1

11

21

31

41

51

61

71

81

91

101

111

121

131

141

151

161

171

181

191

201

211

221

231

241

251

261

271

281

291

301

311

321

331

341

351

361

371

381

391

401

411

421

431

441

451

461

471

481

491

FrameTimes



Player’s Point of View

Occasional freezes

Uncomfortable feeling under recommended specs



Stutter is a Huge Thing

Focus on our experience with Watch Dogs 2

Catching up issues

- Tools

Fixing issues

- Methods / Approaches



Finding Stutter Cases

Watch Dogs 2 is an open world action-adventure game

- Each stutter case should be covered by a benchmark



In-game Benchmark

We had an internal in-game benchmark, but it didn’t cover all of the cases.



Two Benchmarks

In-game Fast Car



0

20

40

60

80

100

120

140

160

180

200

Frames

Frametime graph (ms)



Tools

- GPUView

- Concurrency visualizer (CV)



Stutter issues

1. GPU Memory consumption GPU View / Afterburner

2. CPU Workload Inconsistency GPU View / CV

3. Bursts of creates / destroys GPU View

4. Page faults CV / GPU View



GPU Memory consumption

- Running out of the video memory

- Didn’t account some video memory for the driver (~10%)

Tools:

- GPUView eviction events

- GPUView reference charts



GPU Memory Usage (MB)

1500

1700

1900

2100

2300

2500

2700

Frames



UBI – Memory Consumption

Approaches

•Check VRAM usage (lighting, models, textures, post effects, …)

•Give artists the possibility to tune MIP maps skipping on a per texture basis

•Look for the allocated but not used resources.

•WD2: For the low settings we skipped Global Illumination probes for Far Away objects

Challenges

•Visual quality on the low settings

•Textures in RAM if not enough VRAM



CPU Utilization Inconsistency

100 ms



WD2: We use all CPU threads, but leave some idle time for a driver and other processes

Frame structure:

Main Engine

Worker

Worker

Main Render

UBI – CPU Utilization Inconsistency

Driver Render Render Render Present

Engine Render Windows Idle Idle

Engine Engine Engine Idle Idle

Render Driver Engine Idle Idle



•Multithread rendering (Using deferred context)

•Use Present time to do engine jobs

Main Engine

Worker

Worker

Main Render

UBI – CPU Utilization Inconsistency

Present Render DC:Execute DC:Execute

Engine Driver Render Driver

Engine Engine Engine DC:FillList

Engine DC:FillList Render Windows



Bursts of creates / destroysToo many of creates / destroys of resources on the render thread



Creating resources on the main render thread:

Main Engine

Worker

Worker

Main Render

UBI – Resource Management

Resource Resource Resource Render Present

Engine Render Engine Idle Idle

Engine Engine Engine Idle Idle

Render Engine Idle Idle Idle



Creating resources on the worker threads:

Main Engine

Worker

Worker

Main Render


Render Render Render Render Present

Engine Resource Resource Idle Idle

Engine Engine Engine Engine Engine

Engine Engine Resource Render Idle



Resource jobs between the frames:

FrameFrame


Render Render Render Render Present Render Render

Engine Resource Engine Idle Engine Engine Idle

Engine Engine Engine Idle Engine Engine Engine

Resource Engine Resource Engine Engine




Approaches

•Avoid resource creation on the main render thread



Bursts of creates / destroysToo many of creates / destroys of resources on multiple threads



UBI – Bursts of creates / destroys

Approaches

•Limit resource creation per frame

•The lower minimum limits you set – the less spikes you have

•Use texture pool to decrease Release calls

•Use typeless formats in texture pool

Challenges

•Resource Release can take significant amount of time but is hard to be measured



Page Faults – HDD specific

- HDD specific Page Faults that lead to stutter

- Windows cache’s HDD reads

- Page Faults were only seen on the FIRST run of the game



Page Faults - CV



UBI – Using the System File Cache

Approaches

•Any free memory on Windows PC may be used as the file cache

•Fill windows file cache by game data files before they are requested

•Use logos/intro videos and idle menu time to fill the cache

•Cache the most important files first

Challenges

•Not enough time to cache the entire game

•Not enough free memory to cache the entire game



Result

• It took us 4-5 months: Start 08/2016 – End 12/2016

•Weekly meetings

•Weekly builds with updates and summaries

•During the week: local skype chats and DAILY calls

•Super fast responses on both sides:

1. NVIDIA: catching and reporting issues, suggestions

2. UBI: fixing and sending new builds in a short time



ResultFeel comfortable under recommended specs

0

20

40

60

80

100

120

140

160

180

200

Series1 Series2



GPU Memory eviction



GPU Memory Reference Charts



CPU Workload Inconsistency

100 ms



References

•Stuttering in Game Graphics: Detection and Solutions - Cem Cebenoyan

•Analyzing Stutter – Mining More from Percentiles - Iain Cantlay

•SLI and Stutter Avoidance - Iain Cantlay, Lars Nordskog



Learnings



Take Home

•Apply technical validation criteria before designing

•Adapt code as well as design for best results

•Playtest your game with casual AND enthusiast hardware

•UBI PC Requirements and NVIDIA TRCs are aligned



Take Home

•NVIDIA Technical Requirements Checklist is a great way to get started focusing on the PC as a Platform

•Low performance and Stutter still pose major risks for games today; isolate them early, and iterate aggressively (come talk to us!)

•Partnering with the Game Ready Quality Program will help us support your vision for superior PC Quality



Thank you!

Email [email protected] for a sneak peek

Look out for more information on http://developer.nvidia.com soon!


mailto:[email protected]

http://developer.nvidia.com/

Date post:	28-Jun-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Watch Dogs 2 - PC Adaptation Success Story with...

Documents