© Fraunhofer IIS/EAS
UVM AND VERIFICATION OF SYSTEMC DESIGNS
Thilo Vörtler, Stephan Schulz, Martin Barnasconi (NXP)
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
2
Outline
Introduction and Motivation
UVM-SystemC overview
Applications of UVM-SystemC at different abstraction levels
Summary and outlook
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
3
Introduction and Motivation
Heterogeneous systems are a combination of analog,
digital as well as additional physical domains (e.g.
mechanical)
The SystemC AMS language allows to model such
systems at different levels of abstraction Used for
virtual prototyping
Different kind of verification approaches:
Simulation based
Lab based measurements
Formal verification
Use of standardized verification methodologies like
Universal Verification Methodology, origins in digital
hardware design
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
4
Introduction and Motivation
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
5
Generic test environment
What input patterns to supply for the Design Under Verification (DUV)
What is expected for the output for a properly working design?
When is verification done?
Which parts of a test bench can be reused?
Golden reference model
Design under
verification / testStimulus generator Checker
test environment / test bench
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
6
UVM what is it?
Universal Verification Methodology to create modular, scalable,
configurable and reusable testbenches based on verification components
with standardized interfaces
Class library which provides a set of built-in features dedicated to verification,
e.g., phasing, component overriding (factory), configuration, comparing,
scoreboarding, reporting, etc.
Environment supporting migration from directed testing towards Coverage
Driven Verification (CDV) which consists of automated stimulus generation,
independent result checking and coverage collection
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
7
UVM what is it not…
Infrastructure offering tests or scenario’s out-of-the-box: all verification
behaviour has to be implemented by user
Coverage-based verification templates: application is responsible for coverage
and randomization definition; UVM only offers the hooks and technology
(classes)
Verification management of requirements, test items or scenario’s is outside
the scope of UVM
Test item execution and regression – automation via e.g. the command line
interface or “regression cockpit” is a shell around UVM
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
8
Main concepts of UVM (1)
Clear separation of test stimuli (sequences) and test bench
Sequences are treated as ‘transient objects’ and thus are independent from
the test bench construction and composition
In this way, sequences can be developed and reused independently
Introducing test bench abstraction levels
Communication between test bench components are based on transaction
level modeling (TLM)
Register abstraction layer (RAL) including a register model, adapters, and
predictors is used
Reusable verification components are based on standardized interfaces and
responsibilities
Universal Verification Components (UVCs) offer sequencer, driver and
monitor functionality with clearly defined (TLM) interfaces
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
9
Main concepts of UVM (2)
Non-intrusive test bench configuration and customization
Hierarchy independent configuration and resource database to store and
retrieve properties everywhere in the environment
Factory design pattern introduced to easily replace UVM components or
objects for specific tests
User-defined callbacks to extend or customize UVC functionality
Well defined execution and synchronization process
Simulation based on phasing concept: build, connect, run, extract, check
and report. UVM offers additional refined run-time phases
Objection and event mechanism to manage phase transitions
Independent result checking
Coverage collection, signal monitoring and independent result checking are
running autonomously in a scoreboard
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
10
A bit of history…
In the pre-UVM era (< 2010), various EDA vendors offered a verification methodology in SystemC / SystemVerilog
OVM (Cadence/Mentor), AVM (Mentor), VMM (Synopsys)
Unfortunately, consolidation towards UVM focused on a SystemVerilogstandardization and implementation only, with a focus on RTL verification
Non-standard methods and libraries exist to bridge the UVM and SystemC world
Cadence’s UVM Multi Language library: offers a ‘minimalistic’ UVM-SystemC
Mentor’s UVM-Connect: Mainly TLM communication and configuration
In 2011, a European consortium started building a UVM standard compliant version based on SystemC / C++
Initiators: NXP, Infineon, Fraunhofer IIS/EAS, Magillem, Continental, and UPMC
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
11
Outline
Introduction and Motivation
UVM-SystemC overview
Applications of UVM-SystemC at different abstraction levels
Summary and outlook
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
12
UVM-SystemC overview
SystemC(-AMS)
compliant simulator
SystemC(-AMS)
Language
UVM (-SC / -AMS)
Class library
Universal Verification
Methodology
Verification
management
Language and modeling technology
elements: Tool / simulator
Additional tool layer like “verification
cockpit” (e.g. vManager, vPlan)
UVM technology elements:
• Methodology = what
• Class library = how
UVM-SystemC scope
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
13
Why UVM in SystemC?
Elevate verification beyond block-level towards system-level
System verification and Software-driven verification are executed by teams
not familiar with SystemVerilog and its simulation environment
Trend: Tests coded in C or C++. System and SW engineers use an
(open source) tool-suite for embedded system design and SW dev.
Structured ESL verification environment
The verification environment to develop Virtual Platforms and Virtual
Prototypes is currently ad-hoc and not well architected
Beneficial if the first system-level verification environment is UVM compliant
and can be reused later by the IC verification team
Extendable, fully open source, and future proof
Based on Accellera’s Open Source SystemC simulator
As SystemC is C++, a rich set of C++ libraries can be integrated easily
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
14
Why UVM in SystemC?
Support analogue DUTs with SystemC AMS
Reuse tests and test benches across verification (simulation) and validation
(HW-prototyping) platforms
requires portable language like C++ to run tests on HW prototypes,
measurement equipment, …
Enables Hardware-in-the-Loop simulation and Rapid Control Prototyping
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
15
Advantages of UVM-SystemC
UVM-SystemC library features
UVM components based on SystemC modules
TLM communication API based on SystemC
Phases of elaboration and simulation aligned with SystemC
Packing / Unpacking using stream operators
Template classes to assign RES/RSP types
Standard C++ container classes for data storage and retrieval
Other C++ benefits (exception handling, constness, multiple inheritance,
etc.)
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
16
UVM layered architecture
Spec
Test cases
Scenario
Signal
Test casesTest
Fun
ctio
nal
co
vera
ge
Functional
Command Monitor
ScoreboardSequencer
Driver Monitor
Verification component
Verification environment (test bench)
Deviceunder test
Sequences
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
17
UVM layered architecture
The top-level (e.g. sc_main) contains the test(s), the
DUT and its interfaces
The test bench contains UVM components such as:
Agents/UVCs
Sequences
(Virtual) Sequencers
Scoreboard
Register Model
Adaptors …
top (sc_main)
Testbench (env)
…..agent
UVC1 (env)
MonDrv
Sqr
agent
MonDrv
Sqrconf conf
config
scoreboard
Subscr2
refmodel
Subscr1
Test configregister
sequence
virtualsequencer
Reg model
Adapter
rw
Interf1
UVC2 (env)
Interf2
DUT
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
18
UVM agent
Component responsible to drive and monitor the
DUT for one specific interface e.g. SPI, I²C…
Contains three subcomponents
Sequencer
Driver
Monitor
Contains analysis functionality for basic coverage
and checking
Possible configurations
Active agent: sequencer and driver are enabled
Passive agent: only monitors signals
(sequencer and driver are disabled)
agent
driver monitor
sequencerconfig
seq_item_port
seq_item_export
item_collected_port
trans
seq
vifvif
analysis
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
19
UVM verification component (UVC)
A reusable verification component (UVC)
is a (sub-) environment which packages
one or more agents
The verification component or agents may
set or get configuration parameters
Contains sequences, which contains the
actual transaction data, is processed by
the driver via a sequencer
Each verification component is connected
to the DUT using dedicated interfaces
agent
driver monitor
sequencer
seq_item_port
seq_item_export
item_collected_port
UVM verification component (env)
config
trans
seq
vifvif
config
analysis
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
20
UVM sequences
Sequences are part of the test scenario and define
streams of transactions
The properties (or attributes) of a transaction are
captured in a sequence item
Sequences are not part of the testbench hierarchy,
but are mapped onto one or more sequencers
Sequences can be layered, hierarchical or virtual,
and may contain multiple sequences or sequence
items
transaction
transaction
transaction
sequence
seq
seq1
seq2
trans
trans
seq1
trans
trans
seq2
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
21
UVM virtual sequence
A virtual sequence encapsulates one or more
sequences, which are executed on the sub-
sequencers in each UVC agent, which are all
connected to the parent virtual sequencer
A virtual sequence can be configured as default
sequence in a test, to facilitate automatic
execution on a virtual sequencer or a sequencer
which belongs to a UVC agent
Testbench (env)
…..agent
UVC1 (env)
MonDrv
Sqr
agent
UVC2 (env)
MonDrv
Sqrconf conf
config
virtualsequencer
scoreboard
Subscr2
refmodel
Subscr1
Test configdefault
sequence
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
22
UVM scoreboard
The scoreboard performs
end-to-end checking by
comparing expected and processed transactions
These transactions are retrieved by dedicated
subscribers or listeners, which implement the
write method of the analysis ports of each
monitor, to which these subscribers are
connected
A scoreboard may contain a predictor, which
acts as reference or golden model. Alternatively,
the scoreboard may contain an algorithm to
calculate the expected transaction
Testbench (env) config
Test configdefault
sequence
…..agent
UVC1 (env)
MonDrv
Sqr
agent
UVC2 (env)
MonDrv
Sqrconf conf
virtualsequencer
scoreboard
Subscr2
refmodel
Subscr1
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
23
Top, Tests and Testbench
The top-level (e.g. sc_main) contains the
test(s) and the DUT
The interface to which the DUT is
connected is stored in the configuration
database, so it can be used by the UVCs to
connect to the DUT
The test to be executed is either defined by
the test class instantiation or by the
argument of the member function run_test
DUT
AMS DIG SW
Testbench (env)
…..agent
UVC1 (env)
MonDrv
Sqr
agent
UVC2 (env)
MonDrv
Sqrconf conf
config
virtualsequencer
scoreboard
Subscr2
refmodel
Subscr1
Test
top (sc_main)
configdefault
sequence
in
out
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
24
UVM-SystemC AMS
The UVM-SystemC infrastructure can also handle AMS verification
UVM AMS extensions will not break the existing UVM
Transactions will program analog drivers and monitors
Time annotation to transaction
Drivers generate analog signals, Monitors analyze and extract properties like
amplitude, spectrum, … and transfer them back via transactions
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
25
UVC with AMS driver and monitor using SystemC AMS
Regular UVM-SystemC drivers and
monitors are used by instantiating
SystemC-AMS Timed Data Flow (TDF)
modules in them
The parent driver and monitor establish
the connection from the TDF ports to the
interface via the configuration
mechanism
agent
driver monitor
sequencer
seq_item_port
seq_item_export
item_collected_port
UVM verification component (env)
config
trans
seq
vifvif
config
analysis
AMSmonitor
AMSdriver
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
26
Outline
Introduction and Motivation
UVM-SystemC overview
Applications of UVM-SystemC at different abstraction levels
Summary and outlook
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
27
UVM Design Flow
UVM tests and sequences are derived
from the specification and shall be
used for verification and validation at
different abstraction levels
UVM based test bench should be
automatically generated using a
generator e.g. from IP-XACT
specification of the DUT and UVCs
Specification
Requirements
High Level Test
Sequences
Simulation based
Verification
Lab based
Validation
Evaluation
Coverage criterai
fulfilled?
UVM based testbench
architecture
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
28
Re-use across abstraction levels (1)
Design of a complex system within a SystemC
environment
One-time verification setup with UVM-SystemC
Behavioral model for concept phase, reuse
across SystemC abstraction levels
Detailed model for further implementation
require additional tests
DUT
Testbench (env)
agent
UVC1 (env)
DriverSystemC
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Simulation - SystemC
configdefault
sequence
SystemC - Behavioral
vif
agent
UVC2 (env)
MonitorSystemC
vif
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
29
Re-use across abstraction levels (2)
Continued use of previous verification setup by
running the verification environment using mixed
language simulation / simulator coupling
Exchange of UVM driver/monitor verification
components for suitable HDL abstraction level
Additional tests specific to new model details
DUT
Testbench (env)
agent
UVC1 (env)
DriverHDL
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Mixed Language Simulation
configdefault
sequence
Verilog/VHDL
vif
agent
UVC2 (env)
MonitorHDL
vif
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
30
UVM-Testbench
ds1006.x86
Test of Component
Specification (e.g. for IC’s)
Test of Hardware Components in a not
yet existing environment
Source: dSPACE
Direct SystemC AMS HiL-Simulation
UVM
Env.UVM
Library
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
31
Re-use across abstraction levels (3)
Continued use of previous verification setup by
running the verification environment as a real-time
model on a HIL platform / lab-test equipment
Exchange of UVM driver/monitor verification
components to interact with hardware
Re-use of all tests possible
Source: ZedBoard.org
DUT
Testbench (env)
agent
UVC1 (env)
DriverLab equip
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Real Time Hardware
configdefault
sequence
FPGA/ASIC – 1st Silicon
vif
agent
UVC2 (env)
MonitorLab equip
vif
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
32
Re-use across abstraction levels (4)
download
monitor
integrate
DUT
Testbench (env)
agent
UVC1 (env)
DriverSystemC
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Simulation - SystemC
configdefault
sequence
SystemC - Behavioral
vif
agent
UVC2 (env)
MonitorSystemC
vif
Source: ZedBoard.org
DUT
Testbench (env)
agent
UVC1 (env)
DriverEmulation
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Real Time Hardware
configdefault
sequence
FPGA - Emulation
vif
agent
UVC2 (env)
MonitorEmulation
vif
Source: ZedBoard.org
DUT
Testbench (env)
agent
UVC1 (env)
DriverLab equip
config
virtualsequence
r
scoreboardSubscr
2ref
modelSubscr
1
Test
Real Time Hardware
configdefault
sequence
ASIC – 1st Silicon
vif
agent
UVC2 (env)
MonitorLab equip
vif
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
33
Outline
Introduction and Motivation
UVM-SystemC overview
Applications of UVM-SystemC at different abstraction levels
Summary and outlook
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
34
Summary and outlook
Universal Verification Methodology created in SystemC/C++
Fully compliant with UVM standard
UVM-SystemC language definition and proof-of-concept implementation
are available by Accellera Systems Initiative (currently as beta release)
Allows reuse at different implementation levels
Ongoing developments
Extend UVM-SystemC with constrained randomization capabilities using
SystemC Verification Library (SCV) or CRAVE
Extension of functional coverage features
Add register abstraction layer and callback mechanism
Development of UVM based AMS and system level verification methods
© Fraunhofer IIS/EAS | Thilo Vörtler | 26.11.2015
35