Meeting Demands For Camera and Sensor Interfaces in IoT and
Automotive Applications
Hezi Saar Synopsys, Inc.
Agenda • Implementation of MIPI interfaces in mobile
applications and beyond • Advantages of implementing MIPI interfaces
• MIPI CSI-2, MIPI DSI, MIPI D-PHY, MIPI I3C
• SoC design considerations • Summary
MIPI Display & Camera Interfaces Mobile Applications
• MIPI CSI-2 and MIPI DSI protocols enable application-optimized SoCs primarily used in mobile electronics
• Use of multiple cameras and displays for beyond traditional mobile applications
MIPI Camera & Display Interfaces in IoT
Linux/Rich OS
Ext Flash Memory
Controller LPDDR
eMMC ARC HS Processor
HMI
MIPI DSI
GPU
Communications
Bluetooth Smart
SDIO
USB
Vision
MIPI CSI-2 Vision Processor
On-Chip Memories
ROM SRAM
Sensor & Control Subsystem
Processor
ADC
MIPI I3C / I2C
SPI
Security
Secure Core
Private Key
Public Key Accelerator
True Random Number Generator
Crypto Accelerators
Logic Libraries
• MIPI DSI, MIPI CSI-2, MIPI I3C • LPDDR, eMMC, ADC, BT, USB • Embedded Vision • Security • Sensor Fusion
Point-of-Sale IC
Advanced Driver Assistance Systems (ADAS) A Rapidly Evolving Technology
• Passive Driver Assistance Systems • Back-up camera system
• Distance alert system
• Active Driver Assistance Systems • Back-up camera w/ ID & breaking
• Collision avoidance
Example: Surround View Using MIPI CSI-2 Image Sensors & MIPI DSI Display
Rear Camera
Display
CAN Interface
MPU Proprietary, LVDS or Ethernet Switch
DRAM Flash Memory
Power Supply Front Camera
Module
Left Camera Module
Right Camera Module
Rear Camera Module
Other Camera Module
Vbat
LVDS or Ethernet
Link
MIPI CSI-2 Image Sensors
MIPI DSI Display
Front Camera
Left Camera
Right Camera
Other Camera Module
Typical Automotive SoC High-EndADAS Infotainment
• MIPIDSI,MIPICSI2,MIPII3C,MIPISoundWire• LPDDR4,EthernetAVB,HDMI,PCIe,SATA,ADC
• EmbeddedVision• Security• SensorFusion• RequiresFuncEonalSafety
• MIPIDSI,MIPICSI2,MIPII3C,MIPISoundWire• USB,LPDDR4,EthernetAVB,HDMI,PCIe,SATA,
ADC• eMMC->UFS
• Real-EmeMulEmedia• Security• SensorFusion
I3C
I3C
I3CMIPID-PHY
MIPID-PHY
Microsoft HoloLens Processing Unit
• Processor packaged together with 1GB LPDDR3 and Intel Atom x86 processor
• Uses 5 camera interfaces, depth and motion sensor for image identification and processing, recognizing gestures
(*)Source:EETimesanalysis(*)Source:MicrosoR
MIPI CSI-2 Over MIPI D-PHY
CSI-2Device
D-PHY
CSI-2Host
D-PHY
Clk+
Clk-
L0+
L0-
Clk+
Clk-
L0+
L0-
L1+
L1-
L1+
L1-
FrameBuffer
CSI-2TransmiYer
PacketBuilder
LaneDistribuEon
CCISlaveSCL
SDA
SCL
SDA
CCIMaster
CSI-2Packet
CSI-2Packet
D-PHYHSBurst
D-PHYHSBurst
CSI-2Receiver
PacketDecoder
LaneMerger
CSI-2Packet
CSI-2Packet
FrameBuffer
CRCDT WC ECC
Payloadbytesize DataCRCprocessing
ECCprotecEngtheheader
DataFormatDefiniEon
VirtualChannelIdenEficaEon
PacketBuilder
Virtual Channels
Virtual Channel 0 – Line 0 Virtual Channel 0 – Line 1 Virtual Channel 0 – Line 2 Virtual Channel 0 – Line 3 Virtual Channel 0 – Line 4
Virtual Channel 0 – Line N
Virtual Channel 1 – Line 0 Virtual Channel 1 – Line 1 Virtual Channel 1 – Line 2 Virtual Channel 1 – Line 3 Virtual Channel 1 – Line 4
Virtual Channel 1 – Line M
DSISPacke
t
DSISPacke
t
CRCWC ECC
Payloadbytesize DataCRCprocessing
ECCprotecEngtheheader
DataFormatdefiniEon
VirtualChannelIdenEficaEon
PacketBuilder–LONGPacket
MIPI DSI over MIPI D-PHY Example: DSI Video Example - DPI
D-PHY
D
-PH
Y
Clk+
Clk-
L0+
L0-
Clk+
Clk-
L0+
L0-
L1+
L1-
L1+
L1-
DSITransmiYer
PacketBuilder
LaneDistribuEon
DSIReceiver
PacketDecoder
LaneMerger
VSS HSE
DATA0 ECC
ECCProtecEngtheShortPacket
DataFormatDefiniEon
VirtualChannelIdenEficaEon
PacketBuilder–SHORTPacket
DATA1
D-PHYHS
Burst
D-PHYHS
Burst
DSILPacket
ValidImageLine
DSISPacke
t
VSS
DSISPacke
t
HSED-PHYHS
Burst
D-PHYHS
BurstD-PHYHS
Burst
D-PHYHS
Burst
DSILPacket
ValidImageLine
MIPI D-PHY Architecture The Popular Physical Layer Used for MIPI CSI-2 & MIPI DSI Protocols
• Synchronous Forwarded DDR clock link architecture
• One clock and multiple data lanes configuration
• Static/dynamic de-skew supported through calibration
• Calibration hand-shake not supported
• No encoding overhead
TwoDataLaneConfigura0on
• Low-power and high-speed modes
• Primarily targeting camera and display
• Spread spectrum clocking supported for EMI/EMC considerations
• Large eco-system, proven in millions of phones, cars
Benefits and Evolution of MIPI D-PHY • Higher data rate enables
ultra-high-definition cameras and displays
• Easier adaption of newer technologies
• Backward compatible • Reliable with sufficient
margins • New specs augment
existing ecosystem • Growing market
applications and segments • Longer channel length
MIPI I3C Specification Overview Types of Devices in an I3C System
• MIPI I3C main master • MIPI I3C secondary master(s) • MIPI I3C slave(s) • I2C slave(s)
MIPI I3C Sensor Hub Use Case • The I3C bus has a
secondary master that acts as a hub
• Takes ownership of the I3C bus and communicates to the sensors directly
• As soon as the secondary master has the relevant sensor data available on its I3C bus, it can communicate to the main master, which propagates the data to the CPU
MIPI CSI-2 Use of MIPI I3C for Control & Always-ON Imaging
• Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency
• Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C
MIPI CSI-2 Use of MIPI I3C for Control & Always-ON Imaging
• Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency
• Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C
MIPI CSI-2, MIPI DSI & MIPI D-PHY Technical Considerations
• Bandwidth required
• D-PHY version/speed, number of ports and lanes
• Determine optimal interfaces to Tx/Rx image for memory storage or image
• Configuration & interrupts
• Memory usage
Target Multiple Applications with Flexible SoC Implementation
MIPICSI-2Host
Controller
MIPICSI-2Host
Controller
MIPICSI-2Host
Controller
MIPICSI-2Host
Controller
2LaneMIPID-PHY
2LaneMIPID-PHY
2LaneMIPID-PHY
2LaneMIPID-PHY
Applica7onA Applica7onB
OR
Summary • MIPI CSI-2, DSI, MIPI D-PHY and MIPI I3C
• Leveraged for mobile and beyond – IoT, automotive and AR/VR
• Synopsys provides complete camera, display and sensor interface IP
• Enables new set of applications in automotive, AR/VR, IoT markets • Lowers integration risk for application processors,
bridge ICs & multimedia co-processors
• Future proof IP supporting variety of speeds, proven in silicon • Reduces cost and power for multiple instantiations
• Testability features enable low cost manufacturing
Industry’sfirstMIPII3CDemo
VisittheSynopsysBoothtoSeeOurDemoToday!