SpaceWire and SpaceFibre Interconnect for
High Performance DSPs
Steve Parkes, Martin Dunstan University of Dundee Bruce Yu, Andy Whyte, Chris McCLements, Albert Ferrer Florit,
Alberto Gonzalez Villafranca STAR-Dundee
Contents Next Generation SpaceWire: SpaceFibre SpaceFibre Reference Architecture High Processing Power DSP RC64 FFT Processor Demonstration
2
Next Generation SpaceWire: SpaceFibre
3
Driving Applications for SpaceFibre High data-rates for SAR and high-resolution multi-
spectral imagers require: – 10s Gbits/s data rates
High performance mass memory units require: – Multi-Gbit/s network interconnecting memory modules
Integrated control and payload data handling requires: – Deterministic data delivery for AOCS/GNC – Concurrent with asynchronous payload data delivery – Simple configuration – Galvanic isolation
Space transportation and human space flight requires: – Long distance (100m) – Deterministic data delivery – safety critical – Carry video traffic without interfering with deterministic traffic
4
Other Requirements for SpaceFibre Backwards compatible with SpaceWire Address issues inherent in SpaceWire
– Packet blocking in networks – Limited common mode voltage tolerance – High cable mass – Limited maximum length – No quality of service (QoS) – No deterministic data delivery
All these issues are resolved in SpaceFibre
5
SpaceFibre SpaceFibre is
– A spacecraft on-board data link and network
SpaceFibre runs over – Electrical and fibre optic cables
SpaceFibre designed specifically for spaceflight – Integrated Quality of Service (QoS) – Integrated Fault Detection, Isolation and Recovery (FDIR)
capabilities – Simple configuration
A substantial improvement on SpaceWire – Performance x10 to x100 (multi-lane) – Power per bit x0.2 – Lower mass x0.75 electrical, x0.5 fibre per cable – Robustness: galvanic isolation, FDIR – Capabilities: virtual links, virtual networks, time distribution,
event signalling, deterministic data delivery
6
SpaceFibre Key Features High performance
– 2.5 Gbits/s current flight qualified technology – 3.125 Gbits/s soon (6.25 Gbits/s coming) – Multi-laning of up to 16 lanes (40 Gbits/s)
Integrated QoS – Priority – Bandwidth reservation – Scheduling
Integrated FDIR support – Transparent recovery from transient errors – Error containment in virtual channels and frames – “Babbling Node” protection
Low latency – Broadcast messages
Compatible with SpaceWire at packet level
7
Integrated Network Single integrated network
– Carrying Instrument data Configuration and control information Deterministic traffic High resolution time information Event signals
– Improves reliability, mass, cost, reuse – Backwards compatible with existing SpaceWire equipment
Ideal for interconnecting DSP units
8
Radiation Tolerant SpaceFibre ASIC
9
Designed with EC FP7 funding
SpaceFibre on Microsemi RTG4
10
RTG4: New powerful FPGA from Microsemi Integrated SerDes running at 3.125 Gbits/s Perfect for SpaceFibre SpaceFibre interface 3% to 6% of RTG4 (2 to 8 VCs) SpaceWire interface 1%, RMAP Target 2% of RTG4
SpaceFibre Development Status Open technology Result of work from many companies
– In different member states – With funding from
National agencies, ESA, and EC (FP7)
ECSS standardisation ongoing – Out for public review 3Q2016 – Working group with representatives from industry across
Europe
TRL 4/5 approaching 6 on radiation hard FPGAs IP cores available Ready for implementation in European chip
technologies
11
HPPDSP
High Processing Power Digital Signal Processor
12
Background High data rate payload, requiring DSP with
– High processing power – High throughput
DSP processor – TI SMV320C6727B-SP, 250-MHz, Floating-Point
Total Ionizing Dose tolerance (TID 100Krad) Single Event Latch-up immune (SEL 117 MeV cm2/mg)
– 2000 MIPS/1500 MFLOPS at 250MHz
High throughput using SpW and SpFi interfaces
13
HPPDSP Unit HPPDSP Unit
System Architecture Design
14
DSP SDRAM
Control FPGA
MMU/EDAC
SpW SpFi SpFi M/S
DSP SDRAM
Control FPGA
MMU/EDAC
SpW SpFi SpFi M/S
High Processing Power DSP
15
DSP
IO FPGA
Memory
SpaceFibre
SpaceWire
GPIO
16
Block Diagram of the FPGA Design
DSP Program/DataSDRAM
EDAC ParitySDRAM
UHPI I/F
IODMA
RADMA
SpFi 2 RMAPTARGET
req
SpFi M/S RMAPINITIATOR
req
ADC/DACFIFOs
req
SpWRouter
req
RMAPTARGET
5
1
2 3
4
Boot MgmtFLASH EDAC
req
SCRUBBER FIFO
SEUSIM
EDAC
MMU
ASYNCI/F
INTR
DSPRESET
CHECKER
Memory Mapped
Registers &DSP Task Control
Watchdog
Time Mgmt
LCD Display
Power Control
V & T Monitor
GPIO
External Intr
I2CIO Exp
SharedRAM
FLASHBoot PROM
FLASHFPGA PROM
intr
intr
intr
intr
intr
JTAG
req
Immediate ScrubLocation + Data
ControlFPGA
RMAPTARGET
IODMABus
ConfigurationBus
SlaveAccess Bus
DSPPeripheral
Bus
intr
Slave Access
intr
EMIF bus
UHPI Reset
FLASHProgramme
intr
JTAG
EMIF SDRAM Write Monitor
User EEPROM
ADC/DAC, SpFi, IO/RA DMA,
Boot Mgmt, …
EMIF SDRAM IFDecoder
Data masks
IO DMA
DMA data between DSP memory and: – SpaceFibre – SpaceWire – ADC/DAC
Different operations for Normal, Master, and Slave mode
When a IO DMA is completed, an interrupt is generated
IO DMA and RA DMA compete for access to UHPI
Boot management also goes through IO DMA
On master board all data transfers are copied to SpFi M/S
17
DSP
UHPI I/F
IO DMA
RA DMA
req
RMAP TARGET
SpFi M/S
req
ADC/DAC FIFOs
req
req
SpW Router
5 1
2 3
4
SpFi 2
req
RMAP TARGET
IO DMA Bus
UHPI
intr
RMAP TARGET
RMAP INITIATOR
SpaceWire and SpaceFibre IO SpaceWire
– Internal SpaceWire Router 2 ports to external LVDS drivers/receivers 2 ports to IO DMA bus 1 port to RMAP Target
SpaceFibre – Two interfaces each at 2.5 Gbits/s – Two different SerDes (Xilinx MGT and TI TLK2711) – Master/Slave
For connecting to second HPPDSP 4 virtual channels
– SpaceFibre 2 For high speed IO 2 virtual channels
18
HPPDSP Board
19
DSP
SpFi M/S
SpFi 1
SpFi 2
SpW 1N
SpW 2N
SpW 1R
SpW 2R
GPIO/ LEDs Switches
OLED Display
SDRAM Parity
FMC ADC/DAC
Boot PROM
FPGA PROM
JTAG
SDRAM Memory EMIF
FPGA
UHPI
Power/Temp Monitor
I2C
PCB Layout – Bottom Side
20
FPGA
DSP
Toggle Switch Connectors
SDRAMDATA(LSB)
SpFi 2SerDes
MictorConnector
TempSensor
DC-DC1.2V
SDRAMCheck Bits
TempSensor
Oscillator 30MHz
Oscillator 30MHz
Voltage CurrentSensor
Voltage CurrentSensor
Voltage CurrentSensor
DC-DC2.5V
DC-DC3.3V
MictorConnector
Xilinx JTAG Connector
Power Supplies
IO Expander
IO Expander
HPPDSP – Bottom Side
21
PCB Layout – Top Side
22
Power Supplies
SpWConnectors
SpWConnectors
SpFi Connectors
TI JTAG GPIO ConnectorPower
Jack
Reset Button
SpFi 1SerDes
SpFi 3SerDes
8-way DIP Switch
Xilinx PROMUser
EEPROM
Oscillator 250Mhz ADC/DAC Mezzanine Board
FMC connector
Boot PROM
SDRAMDATA (MSB)
IO Expander
IO Expander
IO Expander
IO Expander
IO Expander Boot PROM
WriteProtection
Jumper
DisplayConnector
KnobConnectorTrigger
Connectors
HPPDSP – Top Side
23
HPPDSP Front Panel
24
SpaceWire Nominal
Ports 1 & 2 SpaceFibre M/S SpaceFibre Display
Push Button/ Knob
Switches
HPPDSP Rear Panel
25
Triggers GPIO DSP JTAG SpaceWire Redundant Ports 1 & 2
Reset
Power
ADC/DAC FMC Board
Ramon Chips RC64
26
RC64 Many Core DSP Processor 64 fast CEVA X1643 DSP with FP
extension and HW scheduler – 300 MHz – 40 GFLOPS, 384 GOPS
Modem and Encrypt accelerators 4 Mbyte on-chip shared memory Fast I/O
– 12x SpaceFibre, – SpaceWire – DDR3, AD/DA LVDS I/F, NVM
Rad-Hard, for space Advanced technology
– TSMC 65nm LP – CCGA / PBGA / COB – 10 Watt
Modular – Payloads can employ many RC64
Versatile – Designed for all space missions – Planned for 2020—2050
Re-programmable in space
Shared Memory
M M M M M M M M
SpFi DDR2/3 AD/DA SpW NVM DMA
DSP DSP
DSP DSP DSP DSP DSP DSP
$ $ $ $ $ $ $ $
scheduler
MO
DEM
ENCRPT
Ramon Chips
FFT Processor
28
FFT Processor FFT board
– I & Q ADCs each running at 2.4 Gsamples/s – 2 GHz bandwidth – 1.5 MHz spectral bins – 2048 point complex FFT – Processing power of 300 GOPS
Rack – Up to 6 FFT processors in a rack – 12 GHz bandwidth – 1.8 TOPS
29
EM FFT Board
Connections to nominal/redundant control processor via back plane ADC interface on front panel Currently under development
30
Pow
er
JTAG
SMAADC 1 AAF
MicrosemiRTG4FPGA
Clock Generator
Power Supplies
SMAADC 3 AAF
Reset
SpWLVDS
Buffer
ADC
SMAADC 2 AAF
SMAADC 4 AAF
REF
CLK
ADC
Flexi
AUX
CLK
SpW LVDS
TRIG Buffer
REF
CLK Buffer
SpWLVDS
Prototype FFT Board
31
SpaceVPX FFT Payload Processor
32
Controller
FFT
FFT
FFT
Controller
FFT
FFT
FFT
Power Sw
itch
Power Supply A
Power Supply B
X X Control, Data and Management Plane
Power Plane Power Plane
Remaining System Management Functions
Control, Data and Management Plane
CLK RST
CLK RST
Demonstration
33
Reference Architecture (Demonstrator)
34
Instrument 2 Interface SpW Control/HK
Data Output
SpaceWire To
SpaceFibre Bridge
SpaceWire Instrument
SpaceWire Instrument
SpW Control/HK Data Output
SpW Control/HK Data Output
Local Instruments
Remote Instruments
Local Instrument
SpaceFibre Router
Instrument 1 Interface SpW Control/HK
Data Output
Local Instrument
Mass Memory Interface Data Bus
To Memory
Mass Memory Unit
HPPDSP Data Input/Output
DSP Processor
Control Processor Interface SpW Control/HK
Data Input/Output
Control Processor
SpaceFibre Router
Reference Architecture (Demonstrator)
35
SpaceWire To
SpaceFibre Bridge
SpaceWire Instrument
SpaceWire Instrument
SpW Control/HK Data Output
SpW Control/HK Data Output
Local Instruments
Remote Instruments
SpaceFibre Router
Mass Memory Interface Data Bus
To Memory
Mass Memory Unit
Downlink Telemetry Interface SpW Control/HK
Data Output
Downlink Telemetry
Control Processor Interface SpW Control/HK
Data Input/Output
Control Processor
SpaceFibre Router
Reference Architecture (Demonstrator)
36
SpaceFibre Router
Downlink Telemetry Interface SpW Control/HK
Data Output
Downlink Telemetry
Control Processor Interface SpW Control/HK
Data Input/Output
Control Processor
SpaceFibre Router
SpaceWire To
SpaceFibre Bridge
SpaceWire Instrument
SpaceWire Instrument
SpW Control/HK Data Output
SpW Control/HK Data Output
Local Instruments
Remote Instruments
Reference Architecture (Demonstrator)
37 Remote Instruments
SpaceFibre Router
Mass Memory Interface Data Bus
To Memory
Mass Memory Unit
Downlink Telemetry Interface SpW Control/HK
Data Output
Downlink Telemetry
Control Processor Interface SpW Control/HK
Data Input/Output
Control Processor
SpaceFibre Router
Reference Architecture (Demonstrator)
38
SpaceFibre Router
Control Processor Interface SpW Control/HK
Data Input/Output
Control Processor
SpaceFibre Router
Remote Instruments
Reference Architecture (Demonstrator)
39
SpaceFibre Router
SpaceFibre Router
Remote Instruments
Reference Architecture (Demonstrator)
40
SpaceFibre Router
SpaceFibre Router
Remote Instruments
41
SpaceFibre Router
SpaceFibre Router
Remote Instruments
42
Conclusions SpaceWire
– An enabler for many space missions
SpaceFibre – Ready for the next generation of space missions – Complementing and extending SpaceWire – Ideal for interconnection of DSP processors
HPPDSP – Programmable DSP processor – With SpaceWire and SpaceFibre links – IO around 500 Mbits/s
Ramon Chips RC64 many core DSP – 64 cores with very high performance – 12 SpaceFibre links on chip
FFT processor – 2.4 Gsamples/s 2k-point FFT implemented in RTG4 FPGA
43
Acknowledgements The research leading to these results has received
funding from – The UK Space Agency, CEOI-ST under University of
Leicester contract number: RP10G0327C11/B11 RP10G0327D13 RP10G0348A02 RP10G0348B206 RP10G0348A207
– The European Space Agency under ESA contract numbers: 17938/03/NL/LvH – SpaceFibre 4000100103 - HPPDSP 4000102641 - SpaceFibre Demonstrator
– The European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement numbers 263148 - SpaceWire-RT (SpaceFibre QoS) 284389 - SpaceFibre-HSSI (VHiSSI chip) 44
