PCS Technical Enrichment Matrix DSP Multimedia Software Architecture Presented By: Patrick Gries 3G...

PCS Technical Enrichment Matrix

DSP Multimedia Software Architecture

Presented By:Patrick Gries

3G DSP Development

3 April 2002

TEM: 3 April 2002 Motorola Confidential Proprietary

2

Agenda Background 3G DSP Challenges Architectural Goals RTOS-101 Architecture Interfaces

Modem MDI IPCM data Handler API

DMP Hi-Fidelity Audio Video Memory Summary/Conclusion

TEM Goals:

RTOS

Talon Architecture


3

Background (1 of 2)Legacy DSP code: DSP56600 or TI320C5x

TI WhiteCap / Motorola WhiteCap Wally Patriot

New Platform: Rainbow (StarCore SC140)No Reuse AvailableDesign for ReuseMultimedia Features RequiredTalon A820 – 1st Motorola 3G Phone


4

Background (2 of 2)


5

3G DSP Challenges (1 of 6) Dual Mode Modem (GSM & WCDMA) Traditional 8 kHz Support

Echo Canceller / Noise Suppressor / Audio Shaper Common Tone Generator Speech Recognition Voice Annotation

4 Vocoders GSM Full-Rate (GSM 06.10) GSM Half-Rate (GSM 06.20) GSM Enhanced Full-Rate (GSM 06.60) Adaptive Multi-Rate (3G TS 26.090)


6

3G DSP Challenges (2 of 6) High-Fidelity Audio Decoders

MP3 (ISO/IEC 11172-3 & 13818-3) AAC (ISO/IEC 13818-7) MIDI WMA

Audio/Graphic Equalization Video

MPEG4 (ISO/IEC 14496-2) H.263 WMV

Playback & Streaming


7

3G DSP Challenges (3 of 6) Beta-level Tools - CodeWarrior Memory Footprint

Total vs. Runtime Runtime Available vs. Runtime Required

MIPS Current Drain


8

3G DSP Challenges (4 of 6)

2G DSP Components

3G DSP Components


9


2G DSP Components

3G DSP Components


10


Multimedia Component Memory

FR

HR

AMR/ EFR

CTGECS/ NSVR2

MP3 AACMIDI

MPEG4

0

20

40

60

80

100

Memory (K bytes)


11

Architectural Goals (1 of 2) Isolate Multimedia Subsystem from

Modem Utilize Commercial RTOS

RTXCdsp from Lineo - Minimal Configuration Process Scheduling Intertask Communication Data Movement

Hierarchical Design for Modularity Tasks - blocking Handlers – data routing Components - processing


12

Architectural Goals (2 of 2) Minimize Use of Global Memory Components Designed for Reuse

Single Function call No RTOS References No Hardware Interface References

Standardized Component APIs 4 pointers (input, output, control, status) Re-entrancy


13

RTOS-101 (1 of 6) RTOS Requirements

Pre-emptive scheduling Low overhead to meet processing

deadlines Minimal footprint

RTXCdsp from Lineo Pre-emptive scheduling Scalable services Easy-to-use kernel services Library-based


14

RTOS-101 (2 of 6) RTXCdsp Configuration

RTXCms only Tasks – data processing Semaphores – intertask communication Queues – data movement & intertask

comm No dedicated timers!


15

RTOS-101 (3 of 6)

RTXCdsp Topology task, threads, both RTXCms API

Task Context Virtual Processors

Object classes Kernel services that

operate on them ISRs

Kernel-aware Private

HW devices

Task 2…

ISR

startup

SysInit

RTXCms

API

API

Task 1

Zone 1

Zone 2

Zone 3


16

RTOS-101 (4 of 6)

Zone 1 Services IS_SignalSema()

Zone 3 Services KS_ExecuteTask() KS_SignalSema() KS_TestSemaW() KS_TestSemaMW() KS_PutQueueData

() KS_GetQueueData

()


17

RTOS-101 (5 of 6)Basic Task Pseudocode

taskA() { … task initialization, user-specified as needed for (;;) { KS_TestSemaW(MYSEMA); … perform event processing as required } }


18

RTOS-101 (5 of 6)


19

Arch - Multimedia Subsystem Perspective


20

Arch - Simplified Rainbow HW Platform

IPCM MCUDSP

VSAP USB

MDIASAP

PCAP

RAINBOW

speechcodec

s tereoDAC

BluetoothI/F device 123


21

Arch – Triggering Mechanisms (1 of 5)

datacontrol

Process X data input

data output

trigger to generate

data

“data ready” event

Basic Multimedia Element


22


Producer A data input

data output/input

trigger to generate

data


Consumer B

data output

trigger to generate

data


Producer / Consumer Relationship


23


ISR data input

data output/input

IRQ

“buffer full” event

Encoderdata

output

trigger to encode

dataProducer / Consumer: Feedforward Control for Encoders


24


Decoder data input

data output/input ISR

data output

IRQ

Producer / Consumer: Feedback Control for Decoders

trigger to decode

data

“buffer empty” event


25

Arch – Triggering Mechanisms (5 of 5) Producers/Sources

ASAP Rx Timeslots 0-3 USB-enabled mic Speech Decoder VA Playback CTG MIDI Decoder MP3 Decoder AAC Decoder

Consumers/Destinations ASAP Tx Timeslots 0-3 USB-enabled speaker VSAP TX timeslots 0 &

1 (stereo speaker, L & R)

Speech Encoder VA Record VR recognition

(frontend)


26

Arch - Software Reference Model


27

Arch - Basic MM Task Manager (input)


28

Arch – MDI: Immediate Processing

mdi_fn1_hdlr()

cmd_parser_task()

audio_mgr_task()

audio mgr cmd

Q

prim1_hdlr()

MDI IRQ


29

Arch – MDI: Deferred Processing

mdi_fn1_hdlr()

cmd_parser_task()

audio_mgr_task()

audio mgr cmd

Q

prim1_hdlr()

MDI IRQ


30

Arch – Task Pseudocodemanager_task() { static const SEMA semalist[] = { DATA_IN_SEMA, DATA_OUT_SEMA, CMD_QNE_SEMA, (SEMA) 0 }; SEMA cause; for(;;) { cause = KS_TestSemaMW(semalist); switch(cause) { case DATA_IN_SEMA: ... process input data break;

case DATA_OUT_SEMA: ... process output data break;

case CMD_QNE_SEMA: ... process message in command queue data break; } } }


31

Arch – Task Functional Grouping

Core 8 kHz audio Every 20ms CTG and VR-FE

includedHigh-fidelity audio

Every 5.8ms, 24-72ms

VideoVR - recognition

Common Attributes Command Queues SEMA-driven

processing Handler layers Isolated

components


32

Arch - Talon Multimedia Architecture (prelim)


33

Arch – Data Buffering & Routing

CTG DL 160

160DL Speech 160

VA Playback

160

Rx SSIx 1

160 Tx SSIx

task

ISR


34

Arch – Codec State Machine

Hybrid ISR Processes samples (private ISR) Signals semaphores (kernel-aware

ISR)Air I/F timing

GSM – every 4.615ms WCDMA – every 10ms


35

Arch – Codec State Machine

waiting for

encode

waiting for

decode

waiting for

decode copy

waiting for

rollover

waiting for resync

waiting for

configure

cnt == ULUL_SEMA

cnt == DLDL_SEMA

cnt == COPYcp tx buf

cnt == 160cnt = 0

cnt == 160cnt = 0

audio == ONcnt = 0


36

Interfaces – Modem

Data structureNot event-drivenAir interface frame reference

GSM – every 4.615 ms WCDMA – every 10 ms ISR for state variable (i.e. sample

counter) check


37

Interfaces – DSP-to-MCU


38

Interfaces – IPCM (1 of 3) Channel Control Block

Base BD pointer Next BD pointer

Buffer Descriptor mode

• DONE (D)• WRAP (W)• CONT (C)• INTERRUPT (I)• EXTENDED (EXT)• FIRST (F)• LAST (L)• ERROR (E)

count buffer pointer extended - parameter

(opt)

CCB

012

31…

mode countbuffer pointer

parameter (opt)

BD


parameter (opt)


39

Interfaces – IPCM (2 of 3) Current Buffer

Descriptor Use 2nd only DONE = 1 CONT = 0 WRAP = 1

DSP buffer Will not wraparound 16-bit transfers

DSP buffer


parameter (opt)

BD


parameter (opt)


40

Interfaces – IPCM (3 of 3) Base Buffer Descriptor

DONE = 1 CONT = 0 WRAP = 0

Current Buffer Descriptor DONE = 1 CONT = 1 WRAP = 1

DSP buffer Will wraparound 16-bit transfers

DSP buffer


parameter (opt)

BD


parameter (opt)


41

Interfaces – Handler Complexity (1 of 2)

McCabe Number Complexity measure of software Based on control flow representation Complexity num_loops Lower is better


42

Interfaces – Handler Complexity (2 of 2)

Some McCabe #’s DL UL audio 7 9 speech 9 12 ctg 3 na va 6 3 ecs/ns na 1 vr2 (fe) na 8 mpeg4a 7 3--------------------------------- MP3 19 AAC 20 MIDI 30

1

2 3

McCabe Number = 3


43

Interfaces - Common API

Input Control Pointer pts to struct to configure how data is to be processed includes init ctrl, ptr to params, ptr to memories

Input Data Pointer linear / modulo

Output Data Pointer pts to where component should write processed data

Output Status Pointer pts to struct where processing status is to be written

NULL ptrs where appropriate


44

DMP – Total Memory for MM Components

total MM required

Total SRAM

unused

0

100

200

300

400

500

600

Memory (K bytes)


45

DMP – Memory Overbudget

Modem needs much more than 50 Kb

Does not yet include WMA/WMVMutually exclusive MM

componentsDynamic Memory Paging (DMP)


46

DMP - Memory for DMP Configurations

ROM

M

PEG4

AAC

MID

I

MP3

FR/V

R2FR/E

CS/H

RFR/E

CS/A

MR

RAM

RAM

0

20

40

60

80

100

120

Memory (K bytes)


47

DMP – dsp_data.sRAM test/init

DMP N

DMP 2

DMP 3

core

app

DMP 4

DMP 5DMP 1

……


48

High-Fidelity Audio


49

HiFi Audio – MP3 / AAC / MIDI

Input IPCM Modulo

Processing Data-driven via “transfer done” SEMA Common API structure

Output Doubly-buffered VSAP via IPCM Linear


50

HiFi Audio - MP3 Component


51

Video – MPEG4 Video

Audio-controlled decoding via timestamp

QCIF output buffer in eDRAM via IPCMLower priority than audioPreemption must handle IPCM transfers


52

Video – A/V Normal Processing


53

Video – A/V Swapping


54

Memory - Talon 2019 (1 of 3)

0

50000

100000

150000

200000

250000

PROG DRAM DROM DMPMemory (bytes)

2019-1007


55


0

50000

100000

150000

200000

250000

Memory (bytes)

WCDMA ModemGSM ModemMultimediaRTOSOtherDMP


56


0

20000

40000

60000

80000

100000

120000

Program Data RAM Data ROMMemory (bytes)

WCDMA ModemGSM ModemMultimediaRTOSOther


57

Summary

Flexible architecture for future features: AMR-WB WMA/WMV Video Telephony (H.263)

MM subsystem modem subsystemsMM components of RTOS & platform


58

Conclusions

Benefited from commercial RTOSMM componentization:

maximizes personnel resources simplifies integration achieves reuse potential

Date post:	18-Jan-2018
Category:	Documents
Upload:	rudolf-hall
View:	226 times
Download:	4 times

PCS Technical Enrichment Matrix DSP Multimedia Software Architecture Presented By: Patrick Gries 3G...

Documents