Hardware/Software Tradeoffs in the Design of a Real Time Distributed Computer System

Mark D. Quinn Gail M. Walsh

Raytheon Company Missile System Laboratories Tewksbury, MA 018760901

Abstract

All too often, during the design of a distributed computer system, the assumption is made that either the hardware will be designed around the software architecture or conversely, the software will be designed around the hardware architecture. Experience has shown, however, in order to meet real time computing requirements, tradeoffs must be made between the hardware and the software architectures. A cooperative effort by hardware and software engineers is an efficient method for system development. Presented herein are our experiences using this approach.

Introduction

A dedicated collaboration team effort produced optimal hardware and software tradeoffs during the development of a distributed computing system with real time requirements. A cost effective design approach was taken to tightly couple video processing and data processing functions into a parallel architecture with a multi-processing Ada real-time operating system. The specifications were generated in total compliance with design to production and system requirements. The distributed computing system was comprised of 14 module types, two of which, the single board computer and the peripheral 110 module, along with the bus structures, were the areas where the sigdicant tradeoffs were made and are discussed in this paper.

System Requirements

The design of the distributing computing system was required to result in a cost effective product which was to be battery operated and met a weight limitation. Implementation of Ada as a source language and throughput processing to support real time video processing were restrictions imposed on the software.

Software Requirements

Several design criteria, such as coupling, cohesion, span of control, scope of decision, implementation language, throughput and memory requirements led to the software design and architecture concept that was chosen for the computer system. Functionally, the software was required to perform launch and flight control, operator/peripheral 110 interface, video and graphics overlay displays, fault detection to the lowest replaceable unit, and to provide an embedded trainer capability. The system microprocessors were required to provide 100 percent growth in both memory and throughput a t the completion of full scale development. In order to meet the memory and throughput requirement, three microprocessors were needed: an interface control computer, a launch/flight control computer and a video control computer. All software resident in the computer system was functionally allocated to a specific microprocessor to provide optimum response to internal/external stimuli. Each microprocessor required a common run-time system, with modifications for a given microprocessor's needs, e.g., interrupt handling, task/priority definitions, 1/0 definitions, etc.. The interface control computer was to be the "master" computer and provide the interface to the other computers and all external interfaces. The video control computer was to control the video module sequencing and to ensure tightly coupled video and data processing for all data received/sent, interacting with the other computers in response to operator requests addressing video and map displays and flight control. The launch/flight control computer was to provide flight control to a missile at a 60 Hz update rate.

Hardware Requirements

Requirements focused on the development of a system architecture which would support real time distributed video and data processing.

Copyright O American Institute of Aeronautics and Astronautics, Inc., 1989. All rights reserved.

The architecture of the distributed computing system was partitioned into two subsystems to satisfy a weight requirement imposed on each chassis. The two subsystems, the video processor and the data processor were to share a common chassis design and both utilize a single board computer (SBC) as subsystem controller. The video processor subsystem was comprised of a number of specialized software reconfigurable video processing modules. The ability to recodigure these processing elements under SBC control provided a flexible video pipeline architecture which allowed for various image processing possibilities. The data processing subsystem provided the system with an intelligent communication manager, comprised of two SBCs as part of its architecture. One SBC provided efficient monitoring and controlling of the man-machine interface and the second SBC provided realtime interaction with a missile interface.

The subsystems resided on two electrically independent VME (Versa Module Europa) bus, each VME bus under mastership of a SBC module. The VME bus provided the system with two key features: compatibility with industry standard VME hardware and software flexibility for module expansion. (Addition of other processors is an effective approach to meet expanding performance requirements.)

The SBC was developed to meet various system level computational, timing and interface requirements. The Motorola 68020 32 bit microprocessor and the Motorola 68882 Floating Point Coprocessor were the key processing elements

on the SBC. The memory structure was designed to provide zero wait state access to RAM and a 1 wait state access to ROM in order to meet real time processing requirements. The architecture provided a user definable memory mapping scheme and a user programmable interrupt structure. The VME interface was designed to provide a user definable request and arbitration scheme meeting VME protocol.

The Peripheral 110 (PIO) module was developed as a method to implement a man machine interface. The P I 0 module was an extension of the SBC memory map and provided the system with a communication and data channel tolfrom the outside world.

Initial Approach

Initially, there were two separate design efforts for the respective disciplines. The software effort focused on task distribution, functional definition and software sizing. An attempt was made to allocate the resources in such a way as to ensure that any critical timing contraints were met with the available resources. Throughput analyses were performed to ascertain that the hardware could support the system timing contraints. A high speed computing system does not guarantee the meeting of time contraints. Other benchmarks were performed to analyze the timing contraints of the implementation language, namely: an Ada runtime system, including basic runtime support, tasking and extended tasking facilities; and an autopilot benchmark, written in Ada and executed on the target computer.

Figure I Computer System Initial Concept

The hardware effort focused on developing a flexible multiprocessor architecture that featured a reconfigurable video pipeline, an intelligent communication manager, an industry standard bus interface and dowed for module expandability. The limitations required a shared memory module for interprocessor communication. The two subsystems communicated via a CPU Expansion bus. Figure I illustrates the initial concept of the computer system.

Individual module designs were driven by a number of system limitations. Due to the weight requirement, module size was limited resulting in the extensive use of Erasable Programmable Logic Devices (EPLD) replacing the standard digital

control logic with fewer components. Once programmed, one EPLD is functionally equivalent to several standard digital components. This provided the system with many user programmable features. The system was battery operated. Thus, CMOS technology was used wherever possible. CMOS is inherently lower power. However, device speed became a critical issue while meeting real time system requirements. Designs were also developed as independently as possible of specific device technology to allow for system upgrades as technology improves. Figure I1 presents the initial design concept for the single board computer and the peripheral I/O module.

mum

Y C 8 I O 2 0 UPROELB8OR

R U 3 2

16 I T PAR

CPU UDAMaWN mu. VYE nus

-1 SINQLF BOARD COMPUTER CONCEPT ' @a020 (PZOMHZ

60882 COPROCESSOR (PZOMHZ 812K RAW

' SIZK PROW 2 R8-232 SERIAL INTERFACCS

' 18 I T PARALLEL W R T W E BUS INTERFACE CPU UPANWON BUS lNRRFACE

Figure II Single Board Computer and Peripheral 1/0 Initial Designs

Meeting of the Minds

After a review of the initial designs, it was apparent that the hardware design could not meet the needs of the software requirements. A cooperative effort was in force., i.e., a dedicated collaboration team was assembled to ensure hardware and software compatibility. The main issues were throughput and memory. Shared memory required considerable VME arbitration overhead. Block transfers was a more efficient method of interprocessor communication. In addition, the software required a method of broadcasting to all three microprocessors. The allocation and ROM and RAM dictated that the execution of code out of ROM. ROM required one wait state for each access which overhead would reduce throughput to an unacceptable level. As a result, it was necessary to execute out of RAM. This introduced another tradeoff, necessitating the software to download the executable code from a mass storage device during system initialization. Thus, throughput contraints required a method of communicating efficiently and a method of broadcasting and memory requirements dictated reallocation of ROM/RAM.

Tradeoffs

The single board computer and the peripheral I/O module, along with the bus structures, were the areas where the significant tradeoffs were made and are herein discussed. Figure III depicts the final computer system design.

Single Board Computer

Two issues drove the design of the single board computer (SBC) hardware: cost effective production and Ada as the implementation language. Tradeoffs were made to generate specifications for a common design which met all processing requirements per the Ada software architecture of three separate processing applications. As a result, the software was required to determine its personality by reading hardwired personality bits located in the backplane. This approach required bootstrap software for all three processing applications to be available on each single board computer. The overall hardware design was tailored for the software architecture which utilized hardware components located throughout the system. The requirement for Ada was the driving factor in determining local memory requirements. The estimate was approximately 60,000 lines of Ada source code, with one single board computer requiring over 450 KBytes of memory. To accommodate a 100 percent required software growth factor, the single board computer was designed to 1 MByte of RAM and 64 KBytes of EEPROM. In order to reduce memory access time, a tradeoff was made to utilize an existing mass storage peripheral as a "virtual memory" for initializing the system requiring a Direct Memory Access controller to handle the task. In addition, the capability of in circuit programming the boot ROM was also provided which reduced labor costs during reprogramming and allowed for flexible

DATA CONTROL ? m U . O R

I m w s I

Figure III Computer System Final Design

software modifications. This common design resulted in a production saving by reducing the single board computer module types from three to one. The final design for the single board computer is illustrated in Figure IV.

Bus Structure

The three single board computers, distributed between two separated subsystems, required a method of interprocessor communication, a method of broadcasting data, and a need for coupling data between the single board computers and certain slave devices. Due to these requirements, two busses, VME and Network, were specified. The tradeoff on the bus network was to have each single board computer interface to two busses, providing the software a simple, efficient bus structure with fast time response. The Network bus was used for interprocessor communication and message broadcasting, and the VME bus was used for communication to the specialized processing elements. A requirement imposed by the video processing modules was that the video control subsystem reside on a Motorola VME bus in order to configure and control the specialized processing elements located as slaves on it. The specialized processing elements were designed with intelligence such that, once initialized by the single board computer software, VME accesses would be kept to a minimum. (i.e., software would only access VME when a slave interrupted it). In order to retain a common single board computer design, the VME

bus was retained in the data processor subsystem. This allowed common software protocol to be used by both subsystems. However, the VME bus hardware provides no capability for communication between subsystems. The Network bus, byte wide operating at 3 MHz, has the capacity of interfacing concurrently to a maximum of eight modules via transmit and receive FIFO buffers. The advantage of the Network bus was an efficient method of block transfers versus byte transfers. However, implementing this bus required software to format messages for transmission and interpret messages received. Thus, the implementation of two bus structures resulted in a very efficient use of the hardware by the software.

Man-Machine Interface

The peripheral 110 module was designed as an interface to a number of serial interfaces. It was essential that the design be extremely efficient for single board computer access because the software had to interact with the data received from the many external devices in a timely manner. Thus, the peripheral 110 hardware was designed to satisfy two requirements: to make each component as accessible as possible to the operational software such that the command, control and data registers of each of the serial interface adapter devices were individually accessible by the software; and, to make the peripheral 110 intelligent such that the operational software did not have to monitor each of the interfaces. Single byte interfaces were to be

ARD COMPUTFR CO-

08020 @ZOYHZ ' -2 COPROCESSOR -HI

1 u e a A a r r E RAY U K CWROY DNA CONTROLLER VYC BUS INTERFACE WITWORK BUS INTCRACI

Figure IV Single Board Computer Final Design

Figure V Peripherial 110 Module Final Design

accessed by the single board computer via the VME bus. Multiple byte interfaces were to be accessed by the single board computer via the Network bus. This allowed the single board computer software to send a command packet to a serial interface and not respond to a receive message until interrupted, thus eliminating the need for polling the interfaces by the software. A Small Computer Systems Interface (SCSI) on the peripheral 110 was used to access to the mass storage element in the system. The single board computer configured the SCSI controller, each register of which was individually addressable, via the VME bus. Data from the mass storage device was written to a dual port RAM for temporary storage allowing the single board computer software to access data through one port while the other port was being loaded by the SCSI interface. The real time software system

requirements forced the peripheral I/O module to be a hardware efficient design. The final design for the peripheral I/O module is presented in Figure V.

Conclusion

In conclusion, the efforts of the hardware/software team in collaborating in tradeoffs produced a hardware design which was cost effective, software compatible, and provided a large capacity for growth. The collaboration yielded a common single board computer designed for cost effective production, a single board computer bus interface designed for simple, efficient, timely responses, and an interrupt driven peripheral I/O module designed to eliminate software polling. The initial time and resources invested by the team demonstrated that a joint hardware/software venture is necessary for a sucessful system level design.