. .................................................................................................................................................................................................................................................... RAMP: RESEARCH ACCELERATOR FOR MULTIPLE PROCESSORS . .................................................................................................................................................................................................................................................... THE RAMP PROJECT’S GOAL IS TO ENABLE THE INTENSIVE, MULTIDISCIPLINARY INNOVATION THAT THE COMPUTING INDUSTRY WILL NEED TO TACKLE THE PROBLEMS OF PARALLEL PROCESSING. RAMP ITSELF IS AN OPEN-SOURCE, COMMUNITY-DEVELOPED, FPGA-BASED EMULATOR OF PARALLEL ARCHITECTURES. I TS DESIGN FRAMEWORK LETS A LARGE, COLLABORATIVE COMMUNITY DEVELOP AND CONTRIBUTE REUSABLE, COMPOSABLE DESIGN MODULES. THREE COMPLETE DESIGNS—FOR TRANSACTIONAL MEMORY, DISTRIBUTED SYSTEMS, AND DISTRIBUTED-SHARED MEMORY—DEMONSTRATE THE PLATFORM’S POTENTIAL. ...... In 2005, the computer hardware industry took a historic change of direction: The major microprocessor companies all an- nounced that their future products would be single-chip multiprocessors, and that future performance improvements would rely on software-specified parallelism rather than additional software-transparent paral- lelism extracted automatically by the micro- architecture. Several of us discussed this mile- stone at the International Symposium on Computer Architecture (ISCA) in June 2005. We were struck that a multibillion- dollar industry would bet its future on solving the general-purpose parallel com- puting problem, when so many have previously attempted but failed to provide a satisfactory approach. To tackle the parallel processing prob- lem, our industry urgently needs innovative solutions, which in turn require extensive codevelopment of hardware and software. However, this type of innovation currently gets bogged down in the traditional de- velopment cycle: N Prototyping a new architecture in hardware takes approximately four years and many millions of dollars, even at only research quality. N Software engineers are ineffective until the new hardware actually shows up, because simulators are too slow to support serious software development activities. Software engineers tend to innovate only after hardware arrives. N Feedback from software engineers on the current production hardware can- not help the immediate next genera- tion because of overlapped hardware development cycles. Instead, the feed- back loop can take several hardware generations to close fully. Hence, we conspired on how to create an inexpensive, reconfigurable, highly parallel platform that would attract researchers from many disciplines—architectures, compilers, operating systems, applications, and others— to work together on perhaps the greatest challenge facing computing in the past John Wawrzynek David Patterson University of California, Berkeley Mark Oskin University of Washington Shih-Lien Lu Intel Christoforos Kozyrakis Stanford University James C. Hoe Carnegie Mellon University Derek Chiou University of Texas at Austin Krste Asanovic ´ Massachusetts Institute of Technology . ...................................................................... 46 Published by the IEEE Computer Society 0272-1732/07/$25.00 G 2007 IEEE
THE RAMP PROJECT’S GOAL IS TO ENABLE THE INTENSIVE, MULTIDISCIPLINARY INNOVATION
THAT THE COMPUTING INDUSTRY WILL NEED TO TACKLE THE PROBLEMS OF PARALLEL
PROCESSING. RAMP ITSELF IS AN OPEN-SOURCE, COMMUNITY-DEVELOPED, FPGA-BASED
EMULATOR OF PARALLEL ARCHITECTURES. ITS DESIGN FRAMEWORK LETS A LARGE,
COLLABORATIVE COMMUNITY DEVELOP AND CONTRIBUTE REUSABLE, COMPOSABLE DESIGN
MODULES. THREE COMPLETE DESIGNS—FOR TRANSACTIONAL MEMORY, DISTRIBUTED
SYSTEMS, AND DISTRIBUTED-SHARED MEMORY—DEMONSTRATE THE PLATFORM’S POTENTIAL.
......In 2005, the computer hardwareindustry took a historic change of direction:The major microprocessor companies all an-nounced that their future products wouldbe single-chip multiprocessors, and thatfuture performance improvements wouldrely on software-specified parallelism ratherthan additional software-transparent paral-lelism extracted automatically by the micro-architecture. Several of us discussed this mile-stone at the International Symposium onComputer Architecture (ISCA) in June2005. We were struck that a multibillion-dollar industry would bet its future onsolving the general-purpose parallel com-puting problem, when so many havepreviously attempted but failed to providea satisfactory approach.
To tackle the parallel processing prob-lem, our industry urgently needs innovativesolutions, which in turn require extensivecodevelopment of hardware and software.However, this type of innovation currentlygets bogged down in the traditional de-velopment cycle:
N Prototyping a new architecture inhardware takes approximately fouryears and many millions of dollars,even at only research quality.
N Software engineers are ineffective untilthe new hardware actually shows up,because simulators are too slow tosupport serious software developmentactivities. Software engineers tend toinnovate only after hardware arrives.
N Feedback from software engineers onthe current production hardware can-not help the immediate next genera-tion because of overlapped hardwaredevelopment cycles. Instead, the feed-back loop can take several hardwaregenerations to close fully.
Hence, we conspired on how to create aninexpensive, reconfigurable, highly parallelplatform that would attract researchers frommany disciplines—architectures, compilers,operating systems, applications, and others—to work together on perhaps the greatestchallenge facing computing in the past
46 Published by the IEEE Computer Society 0272-1732/07/$25.00 G 2007 IEEE
50 years. Because our industry desperatelyneeds solutions, our goal is to develop aplatform that would allow far more rapidevolution than traditional approaches.
RAMP visionOur hallway conversations led us to the
idea of using field-programmable gate arrays(FPGAs) to emulate highly parallel architec-tures at hardware speeds. FPGAs enable veryrapid turnaround for new hardware. You cantape out a FPGA design every day, and havea new system fabricated overnight. Anotherkey advantage of FPGAs is that they easilyexploit Moore’s law. As the number of coresper microprocessor die grows, FPGA densitywill grow at about the same rate. Today wecan map about 16 simple processors onto asingle FPGA, which means we can constructa 1,000-processor system in just 64 FPGAs.Such a system is cheaper and consumes lesspower than a custom multiprocessor, atabout $100 and 1 W per processor.
Because our goal is to ramp up the rate ofinnovation in hardware and software multi-processor research, we named this projectRAMP (Research Accelerator for MultipleProcessors). RAMP is an open-source projectto develop and share the hardware andsoftware necessary to create parallel architec-tures. RAMP is not just a hardware architec-ture project. Perhaps our most importantgoal is to support the software community asit struggles to take advantage of the potentialcapabilities of parallel microprocessors, byproviding a malleable platform throughwhich the software community can collabo-rate with the hardware community.
Unlike commercial multiprocessor hard-ware, RAMP is designed as a research plat-form. We plan to include research featuresthat are impossible to include in real hard-ware systems owing to speed, cost, or prac-ticality issues. For example, the FPGA designcan incorporate additional hardware to mon-itor any event in the system. Being able to addarbitrary event probes, including arbitrarycomputation on those events, provides visi-bility formerly only available in software sim-ulators, but without the inevitable slowdownfaced by software simulators when introduc-ing such visibility.
A second example of how RAMP differsfrom real hardware is reproducibility. Usingthe RAMP Description Language (RDL)framework, different researchers can con-struct the same deterministic parallel com-puting system that will perform exactly thesame way every time, clock cycle for clockcycle. By using processor designs donated byindustry, RAMP users will start with familiararchitectures and operating systems, whichwill provide far more credibility than softwaresimulations that model idealized processorsor that ignore operating-system effects. RDLis designed to make constructing a full com-puter out of RDL-compatible modules easy.Our target speeds of 100 to 200 MHz areslower than real hardware but fast enough torun standard operating systems and large-scale applications that are orders of magni-tude faster than software simulators. Finally,because of the similarities in the design flowof logic for FPGAs and custom hardware, webelieve RAMP is realistic enough to convincesoftware developers to start aggressive de-velopment on innovative architectures andprogramming models and to convince hard-ware and software companies that RAMPresults are relevant.
This combination of cost, power, speed,flexibility, observability, reproducibility, andcredibility will make the platform attractiveto software and hardware researchers in-terested in the parallel challenge. In particu-lar, it allows the research community to revivethe 1980s culture of building experimentalhardware and software systems, which todayhas been almost entirely lost because of thehigher cost and difficulty of building hard-ware.
Table 1 compares alternatives for pursuingparallel-systems research in academia. Thefour options are a conventional shared-memory multiprocessor (SMP), a cluster, asimulator, a custom-built chip and system,and RAMP. The rows are the features ofinterest, with a grade for each alternative andquantification where appropriate. Cost rulesout a large SMP for most academics. Thecosts of both purchase and ownership make alarge cluster too expensive for most academicsas well. Our only alternative thus far has beensoftware simulation, and indeed that has beenthe vehicle for most architecture research in
the past decade. As mentioned, software de-velopers rarely use software simulators, be-cause they run too slowly, and results mightnot be credible. In particular, it’s unclear howcredible results will be to industry when theyare based on simulations of 1,000 processorsrunning small snippets of applications. TheRAMP option is a compromise among thesealternatives: It is so much cheaper thancustom hardware that it will make highlyscalable systems affordable to academics. It isas flexible as simulators, allowing rapidevolution of the state of the art in parallelcomputing. And it is so much faster thansimulators that it could actually temptsoftware people to try out a new hardwareidea.
This speed also lets architects explore amuch larger space in their research and thusdo a more thorough evaluation of their pro-posals. Although architects can achieve highbatch simulation throughput using multipleindependent software simulations distribut-ed over a large computing cluster, this doesnot reduce the latency of obtaining a singlekey result that can move the research for-ward. Nor does it help an application devel-oper trying to debug the port of an applica-tion to the new target system (the emulatedmachine is called the target, and underlyingFPGA hardware is the host). Worse, formultiprocessor targets, simulation speed, inboth instructions per second per core and
total instructions per second, drops as morecores are simulated and as operating-systemeffects are included, and the amount ofmemory required for each node in the hostcompute cluster rises rapidly.
RAMP is obviously attractive to a broad setof hardware and software researchers in paral-lelism. Some representative research projectsthat we believe could benefit from usingRAMP are
N testing the robustness of multiproces-sor hardware and software under faultinsertion;
N developing thread scheduling and dataallocation and migration techniques forlarge-scale multiprocessors;
N developing and evaluating ISAs forlarge-scale multiprocessors;
N creating an environment to emulate ageographically distributed computer,with realistic delays, packet loss, andso on (Internet in a box);
N evaluating the impact of 128-bit andother floating-point representations onconvergence of parallel programs;
N developing and testing hardware andsoftware schemes for improved security;
N recording traces of complex programsrunning on a large-scale multiprocessor;
N evaluating the design of multiprocessorswitches (serial point-to-point, distrib-uted torus, fat trees);
Table 1. Relative comparison of four options for parallel research. From the architect’s perspective, the most
surprising aspect of this table is that not only is performance not the top concern, it is at the bottom of this
list. The platform just needs to be fast enough to run the entire software stack.
Feature SMP Cluster Simulator Custom RAMP
Scalability (1,000 CPUs) C A A A A
Cost (1,000 CPUs) F ($40M) C ($2M-$3M) A+ ($0M) F ($20M) A ($0.1M-$0.2M)
Cost of ownership A D A D A
Power/space (kW, racks) D (120, 12) D (120, 12) A+ (.1, 0.1) A B (1.5, 0.3)
Development community D A A F B
Observability D C A+ A B+Reproducibility B D A+ A A+Reconfigurability D C A+ C A+Credibility of result A+ A+ D A+ B+/APerformance (clock) A (2 GHz) A (3 GHz) F (0 GHz) B (0.4 GHz) C (0.1 GHz)
N developing data-flow architectures forconventional programming languages;
N developing parallel file systems;N testing dedicated enhancements to stan-
dard processors; andN compiling software directly into FPGAs.
We believe that RAMP’s upside potentialis so compelling that the platformwill create a‘‘watering hole’’ effect in academic depart-ments as people from many disciplines useRAMP in their research. As researchers fromsuch diverse fields begin using RAMP, con-versations between disciplines that rarelycommunicate may result, ultimately, inhelping to more quickly develop multipro-cessor systems that are easy to programefficiently. Indeed, to help industry win itsbet on parallelism, we will need the help ofmany people, for the parallel future is not justan architecture change, but likely a change tothe entire software ecosystem.
RAMP design frameworkFrom the earliest stages of the RAMP
project, it was clear that we needed a stan-dardized design framework to enable a largecommunity of users to cooperate and build auseful library of interoperable hardwaremodels. This design framework has severalchallenging goals. It must support both cycle-accurate emulation of detailed parameterizedmachine models and rapid functional-onlyemulations. The design framework shouldhide the details in the underlying FPGA emu-lation substrate from the module designer asmuch as possible, so that groups with dif-ferent FPGA emulation hardware can sharedesigns and RAMP modules for reuse afterFPGA emulation hardware upgrades. In ad-dition, the design framework should notdictate the hardware design language (HDL)that the developers choose. Our approach wasto develop a decoupled machine model anddesign discipline. This discipline is enforcedby the RDL and a compiler to automate thedifficult task of providing cycle-accurate em-ulation of distributed communicating com-ponents.1
The RAMP design framework is based ona few central concepts. A RAMP target modelis a collection of loosely coupled target unitscommunicating with latency-insensitive pro-
tocols over well-defined target channels.Figure 1 gives a simple schematic exampleof two connected units. In practice, a unit willbe a large component corresponding to tensof thousands of gates of emulated hard-ware—for example, a processor with an L1cache, a DRAM controller, or a networkrouter stage. All communication betweenunits is via messages sent over unidirectionalpoint-to-point interunit channels, where eachchannel is buffered to allow units to executedecoupled from one another.
Partitioning of target models is far simplerthan the classic circuit-partitioning problemassociated with traditional FPGA-based cir-cuit emulation. Although units will be large,we expect them to be relatively small com-pared to the FPGA capacity, so they willnever be partitioned across multiple FPGAs.A target model is only partitioned at thechannel interfaces, leaving units intact. Chan-nels connecting units that map to separateFPGAs are implemented using FPGA-to-FPGA physical links. Currently, partitioningis driven by user annotations in RDL, buteventually we expect to build automaticpartitioning tools.
Each unit faithfully models the behavior ofeach target clock cycle in the component. Thetarget unit models can be developed either asregister-transfer-level (RTL) code in a stan-dard HDL (currently Verilog, VHDL, andBluespec are supported) for compilation ontothe FPGA fabric, or as software models thatexecute either on attached workstations or onhard or soft processor cores embedded withinthe FPGA fabric. Many target units takenfrom existing RTL code will execute a singletarget clock cycle in one FPGA physical clockcycle, giving a high simulation speed. How-ever, to save FPGA resources, a unit modelcan be designed to take multiple physical host
clock cycles on the FPGA to emulate onetarget clock cycle, or might even use a varyingnumber of physical clock cycles. Initially, thewhole RAMP host system uses the samephysical clock rate (nominally around100 MHz), with some higher physical clockrates in off-chip I/O drivers.
Unit models are synchronized onlythrough the point-to-point channels. Thebasic principle is that a unit cannot advanceby a target clock cycle until it has receiveda target clock cycle’s worth of activity on eachinput channel, and until the output channelsare ready to receive another target cycle’sworth of activity. This scheme forms a dis-tributed concurrent-event simulator, wherethe buffering in the channels lets units run atvarious physical speeds on the host whileremaining logically synchronized in terms oftarget clock cycles. Unit model designersmust produce the RTL code (or gateware) ofeach unit in their chosen HDL, and specifythe range of message sizes that each input oroutput channel can carry. For each supportedHDL, the RAMP design framework providestools to automatically generate a unit wrapperthat interfaces to the channels and providestarget cycle synchronization. The RTL codefor the channels is generated automatically bythe RDL compiler from an RDL description,which includes a structural netlist specifyingthe instances of each unit and how they areconnected by channels.
The benefit of enforcing a standard channel-based communication strategy between units isthat many features can be provided automat-ically by the RDL compiler and runtime
system. Users can vary the target latency, targetbandwidth, and target buffering on eachchannel at configuration time. The RAMPconfiguration tools will also provide the optionof having channels run as fast as the underlyingphysical hardware will allow, thus supportingfast, functional-only emulation. We are alsoexploring the option of allowing these param-eters to be changed dynamically at target systemboot time to avoid rerunning the FPGA syn-thesis flow when varying parameters for per-formance studies.
The configuration tool will include sup-port for interunit channels to be tapped andcontrolled to provide monitoring and debug-ging facilities. For example, by controllingstall signals from the channels, a unit can besingle stepped. Using a separate, automati-cally inserted debugging network, invisible totarget system software, messages can be in-serted and read out from the channels en-tering and leaving any unit, and all significantevents can be logged. These monitoring anddebugging facilities will provide significantadvantages over running applications oncommercial hardware.
RAMP prototypesAlthough most of the participants in the
project are volunteers, we are on a fairly ag-gressive schedule. Table 2 shows the RAMPproject timeline. We began RAMP develop-ment using preexisting FPGA boards—seethe ‘‘RAMP hardware’’ sidebar. To seed thecollaborative effort, we are developing threeprototype systems: RAMP Red, RAMPBlue, and RAMP White. Each of our initial
Table 2. RAMP timeline.
Date Milestone
6 June 2005 Hallway discussions lead to RAMP vision
13 June 2005 The name ‘‘RAMP’’ coined; BEE22 selected as RAMP-1; a dozen people identified to develop RAMP
January 2006 RAMP retreat and RDL tutorial at Berkeley
March 2006 NSF infrastructure grant awarded
June 2006 RAMP retreat at Massachusetts Institute of Technology; RAMP Red running with eight processors on RAMP-1
boards
January 2007 RAMP Blue running with 256 processors on eight RAMP-1 boards
August 2007 RAMP Red, White, and Blue running with 128 to 256 processors on 16 RAMP-1 boards; accurate clock cycle
accounting and I/O model
December 2007 RAMP-2 boards redesigned based on Virtex-5 and available for purchase; RAMP Web site has downloadable
prototypes contains a complete gateware andsoftware configuration of a scalable multi-processor populated with standard processorcores, switches, and operating systems. Oncethe base system is assembled and softwareinstalled, users will be able to easily run com-plex system benchmarks and then modify thisworking system as desired. Or they can startfrom the ground up, using the basic compo-nents to build a new system. We expect usersto release back to the community any en-hancements and new gateware and softwaremodules. A similar usage model has led to theproliferation of the SimpleScalar framework,which now covers a range of instruction setsand processor designs.
RAMP RedRAMP Red is the first multiprocessor
system with hardware support for transac-tional memory. Transactional memory trans-fers the responsibility for concurrency controlfrom the programmer to the system.3 Itintroduces database semantics to the sharedmemory in a parallel system, which allowssoftware tasks (transactions) to execute atom-ically and in isolation without the use of locks.Hardware support for transactional memoryreduces the overhead of detecting and enfor-cing atomicity violations between concur-rently executing transactions and guaranteescorrect execution under all cases.
RAMP Red implements the StanfordTransactional Coherence and Consistency(TCC) architecture for transactional memo-ry.4 The design uses nine PowerPC 405 hardcores (embedded in the Xilinx Virtex-II-ProFPGAs) connected to a shared main memorysystem through a packet-switched network.The built-in data cache in each PowerPC 405core is disabled and replaced by a customcache (emulated in FPGA) with transactionalmemory support. Each 32-Kbyte cache buf-fers the memory locations that are read andwritten by a transaction during its executionand detects atomicity violations with otherongoing transactions. An interesting featureof RAMPRed is the use of a transaction com-pletion mechanism that eliminates the needfor a conventional cache coherence protocol.
From an application’s perspective, RAMPRed is a fully featured Linux workstation.The operating system actually runs on just
one of the cores, while the remaining eightcores execute applications. A light-weight ker-nel in each application core forwards excep-tions and system calls to the operating systemcore. The programming model is multi-threaded C or Java with locks replaced bytransactional constructs. RAMP Red includesan extensive hardware and software frame-work for debugging, bottleneck identifica-tion, and performance tuning.
The RAMP Red design has been fullyoperational since June 2006. It runs at100 MHz on RAMP-1, which is 100 timesfaster than the same architecture simulatedin software on a 2-GHz workstation. Wee etal. provide more details of the RAMP Reddesign.5 Early experiments with enterprise,scientific, and artificial-intelligence applica-tions have demonstrated the simplicity ofparallel programming with transactionalmemory, and that RAMP Red achievesscalable performance. In the future, RAMPRed will be the basis for further research intransactional memory, focusing mostly onsoftware productivity and system softwaresupport.6
RAMP BlueRAMP Blue is a family of emulated
message-passing machines that can run par-allel applications written for the Message-Passing Interface (MPI) standard, or forpartitioned global-address-space languagessuch as Unified Parallel C (UPC). RAMPBlue can also model a networked servercluster.
The first RAMP Blue prototype was devel-oped at the University of California, Berke-ley; Figure 2 shows its hardware platform.It comprises a collection of BEE2 boardshoused in 2 U chassis and assembled in astandard 19-inch rack. Physical connectionamong the eight boards is through 10-GbpsInfiniband cables (light-colored cables inFigure 2). The BEE2 boards are wired in anall-to-all configuration with a direct connec-tion from each board to all others through10-Gbps links. System configuration, debug-ging, and monitoring take place througha 100-Mbps Ethernet switch with connectionto each board’s control FPGA (dark wires atthe top of Figure 2). For systemmanagementand control, each board runs a full-featured
Linux kernel on one PowerPC 405 hardcoreembedded in the control FPGA. Our initialtarget applications are the UPC versions ofthe NASA Advanced Supercomputing (NAS)Parallel Benchmarks.
The four user FPGAs per BEE2 board areconfigured to hold a collection of 100-MHzXilinx MicroBlaze soft processor cores run-ning uCLinux. We have mapped eight pro-cessor cores per FPGA. The first prototype,with 32 user FPGAs, emulates a 256-waycluster system. In the future, the number ofprocessor cores can be scaled up throughseveral means. We will add more BEE2boards—the simple all-to-all wiring configu-ration will accommodate up to 17 boards.We will also add more cores per FPGA—thecurrent configuration of eight processor coresper FPGA only consumes 40 percent of theFPGA’s logic resources. RAMP Blue imple-ments all necessary multiprocessor compo-nents within the user FPGAs. In addition tothe soft processor cores, each FPGA holdsa packet network switch (one per core) forconnection to cores on the same and otherFPGAs, shared memory controllers, shareddouble-precision floating-point units, and ashared ‘‘console’’ switch for connection to thecontrol FPGA.
In RAMP Blue, each processor is assignedits own DRAM memory space (at least 250Mbytes per processor). The external memoryinterface of the MicroBlaze L1 cache con-nects to external DDR2 (double-data-rate 2)DRAM through a memory arbiter, as eachDRAM channel is shared among a set ofMicroBlaze cores. Because each BEE2 userFPGA has four independent DRAM mem-ory channels, four processor cores wouldshare one channel in the maximum-sizedconfiguration (16 processor cores perFPGA). With each processor running at100 MHz and each memory channel run-ning a 200-MHz DDR 72-bit data interface,each processor can transfer 72 bits of data at100 MHz, which is more than each processorcore can consume even in our maximum-sized configuration. A simple round-robinscheme is used to arbitrate among the cores.
The processor-processor network switchcurrently uses a simple interrupt-driven pro-grammed I/O approach. A Linux driver pro-vides an Ethernet interface so that applica-
tions can access the processor network via tra-ditional socket interfaces. We are planning anext-generation network interface with directmemory access through special ports in thememory controller.
A 256-core (eight per FPGA) version ofthe RAMP Blue prototype has been fullyoperational, running the NAS ParallelBenchmark suite, since December 2006.This initial prototype was not implementedusing RDL, but a newer version based onRDL has been operational since February2007. We are currently measuring andtuning the prototype’s performance.
RAMP WhiteRAMP White is a distributed-shared-
memory machine that will demonstrateRAMP’s open-component nature by in-tegrating modules from RAMP Red, RAMPBlue, and contributions from other RAMPparticipants. The initial version is beingdesigned and integrated at the University of
Texas at Austin. The RAMP White effortbegan in the summer of 2006, somewhatafter Red and Blue, and will be implemen-ted in the following phases.
1. Global distributed-shared memory with-out caches. All requests to remote globalmemory will be serviced directly fromremote memory. Communication willtake place over a ring network.
2. Ring-based snoopy coherency. The basicinfrastructure of the cache-less systemwill be expanded to include a snoopycache that will snoop the ring.
3. Directory-based coherency. A directory-based coherence engine eliminates theneed for each cache to snoop all transac-tions but will use the same snoopy cache.
RAMPWhite will eventually be composedof processor units from the University ofWashington and RAMP Blue teams that willbe connected through a simple ring network(Figure 3). For expediency, the initial RAMPWhite will use embedded PowerPC proces-sors. Each processor unit will contain oneprocessor connected to an intersection unitthat provides connections to a memory con-troller (MCU), a network interface unit(NIU), and I/O if the processor unit supportsit. The NIU will be connected to a simplering network.
The intersection unit switches requests andreplies between the processor, local memory,I/O units, and the network. The initial inter-
section unit is very simple. Memory requestsfrom the processor are divided into localmemory requests, global memory requests(both handled by memory), I/O requests(handled by the I/O module), and remoterequests (handled by the NIU). Remote re-quests from the NIU are forwarded to thememory. Because the initial version ofRAMP White does not cache global loca-tions, incoming remote requests need not besnooped by the processor.
I/O will be handled by a centralized I/Osubsystem mapped into the global addressspace. Each processor will run a separateSMP-capable Linux that will take locks toaccess I/O. The global memory supportthen transparently handles shared I/O. Laterversions will add a coherency support usinga soft cache (emulated in FPGA). RAMPWhite’s first snoopy cache will be based onRAMP Red’s snoopy cache. It is possiblethat some or all of the data in the emulatedcache will actually reside in DRAM if thereis not sufficient space in the FPGA itself. Inthe coherent versions of RAMP White, theintersection unit passes all incoming remoterequests to the coherent cache for snoopingbefore allowing the remote request to pro-ceed to the next stage.
RAMP represents the research commu-nity’s return to building hardware-
software systems. RAMP is designed to em-body the right trade-offs of cost, perfor-mance, density, and visibility for system
research. Moreover, since the system is notfrozen, we can use it to both rapidly evolveand spread successful ideas across thecommunity. Research in hardware architec-
ture, operating systems, compilers, applica-tions, and programming models will allbenefit. We are planning a full public releaseof the RAMP infrastructure in 2007. MICRO
AcknowledgmentsThis work was funded in part by the
National Science Foundation, grant CNS-0551739. Special thanks to Xilinx for itscontinuing financial support and dona-tion of FPGAs and development tools.We appreciate the financial support pro-vided by the Gigascale Systems ResearchCenter. Thanks to IBM for its financialsupport through faculty fellowships anddonation of processor cores, and to SunMicrosystems for processor cores. There isan extensive list of industry and academicfriends who have given valuable feedbackand guidance. We especially thank Arvind(Massachusetts Institute of Technology)and Jan Rabaey (University of California,Berkeley) for their advice. The workpresented in this article is the effort of theRAMP students and staff: Hari Angepat,Dan Burke, Jared Casper, Chen Chang,Pierre-Yves Droz, Greg Gibeling, AlexKrasnov, Ken Lutz, Martha Mercaldi, NjuNjoroge, Andrew Putnam, Andrew Schutlz,and Sewook Wee.
John Wawrzynek is a professor of electricalengineering and computer sciences at theUniversity of California, Berkeley. Hecurrently teaches courses in computer archi-tecture, VLSI system design, and reconfigur-able computing. He is codirector of theBerkeley Wireless Research Center andprincipal investigator of the Research Accel-erator for Multiple Processors (RAMP)project. He holds a PhD and MS incomputer science from the California In-stitute of Technology and a MS in electricalengineering from the University of Illinois,Urbana-Champaign. He is a member of theIEEE and the ACM.
David Patterson is the Pardee Professor andDirector of RAD Lab at the University ofCalifornia, Berkeley. His research interestsare in design and automatic management ofdatacenters and in hardware-software archi-tectures of highly parallel microprocessors.He is a fellow of IEEE; recipient of the IEEEvon Neumann and IEEE Mulligan Educa-tion medals; fellow and past president of theACM; and a member of the AmericanAcademy of Arts and Sciences, the NationalAcademy of Engineering, the NationalAcademy of Sciences, and the Silicon ValleyEngineering Hall of Fame.
Mark Oskin is an assistant professor at theUniversity of Washington. His research inter-ests include computer systems architecture,performance modeling, and architectures foremerging technologies. With a large team ofgraduate students, he is the coinventor ofWaveScalar, the first dataflow machine ca-pable of executing applications written inimperative languages, and brick and mortar,a low-cost technique for manufacturing SoCdevices. He received his PhD in computer
Shih-Lien Lu has a BS in electricalengineering and computer sciences fromthe University of California, Berkeley, andan MS and a PhD, both in computer scienceand engineering, from the University ofCalifornia, Los Angeles. He served on thefaculty of the Electrical and ComputerSciences Department at Oregon State Uni-versity from 1991 to 2001. In 1999, he tooka two-year leave from OSU and joined Intel.He is currently a principal research scientistin the microarchitecture lab of Intel’sMicroprocessor Technology Lab in Oregon.His research interests include computermicroarchitecture, circuits, and FPGA sys-tems design.
Christoforos Kozyrakis is an assistant pro-fessor of electrical engineering and computerscience at Stanford University. His researchfocuses on architectural support for parallelcomputing, system security, and energymanagement. He is currently working ontransactional memory techniques that cangreatly simplify parallel programming for theaverage developer. He has a PhD in com-puter science from the University of Califor-nia, Berkeley.
James C. Hoe is an associate professor ofelectrical and computer engineering atCarnegie Mellon University. His researchinterests include computer architecture andhigh-level hardware description and synthe-sis. He has a PhD in electrical engineering
and computer science from the Massachu-setts Institute of Technology. He isa member of the IEEE and the ACM.
Derek Chiou is an assistant professor at theUniversity of Texas at Austin. His researchinterests include computer system simulation,computer architecture, parallel computerarchitecture, and Internet router architecture.He received his PhD, SM, and SB degrees inelectrical engineering and computer sciencefrom the Massachusetts Institute of Technol-ogy. He is a senior member of the IEEE anda member of the ACM.
Krste Asanovic is an associate professor inthe Department of Electrical Engineering andComputer Science at MIT and a member ofthe MIT Computer Science and ArtificialIntelligence Laboratory. His research interestsinclude computer architecture and VLSIdesign. Asanovic has a BA in electrical andinformation sciences from the University ofCambridge and a PhD in computer sciencefrom the University of California, Berkeley.He is a member of the IEEE and the ACM.
Direct questions and comments aboutthis article to John Wawrzynek, 631 SodaHall, Computer Science Division, Univer-sity of California, Berkeley, CA 94720-1776; [email protected].
For further information on this or anyother computing topic, please visit ourDigital Library at http://www.computer.org/publications/dlib.
