Evaluation of the Raw Microprocessor: An Exposed-Wire-Delay Architecture for ILP and Streams Michael Bedford Taylor, Walter Lee, Jason Miller, David Wentzlaff, Ian Bratt, Ben Greenwald, Henry Hoffmann, Paul Johnson, Jason Kim, James Psota, Arvind Saraf, Nathan Shnidman, Volker Strumpen, Matt Frank, Saman Amarasinghe, and Anant Agarwal CSAIL, Massachusetts Institute of Technology ABSTRACT This paper evaluates the Raw microprocessor. Raw addresses the challenge of building a general-purpose architecture that performs well on a larger class of stream and embedded computing appli- cations than existing microprocessors, while still running existing ILP-based sequential programs with reasonable performance in the face of increasing wire delays. Raw approaches this challenge by implementing plenty of on-chip resources – including logic, wires, and pins – in a tiled arrangement, and exposing them through a new ISA, so that the software can take advantage of these resources for parallel applications. Raw supports both ILP and streams by rout- ing operands between architecturally-exposed functional units over a point-to-point scalar operand network. This network offers low latency for scalar data transport. Raw manages the effect of wire delays by exposing the interconnect and using software to orches- trate both scalar and stream data transport. We have implemented a prototype Raw microprocessor in IBM’s 180 nm, 6-layer copper, CMOS 7SF standard-cell ASIC process. We have also implemented ILP and stream compilers. Our evaluation attempts to determine the extent to which Raw succeeds in meeting its goal of serving as a more versatile, general-purpose processor. Central to achieving this goal is Raw’s ability to exploit all forms of parallelism, including ILP, DLP, TLP, and Stream parallelism. Specifically, we evaluate the performance of Raw on a diverse set of codes including traditional sequential programs, streaming ap- plications, server workloads and bit-level embedded computation. Our experimental methodology makes use of a cycle-accurate sim- ulator validated against our real hardware. Compared to a 180 nm Pentium-III, using commodity PC memory system components, Raw performs within a factor of 2x for sequential applications with a very low degree of ILP, about 2x to 9x better for higher levels of ILP, and 10x-100x better when highly parallel applications are coded in a stream language or optimized by hand. The paper also proposes a new versatility metric and uses it to discuss the generality of Raw. 1. INTRODUCTION Fast moving VLSI technology will soon offer billions of transis- tors, massive chip-level wire bandwidth for local interconnect, and a modestly larger number of pins. However, there is growing evidence that wire delays become relatively more significant with shrinking feature sizes and clock speeds [6, 29, 1]. Processors need to con- vert the abundant chip-level resources into application performance, while mitigating the negative effects of wire delays. The advance of technology also expands the number of applica- tions that are implementable in VLSI. These applications include sequential programs that can run on today’s ILP-based micropro- cessors, as well as highly parallel algorithms that are currently implemented directly using application-specific integrated circuits (ASICs). One such circuit is found in the Nvidia Ti 4600 graphics accelerator, which executes hundreds of parallel operations per cy- cle at 300 MHz. Many of these parallel ASICs implement algorithms with computational demands that are far beyond the capabilities of today’s general-purpose microprocessors. The Raw project addresses the challenge of whether a future general-purpose microprocessor architecture could be built that runs a greater subset of these ASIC applications while still running the same existing ILP-based sequential applications with reasonable performance in the face of increasing wire delays. To obtain some intuition on how to approach this challenge, we conducted an early study [4, 48] on the factors responsible for the significantly better performance of application-specific VLSI chips. We concluded that there were four main factors: specialization; exploitation of paral- lel resources (gates, wires and pins); management of wires and wire delay; and management of pins. 1. Specialization: ASICs specialize each “operation” at the gate level. In both the VLSI circuit and microprocessor context, an op- eration roughly corresponds to the unit of work that can be done in one cycle. A VLSI circuit forms operations by combinational logic paths, or “operators”, between flip-flops. A microprocessor, on the other hand, has an instruction set that defines the operations that can be performed. Specialized operators, for example, for implementing an incompatible floating point operation, or implementing a linear feedback shift register, can yield an order of magnitude performance improvement over an extant general purpose processor that may re- quire many instructions to perform the same one-cycle operation as the VLSI hardware. As an example, customized Tensilica ASIC pro- cessors take advantage of specialization by augmenting a general purpose processor core with specialized instructions for specific ap- plications. 2. Exploitation of Parallel Resources: ASICs further exploit plen- tiful silicon area to implement enough operators and communica- tions channels to sustain a tremendous number of parallel operations in each clock cycle. Applications that merit direct digital VLSI cir- cuit implementations typically exhibit massive, operation-level par- allelism. While an aggressive VLIW implementation like Intel’s Ita- nium II [32] executes six instructions per cycle, graphics accelera- tors may perform hundreds or thousands of word-level operations per cycle. Because they operate on very small word operands, logic emulation circuits such as Xilinx II Pro FPGAs can perform hun- dreds of thousands of operations each cycle. The recent addition of MMX and SSE-style multigranular instructions that operate on multiple subwords simultaneously marks an effort to improve the efficiency of microprocessors by exploiting additional parallelism available due to smaller word sizes. The Itanium II die photo reveals that less than two percent of the die area is dedicated to its 6-way issue integer execution core.
Transcript
Evaluation of the Raw Micr opr ocessor:An Exposed-Wire-Dela y Architecture for ILP and Streams
Michael Bedford Taylor, Walter Lee, Jason Miller, David Wentzlaff,Ian Bratt, Ben Greenwald, Henry Hoffmann, Paul Johnson, Jason Kim, James Psota,
Arvind Saraf, Nathan Shnidman, Volker Strumpen, Matt Frank,Saman Amarasinghe, and Anant Agarwal
CSAIL, Massachusetts Institute of Technology
ABSTRACTThis paperevaluatesthe Rawmicroprocessor. Rawaddressesthechallenge of building a general-purposearchitecture that performswell on a larger classof streamand embeddedcomputingappli-cationsthan existing microprocessors, while still running existingILP-basedsequentialprogramswith reasonableperformancein thefaceof increasingwire delays. Rawapproachesthis challenge byimplementingplentyof on-chip resources– including logic, wires,andpins– in a tiled arrangement,andexposingthemthrougha newISA,so that thesoftware can take advantage of theseresourcesforparallel applications.Rawsupportsboth ILP andstreamsby rout-ing operandsbetweenarchitecturally-exposedfunctionalunitsovera point-to-pointscalar operand network. This networkoffers lowlatencyfor scalar data transport. Rawmanages the effect of wiredelaysby exposingthe interconnectand usingsoftware to orches-tratebothscalarandstreamdatatransport.
We haveimplementeda prototypeRawmicroprocessorin IBM’s180nm,6-layercopper, CMOS7SFstandard-cell ASICprocess.Wehavealso implementedILP and streamcompilers. Our evaluationattemptsto determinetheextentto which Rawsucceedsin meetingits goal of servingas a more versatile, general-purposeprocessor.Central to achieving this goal is Raw’s ability to exploit all formsof parallelism, including ILP, DLP, TLP, and Streamparallelism.Specifically, we evaluatethe performanceof Rawon a diversesetof codesincluding traditional sequentialprograms,streamingap-plications, serverworkloadsand bit-level embeddedcomputation.Our experimentalmethodology makesuseof a cycle-accurate sim-ulator validatedagainstour real hardware. Compared to a 180nmPentium-III,usingcommodityPC memorysystemcomponents,Rawperformswithin a factorof 2xfor sequentialapplicationswith a verylow degreeof ILP, about2x to 9xbetterfor higherlevelsof ILP, and10x-100xbetter whenhighly parallel applicationsare codedin astreamlanguage or optimizedby hand. Thepaperalsoproposesanew versatility metricandusesit to discussthegenerality of Raw.
1. INTRODUCTIONFastmoving VLSI technologywill soonoffer billions of transis-
The advanceof technologyalsoexpandsthe numberof applica-tions that are implementablein VLSI. Theseapplicationsincludesequentialprogramsthat can run on today’s ILP-basedmicropro-cessors,as well as highly parallel algorithms that are currentlyimplementeddirectly using application-specificintegratedcircuits
(ASICs). Onesuchcircuit is found in the Nvidia Ti 4600graphicsaccelerator, which executeshundredsof paralleloperationspercy-cleat300MHz. Many of theseparallelASICsimplementalgorithmswith computationaldemandsthatarefar beyond thecapabilitiesoftoday’s general-purposemicroprocessors.
The Raw project addressesthe challengeof whethera futuregeneral-purposemicroprocessorarchitecturecouldbebuilt thatrunsa greatersubsetof theseASIC applicationswhile still runningthesameexisting ILP-basedsequentialapplicationswith reasonableperformancein the faceof increasingwire delays.To obtainsomeintuition on how to approachthis challenge,we conductedanearlystudy[4, 48] on the factorsresponsiblefor the significantlybetterperformanceof application-specificVLSI chips.We concludedthattherewerefour main factors: specialization;exploitation of paral-lel resources(gates,wiresandpins);managementof wiresandwiredelay;andmanagementof pins.
1. Specialization: ASICs specializeeach“operation” at the gatelevel. In both the VLSI circuit andmicroprocessorcontext, an op-erationroughlycorrespondsto theunit of work thatcanbedoneinonecycle. A VLSI circuit formsoperationsby combinationallogicpaths,or “operators”,betweenflip-flops. A microprocessor, on theotherhand,hasaninstructionsetthatdefinestheoperationsthatcanbeperformed.Specializedoperators,for example,for implementingan incompatiblefloating point operation,or implementinga linearfeedbackshift register, canyield anorderof magnitudeperformanceimprovementover anextantgeneralpurposeprocessorthatmayre-quiremany instructionsto performthesameone-cycle operationastheVLSI hardware.As anexample,customizedTensilicaASIC pro-cessorstake advantageof specializationby augmentinga generalpurposeprocessorcorewith specializedinstructionsfor specificap-plications.
2. Exploitation of Parallel Resources:ASICsfurtherexploit plen-tiful silicon areato implementenoughoperatorsand communica-tionschannelsto sustainatremendousnumberof paralleloperationsin eachclock cycle. Applicationsthatmerit directdigital VLSI cir-cuit implementationstypically exhibit massive, operation-level par-allelism.While anaggressive VLIW implementationlike Intel’s Ita-nium II [32] executessix instructionsper cycle, graphicsaccelera-tors may perform hundredsor thousandsof word-level operationspercycle. Becausethey operateon very smallword operands,logicemulationcircuits suchasXilinx II Pro FPGAscanperform hun-dredsof thousandsof operationseachcycle. The recentadditionof MMX andSSE-stylemultigranularinstructionsthat operateonmultiple subwords simultaneouslymarksan effort to improve theefficiency of microprocessorsby exploiting additionalparallelismavailabledueto smallerwordsizes.
The Itanium II die photo reveals that less than two percentofthe die areais dedicatedto its 6-way issueinteger executioncore.
Clearly, the ALU areais not a significantconstrainton the execu-tion width� of amodern-daywide-issuemicroprocessor. Ontheotherhand,thepresenceof many physicalexecutionunits is a minimumprerequisiteto theexploitationof thesamemassive parallelismthatASICsareableto exploit.
3. Managementof Wir esand Wir e Delay: ASIC designerscanplaceand wire communicatingoperationsin ways that minimizewire delay, minimizelatency, andmaximizebandwidth.In contrast,it is now well known that the delayof the interconnectinside tra-ditional microprocessorslimits scalability [36, 1, 15, 38, 45]. Ita-niumII’ s6-way integerexecutionunit presentsevidencefor this– itspendsover half of its critical pathin thebypasspathsof theALUs.ASIC designersmanagewire delayinherentin largedistributedar-raysof function units in multiple steps. First, they placecloseto-getheroperationsthat needto communicatefrequently. Second,when high bandwidthis needed,they createmultiple customizedcommunicationchannels.Finally, they introducepipelineregistersbetweendistantoperators,therebyconvertingpropagationdelayintopipelinelatency. By doingso,thedesigneracknowledgestheinher-ent tradeoff betweenparallelismand latency: leveragingmore re-sourcesrequiressignalsto travel greaterdistances.TheAlpha21264is an exampleof a microprocessorthat acknowledgesthis tradeoffon a small scale: it incursa one-cycle latency for signalsto travelbetweenits two integerclusters.
4. Managementof Pins: ASICscustomizetheusageof their pins.Ratherthanbeingbottlenecked by a cache-orientedmulti-level hi-erarchicalmemorysystem(andsubsequentlyby a genericPCI-styleI/O system),ASICsutilize theirpinsin waysthatfit theapplicationsat hand, maximizing realizableI/O bandwidthor minimizing la-tency. Thisefficiency appliesnot justwhenanASIC accessesexter-nal DRAMs, but alsoin theway that it connectsto high-bandwidthinput devices like wide-word analog-to-digitalconverters,CCDs,and sensorarrays. Thereare currently few easyways to arrangefor thesedevicesto streamdatainto ageneral-purposemicroproces-sor in a high-bandwidthway, especiallysinceDRAM mustalmostalwaysbeusedasanintermediatebuffer. In somesenses,micropro-cessorsstrive to minimize, ratherthanmaximize,the usageof pinresources,by hiding themthrougha hierarchyof caches.
Our goal was to build a microprocessorthat could leveragetheabove four factors,andyet implementthegamutof general-purposefeaturesthatwe expectin a microprocessorsuchasfunctionalunitvirtualization, unpredictableinterrupts, instruction virtualization,anddatacaching. The processoralsoneededto exploit ILP in se-quentialprograms,aswell asspaceandtime multiplex (i.e.,contextswitch) threadsof control. Furthermore,thesemultiple threadsofcontrolshouldbeableto communicateandcoordinatein anefficientfashion.
This paperevaluatesthe Raw microprocessoranddiscussesoursuccessin achieving thesegoals.Raw takesthefollowing approachto leveragingthefour factorsbehindthesuccessof ASICs.
1. Raw implementsthe most commonoperationsneededby ILPor streamapplicationsin specializedhardwaremechanisms.Mostof theprimitive mechanismsareexposedto softwarethrougha newISA. Thesemechanismsincludetheusualintegerandfloatingpointoperations,specializedbit manipulationoperations,scalaroperandrouting betweenadjacentfunction units, operandbypassbetweenfunctionunits,registersandI/O queues,anddatacacheaccess(i.e.,dataloadwith tagcheck).
2. Raw implementsa largenumberof theseoperatorswhich exploitthecopiousVLSI resources– includinggates,wiresandpins– and
exposesthemthrougha new ISA, suchthat the softwarecan takeadvantageof themfor bothILP andhighly parallelapplications.
3. Raw managesthe effect of wire delaysby exposingthe wiringchanneloperatorsto thesoftware,so that the softwarecanaccountfor latenciesby orchestratingbothscalarandstreamdatatransport.By orchestratingoperandflow ontheinterconnect,Raw canalsocre-atecustomizedcommunicationspatterns.Takentogether, thewiringchanneloperatorsprovide the abstractionof a scalaroperandnet-work [45] thatoffers very low latency for scalardatatransportandenablestheexploitationof ILP.
4. Raw softwaremanagesthe pins for cachedatafetchesand forspecializedstreaminterfacesto DRAM or I/O devices.
WehaveimplementedaprototypeRaw microprocessorin theSA-27E ASIC flow, which usesIBM’ s CMOS 7SF, an 180nm,6-layercopperprocess. We received 120 chips from IBM in Octoberof2002.We have built a prototypeRaw motherboardcontainingasin-gle Raw chip, SDRAM chips,I/O interfacesandinterfaceFPGAs.We have also implementedILP and streamcompilersfor sequen-tial programsand streamapplicationsrespectively. Our develop-ment tools include a validated,cycle-accuratesimulator, an RTL-level simulator, anda logic emulator.
Our evaluationattemptsto determinethe extent to which Rawsucceedsin meetingits goalof servingasa moreversatile,general-purposeprocessor. Specifically, weevaluatetheperformanceof Rawonapplicationsdrawn from severalclassesof computationincludingILP, streamsandvectors,server, andbit-level embeddedcomputa-tion. Ourinitial resultsareveryencouraging(seeFigure3 for aquicksamplingof our results).For eachof theapplicationclasses,wefindthatRaw is ableto performator closeto thelevel of thebest-in-classmachine– i.e.,thebestspecializedmachinefor thegivenapplicationclass.For example,for sequentialapplicationswith varyingdegreesof ILP, theperformanceof Raw rangesfrom 0.5x to 9x to thatof aPentiumIII (P3)processorimplementedin thesametechnologygen-erationasRaw. For streamingcomputations,Raw’s performanceis10xto 100xbetterthantheP3,andcomparableto thatof specializedstreamandvectorengineslike Imagine[17] andVIRAM [21].
Wepresentametriccalledversatilitythatfolds into asinglescalarnumberthe performanceof an architectureover many applicationclasses,andshow thatRaw’s is seven timesbetterthanthat for theP3.Our futurework will expandthenumberof applicationsin eachclassandseeif this trendcontinuesto hold.
Therestof this paperis organizedasfollows. Section2 providesan overview of the Raw architectureand its mechanismsfor spe-cialization,exploitationof parallelresources,orchestrationof wires,andmanagementof pins.Section3 describestheimplementationofRaw, andSection4 providesdetailedresults. Section5 introducesour versatility metric and analyzesthe results. Section6 followswith a detaileddiscussionof relatedwork, andSection7 concludesthepaper.
2. ARCHITECTURE OVERVIEWThis sectionprovides a brief overview of the Raw architecture.
A more detaileddiscussionof the architectureis available else-where[43, 44,45].
FPU, a 32 KB datacache,two typesof communicationrouters–static� anddynamic,and32KB and64KB of software-managedin-structioncachesfor theprocessingpipelineandstaticrouterrespec-tively. Eachtile is sizedso that the amountof time for a signalto travel througha small amountof logic andacrossthe tile is oneclock cycle. We expect that the Raw processorsof the future willhave hundredsor eventhousandsof tiles.
On-chip Networks Thetilesareinterconnectedby four 32-bit full-duplex on-chipnetworks,consistingof over 12,500wires (seeFig-ure1). Two of thenetworksarestatic(routesarespecifiedatcompiletime) andtwo aredynamic(routesarespecifiedat run time). Eachtile is connectedonly to its four neighbors.Every wire is registeredat the input to its destinationtile. This meansthat the longestwirein the systemis no greater than the lengthor width of a tile. Thispropertyensureshigh clock speeds,andthecontinuedscalabilityofthearchitecture.
The designof Raw’s on-chip interconnectandits interfacewiththeprocessingpipelineareits key innovativefeatures.Theseon-chipnetworksareexposedto thesoftwarethroughtheRaw ISA, therebygiving the programmeror compiler the ability to directly programthe wiring resourcesof the processor, and to carefully orchestratethetransferof datavaluesbetweenthecomputationalportionsof thetiles – muchlike theroutingin anASIC. Effectively, thewire delayis exposedto theuserasnetwork hops.To go from cornerto cornerof the processortakes6 hops,which correspondsto approximatelysix cyclesof wire delay. To minimizethelatency of inter-tile scalardatatransport,which is critical for ILP, the on-chip networks arenotonly register-mappedbut alsointegrateddirectly into thebypasspathsof theprocessorpipeline.
Raw’son-chipinterconnectsbelongto theclassof scalaroperandnetworks [45], which lendan interestingway of looking at modernday processors.The register file usedto be the centralcommuni-cation mechanismbetweenfunctional units in a processor. Start-ing with the first pipelinedprocessors,the bypassnetwork hasbe-comelargely responsiblefor the communicationof active values,andthe registerfile is moreof a checkpointingfacility for inactivevalues.TheRaw networks (thestaticnetworks in particular)areinonesense2-D bypassnetworksservingasbridgesbetweentheby-passnetworksof separatetiles.
Thestaticrouterin eachtile containsa 64KB software-managedinstructioncacheanda pair of routing crossbars.Compilergener-atedrouting instructionsare64 bits andencodea small command(e.g.,conditionalbranchwith/withoutdecrement)andseveralroutes– onefor eachcrossbaroutput. Thesesingle-cycle routing instruc-tions areoneexampleof Raw’s useof specialization.Becausetherouterprogrammemoryis cached,thereis no practicalarchitecturallimit on the numberof simultaneouscommunicationpatternsthatcanbe supportedin a computation.This feature,coupledwith theextremelylow latency andlow occupancy of in-orderinter-tile ALU-to-ALU operanddelivery (3 cyclesnearestneighbor)distinguishesRaw from prior systolicor messagepassingsystems[3, 12,23].
Raw’s two dynamicnetworks supportcachemisses,interrupts,dynamicmessages,and other asynchronousevents. The two net-worksusedimension-orderedroutingandarestructurallyidentical.Onenetwork, the memorynetwork, follows a deadlock-avoidancestrategy to avoid end-pointdeadlock.It is usedin a restrictedman-nerby trustedclientssuchasdatacaches,DMA andI/O. Thesecondnetwork, thegeneralnetwork, is usedby untrustedclients,andrelieson a deadlockrecovery strategy [23].
Raw supportscontext switches.Ona context switch,thecontentsof the processorregistersandthe generalandstaticnetworks on asubsetof theRaw chip occupiedby theprocess(possiblyincludingmultiple tiles)aresavedoff, andtheprocessandits network datacan
berestoredat any time to anew offseton theRaw grid.
Dir ect I/O Interfaces On the edgesof the network, the networkchannelsare multiplexed down onto the pins of the chip to formflexible I/O portsthat canbe usedfor DRAM accessesor externaldevice I/O. In order to togglea pin, the userprogramsoneof theon-chipnetworksto routeavalueoff thesideof thearray. Our1657pin CCGA (ceramiccolumn-gridarray) packageprovides us withfourteenfull-duplex, 32-bit I/O ports. Raw implementationswithfewer pinsaremadepossiblevia logical channels(asis alreadythecasefor two out of thesixteenlogical ports),or simply by bondingoutonly a subsetof theports.
The static and dynamicsnetworks, the datacacheof the com-puteprocessors,andtheexternalDRAMs connectedto theI/O portscompriseRaw’s memorysystem.Thememorynetwork is usedforcache-basedmemorytraffic, while the static andgeneraldynamicnetworksareusedfor stream-basedmemorytraffic. Memory inten-sive domainscanhave up to 14 full-duplex full-bandwidthDRAMmemorybanksto be placedon the 14 I/O portsof the chip. Mini-malembeddedRaw systemsmayeliminateDRAM altogether– theymayusea singlebootROM sothatRaw canexecuteout of theon-chip memory. In additionto transferringdatadirectly to the tiles,off-chip devicesconnectedto theI/O portscanroutethroughtheon-chip networks to otherdevices in order to performgluelessDMAandpeer-to-peercommunication.1
ISA Analogs to Physical Resources By creatingfirst classarchi-tecturalanalogsto thephysicalchipresources,Raw attemptsto min-imize the ISA gap– that is, the gapbetweenthe resourcesthat aVLSI chiphasavailableandtheamountof resourcesthatareusableby software. Unlike conventionalISAs, Raw exposesthe quantityof all threeunderlyingphysicalresources(gates,wiresandpins) tothe ISA. Furthermore,it doesthis in a mannerthat is backwards-compatible– the instructionsetdoesnot changewith varying de-greesof resources.
Physical Entity Raw ISA analog Conventional ISA AnalogGates Tiles,new instructions New instructions
Wir es,Wir e delay Routes,Network Hops nonePins I/O ports none
Table1: How Raw converts increasingquantities of physicalentities intoISA entities
Table 1 contraststhe way the Raw ISA and conventional ISAsexposephysical resourcesto the programmer. Becausethe RawISA hasmoredirectinterfaces,Raw processorscanhavemorefunc-tionalunits,andhavemoreflexibleandmoreefficientpin utilization.High-endRaw processorswill typically havemorepins,becausethearchitectureis betterat turningpin countinto performanceandfunc-tionality. Finally, Raw processorsare more predictableand havehigherfrequenciesbecauseof theexplicit exposureof wire delay.
This approach,exposure,makes Raw scalable. Creatingsubse-quent,more powerful, generationsof the processoris straightfor-ward:wesimplystampoutasmany tilesandI/O portsasthesilicondie andpackageallow. Thedesignhasno centralizedresources,noglobalbuses,andnostructuresthatgetlargerasthetile or pin countincreases.Finally, the longestwire, thedesigncomplexity, andtheverificationcomplexity areall independentof transistorcount.
3. IMPLEMENT ATIONThe Raw chip is a 16-tile prototypeimplementedin IBM’ s 180
nm 1.8V 6-layerCMOS7SFSA-27Ecopperprocess.AlthoughtheRaw array is only 16 mm x 16 mm, we usedan 18.2 mm x 18.2�In fact,wearebuilding a4x4 IP packet routerusingasingleRaw chipandits peer-to-
peercapability.
DR
AM
DR
AM
DR
AM
PC�
I x 2
PCI x 2
DR�
AM
D/A
DR�
AM
DRAM
DR�
AM
DRAM
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
Com�
put e Pipelin e
SMEMPC
X
DATA�
CACHEPCIM EM
DR
AM
DR
AM
DR
AM
PC�
I x 2
PCI x 2
DR�
AM
D/A
DR�
AM
DRAM
DR�
AM
DRAM
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
DR
AM
DR
AM
DR
AM
PC�
I x 2
PCI x 2
DR�
AM
D/A
DR�
AM
DRAM
DR�
AM
DRAM
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S� C
�S�C
�S�C
�S�C
�S� C
�S�C
�S�C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S� C
�S�C
�S�C
�S�C
�S� C
�S�C
�S�C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S� C
�S�C
�S�C
�S�C
�S� C
�S�C
�S�C
�S�C
�S�
C�
S�C
�S� C
�S�C
�S� C
�S�C
�S�C
�S�C
�S� C
�S�C
�S�C
�S�C
�S�
Com�
put e Pipelin e
SMEMPC
X
DATA�
CACHEPCIM EM
IF RFDA TL
M1
F P
E
U WB
r26
r27
r25
r24
InputFIFOsfromSt
aticRouter
r26
r27
r25
r24
Outpu
t�
FIFOsto�St
aticRouter�
0-
cycle“local bypassnetwork”
M2
TV
F4IF RFD
A TL
M1
F P
E
U WB
r26
r27
r25
r24
InputFIFOsfromSt
aticRouter
r26
r27
r25
r24
Outpu
t�
FIFOsto�St
aticRouter�
0-
cycle“local bypassnetwork”
M2
TV
F4IF RFD
A TL
M1
F P
E
U WB
r26
r27
r25
r24
InputFIFOsfromSt
aticRouter
r26
r27
r25
r24
Outpu
t�
FIFOsto�St
aticRouter�
0-
cycle“local bypassnetwork”
M2
TV
F4
Figure 1: The Raw microprocessorcomprises16 tiles. Each tile hasa computeprocessor, routers,network wires,and instruction and data memories.
mm die to allow usto usethehighpin-countpackage.The1657pinceramiccolumngrid arraypackage(CCGA) providesuswith 1080highspeedtransceiver logic (HSTL) I/O pins.Ourmeasurementsin-dicatethatthechipcoreaverages18.2wattsat425MHz.Wequiesceunusedfunctionalunitsandmemoriesandtri-stateunuseddataI/Opins. We targeteda 225 MHz worst-casefrequency in our design,which is competitive with other180 nm lithographyASIC proces-sors,likeVIRAM, Imagine,andTensilica’sXtensaseries.Thenom-inal runningfrequency is typically higher– theRaw chip core,run-ning at room temperature,reaches425MHz at 1.8V, and500 MHzat2.2V. Thiscomparesfavorablyto IBM-implementedmicroproces-sorsin thesameprocess;thePowerPC405GPrunsat266-400MHz,while thefollow-on PowerPC440GPreaches400-500MHz.
Wepipelinedourprocessoraggressively andtreatedcontrolpathsvery conservatively in order to ensurethat we would not have tospendsignificantperiodsclosing timing in the backend. Despitetheseefforts, wire delayinsidea tile wasstill largeenoughthatwewereforcedto createaninfrastructureto placethecellsin thetimingandcongestioncritical datapaths.More detailson theRaw imple-mentationareavailablein [44].
A difficult challengefor uswasto resistthetemptationsof makingtheabsolutelyhighestperformance,highestfrequency tile processor,andinsteadto concentrateontheresearchaspectsof theproject,suchas the designof Raw’s scalaroperandnetwork. As onecan inferfrom Section5, moving from a one-way issuecomputeprocessorto a two-way issuecomputeprocessorwould have likely improvedour performanceon low-ILP applications. Our estimatesindicatethatsuchacomputeprocessorwouldhaveeasilyfit in theremainingwhite spaceof the tile. The frequency impactof transitioningfrom1-issueto 2-issueis generallyheldto besmall.
With our collaboratorsat ISI-East,we have designeda prototypemotherboard(shown in Figure2) aroundtheRaw chip thatwe useto explore a numberof applicationswith extremecomputationandI/O requirements.A largersystem,consistingof 64Raw chips,con-nectedto form a virtual 1024tile Raw processor, is alsobeingfabri-catedin conjunctionwith ISI-East.
the Raw microprocessor. We begin by explaining our experimen-tal methodology. Thenwe presentsomebasichardwarestatistics.The remainderof the sectionfocuseson evaluatinghow well Rawsupportsarangeof programmingmodelsandapplicationtypes.Thedomainswe examineincludeILP computation,streamandembed-dedcomputation,server workloads,andbit-level computation.The
performanceof Raw in theseindividual areasarepresentedascom-parisonto a reference600MHz PentiumIII.
We notethatRaw achievesgreaterthan16x speedup(eitherver-susaPentiumor versusasingletile) for severalapplicationsbecauseof compoundingor additive effectsfrom several factorslistedin Ta-ble2. Thefollowing is abrief discussionof theseeffects.
1. Whenall 16 tiles canbeused,thespeedupcanbe16-fold.2. If � ,
�, and � arevariablesin memory, thenan operationof
the form ������� � in a load-storeRISC architecturewill requirea minimum of 4 operations– two loads,one add, and one store.StreamarchitecturessuchasRaw canaccomplishtheoperationin asingleoperation(for a speedupof 4x) becauseprocessorcanissuebulk datastreamrequestsand then processdatadirectly from thenetwork withoutgoingthroughthecache.
3. Whenvectorlengthsexceedthecachesize,streamingdatafromoff-chip DRAM directly into theALU achieves7.5x thethroughputof cacheaccesses(eachcachemisstransports8 wordsin 60 cycles,while streamingcan achieve oneword per cycle). The streamingeffect is evenmorepowerful with stridedrequeststhatuseonly partof a full cacheline. In this case,streamingthroughputis 15 timesgreaterthangoingthroughthecache.
4. Raw has60x theI/O bandwidthof theP3.Furthermore,Raw’sdirectprogrammaticinterfaceto thepinsenablesefficientutilization.
5. Whenmultiple tiles areusedin a computation,the effectivenumberof registersandcachelines is increased,allowing a greaterworking set to be accessedwithout penalty. We approximatethepotentialspeedupfrom this effect as2-fold.
6. Finally, specializedbit manipulationinstructionscanoptimizetablelookups,shifts,andlogicaloperations.We estimatethepoten-tial speedupfrom this effect as3-fold.
4.1 Experimental methodologyValidated Simulator Theevaluationfor this papermakesuseof avalidatedcycle-accuratesimulatorof the Raw chip. Using theval-idatedsimulatorasopposedto actualhardwareallows us to betternormalizedifferenceswith a referencesystem,e.g.,DRAM mem-ory latency, andinstructioncacheconfiguration.It alsoallows usto
Figure 2: Photosof the Raw chip and Raw prototypemotherboard respectively.
explorealternative motherboardconfigurations.We verifiedthatthesimulatorandthegate-level RTL netlisthave exactly thesametim-ing anddatavaluesfor all 200,000linesof our hand-writtenassem-bly testsuite,aswell asfor anumberof C applicationsandrandomlygeneratedtests.Every stall signal,registerfile write, SRAM write,on-chipnetwork wire, cachestatemachinetransition,interruptsig-nal,andchipsignalpin matchesin valueoneverycyclebetweenthetwo. This gate-level RTL netlistwasthenshippedto IBM for man-ufacturing. Upon receiptof the chip, we compareda subsetof thetestsontheactualhardwareto verify thatthechipwasmanufacturedaccordingto spec.
Selectionof a ReferenceProcessor In our evaluation,we realizedthe importanceof tying our performancenumbersto an existingcommercialsystem. For fairness,this comparisonsystemmustbeimplementedin aprocessthatusesthesamelithographygeneration,180nm. Furthermore,thereferenceprocessorneedsto bemeasuredatasimilarpoint in its lifecycle,i.e.,ascloseto first siliconaspossi-ble. This is becausemostcommercialsystemsarespeedpathor pro-cesstunedafterfirst silicon is created[7]. For instance,the180nmP3initial productionsiliconwasreleasedat500-733MHz andgrad-ually wastuneduntil it reacheda final productionsilicon frequencyof 1 GHz. The first silicon valuefor theP3 is not publicly known.However, the frequenciesof first-siliconandinitial productionsili-conhave beenknown to differ by asmuchas2x.
TheP3 is especiallyideal for comparisonwith Raw becauseit isin commonuse,becauseits fabricationprocessis well documented,andbecausethe common-casefunctionalunit latenciesarealmostidentical. The backendsof the processorssharea similar level ofpipelining, which meansthat relative cycle-countscarry somesig-nificance.ConventionalVLSI wisdomsuggeststhat,whennormal-ized for process,Raw’s single-portedL1 datacacheshouldhaveapproximatelythe sameareaanddelayas the P3’s two-portedL1datacacheof half thesize. For sequentialcodeswith working setsthat fit in the L1 caches,the cycle countsshouldbe quite similar.And given that the fortunesof Intel have rested(and continuetorest, with the Pentium-Mreincarnation)upon this architectureforalmosttenyears,thereis reasonto believe that the implementationis agoodone.In fact,theP3,uponreleasein 4Q’99,hadthehighestSpecInt95valueof any processor[13].
Itanium andPentium4 (P4) cameas closesecondsto our final
choice. Our decisionto avoid themcamefrom our needto matchthe lifecycle of the referencesystemto Raw’s. Intel’s market pres-surescauseit to delaythe releaseof new processorssuchasP4 orItaniumuntil they havebeentunedenoughto competewith theexist-ing Pentiumproductline. Consequently, whentheseprocessorsarereleased,they maybecloserto final-siliconthanfirst-silicon.For ex-ample,it is documentedin thepressthat ItaniumI wasdelayedfortwo yearsbetweenfirst-siliconannouncementandinitial productionsilicon availability. Finally, the implementationcomplexity of Rawis moresimilar to theP3thanP4or Itanium.
Comparison of Silicon Implementations Table3 comparesthetwo chips and their fabricationprocesses,IBM’ s CMOS 7SF[27,37] andIntel’s P858[51]. CMOS7SFhasdenserSRAM cellsandlessinterconnectresistivity, due to coppermetalization. P858,onthe otherhand,attemptsto compensatefor aluminummetalizationby using a lower-k dielectric, SiOF, and by carefully varying theaspectratiosof thewires.
DynamicLogic, CustomMacros no yes(SRAMs,RFs)SpeedpathTuningsinceFirst Silicon no yes
Initial Frequency 425MHz 500-733MHzDie Area2 331mm
�106mm
�SignalPins � 1100 � 190Vdd used 1.8V 1.65VNominalProcessVdd 1.8V 1.5V
Table 3: Comparison of Implementation Parameters for Raw and P3-Coppermine.
The Ring Oscillatormetric measuresthe delayof a fanout-of-1(FO1) inverter. It hasbeensuggestedthatanapproximateFO4de-lay canbe found by multiplying the FO1 delayby 3 [15]. Thus,P858gatesappearto be significantly (2.1x) fasterthanthe CMOS
7SF gates. This is to be expected,as IBM termsCMOS 7SF a“value”� process.IBM’ s non-ASIC,high-performance,180nm pro-cess,CMOS8S,is competitive with P858[8], andhasring oscilla-tor delaysof 11psandbetter. Furthermore,production180nmP3’shave theirvoltagesset10%higherthanthenominalprocessvoltage,which typically improvesfrequency by 10%or more.
A recentbook,[7], listsanumberof limitationsthatASIC proces-sor implementationsfaceversusfull-customimplementations.Wementionsomeapplicableoneshere. First, becausethe ASIC flowpredeterminesaspectsof a chip, basicoverheadsarerelatively highin comparisonto full-customdesigns.Two of Raw’s largestover-headswere the mandatoryscanflip-flops (18� ), and clock skewand jitter (13� ). Second,ASIC flows tend to producelogic thatis significantlylessdensethancorrespondingcustomflows. Third,ASIC flows prevent useof customor dynamiclogic, except for alimited menu(up to 2 readportsand2 write ports)of fixedpipeline-depthregisterfiles andSRAMs, which aremachinegenerated.A40-80� improvementin frequency often is attributedto the useofdynamiclogic. Processandspeedpathtuningaccountfor 35%. Fi-nally, speed-binningyieldsapproximately20%.
We compensateonly for the last two factorsin this paper. Weselecteda 600MHz P3asthereferencesystem.The600MHz P3,releasedprior to processtuning,andafter limited speedpathtuning,is solidly in themiddleof theP3initial productionfrequency range,presumablyrepresentinganaverage-speedbinpart.
Webelieve thata Raw implementationwith thesameengineeringeffort andprocesstechnologyasthe Intel P3would besmallerandsignificantlyfaster. However, we make no attemptto normalizeforthesefactors.
Normalization Details With theselectionof a referenceCPUim-plementationcomesa selectionof anenclosingcomputer. We useda pair of 600MHz Dell Precision410’s to run our referencebench-marks. We outfitted thesemachineswith identical 100 MHz 2-2-2 PC100256MB DRAMs, andwroteseveral microbenchmarkstoverify thatthememorysystemtimingsmatched.
To comparetheRaw andDell systemsmoreequally, we usedtheRaw simulator’s extensionlanguageto implementa cycle-matchedPC100DRAM modelandachipset4. Thismodelhasthesamewall-clock latency andbandwidthastheDell 410. However, sinceRawrunsat a slower frequency thantheP3,thelatency, measuredin cy-cles,is less.WeusethetermRawPC to describeasimulationwhichuses8 PC100DRAMs, occupying 4 portson the left handsideofthechip,and4 on theright handside.
BecauseRaw is alsodesignedfor streamingapplications,we alsowantedto measureapplicationsthat usethe full pin bandwidthofthe chip. In this case,we usea simulationof CL2 PC 3500DDRDRAM, which provides enoughbandwidthto saturateboth direc-tions of a Raw port. In this case,we use16 PC 3500DRAMs, at-tachedto all 16logicalportsonthechip,in conjunctionwith amem-ory controller, implementedin thechipset,thatsupportsanumberofstreamrequests.A Raw tile cansendamessageover thegeneraldy-namicnetwork to thechipsetto initiate largebulk transfersfrom theDRAMs into andout of thestaticnetwork. Simpleinterleaving andstriding is supported,subjectto the underlyingaccessand timingconstraintsof theDRAM. We call this configurationRawStreams.
The placementof a DRAM on a Raw port doesnot excludetheuseof otherdevices on that port – the chipsetshave a simplede-
�Note that despitethe areapenaltyfor an ASIC implementation,it is almostcertain
that theRaw processoris a biggerdesignthantheP3. Our evaluationdoesnot aim tomake a cost-normalizedcomparison,but ratherseeksto demonstratethe scalabilityofourapproachfor futuremicroprocessordesigns.�
Thesupportchipstypically usedto interfaceaprocessorto its memorysystemandI/Operipherals.
Exceptwhereotherwisenoted,weusedgcc3.3-O3 to compileCandFortrancodefor bothRaw5 andtheP36. For programsthatdoCor Fortranstdiocalls,we usenewlib 1.9.0for bothRaw andtheP3.Finally, to eliminatethe impactof disparatefile andoperatingsys-tems,the resultsof I/O systemcalls for theSpecbenchmarkswerecapturedandembeddedinto thebinariesasstaticdatausing[42].
We performedone final normalization. Our preliminary Rawsoftware-managedinstruction-cachingsystemhas not been opti-mized,which madeit difficult to comparethe two systems.To en-ablecomparisonswith theP3,weaugmentedthecycle-accuratesim-ulator so that it employs conventional2-way associative hardwareinstructioncaching. Theseinstructioncachesaremodelledcycle-by-cycle in the samemannerasthe restof the hardware. Like thedatacaches,they servicemissesover thememorydynamicnetwork.Resourcecontentionbetweenthecachesis modeledaccordingly.
4.2 BasicdataTables4 and5 show functionalunit timingsandmemorysystem
characteristicsfor bothsystems,respectively. Table6 shows Raw’smeasuredpower consumption[19]. Table7 listsa breakdown of theend-to-endmessagelatency on Raw’s scalaroperandnetwork. Thelow 3-cycle inter-tile ALU-to-ALU latency andzerocycle sendandreceiveoccupanciesarecritical for obtaininggoodILP performance.
4.3 ILP ComputationThis sectionexamineshow well Raw is ableto supportconven-
tional sequentialapplications. Typically, the only form of paral-lelism available in theseapplicationsis ILP-level parallelism. For TheRaw gccbackend,basedontheMIPSbackend,targetsasingletile’scomputeand
Much like a VLIW architecture,Raw is designedto rely on thecompilerto find andexploit ILP. We have developedRawcc [5, 24,25] to explore thesecompilationissues.Rawcc takessequentialCor Fortranprogramsandorchestratesthemacrossthe Raw tiles intwo steps.First,Rawccdistributesthedataandcodeacrossthetilesto attemptto balancethe tradeoff betweenlocality andparallelism.Then,it schedulesthecomputationandcommunicationto maximizeparallelismandminimizecommunicationstalls.
Rawccwasdevelopedasaprototypingenvironmentfor exploringcompilationtechniquesfor Raw. As such,unmodifiedSpecapplica-tionsstretchits robustness.Weareworkingonimproving therobust-nessof Rawcc. Additionally, wehavemadeprogressona follow-onparallelizingcompilerwhichhasmorefocusonrobustnessandcodequality. Thespeedupsattainedin Table8 shows thepotentialof au-tomaticparallelizationandILP exploitationon Raw. Of thebench-markscompiledby Rawcc,Raw is ableto outperformtheP3for allthescientificbenchmarksandseveralirregularapplications.
# Raw Cycles SpeedupvsP3Benchmark Source Tiles on Raw Cycles Time
Table8: Performanceof sequentialprogramson Raw and on a P3.
Table 9 shows the speedupsachieved by Rawcc as the numberof tiles variesfrom two to 16. The speedupsarecomparedto per-formanceof a single Raw tile. Overall, the sourceof speedupscomesprimarily from tile parallelism(seeTable2), but several ofthe densematrix benchmarksbenefitfrom increasedcachecapac-ity aswell (which explainsthesuper-linearspeedups).In addition,Fpppp-kernelbenefitsfrom increasedregistercapacity, which leadsto fewer spills.
For completeness,we alsocompileda selectionof theSpec2000benchmarkswith gcc for a single tile, and ran them using theMinneSPEC’s [20] LgReddatasetsto reducethe length of simu-lations.Theresults,shown in Table10, representa lower boundforthe performanceof thosecodeson Raw, as they only use1/16 ofthe resourceson the Raw chip. The numbersarequite surprising;on average;the simple in-orderRaw tile with no L2 cacheis only
Table 9: Speedupof the ILP benchmarksrelative to single-tile Raw.1.4x slower by cyclesand2x slower by time thanthe full P3. Thissuggeststhatin theeventthattheparallelismin theseapplicationsistoo small to be exploited acrossRaw tiles, a simpletwo-way Rawcomputeprocessormightbesufficient to make theperformancedif-ferenceeasilybehiddenby otheraspectsof thesystem.
# Raw Cycles SpeedupvsP3Benchmark Source Tiles on Raw Cycles Time
computationsarisenaturallyoutof real-timeI/O applicationsaswellasfrom embeddedapplications.Thedatasetsfor theseapplicationsareoften large andmay even be a continuousstreamin real-time,which makes themunsuitablefor traditional cachebasedmemorysystems.Raw providesamorenaturalsupportfor streambasedcom-putationby allowing datato befetchedefficiently througha registermapped,softwareorchestratednetwork.
We presenttwo setsof results. First we show the performanceof programswritten in StreamIt,a high level streamlanguage,andautomaticallycompiledto Raw. Then,we show theperformanceofsomehandwrittenapplications.
4.4.1 StreamItStreamIt is a high-level, architecture-independentlanguagefor
high-performancestreamingapplications. StreamItcontainslan-guageconstructsthat improve programmerproductivity for stream-ing, including hierarchicalstructuredstreams,graphparameteriza-tion, andcircular buffer management;theseconstructsalsoexposeinformationto thecompilerandenablenovel optimizations[47]. Wehave developeda Raw backendfor theStreamItcompiler, which in-cludesfully automaticloadbalancing,graphlayout,communicationscheduling,androuting[11].
We evaluate the performanceof RawPC on several StreamItbenchmarks,which representlargeandpervasive DSPapplications.Table11 summarizestheperformanceof 16 Raw tiles vs. a P3. Forbotharchitectures,we useStreamItversionsof thebenchmarks;wedo not compareto hand-codedC on the P3 becauseStreamItper-formsat least1-2X betterfor 4 of the6 applications(this is dueto
aggressive unrolling andconstantpropagationin theStreamItcom-piler).# Thecomparisonreflectstwo distinctinfluences:1) thescalingof Raw performanceasthenumberof tiles increases,and2) theper-formanceof a Raw tile vs. a P3for thesameStreamItcode.To dis-tinguishbetweentheseinfluences,Table12showsdetailedspeedupsrelative to StreamItcoderunningona 1-tile Raw configuration.
CyclesPer Output SpeedupvsP3Benchmark on Raw Cycles Time
Table12: Speedup(in cycles)of StreamIt benchmarksrelative to a 1-tileRaw configuration. From left, the columnsindicate the StreamIt versionon a P3,and on Raw configurations with oneto 16 tiles.
Theprimaryresultillustratedby Table12is thatStreamItapplica-tionsscaleeffectively for increasingsizesof theRaw configuration.For FIR,FFT, andBitonic, thescalingis approximatelylinearacrossall tile sizes(FIR is actually super-linear due to decreasingregis-ter pressurein largerconfigurations).For Beamformer, Filterbank,andFMRadio,the scalingis slightly inhibited for small configura-tions. This is because1) theseapplicationsare larger, and IMEMconstraintspreventanunrollingoptimizationfor smalltile sizes,and2) they have moredataparallelism,yielding speedupsfor largecon-figurationsbut inhibiting smallconfigurationsdueto aconstantcon-trol overhead.
Thesecondinfluenceis theperformanceof a P3vs. a singleRawtile on thesameStreamItcode,asillustratedby thesecondcolumnin Table 12. In most cases,performanceis comparable.The P3performsbetterin two casesbecauseit canexploit ILP: Beamformerhasindependentreal/imaginaryupdatesin theinnerloop,andFIR isa fully unrolledmultiply-accumulateoperation.In othercases,ILPis obscuredby circularbuffer accessesandcontroldependences.
In all, StreamItapplicationsbenefitfrom Raw’s exploitation ofparallelresourcesandmanagementof wires(seeTable19 for sum-mary). The abundantparallelismand regular communicationpat-ternsin streamprogramsarean idealmatchfor theparallelismandtightly orchestratedcommunicationonRaw. As streamprogramsof-tenrequirehigh bandwidth,register-mappedcommunicationservesto avoid costlymemoryaccesses.Also,autonomousstreamingcom-ponentscanmanagetheir localstatein Raw’sdistributeddatacachesandregisterbanks,therebyimproving locality. Theseaspectsarekeyto thescalabilitydemonstratedin theStreamItbenchmarks.
a wide rangeof streambasedapplicationsto take advantageof Rawasanembeddedprocessor. This sectionpresentstheresults.Theseincludeasetof linearalgebraroutinesimplementedasStreamAlgo-rithms,theSTREAMbenchmark,andseveralotherembeddedappli-cationsincludinga real-time1020-nodeacousticbeamformer. Thebenchmarksaretypically written in C andcompiledwith gcc,with
inline assemblyfor a subsetof inner loops. Someof the simplerbenchmarkslike the STREAM benchmarkandthe FIR weresmallenoughthatcodingentirelyin assemblywasmostexpedient.
StreamAlgorithms Table13 presentstheperformanceof a setoflinearalgebraalgorithmson RawPCversustheP3.
MFlops SpeedupvsP3Benchmark ProblemSize on Raw Cycles Time
TheRaw implementationsarecodedasStreamAlgorithms[16],which emphasizecomputationalefficiency in spaceand time andaredesignedspecificallyto take advantageof tiled microarchitec-tureslike Raw. They have threekey features. First, streamalgo-rithms operatedirectly on datafrom the interconnectand achieveanasymptoticallyoptimal $&%'%(� computeefficiency for largenum-bersof tiles. Second,streamalgorithmsuseno morethana small,boundedamountof storageon eachprocessingelement. Third,dataarestreamedthroughthe computefabric from andto periph-eralmemories.
With the exceptionof Convolution, we compareagainstthe P3runningsingleprecisionLapack(LinearAlgebraPackage).We useclapackversion 3.0 [2] and a tuned BLAS implementation,AT-LAS [50], version3.4.2. We disassembledthe ATLAS library toverify that it usesP3 SSEextensionsappropriatelyto achieve highperformance.SinceLapackdoesnotprovideaconvolution,wecom-pareagainsttheIntel IntegratedPerformancePrimitives(IPP).
As can be seenin Table 13, Raw performssignificantly betterthantheP3on theseapplicationsevenwith optimizedP3SSEcode.Raw’s betterperformanceis dueto load/storeelimination(seeTa-ble 2), andthe useof parallelresources.StreamAlgorithms oper-atedirectly on valuesfrom thenetwork andavoid loadsandstores,therebyachieving higher utilization of parallel resourcesthan theblockedcodeon theP3.
STREAM benchmark TheSTREAM benchmarkwascreatedbyJohnMcCalpin to measuresustainablememorybandwidthandthecorrespondingcomputationrate for vector kernels [30]. Its per-formancehasbeendocumentedon thousandsof machines,rangingfrom PCsanddesktopsto MPPsandothersupercomputers.
Bandwidth (GB/s)ProblemSize P3 Raw NEC SX-7 Raw/P3
Table14: Performance(by time) of STREAM benchmark.
We hand-codedanimplementationof STREAM on RawStreams.We alsotweaked the P3 versionto usesingleprecisionSSEfloat-ing point, improving its performance.TheRaw implementationem-ploys 14 tiles andstreamsdatabetween14 processorsand14 mem-ory portsthroughthe staticnetwork. Table14 displaysthe results.Raw is 55x-92xbetterthantheP3. Thetablealsoincludestheper-formanceof STREAMonNECSX-7Supercomputer, whichhasthehighestreportedSTREAM performanceof any single-chipproces-sor. NotethatRaw surpassesthatperformance.Thisextremesingle-chip performanceis achievedby takingadvantageof threeRaw ar-chitecturalfeatures:its amplepin bandwidth,theability to preciselyroutedatavaluesin andoutof DRAMs with minimal overhead,anda carefulmatchbetweenfloatingpointandDRAM bandwidth.
Other stream-basedapplications Table 15 presentsthe perfor-mance� of somehandwrittenstreamapplicationsonRaw. Wearede-velopinga real time 1020microphoneAcousticBeamformerwhichwill usethe Raw systemfor processing.On this application,Rawruns16 instantiationsof thecodeandthemicrophonesarestripedina dataparallelmanneracrossthearray. Raw’s softwareexposedI/Ois alsomuchmoreefficient thangettingthestreamdatafrom DRAMin thecaseof theP3. Inputtingandoutputtingdatafrom DRAM isthebestcasefor theP3. TheP3resultswould bemuchworsein anactualsystemwherethe datawould comeover a PCI bus. For theFIR, we comparedto the Intel IPP. Resultsfor CornerTurn, BeamSteering,andCSLCarediscussedin thepreviously published[41].
Machine Cycles SpeedupvsP3Benchmark Config. on Raw Cycles Time
Table15: Performanceof hand written streamapplications.
4.5 ServerTo measuretheperformanceof Raw onserver-likeworkloads,we
conductthe following experimenton RawPC to obtain SpecRate-like metrics. For eachof a subsetof Spec2000 applications,weexecutean independentcopy of it on eachof the 16 tiles, andwemeasuretheoverall throughputof thatworkloadrelative to a singlerun on theP3.
Table16presentstheresults.Notethatthespeedupof Raw versusP3is equivalentto thethroughputof Raw relativeto P3’sthroughput.As anticipated,RawPCoutperformstheP3by a largemargin, withanaveragethroughputadvantageof 10.8x(by cycles)and7.6x (bytime). Thekey Raw featurethatenablesthisperformanceis thehighpin bandwidthavailableto off-chip memory. RawPCcontainseightseparatememoryportsto DRAM. Thismeansthatevenwhenall 16tilesarerunningapplications,eachmemoryportandDRAM is onlysharedamongtwo applications.
Cycles SpeedupvsP3Benchmark on Raw Cycles Time Efficiency
Table 16: Performanceof Raw on server workloads relative to the P3.
Table 16 shows the efficiency of RawPC’s memorysystemforeachserver workload. Efficiency is the ratio betweenthe actualthroughputand the ideal 16x speedupattainableon 16 tiles. Lessthantheidealthroughputis achievedbecauseof interferenceamongmemoryrequestsoriginatingfrom tiles thatsharethesameDRAMbanksand ports. We seethat the efficiency is high acrossall theworkloads,with anaverageof 93%.
4.6 Bit-Level ComputationWe measurethe performanceof RawStreamson two bit-level
computations[49]. Table17 presentsthe resultsfor the P3, Raw,FPGA, and ASIC implementations. We comparewith a Xilinx
Table 17: Performance of two bit-level applications: 802.11aConvolu-tional Encoder and 8b/10bEncoder. The hand codedRaw implementa-tions arecompared to referencesequentialimplementationson the P3.Virtex-II 3000-5FPGA,which is built on the sameprocessgener-ationastheRaw chip,andfor theASIC implementationswesynthe-sizeto theIBM SA-27EprocessthattheRaw chipis implementedin.For eachbenchmark,we presentthreeproblemsizes:1024,16384,and65536samples.Theseproblemsizesareselectedto fit in theL1, L2, and miss in the cacheon the P3, respectively. We usearandomizedinputsequencein all cases.
Onthesetwo applications,Raw is ableto excelby exploiting fine-grain pipelineparallelism. To do this, the computationswerespa-tially mappedacrossmultiple tiles. Both applicationsbenefitedbymorethan2x from Raw’sspecializedbit-level manipulationinstruc-tions,which reducethe latency of critical feedbackloops. Anotherfactorin Raw’shighperformanceon theseapplicationsis Raw’sex-posedstreamingI/O. This I/O modelis in sharpcontrastto havingto move datathoughthecachehierarchyona P3.
Wealsopresentin Table18resultsfor theoperationon16parallelinput streams.This is to simulatea possibleworkloadthata base-stationcommunicationschip mayneedto completeby encoding16simultaneousconnections.For this throughputtest,a moreareaef-ficient implementationwasusedon Raw. This implementationhaslower peakperformance,but by instantiating16 instances,a higherthroughputperareais achieved.
Cycles SpeedupvsP3Benchmark ProblemSize on Raw Cycles Time
Table 18: Performance of two bit-level applications for 16 streams:802.11aConvolutional Encoder and 8b/10bEncoder. This test simulatesa possibleworkload for a base-stationwhich processesmultiple commu-nication streams.
5. ANALYSISSections4.3 through4.6 presentedperformanceresultsfor Raw
for several applicationclassesandshowed thatRaw’s performancewas within a factor of 2x of the P3 for low-ILP applications,2x-9x betterthantheP3 for high-ILP applications,and10-100xbetterfor streamor embeddedcomputations. Table 19 summarizestheprimaryfeaturesthatareresponsiblefor performanceimprovementsonRaw.
In this section,we comparetheperformanceof Raw to otherma-chinesthathave beendesignedspecificallywith streamsor embed-dedcomputationin mind. We alsoattemptto explorequantitativelythedegreeto whichRaw succeedsin beingamoreversatilegeneral-purposeprocessor. To do so,we selecteda representative subsetofapplicationsfrom eachof our computationalclasses,andobtainedperformanceresultsfor Raw, P3andmachinesespeciallysuitedforeachof thoseapplications.Wenotethattheseresultsareexploratoryin natureandnot meantto be taken asany sort of proof of Raw’sversatility, ratherasanearlyindicationof thepossibilities.
Best-in-class envelope) Raw* P3+ NEC SX-7, VIRAM- Imagine. 16-P3 server farm/ FPGA0 ASIC
Figure 3: Performanceof various architecturesover several applications classes.Each point in the graph representsthe speedupof an architectureover the P3 for a given application. The best-in-classenvelopeconnectsthe bestspeedupsfor eachapplication. The dashedline connectsthe speedupsof Raw for all the applications. A versatilearchitecturehasspeedupscloseto the best-in-classenvelopefor all application classes.Imagine and VIRAMresultsareobtained fr om [41] and [34]. Bit-level resultsfor FPGA and ASIC implementationsare obtained fr om [49].
Figure3 summarizestheseresults. We canmake several obser-vationsfrom thefigure. First, it is easyto seethat theP3doeswellrelative to Raw for applicationswith low degreesof ILP, while theoppositeis truefor applicationswith higherdegreesof ILP, suchasVpenta. For streamsandvectors,the performanceof Raw is com-parableto that of streamandvectorarchitectureslike VIRAM andImagine. All threeoutperformthe P3 by factorsof 10x to 100x.Raw, usingtheRawStreamsconfiguration,beatsthehighestreportedsingle-chipSTREAM memorybandwidthchampion,theNEC SX-7 Supercomputer, and is 55x-92x betterthan the P3. EssentialtoRaw’s performanceon this benchmarkis theamplepin bandwidth,theability to preciselyroutedatavaluesin andout of DRAM withminimal overhead,anda carefulmatchbetweenfloating point andDRAM bandwidth.
Stream:StreamAlgo. Mxm, LU fact.,Triang.solver, QR fact.,Conv. X X XStream:STREAM Copy, Scale,Add, Scale& Add X XStream:Other Acoustic Beamforming,FIR, FFT,
BeamSteeringX X X
CornerTurn X XCSLC X X
Server 172.mgrid, 173.applu, 177.mesa,183.equake,188.ammp,301.apsi,175.vpr,181.mcf,197.parser, 256.bzip2,300.twolf
X X
Bit-Level 802.11aConvEnc,8b/10bEncoder X X X
Table 19: Raw feature utilization table. S = Specialization.R = Exploit-ing Parallel Resources.W = Managementof Wire Delays.P = Manage-ment of Pins.
We chosea server farm with 16 P3sasour best-in-classserversystem.Noticethata single-chipRaw systemcomeswithin a factorof threeof this server farm for most applications. (Note that thisis only a pureperformancecomparison,andwe have not attemptedto normalizefor cost.) We choseFPGAsandASICs asthebestinclassfor our embeddedbit-level applications.Raw’s performanceis comparableto that of an FPGA for theseapplications,and is afactorof 2x to 3x off from anASIC. (Again, notethatwe areonlycomparingperformance– ASICs usesignificantly lower areaand
power than Raw [49].) Raw performswell on theseapplicationsfor thesamereasonsthatFPGAsandASICsdo – namely, a carefulorchestrationof thewiring or communicationpatterns.
Thus far, our analysisof Raw’s flexibility hasbeenqualitative.Giventherecentinterestin flexible, versatileor polymorphicarchi-tecturessuchasTarantula[9], Scale[22], Grid [33], andSmartMem-ories[28], which attemptto performwell over a wider rangeof ap-plicationsthanextantgeneralpurposeprocessors,it is intriguing tosearchfor a metric that can capturethe notion of versatility. Wewould like to offer up a candidateanduseit to evaluatetheversatil-ity of Raw quantitatively. In a mannersimilar to thecomputationofSpecRates,wedefinetheversatility of a machineM asthegeometricmeanover all applicationsof theratio of machineM’s speedupfora givenapplicationrelativeto the speedupof the bestmachine forthatapplication.7
For the applicationsetgraphedin Figure3, Raw’s versatility is0.72,while that of the P3 is 0.14. The P3’s relatively poor perfor-manceon streambenchmarkshurtsits versatility. Although Raw’s0.72 numberis relatively good, even our small sampleof appli-cationshighlights two clear areaswhich merit additionalwork inthe designof polymorphicprocessors.One is for embeddedbit-level designs,whereASICsperform2x-3x betterthanRaw for oursmall applicationset. Certainly thereare countlessother applica-tionsfor whichASICsoutstripRaw by muchhigherfactors.Perhapsthe additionof small amountsof bit-level programmablelogic a laPipeRench[10] or Garp[14] canbridgethegap.
Computationwith low levels of ILP is anotherareafor furtherresearch.We will refer to Figure4 to discussthis in moredetail.Thefigure plots thespeedups(in cycles)of Raw anda P3 with re-spectto executionon a singleRaw tile. Theapplicationsarelistedon thex-axisandsortedroughly in theorderof increasingILP. Thefigure indicatesthat Raw is able to convert ILP into performancewhenILP exists in reasonablequantities.This indicatesthe scala-
>Sincewe are taking ratios, the individual machinespeedupscan be computedrela-
tive to any onemachine,sincetheeffect of thatmachinecancelsout. Accordingly, thespeedupsin Figure3 areexpressedrelative to theP3without lossof generality. Further,like SpecRates’ratherarbitrarychoiceof theSunUltra5 asa normalizingmachine,thenotion of versatility canbe generalizedto future machinesby choosingequallyarbi-trarily thebest-in-classmachinesgraphedin Figure3 asour referencesetfor all time.Thus,sincethebest-in-classmachinesarefixed,theversatilitiesof futuremachinescanbecomegreaterthan1.0.
bility of Raw’s scalaroperandnetwork. The performanceof Rawis lower� thanthe P3 by about33 percentfor applicationswith lowdegreesof ILP for several reasons.First, the threeleftmostappli-cationsin Figure4 wererun on a singletile. We hopeto continuetuning our compiler infrastructureanddo betteron someof theseapplications.Second,a near-term commercialimplementationof aRaw-like processormight likely usea two-way superscalarin placeof our single-issueprocessorwhich would beableto matchtheP3for integerapplicationswith low ILP. (See[31] for detailson grain-sizetradeoffs in Raw processors).
Figure4: Speedup(in cycles)achievedby Raw and the P3over executingon a singleRaw tile.
6. RELATED WORKRaw distinguishesitself from othersby beinga modelessarchi-
tectureandsupportingall formsof parallelism,includingILP, DLP,TLP andstreams.Several projectshave attemptedto exploit spe-cific forms of parallelism. Theseinclude systolic (iWarp [12]),vector (VIRAM [21]), stream (Imagine [17]), shared-memory(DASH [26]), and messagepassing(J machine[35]). Thesema-chines,however, werenot designedfor ILP. In contrast,Raw wasdesignedto exploit ILP effectively in additionto theseotherformsof parallelism. ILP presentsa hard challengefor thesemachinesbecauseit requiresthat the architecturebe able to transportscalaroperandsbetweenlogic unitswith very low latency, evenwhentherearea large numberof highly irregular communicationpatterns.Arecentpaper, [45], employs a 5-tuple to characterizethe cost ofsendingoperandsbetweenfunction units in a numberof architec-tures.Table7 lists thecomponentsof this 5-tuplein order. Qualita-tively, larger5-tuplevaluesrepresentproportionallymoreexpensiveoperandtransportcosts.Thelargevaluesin thenetwork 5-tuplesforiWarp JK$MLONPLRQMLS%TLR$VU , sharedmemory JW$PLO$'XPLRYMLR$&ZTL[$\U , andmessagepassingJ^]PLR_MLO$PLR$MLO$MY�U , comparedto the low numbersin the 5-tuplesof machinesthat can exploit ILP, e.g., superscalarJ`%aLS%TLb%aLS%TLb%cU , Raw J`%TLO$MLR$PLO$MLb%cU , Grid Jd%aLS%TLR$Me'YPLb%aLb%cU ,and ILDP Jf%TLO$MLb%TLR$MLS%�U quantitatively demonstratethe differ-ence. The low 5-tupleof Raw’s scalaroperandnetwork comparedto thatof iWarpenablesRaw to exploit diverseformsof parallelism,andis adirectconsequenceof theintegrationof theinterconnectintoRaw’s pipelineandRaw’searlypipelinecommitpoint. Wewill fur-therdiscussthecomparisonwith iWarphere,but see[45] for moredetailsoncomparingnetworksfor ILP.
Raw supportsstatically orchestratedcommunicationlike iWarpor NuMesh[39]. iWarp and NuMeshsupporta small numberoffixed communicationpatterns,and can switch betweenthesepat-ternsquickly. However, establishinga new patternis moreexpen-sive. Raw supportsstaticallyorchestratedcommunicationby usinga programmableswitchwhich issuesaninstructioneachcycle. Theinstructionspecifiestheroutesthroughtheswitchduringthatcycle.
Becausetheswitchprogrammemoryin Raw is large,andvirtualizedthroughcaching,thereis nopracticalarchitecturallimit on thenum-ber of simultaneouscommunicationpatternsthat canbe supportedin a computation. This virtualizationbecomesparticularly impor-tantfor supportingILP, becauseswitchprogramsbecomeaslargeorevenlargerthanthecomputeprograms.
Processorslike Grid [33] and ILDP [18] are targetedspecifi-cally for ILP and proposeto use low latency scalaroperandnet-works.Raw sharesin their ILP philosophy, andimplementsastatic-transport,point-to-pointscalaroperandnetwork, while Grid usesadynamic-transport,point-to-pointnetwork, andILDP usesa broad-cast baseddynamic-transportnetwork. Both Raw and Grid per-form compiletime instructionassignmentto computenodes,whileILDP usesdynamicassignmentof instructiongroups. Raw usescompile-timeoperandmatching,while Grid usesdynamicassocia-tive operandmatchingqueues,and ILDP’s dynamicschemeusesfull-empty bits on distributedregisterfiles. Accordingly, usingtheAsTrO categorization(Assignment,Transport,Ordering)from [46],Raw, Grid andILDP canbeclassifiedasSSS,SDD,andDDD archi-tecturesrespectively, whereS standsfor staticandD for dynamic.BoththeGrid andILDP designsprojectlowernetwork 5-tuplesthanRaw, but the final numbersshouldbe forthcomingas their imple-mentationsmature.Taken together, Grid, ILDP andRaw representthree distinct points in the scalaroperandnetwork designspace,rangingfrom the morecompile-timeorientedapproachasin Raw,to thedynamicapproachasin ILDP.
Raw took inspirationfrom theMultiscalarprocessor[40], whichusesa separateone-dimensionalnetwork to forward registervaluesbetweenALUs. Raw generalizesthebasicidea,andsupportsa two-dimensionalprogrammablemeshnetwork bothto forwardoperandsandfor otherformsof communication.
BothRaw andSmartMemories[28] sharethephilosophyof anex-posedcommunicationarchitecture,andrepresenttwo designpointsin the spaceof tiled architecturesthat can supportmultiple formsof parallelism. Raw useshomogeneous,programmablestatic anddynamicmeshnetworks,while SmartMemoriesusesprogrammablestatic communicationwithin a local collection of nodes, and adynamic network betweenthesecollectionsof nodes. The nodegranularitiesare also different in the two machines. Perhapsthemostsignificantarchitecturaldifference,however, is thatRaw (likeScale[22]) is modeless, while SmartMemoriesandGrid havemodesfor differentapplicationdomains. Anotherarchitecturethat repre-sentsa naturalextremepoint in modesis Tarantula[9], which im-plementstwo distinct typesof processingunits for ILP andvectors.Raw’sresearchfocusis ondiscoveringandimplementingaminimalsetof primitivemechanisms(e.g.,scalaroperandnetwork) usefulforall formsof parallelism,while themodesapproachimplementsspe-cial mechanismsfor eachform of parallelism.We believe themod-elessapproachis moreareaefficient andsignificantlylesscomplex.Lookedatanotherway, for a givenarea,machineswith modesmustdemonstratequantitatively betterversatilitynumbersthanmodelessmachinesto justify their increasedcomplexity. Much like for GUIsin the late 70’s, we believe the issueof modesversusmodelessforversatileprocessorsis likely to bea controversialtopic of debateintheforthcomingyears.
Finally, like VIRAM and Imagine, Raw supportsvector andstreamcomputations,but doesso very differently. Both VIRAMand Imaginesport large memoriesor streamregister files on onesideof the chip connectedvia a crossbarinterconnectto multiple,deepcomputepipelineson the other. The computationalmodel isonethatextractsdatastreamsfrom memory, pipesthemthroughthecomputepipelines,andthendepositsthembackin memory. In con-trast,Raw implementsmany co-locatedsmallermemoriesandcom-
puteelements,interconnectedby a meshnetwork. The Raw com-putationalg modelis moreASIC-like in that it streamsdatathroughthe pins andon-chip network to the ALUs, continuesthroughthenetwork to moreALUs, andfinally throughthenetwork to thepins.Raw’s ALUs alsocanstoredatatemporarilyin the local memoriesif necessary. Webelievethelower latenciesof thememoriesin Raw,togetherwith the tight integrationof the on-chipnetwork with thecomputepipelines,make Raw moresuitablefor ILP.
7. CONCLUSIONThis paperdescribesthe architectureand implementationof the
Raw microprocessor. Raw’s exposedISA allows parallel applica-tions to exploit all of the chip resources,including gates,wiresandpins. Raw supportsILP by schedulingoperandsover a scalaroperandnetwork that offers very low latency for scalardatatrans-port. Raw’s compilermanagestheeffect of wire delaysby orches-trating both scalarand streamdatatransport. The Raw processordemonstratesthat existing architecturalabstractionslike interrupts,caches,andcontext-switchingcancontinueto be supportedin thisenvironment,evenasapplicationstake advantageof thelow-latencyscalaroperandnetwork andthelargenumberof ALUs.
Our resultsdemonstratethat the Raw processorperformsat orcloseto the level of the bestspecializedmachinefor eachapplica-tion class. Whencomparedto a PentiumIII, Raw displaysonetotwo ordersof magnitudemoreperformancefor streamapplications,while performingwithin a factorof two for low-ILP applications.Itis our hopethat theRaw researchwill provide insight for architectswho are looking for ways to build versatileprocessorsthat lever-agethevastsiliconresourceswhile mitigatingtheconsiderablewiredelaysthatloomon thehorizon.
Acknowledgments We thankour StreamItcollaborators,specif-ically M. Gordon,J. Lin, and B. Thies for the StreamItbackendand the correspondingsectionof this paper. We are grateful toour collaboratorsfrom ISI East including C. Chen,S. Crago,M.French,L. WangandJ. Suhfor developing the Raw motherboard,firmwarecomponents,andseveralapplications.T. Konstantakopou-los, L. Jakab,F. Ghodrat,M. Seneski,A. Saraswat, R. Barua,A.Ma, J. Babb,M. Stephenson,S. Larsen,V. Sarkar, andseveral oth-erstoo numerousto list alsocontributedto thesuccessof Raw. TheRaw chipwasfabricatedin cooperationwith IBM. Raw is fundedbyDarpa,NSF, ITRI andtheOxygenAlliance.
REFERENCES[1] V. Agarwal, et al. Clock RateversusIPC: TheEndof theRoadfor Conventional
Microarchitectures.2000ISCA, pp.248–259.[2] E. Anderson,et al. LAPACK: A PortableLinear Algebra Library for High-
PerformanceComputers.1990ICS, pp.2–11.[3] M. Annaratone,etal.TheWarpComputer:Architecture,ImplementationandPer-
[21] C. Kozyrakis,et al. A New Directionfor ComputerArchitectureResearch.IEEEComputer30, 9 (September1997),pp.24–32.
[22] R. Krashinsky, etal. TheVector-ThreadArchitecture.2004ISCA.[23] J.Kubiatowicz. IntegratedShared-MemoryandMessage-PassingCommunication
in theAlewifeMultiprocessor. PhDthesis,MIT, 1998.[24] W. Lee,et al. Space-Time Schedulingof Instruction-Level Parallelismon a Raw
Machine.1998ASPLOS, pp.46–54.[25] W. Lee,etal. ConvergentScheduling.2002MICRO, pp.111–122.[26] D. Lenoski, et al. The StanfordDASH Multiprocessor. IEEE Computer25, 3
MixedSignal& eDRAM applications.1999IEDM, pp.849–852.[28] K. Mai, et al. SmartMemories: A Modular ReconfigurableArchitecture.2000
ISCA, pp.161–171.[29] D. Matzke. Will PhysicalScalabilitySabotagePerformanceGains? IEEE Com-
puter30, 9 (September1997),pp.37–39.[30] J. McCalpin.STREAM: SustainableMemoryBandwidthin High Perf.Comput-
ers.http://www.cs.virginia.edu/stream.[31] C. A. Moritz, et al. SimpleFit: A Framework for AnalyzingDesignTradeoffs in
Raw Architectures.IEEE Transactionson Parallel andDistributedSystems(July2001),pp.730–742.
[32] S. Naffziger, et al. The Implementationof the Next-Generation64b ItaniumMi-croprocessor. 2002ISSCC, pp.344–345,472.
[33] R. Nagarajan,et al. A DesignSpaceEvaluationof Grid ProcessorArchitectures.2001MICRO, pp.40–51.
[34] M. Narayanan,et al. GeneratingPermutationInstructionsfrom a High-Level De-scription.TR UCB-CS-03-1287,UC Berkeley, 2003.
[35] M. Noakes, et al. The J-MachineMulticomputer: An ArchitecturalEvaluation.1993ISCA, pp.224–235.
[36] S. Palacharla.Complexity-EffectiveSuperscalar Processors. PhD thesis,Univer-sity of Wisconsin–Madison,1998.
[37] N. Rovedo,et al. IntroducingIBM’ s First CopperWiring FoundryTechnology:Design,Development,andQualificationof CMOS7SF, a .18micronDual-OxideTechnologyfor SRAM,ASICs,andEmbeddedDRAM. Q42000IBM MicroNews,pp.34–38.
[38] J.Sanchez,etal. ModuloSchedulingfor a Fully-DistributedClusteredVLIW Ar-chitecture.2000MICRO, pp.124–133.
[39] D. Shoemaker, et al. NuMesh: An ArchitectureOptimizedfor ScheduledCom-munication.Journalof Supercomputing10, 3 (1996),pp.285–302.
[40] G. Sohi,et al. MultiscalarProcessors.1995ISCA, pp.414–425.[41] J.Suh,et al. A PerformanceAnalysisof PIM, StreamProcessing,andTiled Pro-
cessingon Memory-Intensive Signal ProcessingKernels.2003 ISCA, pp. 410–419.
[42] M. B. Taylor. Deionizer: A Tool For Capturing And Embedding I/OCalls. TechnicalMemo, CSAIL/Laboratoryfor ComputerScience,MIT, 2004.http://cag.csail.mit.edu/� mtaylor/deionizer.html.
[43] M. B. Taylor. The Raw Processor Specification. Technical Memo,CSAIL/Laboratoryfor ComputerScience,MIT, 2004.
[44] M. B. Taylor, et al. TheRaw Microprocessor:A ComputationalFabric for Soft-wareCircuitsandGeneral-PurposePrograms.IEEEMicro (Mar 2002),pp.25–35.
[45] M. B. Taylor, et al. ScalarOperandNetworks: On-Chip Interconnectfor ILP inPartitionedArchitectures.2003HPCA, pp.341–353.
[46] M. B. Taylor, etal. ScalarOperandNetworks:Design,Implementation,andAnal-ysis.TechnicalMemo,CSAIL/LCS,MIT, 2004.
[47] W. Thies,etal.StreamIt:A Languagefor StreamingApplications.2002CompilerConstruction, pp.179–196.
[48] E. Waingold,et al. BaringIt All to Software:Raw Machines.IEEE Computer30,9 (September1997),pp.86–93.
[49] D. Wentzlaff. ArchitecturalImplicationsof Bit-level Computationin Communi-cationApplications.Master’s thesis,LCS,MIT, 2002.
