Testing of the Cache Memory in the

OpenSPARC T1

Matteo Ainardi - 147695Matteo Bosio - 148451

Salvatore Campione - 145781Vittorio Giovara - 149374Alberto Grand - 149389

June 20, 2008

Contents

1 Introduction 2

2 Synthesize the OpenSPARC processor T1 32.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 First approach: Basic processor in Modelsim . . . . . . . . . . . 42.3 Second approach: Full processor in Synopsys . . . . . . . . . . . 4

3 The Sparc Cross Compiler 63.1 What is the Cross Compilation . . . . . . . . . . . . . . . . . . . 63.2 The gcc-sparc-linux compiler . . . . . . . . . . . . . . . . . . . 6

3.2.1 Build explanation . . . . . . . . . . . . . . . . . . . . . . 63.2.2 The building process . . . . . . . . . . . . . . . . . . . . . 7

3.3 Environment Loading . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Building blocks of the OpenSPARC T1 core 10

Bibliography 12

1

Chapter 1

Introduction

Our team was assigned to generate a working testbench for evaluation andtesting purpose of the memory cache of the OpenSPARC T1.

The OpenSPARC T1 is a completely open source processor, produced andsupported by SUN. Because of the availability of the source code, our plan wasto synthesize the core processor and perform a simulation of the cache usingspecial instruction sequence obtained by the use of a Sparc cross compiler; inorder to understand the correct sequence of instructions a building block schemeof the processor was to be realized.

Unfortunately we encountered many difficulties during the development ofthe plan as the task proved excessively difficult, especially the synthesis part,which has not been fully carried out.

In this document we outline the final status of the project, the step we havefollowed and the results of the operations.

2

Chapter 2

Synthesize the OpenSPARCprocessor T1

2.1 Overview

The OpenSPARC material (which comprises documentation, verilog sources andtestbenches) can be downloaded freely from the web at [1].

The processor can be synthesized in two different ways: core1 configurationand chip8 configuation.

The core1 environment consists of:

1. One SPARC CPU core

2. Cache

3. Memory

4. Crossbar

The chip8 environment consists of:

1. A full OpenSPARC T1 chip, including all eight cores

2. Cache

3. Memory

4. Crossbar

5. I/O subsystem

Each of the previous configurations can be obtained by synthesizing differentverilog files of the project.

3

2.2 First approach: Basic processor in Model-sim

The first expected result consists of obtaining a complete functional version ofthe processor in its basic functions; for this reason, we tried to compile andsynthesize a minimum configuration set of the processor using ModelSIM. Aftercreating a new project in ModelSIM and inserting all the files (both the Verilogfiles and the needed libraries) inside it, we started with a syntax check on thecode, preparing for the simulation.

Unfortunately, a lot of errors appeared. We recognized some errors were dueto missing includes and so we tried to fix them; however, after overcoming thesyntax errors, we faced major problems in the simulation part.

When trying to simulate we had to cope with technological library problems(i.e. no hardware basic blocks matched the requested) since the code was writtenfor synopsys, as we found out after thorough inspection.

Since we noticed that we wouldn’t have reached a complete simulation, wedecided to change the configuration environment.

2.3 Second approach: Full processor in Synop-sys

First of all, we logged on the Politecnico server used for Synopsys synthesisat mordor.polito.it To start the synthesis process, we first had to set theenvironment variables for the machine we were using. Script.sh is shownbelow:

#!/usr/bin/bash

export DV_ROOT="/cadtools/opensparc"export MODEL_DIR="/cadtools/opensparc/model"export CC_BIN="/usr/dist/pkgs/sunstudio_‘uname -p‘/SUNWspro/bin"export SYN_HOME="/cadtools/synopsys/2002.05-SP1";export LM_LICENSE_FILE="/import/EDAtools/licenses/synopsys_key:

/import/EDAtools/licenses/ncverilog_key";export TRE_ENTRY="/";export TRE_LOG="nobody";export TRE_SEARCH="$DV_ROOT/tools/env/tools.iver";export ENVDIR="$DV_ROOT/tools/env";export PERL_MODULE_BASE="$DV_ROOT/tools/perlmod";export SYN_LIB="$SYN_HOME/libraries/syn";export SYN_BIN="$SYN_HOME/sparcOS5/syn/bin";export PERL_VER="5.8.7";export PERL_PATH="$DV_ROOT/tools/perl-$PERL_VER";export PERL5_PATH="$DV_ROOT/tools/perl-$PERL_VER/lib/perl5";export PERL_CMD="$PERL_PATH/bin/perl";export PATH="$DV_ROOT/tools/bin:$NCV_HOME/tools/bin:$VCS_HOME/bin

:$VERA_HOME/bin:$SYN_BIN/:$CC_BIN/:$PATH";

4

The synthesis process was started by the command rsyn -all; we had todisable the server connection timeout, since it took several hours to accomplishthe task (about 23 hours).

In the end, no error messages appeared, however, when we verified thesynthesis output, only some blocks were successfully synthesized, while othersshowed some errors.

The output of the synthesis is located in the following path (of the server)/cadtools/opensparc/design/sys/iop; it is possible to recognise the syn-thetized modules by looking in the synoptis/gate directory which containsthe physical descritption of the path gate.

At this point we decided (together with the teacher assistant) to give upsince we had no idea how to continue the workflow.

5

Chapter 3

The Sparc Cross Compiler

3.1 What is the Cross Compilation

Perfoming Cross Compilation is the technique for creating executable code fora platform different from the platform the code is generated on; the softwarethat performs cross compilation is called Cross Compiler.

Hereinafter the machine on which the compiler is located will be called host,the machine on which the code will run target.

This special compiler must be compiled with specific instruction on the targetarchitecture; moreover, the standard libraries must already be available both forthe target and host machine. When building a cross compiler it is intended thatthe whole suite of programs are built, including the assembler, the binutils1 thecore compiler, the language extensions (optional) and the kernel environment(of the host platform).

3.2 The gcc-sparc-linux compiler

As the GNU GCC is one of the most diffused compilers, with extremely gooddocumentation, not counting being open source, it has been chosen as startingpoint to obtain the SPARC cross compiler. The host machine run GentooLinux since it is one of the Linux distrubutions most suited for compilation anddevelopment.

Compiling the GCC is rather difficult, and its cross versions are none easier:however with some command line tuning it is possibile to sequencially build allthe needed parts.

3.2.1 Build explanation

As a clean build of the compiler is require, the sysroot approach is used: theGCC will consider the source directory ad the root of a tree that contains a(subset of) the root filesystem of the target operating system, requiring thattarget system headers, libraries and run-time objects are placed there.

It is important to note that both the GCC and the libraries must be compiledtwice, particially and completely, in two different occasions. This is because of

1binutils are a set of tools for creating, analyising and using custom libraries

6

dependecies among the GCC toolchain and the libraries that need to be satisfiedbefore attempting any build.

Having said that, in order to obtain a working cross compiler, it is necessaryto build the following software in this order:

1. Binutils

2. Kernel headers

3. Glibc headers

4. GCC stage1

5. Glibc

6. GCC stage2

Gcc stage1 includes only the C language while stage2 comprises all the otherlanguages (C/C++/Java/ADA etc). For a correct configuration the compilerused for building the cross compiler must support the -mlong-double-128 flag(obtained by configuring GCC with --with-long-double-128).

3.2.2 The building process

Before beginning, it is important to setup two environment variables, that willfaciliate code reproduction in the following steps; the first one set the targetarchitecture for the programas, the second one is for the kernel architecture.

$ export CTARGET=sparc-linux$ export ARCH=sparc

First of all, binutils must be configured, built and installed, taking care ofremoving unnecessary libraries.

$ cd /path_to_binutils_source/$ ./configure --target=$CTARGET --prefix=/usr --with-sysroot=/usr/$CTARGET$ make$ make install DESTDIR=$PWD/install-root$ rm -rf install-root/usr/{info,lib,man,share}# cp -a install-root/* /

Then compile the kernel, even though it will result in an error, as the gcccross compiler isn’t ready yet, but what is really needed are the kernel headers.

$ cd /usr/src/linux$ yes "" | make ARCH=$ARCH oldconfig prepare# mkdir -p /usr/$CTARGET/usr/include# cp -a include/linux include/asm-generic /usr/$CTARGET/usr/include# cp -a include/asm-$ARCH /usr/$CTARGET/usr/include/asm

The GNU C Libraries (Glibc) suffers from the same problem as above, butyet again only the headers are needed at this time.

7

$ cd /path_to:glibc_source/$ mkdir build$ cd build$ ../configure --host=$CTARGET --prefix=/usr --with-headers= \

/usr/$CTARGET/usr/include --disable-sanity-checks# make -k install-headers install_root=/usr/$CTARGET# mkdir -p /usr/$CTARGET/usr/include/gnu# touch /usr/$CTARGET/usr/include/gnu/stubs.h# cp bits/stdio_lim.h /usr/$CTARGET/usr/include/bits/

Now it is possibile to build the core of the GCC compiler (stage1), linkingthe libraries for the target architecture and removing the unnecessary ones.

cd /path_to_gcc_source/# ln -s usr/include /usr/sparc/sys-include$ mkdir build$ cd build$ ../configure --target=sparc --prefix=/usr --with-sysroot=/usr/sparc \

--enable-languages=c --disable-shared --disable-checking \--disable-werror --disable-libmudflap --disable-libssp

$ make$ make install DESTDIR=$PWD/install-root$ rm -rf install-root/usr/{info,include,lib/libiberty.a,man,share}# cp -a install-root/* /

After this step, the C libraries can be completely compile,

$ rm -rf build$ mkdir build$ cd build$ ../configure --host=$CTARGET --prefix=/usr$ make# make install install_root=/usr/$CTARGET

as well as the GCC, including any other language (stage2).

$ ./configure --target=$CTARGET --prefix=/usr --with-sysroot=/usr/$CTARGET \--enable-languages=c,c++ --enable-shared --disable-checking \--disable-werror

$ make# make install

At this point, if no error occurred, the cross compiler can be invoked fromcommand line with gcc-sparc-linux.

3.3 Environment Loading

One final section of code should be produced, separately from the compiler:the loader, which takes care of storing the generated code into the instructionmemory which subsequentially loads it in the processor registers, the ArithmeticLogic Unit and the memory cache.

8

The loader can be implemented with any hardware description languagelike VHDL or Verilog, using a read from file function and converting data intobus values. However this procedure requires high level of details of the finalarchitecture, as well as the synthezised unit under test, for some importanttuning; since neither of these were available it has been not possible to writeany memory loader.

9

Chapter 4

Building blocks of theOpenSPARC T1 core

For the purposes of designing a test-bench for the cache memory of the OpenSPARCT1 processor, an in-depth knowledge of its architecture and of the buildingblocks it comprises is required. One of the teams in our group was in charge ofproducing a block scheme of the OpenSPARC core having such a level of detailas to permit a test design.

Our first reference was the OpenSPARC T1 Microarchitecture Specifica-tion, available on the OpenSPARC website. It provides a good overview of thearchitecture, as well as more detailed information about its implementation.Nonetheless, the description was at times a rather high-level and qualitativeone and only scratched the surface of some components, which was usually notenough for our goal.

As a consequence, we later decided to take a closer look at the Verilog sourcecode available at [4]. By means of the sidebar it is possible to navigate throughthe source code. Since we were only interested in the architecture of a singlecore, we started our inspection from the cmp top → OpenSPARC T1 → sparcmodule within the source browser hierarchy and we progressively analysed allthe lower-level source files.

The following criterion was adopted to choose only the most meaningfulsignals among the many:

• data buses were added to the block scheme if their width was relevant(usually larger than 6 or 7 bits) or if their presence was required for someother reason;

• control signals were usually not inserted in the block scheme;

• scan-in/-out signals for test purposes were included.

The block scheme we obtained is depicted in the following figure.

10

Figure 4.1: Building blocks of the OpenSPARC T1 core

11

Bibliography

[1] OpenSPARC Official Sitehttp://www.opensparc.net

[2] Wikipedia, the free encyclopedia, Cross compiler,http://en.wikipedia.org/wiki/Cross_compiler

[3] Mike “Vapier” Frysinger, Cross Compils Guts,http://dev.gentoo.org/~vapier/CROSS-COMPILE-GUTS

[4] OpenSPARC Verilog Hierarchyhttp://opensparc-t1.sunsource.net/nonav/source/verilog/html/verilog.html

12