Lecture 0: Introduction

EEN 312: Processors: Hardware, Software, and Interfacing

Department of Electrical and Computer Engineering

Fall 2012, Dr. Rozier (UM)

Welcome to EEN 312!

ROSE-E-AProfessor Eric Rozier

Who am I?

• BS in Computer Science from William and Mary

Who am I?

• BS in Computer Science from William and Mary

• Studied models of agricultural pests (flour beetles).

Who am I?

• BS in Computer Science from William and Mary

• Studied models of agricultural pests (flour beetles).

• And load balancing of super computers.

Who am I?

• First job – NASA Langley Research Center

Who am I?

• First job – NASA Langley Research Center

• Researched problems in aeroacoustics

Who am I?

• First job – NASA Langley Research Center

• Researched problems in aeroacoustics– Primarily on the XV-15

Who am I?

• First job – NASA Langley Research Center

• Researched problems in aeroacoustics– Primarily on the XV-15– Precursor to the better

known V-22

Who am I?

• PhD in CS/ECE from the University of Illinois

Who am I?

• PhD in CS/ECE from the University of Illinois

• Studied non-linear dynamics of transactivation networks in economically important species…

Who am I?

Who am I?

• PhD in CS/ECE from the University of Illinois

• Worked with the NCSA on problems in super computing, reliability, and big data.

Who am I?

• PhD in CS/ECE from the University of Illinois

• Worked with the NCSA on problems in super computing, reliability, and big data.

• Research led to patented advances with IBM

Who am I?

• Served as a visiting scientist and IBM Fellow at the IBM Almaden Research Center in San Jose, CA

• Helped advance state of the art in fault-tolerance, and our understanding of why systems fail

Who am I?

• Postdoctoral work at the Information Trust Institute– Worked on Blue Waters

Super Computer, first sustained Petaflop machine

– Designed new fault-tolerant methods for data protection on large-scale systems

Who am I?

• Assistant Professor at UM ECE

Who am I?

• Assistant Professor at UM ECE– Head of the Trustworthy

Systems Lab

Who am I?

• Assistant Professor at UM ECE– Head of the Trustworthy

Systems Lab– Working on problems in:

• Cloud computing• Big Data• Reliability• Security• Compliance

How to get in touch with me?

• Office– Department of Electrical and Computer Engineering– Fifth Floor, Room 517

• Contact Information– Email: e.rozier@miami.edu– Phone: 8-9752

• Currently looking for motivated students– Research projects and papers

Office Hours

• Office– Department of Electrical and Computer Engineering– Fifth Floor, Room 517

Day Hours

Tuesday 10:00a – 11:00a

Thursday 10:00a – 11:00a

Or by appointment

COURSE SUMMARY AND OVERVIEW

EEN 312

• Processors: Hardware, Software, and Interfacing– Class: MM 102– Lab: McArthur Engineering Building 402

• Class website– http://performalumni.org/erozier2/een312.html

The syllabus…

Grades

Grade Component Percentage

Midterm I 10%

Midterm II 10%

Laboratory Projects 50%

Final Exam 30%

Grades

• Guaranteed Grades

• A+’s are assigned on the basis of exceptional work, scoring 99 or 100 for the entire course.

Labs

• Labs are a HUGE component of this course– Lab sessions will be held based on the session you have

been assigned and registered for. – Labs for this class will be very demanding. It is unlikely you

will finish them during the assigned sessions. – You will need to make good use of your assigned

laboratory time to seek guidance from your TAs, but you should expect to spend significant time outside of lab working on your lab assignments.

Active Learning

• After 2 weeks we tend to remember:– Passive learning

• 10% of what we read• 20% of what we hear• 30% of what we see• 50% of what we hear and see

– Active learning• 70% of what we say• 90% of what we say and do

Bloom’s Taxonomy

EvaluationSynthesisAnalysis

ApplicationComprehension

Knowledge

Training Good Engineers

• Understanding processors isn’t our only goal– Critical Reading– Critical Reasoning

• Ask questions!• Think through problems!• Challenge assumptions!

312

312

304

118

Course overview

• Understanding the abstractions beneath your applications and programs.

• We will focus on:– How programs are translated into machine

language.– How hardware executes machine

instructions.– How computers are organized and

designed.

Course Components

• Class time– High level concepts– Hands on exercises and application– Discussions

• Labs– The heart of the course– 1-2 weeks each– Indepth exploration of an aspect of system design and organization

• Exams– 2 Midterms + 1 Final– Test your understanding of concepts and mathematics

Textbook

Textbook

• Be sure to get the 4th edition!• Available from the bookstore

– New: $89.95– Used: $67.50

• Available online– Softback: $61.98 (Amazon)– Kindle: $71.99– Kindle Rental: ~$35

• The textbook is essential for this course.

Laboratories

Laboratories

• TAs– Yilin Yan

• y.yan4@umiami.edu– Murat Aykin

• m.aykin@umiami.edu

• Lab Sections– Wednesday 2:30 – 4:50p– Friday 2:30 – 4:50p

Lab Procedure

• Labs will be completed in groups of 2-3.– You may complete labs as a group, but you must

each hand in a separate lab assignment.– You may change groups with each lab.

Raspberry Pi

Lab Pis

Lab Pis

• We have a set of 16 Raspberry Pis available for the class. Each group will be assigned one for each lab.– Don’t use an unassigned Pi!– Some of our labs will have the potential to reboot

the platform, or worse! One group per Pi!

• Pis used for the lab are accessible from the school network.

Laboratory Assignments

• The labs for this class will require a lot of time. • Start them early. • Labs will be assigned in class on Tuesday before

the first lab session. – It is recommended you prepare any questions for

your first laboratory session in advance!

• Labs are typically due at the beginning of your lab session, 2 weeks after they are assigned.

Laboratory Assignments

• Each student is allocated 3 slip days for the semester. – A slip day can be used to extend the due date for a laboratory by 1

day, no questions asked.– You should indicate on your submitted assignment how many slip days

are being used.

• No other extensions will be granted except in the case of a documented emergency.– Late work suffers

• A -20% on the first day it is late.• A -40% on the second day it is late.• A -60% on the third day it is late.• No credit for four days or more late.

Examinations

• Examinations– Midterm I – February 13th in class– Midterm II – March 3rd in class– Final Exam – May 1st from 11:00a – 1:30p in MM-

102.

Course Plan

• University of Miami Honor Code is in effect– Open hands policy on assignments

• Late policy– Late assignments are only accepted if

arrangements are made ahead of time

• Electronic device policy– Laptops and tablets are ok as long as they’re being

used for class– Silence cell phones please

Week

1 Introduction, Computer Organization, Performance Lab 0

2 Instructions, operands, load/store, and numbers Lab 1

3 Branches, conditions, loops, procedures and the stack

4 Arithmetic, ALUs, Processors Lab 2

5 Data path, control, pipelining MIDTERM I

6 Jumps, branches, and pipelines Lab 3

7 Pipeline hazards, branch prediction, exceptions

8 Memory hierarchy, caches, addressing Lab 4

3/11 Spring Break

9 Cache performance, block replacement, caching algorithms Lab 4

10 Virtual memory, paging, page faults, protection Lab 5

11 Intro to storage systems MIDTERM II

12 Storage systems, reliability, deduplication, RAID, flash and PCM Lab 6

13 Connecting processors, memory, and I/O

14 Parallel processing, concurrency, and course synthesis No Lab

ON ABSTRACTIONS

Abstraction and Reality

• Most courses in CS/ECE emphasize abstractions– Abstract data types– Abstract analysis

• Abstractions have limits– Reality raises its ugly heads as bugs in design and

implementation.– Understanding the details of underlying systems

becomes important!

Some Realities

What is an int?What is a float?

Some Realities

Reality #1•An int is not an integer!•A float is not a real number!•Example: Is x^2 >= 0?

Some Realities

• An int is not an integer!• A float is not a real number!• Example: Is x^2 >= 0?

– Floats? Yes.

– Ints?• 40000 * 40000 -> 1600000000• 50000 * 50000 -> ??

Doesn’t behave like an integer!

Some Realities

• Is addition ?communicative?– Does x + y = y + x?

– Ints? Yes.– Floats? No!

– ADD SOMETHING HERE

Computer Arithmetic

• It isn’t random!– Operations have mathematical properties they

adhere to.

• May not be the ones we assume as “usual”– Finite representation in the hardware matters!

• Observation:– Understanding the hardware implementation is

necessary.

What kind of abstractions are we using?

• Code found in BSD implementation of getpeername

/* Kernel memory region holding user-accessible data */#define KSIZE 1024char kbuf[KSIZE];

/* Copy at most maxlen bytes from kernel region to user buffer */int copy_from_kernel(void *user_dest, int maxlen) { /* Byte count len is minimum of buffer size and maxlen */ int len = KSIZE < maxlen ? KSIZE : maxlen; memcpy(user_dest, kbuf, len); return len;}

Intended Usage

• Code found in BSD implementation of getpeername

/* Kernel memory region holding user-accessible data */#define KSIZE 1024char kbuf[KSIZE];

/* Copy at most maxlen bytes from kernel region to user buffer */int copy_from_kernel(void *user_dest, int maxlen) { /* Byte count len is minimum of buffer size and maxlen */ int len = KSIZE < maxlen ? KSIZE : maxlen; memcpy(user_dest, kbuf, len); return len;}

#define MSIZE 528

void getstuff() { char mybuf[MSIZE]; copy_from_kernel(mybuf, MSIZE); printf(“%s\n”, mybuf);}

Malicious Usage

• Code found in BSD implementation of getpeername

/* Kernel memory region holding user-accessible data */#define KSIZE 1024char kbuf[KSIZE];

/* Copy at most maxlen bytes from kernel region to user buffer */int copy_from_kernel(void *user_dest, int maxlen) { /* Byte count len is minimum of buffer size and maxlen */ int len = KSIZE < maxlen ? KSIZE : maxlen; memcpy(user_dest, kbuf, len); return len;}

#define MSIZE 528

void getstuff() { char mybuf[MSIZE]; copy_from_kernel(mybuf, -MSIZE); . . .}

Some Realities

Reality #2•Knowing assembly is essential to your future!

Some Realities

Reality #2•Knowing assembly is essential to your future!

– You will probably NEVER write assembly programs outside of this class.

• Compilers are better at it than you are.

– But…

Some Realities

Reality #2•Knowing assembly is essential to your future!

– You will probably NEVER write assembly programs outside of this class.

• Compilers are better at it than you are.

– Understanding assembly is key to understanding machine execution.

• Behavior of programs with bugs.• Performance tuning.• System Software.• Malware analysis.

Some Realities

Reality #3•Memory matters!

– Random access memory is an abstraction with little basis in the physical world.

– Memory is not unbounded.• Memory must be allocated and managed.• Many applications are memory dominated.

– Memory reference bugs are very difficult.– Memory performance is non-uniform.

Memory Referencing Bug Example

• Result varies based on architecture.

double fun(int i){ volatile double d[1] = {3.14}; volatile long int a[2]; a[i] = 1073741824; /* Possibly out of bounds */ return d[0];}

fun(0) ➙ 3.14fun(1) ➙ 3.14fun(2) ➙ 3.1399998664856fun(3) ➙ 2.00000061035156fun(4) ➙ 3.14, then segmentation fault

Memory Referencing Bug Exampledouble fun(int i){ volatile double d[1] = {3.14}; volatile long int a[2]; a[i] = 1073741824; /* Possibly out of bounds */ return d[0];}

fun(0) ➙ 3.14fun(1) ➙ 3.14fun(2) ➙ 3.1399998664856fun(3) ➙ 2.00000061035156fun(4) ➙ 3.14, then segmentation fault

Location accessed by fun(i)

Explanation: Saved State 4

d7 ... d4 3

d3 ... d0 2

a[1] 1

a[0] 0

Memory Referencing Errors

• C/C++ do not provide memory protection– Out of bound references– Invalid pointer values– Abuses of allocation

• Can lead to hard to debug situations– Dependent on architecture and compiler– Action at a distance

Memory Performance Example

• How big of a difference does this simple change make?

• 21x times slowdown!

void copyji(int src[2048][2048], int dst[2048][2048]){ int i,j; for (j = 0; j < 2048; j++) for (i = 0; i < 2048; i++) dst[i][j] = src[i][j];}

void copyij(int src[2048][2048], int dst[2048][2048]){ int i,j; for (i = 0; i < 2048; i++) for (j = 0; j < 2048; j++) dst[i][j] = src[i][j];}

Memory Performance Example

Some Realities

Reality #4•There is more to performance than asymptotic complexity.

– Constants matter too!– Exact op count doesn’t even full describe the

situation!– Must optimize at all levels, algorithm, data

representation, functions, loops.

Some Realities

Reality #4•There is more to performance than asymptotic complexity.

– Need to understand how systems work to optimize them!

• How are programs compiled?• How are they executed?• How do we measure performance?• How do we find bottlenecks?• How do we improve performance without affecting the code?

Performance of Matrix Multiply

• Same computer. • Same compiler.• Same flags.• Exactly the same number of operations• Why does this happen?

160x

Triple loop

Best code (K. Goto)

Performance of Matrix Multiply

• Reasons for 20x improvement: Blocking or tiling, loop unrolling, array scalarization, instruction scheduling, search to find best choice.

• Effect: Fewer register spills, L1/L2 cache misses, and TLB misses.

Memory hierarchy and other optimizations: 20x

Vector instructions: 4x

Multiple threads: 2x

Some Realities

Reality #5•Computers do more than execute programs

– Need to get data in and out• I/O systems are critical to performance and reliability.

– Communicate over networks• How to cope with unreliable media?• Dealing with concurrency?• Cross platform issues?

COMPUTER ORGANIZATION

Classes of Computers

• Desktop Computers– General purpose, run a variety of software for many

applications– Subject to cost/performance tradeoffs

• Server Computers– Network based– High capacity, performance, and reliability– Range from small servers to super computers

• Embedded Computers– Parts of systems, cyberphysical controllers– Power/performance/cost constraints

Market Trends

Components of a Computer

• All computers have similar philosophies of organization.

• Get input, perform computation, produce output.– User interface: Display,

keyboard, sensors– Storage devices: Hard disk,

CD/DVD, flash– Communication: Network,

wifi, etc.– Compute: CPU, GPU, Memory

Internals of a Computer

Internals of a Processor (CPU)

• Datapath: Performs operations on data.• Control: sequences datapath, memory• Cache memory

– Small fast SRAM memory for immediate access to data.

Abstractions and the CPU

• Abstractions help us deal with complexity and hide low level details.

• Instruction set architecture (ISA)– The hardware/software interface

• Application binary interface– ISA + system software interface

Why Abstractions?

• What is an instruction?

Why Abstractions?

• What is an instruction?– A collection of bits the computer understands and

can “execute” or perform.

– Example:00000010001100100100000000100000

– Tells a computer to add two numbers.– How does the computer know?

Why Abstractions

00000010001100100100000000100000

Op code rs rt rd shamt function code

What the heck does this even mean?

add $t0, $s1, $s2

• op code - Code for the basic operation of the instruction• rs - The first register source operand• rt - The second register source command• rd - The register destination operand, gets the result of the

operands• shamt - Shift amount• function code - Function code, selects the specific variant of the

operation indicated by the op code.

The Hardware

Why abstractions

This:add $t0, $s1, $s2

Is easier than this:00000010001100100100000000100000

Why abstractions

• You know what is even easier?

This:c = a + b;

• For a human at least…

Abstraction Layers

• High-level language– Level of abstraction is

close to the problem domain.

– Allows us to be productive!

– Allows the code to be machine portable

• Different machines have different instructions!

Abstraction Layers

• Assembly language– Assembly language is

created from a compiler.– Compiler takes a high-

level language and compiles the instructions necessary to accomplish the indicated algorithm.

– Assembly language is a symbolic version of binary instructions.

Abstraction Layers

• Machine Language– Created by the

assembler.– Translates from symbolic

assembly language into the binary representation in machine language which the computer actually understands.

WRAP UP

For next time

• Read Chapter 1, Sections 1.1 – 1.5, and 1.8.

• Start Lab 0 early!!!

Many thanks to Drs. Lee and Seshia for their text and materials