CS698Y: Modern Memory SystemsLecture-15 (DRAM Organization)

Biswabandan Panda biswap@cse.iitk.ac.in

https://www.cse.iitk.ac.in/users/biswap/CS698Y.html

Modern Memory Systems Biswabandan Panda, CSE@IITK 2

DRAM Organization

Channel

Rank

Chip

Bank

Row

Column

Rank 0 with 8 chips

Rank 1 with 8 chips

DIMM

Modern Memory Systems Biswabandan Panda, CSE@IITK 3

Ranks, Banks, Rows, and Columns

Rank 0 Rank 1

DIMM

Bank 0 Bank 1

Bank 2 Bank 3

Multiple Chips

Modern Memory Systems Biswabandan Panda, CSE@IITK 4

Ranks, Banks, Rows, and Columns

Rank 0

64 bits

16

16

16

16

Bank

Row

Column

Chip

Bank 0 Bank 1

Bank 2 Bank 3

16-bit interface: 16 bits from each chip in one go

Modern Memory Systems Biswabandan Panda, CSE@IITK 5

Let’s Dig Deep

Each rank has 64-bit wide data bus

If a rank is of width x8 then # DRAM chips ??

What about x4, # DRAM chips ??

If a rank is of width x8 then # DRAM chips ?? 8

What about x4, # DRAM chips ?? 16

Modern Memory Systems Biswabandan Panda, CSE@IITK 6

Where is Your 1 bit?

Row or Page

Wordline

Bitline

Modern Memory Systems Biswabandan Panda, CSE@IITK 7

Row (page) and Row buffer (Sense Amplifier)

Column mux

RowDecoder

Row Buffer

Each bank has a row buffer

Stores the last used row

Logically

Actually

Multiple local row buffers

Multiple Sub-arrays

2KB

Modern Memory Systems Biswabandan Panda, CSE@IITK 8

SALP [ISCA ‘12]

Row

Row-Buffer

RowRowRow

32k rows

Logical Bank

A single row-buffer cannot drive all rows

Global Row-Buf

Physical Bank

Local Row-Buf

Local Row-BufSubarray1

Subarray64

Many local row-buffers, one at each subarray

For the sake of simplicity, we will use the logical view

Modern Memory Systems Biswabandan Panda, CSE@IITK 9

Bank: Collection of DRAM Arrays

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

Column mux

RowDecoder

Row Buffer

• DRAM Width

• x4 device

• x8 device

• Other possible widths

• x16

• x32

• x48

• x72

Modern Memory Systems Biswabandan Panda, CSE@IITK 10

An Example – 4GB DIMM

2Gb * 8 DRAM Chips (one side of the rank)

Total 16 chips + 2 chips for ECC (for both the ranks)

64 bit + 8 bit ECC interface (72 bit wide DIMM)

Transferring a 64B cache line will take 8 transfers of 8B each

8B will come from 8 chips (8 bits from one chip)

1 bit from each DRAM array assuming 8 DRAM arrays per bank

Modern Memory Systems Biswabandan Panda, CSE@IITK 11

Another View

Modern Memory Systems Biswabandan Panda, CSE@IITK 12

DRAM Channels

2 channels: 1 channel per controller

Modern Memory Systems Biswabandan Panda, CSE@IITK 13

DRAM to LLC

0xFFFF…F

0x00

0x40

...

64B cache block

Physical memory space

Rank 0Chip 0 Chip 1 Chip 7

<0

:7>

<8

:15

>

<5

6:6

3>

Data <0:63>

. . .

Modern Memory Systems Biswabandan Panda, CSE@IITK 14

DRAM to LLC

0xFFFF…F

0x00

0x40

...

64B cache block

Physical memory space

Rank 0Chip 0 Chip 1 Chip 7

<0

:7>

<8

:15

>

<5

6:6

3>

Data <0:63>

Row 0Col 0

. . .

Modern Memory Systems Biswabandan Panda, CSE@IITK 15

DRAM to LLC

0xFFFF…F

0x00

0x40

...

64B cache block

Physical memory space

Rank 0Chip 0 Chip 1 Chip 7

<0

:7>

<8

:15

>

<5

6:6

3>

Data <0:63>

8B

Row 0Col 0

. . .

8B

Modern Memory Systems Biswabandan Panda, CSE@IITK 16

DRAM to LLC

0xFFFF…F

0x00

0x40

...

64B cache block

Physical memory space

Rank 0Chip 0 Chip 1 Chip 7

<0

:7>

<8

:15

>

<5

6:6

3>

Data <0:63>

8B

8B

Row 0Col 1

. . .

8 cycles (DRAM IO): 1 cycle transfers 8 bytes from a column

Modern Memory Systems Biswabandan Panda, CSE@IITK 17

DRAM Address Mapping (1 Channel)

Low Col. High ColumnRow (14 bits) Byte in bus (3 bits)Bank (3 bits)

3 bits8 bits

Column (11 bits)Bank (3 bits)Row (14 bits) Byte in bus (3 bits)

2GB DRAM, 8 Banks, 16K rows, 2K Columns per bank

Row Interleaving: Consecutive rows in consecutive banks

Cache Interleaving: Consecutive cache blocks in consecutive banks

What about Multiple Channels?

ACK.

Some of the slides are from Prof. Mutlu’s lectures T.V. Kalyan (IBM India)