Introduction to Solid State Drives

Introduction to Solid State Drive

Technology

Saturday, November 16, 13

Class Overview

• The Evolution of Storage Technology

• Spinning Disks

• Storage Metrics

• Solid State Technology


better understanding of spinning disks, understand high & low level flash, issues with SSDs

The Evolution of Storage Technology


Pre-History

Density/Time Speed/Time


Spinning Disks


The Parts of a Hard Drive


PlattersThe Parts of a Hard Drive


Actuator Arms and Heads



Controllerand Interface



Voltron Force Assemble!


Disk Interface

• Removable (USB/CF)

• SATA1 / II / III

• Nearline SAS

• SAS

• Fibre Channel

• PCI-e


Spinning Disk Removable Media

• Advantages:

• Nigh Universal

• Disadvantages:

• Slower

• Fragile

• Easily lost.

• Abstraction Layer

Disk Interface


USB

SATA I / II / III

• Speeds: 1.5 / 3 / 6Gb/s

• Requires AHCI for things like NCQ

• Subset of SAS

• Shares IDE command set

Disk Interface


AHCI - Advanced Host Controller Interface (IDE is ok for TRIM)NCQ on SSD ensure SSD has things to do while host is latent(Intel can queue 32 requests) - logo from SATA-IO (intnl org)

SATA 3.1

• Approved July 2011

• Universal Storage Module

• mSATA

• QTRIM

Disk Interface

(This time, it’s personal)


QTRIM - queued TRIM commands, USM is a mobile drive standard

SAS / Nearline SAS

• SAS

• Enhanced CRC checking

• 512/520/528 bit blocks

• Low density, high reliability

• Nearline SAS

• ...not so much


Serially Attached SCSI - 16 bits of CRC

Disk Geometry


PlattersDisk

Geometry


TracksDisk

Geometry


Cylinders Disk

Geometry


Cylinders Disk

Geometry


SectorsDisk

Geometry


Logical Block Addressing

• First introduced as an abstraction layer

• Replaced CHS addressing

• Address Space is Linear (block 0 - n)

• Size of address space depends on the standard at time of manufacture.

DiskGeometry


Currently at 48-bit LBA -

Variables Affecting Spinning Disk IO Rate


Platter Rotational Speed


Seek Speed


Data Density


Controller CacheVariables Affecting Spinning

Disk Speed


Size / battery backed / hybrid drives

Spinning Disk Damage Vectors


Movement

• Movement vertical or parallel to platter

• Measured in G forces

• Head Crashes

• Spinning Down

• Head uses “Landing Strip”

• Repeated platter contact causes damage to the read/write head



Used to manually park the heads | Putting your computer to sleep can cause the head to park | nanocoating on the bumpy landing strip

Protection against movement

• “Active Drive Protection”: Free-fall sensor

• Has a lift arm to lift the head away from the platter

• Some protection systems are in the drive, some are in the controller

• Don’t mix the two


Apple: Sudden motion sensor, Lenovo/IBM: Hard Drive Active Protection System

Next slide: “You know, vibrations are movement...”

You know, vibrations are movement...


Yes, vibrations are important, too


Shelf Life

• Oil / Lubricants in bearings

• Temperature fluctuations

• Magnetic “events” (bit rot)

• Outgassing / vapor removal



Long-term “archival quality” drives with long-life lubricant Long term “cold storage” arrays which periodically spin up drives every few weeks to clean & scrub the data

Spinning Disks in RAID

• Redundant Array of Inexpensive Disks

• Common RAID levels:

• 0,1,5,6,10

• Software / Hardware


Important Considerations

• Redundancy

• Capacity

• Speed

• Robust

SpinningDisks

In RAID


Speed: Dedicated hardware? Single point of failure? Parity Calculation? How long to rebuild a drive? NUMBER OF SPINDLES!! Redundancy: How many drive failures? URE errors? Capacity: Parity stripe or mirrors? (harder better faster stronger)

Advantages

• Linear Speed

• Price (Per Gigabyte)

• Well-Understood

SpinningDisks

In RAID


Disadvantages

• Random Speed

• Price (per IOPS)

• Failure Rate

• Rebuild Speed

SpinningDisks

In RAID


Storage Metrics


IOPS

• What are they?

• What aren’t they?


The Simplified Equation

IOPS = 1/(((R+W)/2)/1000) + (L/1000)

RWL

= Average Read Time= Average Write Time= Average Latency


Rule of Thumb Assumptions

RPM IOPS5400 50-807200 80-10010k 130-15015k 180-200


Determining IOPS

• Per Drive

• Manufacturer’s Stated Numbers

• Rule of Thumb

• Per RAID Array

• Write penalty


IO Profiling

• Active Tools

• Bonnie++

• dd

• Intel NAS Toolkit

• Passive Tools

• io(stat/meter/top), atop

• Resource Monitor / Process Explorer


http://www.intel.com/products/server/storage/NAS_Perf_Toolkit.htm

Solid State Drive Technology


NOR Flash

• Reads and writes are atomic single-bit

• Expensive

• Small specific use cases


Won’t talk about NOR much.

NAND Flash

• Reads are based on “read blocks” (4k)

• Writes are based on “erasure blocks”

• Cheap (and getting cheaper)

• Broad use cases


Read / Write Profiles

• Logical addresses abstracted from LBA

• No seek time

• Reads are generally very fast

• Writes are typically slower


Random and Linear IO have identical access timeNext slide: The magic

The Magic

InsulatingBarrier

Pure Silicon

Doped silicon capable of holding an electrical charge


Barrier is a dielectric film (silicon oxide)

Quantum Tunneling

(transmission coefficient for a particle tunneling through a single potential barrier)


Hot Carrier InjectionStorage / Erase uses Fowler-Nordheim Tunnel Injection / Release

Doped Silicon

Single Layer Cell (SLC)

Multi-Layer Cell (MLC)

Triple-Layer Cell (TLC)


Use charge pumps to get through the barrier Each charge level has a binary state - 1 or 0

Gradual Destruction

Energy increaseswith cell layers

Multiple cells needmultiple writes

Barrier accumulateselectrons

Electrical potential difference of barrier and cells disappears


Difficulty Going ForwardTLCTLC

000 100

001 101

010 110

011 111

SLC

0

1

MLC

00

01

10

11

4LC4LC4LC4LC

0000 0100 1000 1100

0001 0101 1001 1101

0010 0110 1010 1110

0011 0111 1011 1111


Density

• 3-Dimensional

• Charge levels

• Size of cells

• “Dot Pitch” (Cells Per Inch)

• 5nm, 3nm, 2nm

• Varies with “level” count


SLC / ESLC

• Low Density

• Single (bit) Level Cell

• Quick: 25µs Read / 200-300µ Write

• More robust & long wear time

• Write endurance near 100,000 cycles


Capacity expensive | only in 5nm / 3nm densities |

MLC / EMLC

• Reasonably High Density

• Two (bit) Level Cell

• Decently fast: 50µs Read / 600-900µs Write

• Medium lifetime

• Write endurance near 3,000 cycles


TLC

• Very High Density

• Three (bit) Level Cell

• Decently fast: 75µs Read / 900-1350µs Write

• Not very robust or durable :-(

• Write endurance ~ 1,000 cycles


Write Amplification and Garbage Collection


Block Sizes

• Read Block

• 4k (aka “page”)

• Erasure Block

• (Large) multiple of 4k

• aka “block” 256KB erasure

block size


e-ink parallel

Write Amplification

Written Data

Empty Cell


next - want to change the data in the upper right quadrant

Write Amplification

Written Data

Empty Cell

New Data

Old Data


next - big chunk of new data to write

Write Amplification

Written Data

Empty Cell

Old Data

New Data


Where does this go? We’re out of empty erasure blocks!

Write Amplification

Written Data

Empty Cell

New data writtenover old cell

w/o TRIM


Write Amplification

Written Data

Empty Cell

New data writtenover old cell

w/ TRIM


Garbage Collection


Garbage Collection


Garbage Collection


Garbage Collection


Garbage Collection


IO Performance Profiles


Remember:

• Spinning Disks

• Linear is fast

• Random is slow

• Read marginally faster than writes (sometimes)


writes slower when switching tracks

With SSDs:

• Reads are fast

• Writes are slow(ish)

• Random or linear doesn’t matter (as much)


SSD Performance Overview

• Depends on

• Number of flash chips in use

• Number of busses from the processor

• Performance of controller CPU

• Contention

• Bus speed

• Number of erasure blocks used

• Number of previous writes to flash cells


• Chips

• IO Busses

• CPU Cores


Causes of Contention

• Legitimate use

• Garbage collection

• Legitimate (but latent) useage

• IO Blender!

(Bender Blender: http://bit.ly/10vc7Sf)Saturday, November 16, 13

Latent: updatedb? atime? app-level garbage collection? (t-shirt at threadless)

http://bit.ly/10vc7Sf

http://bit.ly/10vc7Sf

Bus Speed

• SATA - 3 or 6 Gb/s?

• IOPS Calc

• Can your controller handle your disks?


Read

• Very fast

• No seek time

• moderately improved over spinning disk (linear - random greatly)

• Causes no damage to the media

• Generally scales up with capacity


Write

• Usually fast (depending on drive usage)

• No seek time

• highly improved over spinning disk

• Causes no damage to the media

• Generally scales up with capacity


Spinning DiskRead/Write Matrix

Read Write

Linear

Random


SSDRead/Write Matrix

Read Write

Linear

Random


Solid State in Practice


Solid State Form Factors


Removable Media


Drives


PCI Cards


Next slide: parts of an SSD

Parts of an SSD


Interface

USB

PCI

SATA/SAS


IDE (sadly?)

ControllerMain

Processor

I/O Bus Lanes

RAMCache

Battery / SuperCapacitor


Flash Chips


If individual chip capacity is finite, how do bigger drives increase capacity? What does this mean for performance?

Flash Controllers

• Flash Translation Layer (FTL)

• Stripe Writes

• Interpret bus instructions

• Wear Leveling

• Garbage Collection


Do the heavy lifting - single largest problem with flash drives, without a doubt.

Flash Translation LayerLBA (0...n blocks)

F L A S H C H I P S


SSD Aspects & Concerns


Longevity

• Primarily determined by the class of flash

• (e)SLC, (e)MLC, TLC

• Related to wear-leveling

• Under-reported capacity

• Short-stroking improves lifetime (not speed)


Partition Alignment

• Performance and longevity

• As big (or bigger) issue than it was in spinning disks

• Native 4k read blocks

• Far larger erasure blocks

• larger than is practical for alignment


TRIM

• As a command, refers to ATA-8 spec

• SCSI equivalent is UNMAP, but both are often referred to as TRIM.

• Does not immediately delete unused blocks

• Allows for GC


Linux calls this “discard” - TRIM refers

Linux TRIM Support

• EXT4 / XFS / JFS / BTRFS - Native using ‘discard’ option

• Consider NOOP or Deadline IO scheduler

• fstrim (part of util-linux) for R/W vols

• zerofree for R/O vols


fstrim & zerofree - userland - important for thin-provisioned volumes on SAN arrays which support it. Check docs on schedulers for details - deadline prefers read queues (under /sys/block)

OSX Trim Support

• Comes by default on factory-installed SSDs

• Trim-Enabler

• http://www.groths.org/trim-enabler/


http://www.groths.org/trim-enabler/



ZFS and SSDs

• ZFS Intent Log (ZIL)

• Adaptive Replacement Cache (ARC)

• arc_summary can help you decide


ZIL is almost like a journal - ARC is a RAM cache that has disk backing it. SSDs can be L2ARC - https://code.google.com/p/jhell/wiki/arc_summary

Filesystems in General

• Standard journaling filesystems

• Mount options (atime/relatime, etc), /tmp->tmpfs

• Next-Gen

• ZFS / BTRFS

• Distributed Filesystems

• DRBD


ZFS - SSD cache pool | ZFS/BTRFS are COW | DRBD no trim

Monitor Health w/ S.M.A.R.T.

• S.M.A.R.T. information

• vendor-specific

• Includes flash erase count

• smartctl on Linux and Mac

• Dozens of tools on Windows (check wiki)


Forensics

(http://bit.ly/fast11-wei-paper)

...Our results lead to three conclusions:

First, built-in commands are effective, but manufacturers sometimes implement them incorrectly.

Second, overwriting the entire visible address space of an SSD twice is usually, but not always, sufficient to sanitize the drive.

Third, none of the existing hard drive-orientedtechniques for individual file sanitization are effective on SSDs

Reliably Erasing Data From Flash-Based Solid State DrivesMichael Wei∗, Laura M. Grupp∗, Frederick E. Spada†, Steven Swanson∗

∗Department of Computer Science and Engineering, University of California, San Diego†Center for Magnetic Recording and Research, University of California, San Diego


http://bit.ly/fast12-wei-paper

http://bit.ly/fast12-wei-paper

SSD-enhanced RAID Array Considerations


Hardware / Software

• Dedicated CPU Power

• Battery-backed storage

Hardware RAID Controllers

• Trust (eyes on code)

• Excessive cost of HW

Software RAID Controllers

• Commercial Support

• Proprietary Tech

• Portability

• Spare CPU Cycles


single point of failure

TRIM / GC?

• Does the RAID software/device know enough to pass along TRIM?

• Will the array eventually crawl because of ongoing GC issues?


No software RAID that I know of supports it. Intel chipset for RAID0 with TRIM

Access Bandwidth

• How much data can a single drive transmit?

• How many drives are in the array?

• What is the aggregate bus speed to the array controller?

• What is the bus speed to the host(s)?


SSD Throughput Example

From Tech Radar: http://bit.ly/100UhvY

4.15Gb/s


http://bit.ly/100UhvY

http://bit.ly/100UhvY

Controller / Bus

• Speed / Ports

• How mature / reliable / tested?

RememberMe?


Just because buses exist in a storage array oesn’t make them magic and infinite in size

Tiering / Caching

Very slow, cheap disks

Faster spinning disks

SSD tier - hot blocks

Very fast SDRAM


Future Technology


Enhanced Capacity


Kowloon Walled City

Enhanced Longevity


Telomeres in chromosomes

Smart SSDs


Active Flash


What should I buy?


Questions?


Date post:	29-Nov-2014
Category:	Technology
Upload:	matt-simmons
View:	7,238 times
Download:	4 times