Solaris Volume ManagerSolaris Volume Manager

M. DesoukyM. Desouky

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

22

Table of ContentsTable of Contents1.1. RAID OverviewRAID Overview

2.2. Introduction to SDS SoftwareIntroduction to SDS Software

3.3. SDS InstallationSDS Installation

4.4. SDS User InterfacesSDS User Interfaces

5.5. Metadevice State DatabaseMetadevice State Database

6.6. Concatenation & StripesConcatenation & Stripes

7.7. Mirrors & Hot SparesMirrors & Hot Spares

8.8. RAID-5 MetadevicesRAID-5 Metadevices

9.9. File System OperationsFile System Operations

10.10. DisksetsDisksets

RAIDRAIDUniversity of CaliforniaUniversity of California

Berkeley 1987Berkeley 1987

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

44

DiscussionDiscussionDisk Speed LimitationsDisk Speed Limitations

File System Is FullFile System Is Full

Disk Fault ToleranceDisk Fault Tolerance

24x7 Systems24x7 Systems

Multiple Disks:Multiple Disks: Higher probability of disk failureHigher probability of disk failure MTBFMTBF File System Spanning: (Max: 1 TB)File System Spanning: (Max: 1 TB) Longer reboot timesLonger reboot times

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

55

Managing Large No of DisksManaging Large No of Disks

Decrease the number of file systemsDecrease the number of file systems

Avoid data loss due to disk failureAvoid data loss due to disk failure

Balance I/O across disks, performanceBalance I/O across disks, performance

Avoid file systems check at bootAvoid file systems check at boot

Grow file systems onlineGrow file systems online

Apply dual-host failoverApply dual-host failover

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

66

Managing DisksManaging DisksTechniques:Techniques: ConcatenationConcatenation StripingStriping Expanding File SystemsExpanding File Systems Hot SparesHot Spares RAID 5RAID 5 MirroringMirroring DisksetsDisksets

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

77

RAID LevelsRAID Levels

1.1. RAID 0RAID 0

2.2. RAID 1RAID 1

3.3. RAID 0+1RAID 0+1

4.4. RAID 3RAID 3

5.5. RAID 5RAID 5

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

88

RAID 0, ConcatenationRAID 0, Concatenation

FS 1

FS 2

FS 3

RAIDManagement

Software

VirtualFS

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

99

RAID 0, StripingRAID 0, Striping

FS 1

FS 2

FS 3

RAIDManagement

Software FS 3

FS 2

FS 1

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1010

RAID 1, MirroringRAID 1, Mirroring

FS 4

FS 3

FS 2

FS 1

FS 4

FS 3

FS 2

FS 1

RAIDManagement

SoftwareFS 4

FS 3

FS 2

FS 1

Mirror

Data

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1111

RAID 0+1RAID 0+1

RAIDManagement

Software

Mirroring

RAIDManagement

Software

Striping

FS 4

FS 3

FS 2

FS 1

FS 8

FS 7

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

RAIDManagement

Software

Striping

FS 8

FS 7

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

FS 8

FS 7

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

FS 8

FS 7

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

FS 8

FS 7

FS 6

FS 5

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1212

RAID 3, Striping with ParityRAID 3, Striping with Parity

RAIDManagement

Software

FS 4

FS 1

FS 5

FS 2

FS 6

FS 3

P(5-6)

P(2-4)

P(1-3)

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1313

RAID 5, Distributed ParityRAID 5, Distributed Parity

P(10-12)

FS 7

FS 4

FS 1

FS 10

P(7-9)

FS 5

FS 2

FS 11

FS 8

P(4-6)

FS 3

FS 12

FS 9

FS 6

P(1-3)

RAIDManagement

Software

FS 12

FS 11

FS 10

FS 9

FS 8

FS 7

FS 6

FS 5

FS 4

FS 3

FS 2

FS 1

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1414

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1515

IntroductionIntroductionDiskSuite Manage:DiskSuite Manage: Large Number of DisksLarge Number of Disks Data on DisksData on Disks

DiskSuite TasksDiskSuite Tasks Storage Capacity: metadevicesStorage Capacity: metadevices Data Availability: Mirrors, RAID5 metadevicesData Availability: Mirrors, RAID5 metadevices

DiskSuite, Virtual DisksDiskSuite, Virtual Disks MetadeviceMetadevice

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1616

Planning DiskSuitePlanning DiskSuiteSetting Priorities:Setting Priorities:

Performance:Performance:Availability: MirroringAvailability: Mirroring

Hardware Cost: StripingHardware Cost: Striping

Availability:Availability:Performance: MirroringPerformance: Mirroring

Hardware Cost: RAID5Hardware Cost: RAID5

Hardware Cost:Hardware Cost:Availability: RAID5Availability: RAID5

Performance: StripingPerformance: Striping

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1717

State Database ReplicasState Database ReplicasReplicas Stores:Replicas Stores: DiskSuite ConfigurationDiskSuite Configuration State InformationState Information

Planning Replicas:Planning Replicas: One Disk: 3 replicas on one sliceOne Disk: 3 replicas on one slice Two-Four Disks: 2 replicas on eachTwo-Four Disks: 2 replicas on each Five or more Disks: 1 on eachFive or more Disks: 1 on each

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1818

State Database ReplicasState Database ReplicasCreating Replicas:Creating Replicas: Small slices (520 MB)Small slices (520 MB) Slices should NOT contain data!Slices should NOT contain data! Backup your data!Backup your data! Allocating Space from swap partition!Allocating Space from swap partition!

swap -lswap -l

swap -dswap -d

swap -aswap -a metadb -a -f -c metadb -a -f -c -a Add-a Add -c Number of replicas-c Number of replicas -f Force-f Force

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

1919

Creating ReplicasCreating ReplicasExamples:Examples:

Creating 2 Replicas on 3 Disks: Creating 2 Replicas on 3 Disks: metadb -a -f -c 2 c0t1d0s3 c1t1d0s3 c2t1d0s3metadb -a -f -c 2 c0t1d0s3 c1t1d0s3 c2t1d0s3

Creating 3 replicas on 1 Disk:Creating 3 replicas on 1 Disk: metadb -a -f -c 3 c0t0d0s3metadb -a -f -c 3 c0t0d0s3

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2020

DiskSuite ObjectsDiskSuite ObjectsObject:Object: Assigning physical device to a DiskSuite nameAssigning physical device to a DiskSuite name

Object:Object: ReplicasReplicas MetadeviceMetadevice

StripesStripes

ConcatenationsConcatenations

MirrorsMirrors

RAID5RAID5 Hot Spare PoolsHot Spare Pools DisksetsDisksets

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2121

Objects, ReplicasObjects, ReplicasCreating Additional Replica:Creating Additional Replica: metadb -a c0t2d0s0metadb -a c0t2d0s0 metadbmetadb

Creating 2 Additional Replicas:Creating 2 Additional Replicas: metadb -a -c 2 c0t2d0s0metadb -a -c 2 c0t2d0s0

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2222

Objects, StripesObjects, StripesOne Stripe of 2 slices:One Stripe of 2 slices: metainit d10 1 2 c0t1d0s2 c0t2d0s2metainit d10 1 2 c0t1d0s2 c0t2d0s2

One Stripe of 3 slices:One Stripe of 3 slices: metainit d20 1 3 c0t1d0s2 c0t2d0s2 c0t3d0s2metainit d20 1 3 c0t1d0s2 c0t2d0s2 c0t3d0s2

Deleting Meta DevicesDeleting Meta Devices metaclear -ametaclear -a

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2323

Objects, ConcatenationObjects, ConcatenationCreating a Concatenation of 2 Slices:Creating a Concatenation of 2 Slices: metainit d25 2 1 c0t1d0s2 1 c0t2d0s1metainit d25 2 1 c0t1d0s2 1 c0t2d0s1

Creating a Concatenation of 4 Slices:Creating a Concatenation of 4 Slices: metainit d40 4 1 c0t1d0s2 1 c0t2d0s2 1 metainit d40 4 1 c0t1d0s2 1 c0t2d0s2 1

c0t2d0s3 1 c0t2d1s3c0t2d0s3 1 c0t2d1s3

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2424

Objects, MirrorsObjects, MirrorsMirror:Mirror: On different disksOn different disks Slices should be the same size!Slices should be the same size!

Types of Mirrors:Types of Mirrors: One WayOne Way Two WayTwo Way Three WayThree Way

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2525

Objects, MirrorsObjects, MirrorsCreating Two-Way Mirror:Creating Two-Way Mirror: metainit -f d51 1 1 c0t0d0s2 (/home)metainit -f d51 1 1 c0t0d0s2 (/home) metainit d52 1 1 c1t0d0s2 (unmounted)metainit d52 1 1 c1t0d0s2 (unmounted) metainit d50 -m d51metainit d50 -m d51 umount /home (What if you can’t umount!)umount /home (What if you can’t umount!) vi /etc/vfstab: /dev/md/dsk/d50vi /etc/vfstab: /dev/md/dsk/d50 newfs /dev/md/dsk/d50newfs /dev/md/dsk/d50 mount /home (on d50)mount /home (on d50) metattach d50 d52metattach d50 d52

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2626

Objects, MirrorsObjects, MirrorsCreating a Mirror from root /Creating a Mirror from root / metainit -f d11 1 1 c0t3d0s0metainit -f d11 1 1 c0t3d0s0 metainit d12 1 1 c1t3d0s0metainit d12 1 1 c1t3d0s0 metainit d10 -m d11metainit d10 -m d11 metattach d10 d12metattach d10 d12

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2727

Deleting Mirror MetadeviceDeleting Mirror Metadevice

Detaching a Mirror Meta deviceDetaching a Mirror Meta device metadetach d50 d51metadetach d50 d51

Deleting the Meta DevicesDeleting the Meta Devices

metaclear -ametaclear -a

RAID Overview

SDS Software

SDS Installation

SDS User Interfaces

MD State Database

Concats & Stripes

Mirrors

Hot Spares

RAID-5 MD

FS Operations

Disksets

2828

Objects, RAID5Objects, RAID5Creating a RAID5 metadevice of 3 slicesCreating a RAID5 metadevice of 3 slices metainit d45 -r c2t3d0s2 c3t0d0s2 c4t0d0s2metainit d45 -r c2t3d0s2 c3t0d0s2 c4t0d0s2