Date post: | 26-Mar-2015 |
Category: |
Documents |
Upload: | leslie-steele |
View: | 217 times |
Download: | 1 times |
Page 1 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Analyzing NFS Client Performance with IOzone
Don Capps Tom McNeal
Performance Architect Independent Consultant
HP TMCN Consulting
[email protected] [email protected]
1
Page 2 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Benchmark Overview
Characteristics of IOzoneActivities
Page 3 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Load Generation• File System I/O requests– File sizes vary from 64K to 512M– Record sizes vary from 4K to 16M– Each increase doubles previous size– Large file system calls supported
• System variants supported– Memory mapped files– fread(), fwrite()– pread(), pwrite()
Page 4 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Sequential Reads/Writes
• Reads & Rereads
• Writes & Rewrites
• Backwards sequential read
• “Stride” read– Uses constant intervals for sequential
reads from beginning to end
Page 5 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Other Reads/Writes
• Randomized Reads/Writes
• Record Rewrite (from offset 0)
• fread() – Reads and Rereads– Serialized, Buffered & Blocked IO
• fwrite() – Writes and Rewrites– Serialized, Buffered & Blocked IO
Page 6 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Recommended Variants for NFS Clients
• All tests, all record sizes
• Commit time included in measurements
• IO targeted at mounted file– Unmount clears out caches between tests
– Target file specified in mounted directory
./iozone –azc –U /mnt/testdir –f /mnt/testdir/testfile
Page 7 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Benchmark Results
Characteristics of IOzone
Reports
Page 8 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Graphical Reports
• Generate Excel output text
• Named file has graphs and data– 3D Surface Charts for all tests
– Includes text output used for graphs
• Standard output sent to log file– Generally useful for debugging problems
./iozone –R -b exceloutput.xls > logfile
Page 9 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Surface Plot Graphs
64
128
256
512
1024
2048
4096
8192
1638
4
3276
8
6553
6
1310
72
2621
44
5242
88
1048
576
4
32
256
2048
16384
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000
110000
120000
130000
140000
150000
160000
170000
180000
Kbytes/sec
File size in Kbytes
Record size in Kbytes
Write performance
170000-180000
160000-170000
150000-160000
140000-150000
130000-140000
120000-130000
110000-120000
100000-110000
90000-100000
80000-90000
70000-80000
60000-70000
50000-60000
40000-50000
30000-40000
20000-30000
10000-20000
0-10000
CPU Cache effect Memory /buffer cache
Not measured
Physical disk I/O
Page 10 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Surface Plot Graphs II
64
128
256
512
1024
2048
4096
8192
1638
4
3276
8
6553
6
1310
72
2621
44
5242
88
1048
576
4
32
256
2048
16384
0
30000
60000
90000
120000
150000
180000
210000
240000
270000
300000
330000
360000
390000
420000
Kbytes/sec
File size in Kbytes
Record size in Kbytes
Read Performance
390000-420000
360000-390000
330000-360000
300000-330000
270000-300000
240000-270000
210000-240000
180000-210000
150000-180000
120000-150000
90000-120000
60000-90000
30000-60000
0-30000
CPU Cache effect Memory /buffer cache
Not measured
Physical disk I/O
Page 11 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
System & Network Variants
Environmental Control and Interactions
Page 12 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
System Level Variations
• SMP Issues– Processor cache purges
– Processor affinity (for a given # of cpus)
– Lower bound of number of cpus
• Cache Management– CPU Cache size
– CPU Cache line size
-p, -P #, -l n, -S size, -L size
Page 13 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
System parameters
• Client BIOD Daemons
• Server NFSD Daemons
• Number of file system nodes– rnode/inode/vnode/file handles
• Directory Name Lookup Cache
• Network buffer sizes
Page 14 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
File System Variations
• O_SYNC file option for all tests
• File locking required for all IO
• Flush timings included– fsync() and fflush()
• Large file offsets – File system calls determined at make time
– Alternate max file size may be specified
-o, -W, -e, -g #
Page 15 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
File System Variations II
• Memory mapped file IO – mmap() interface
– MS_ASYNC or MS_SYNC usage available
• Posix asynchronous IO
-B, -D, -G, -H n, -k n
Page 16 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Network Variations• UDP/TCP Protocol
• Client transfer sizes
• Network speed, duplex settings– Autonegotiation is often “interesting”
• IP issues– Jumbo frames with gigabit ethernet
– Stream heads, Socket buffer sizes
Page 17 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Clustered Clients
Managing and Measuring
a Cluster
Page 18 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Client Specification
• Clients specified in a file
• Clients must be accessible– Remote shells enabled through .rhosts
• DNS ®
• IOzone revision 3.128 or later
• Stonewalling helpful (removed by –x)
-+m filename
Page 19 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
“Stonewalling”• Client tests initiated in tandem– All clients kept equally busy
• When one finishes, they all finish– Tests halted when the first client completes
• Emulates high performance parallel processing clusters– Beowulf clusters at LLNL, PNNL, Los
Alamos
Page 20 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Summary
Examplesand
References
Page 21 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Summary• NFS Client measurement standard
• Gather standard data first– What is right for your client?
• Review Variations and Features– Review System, FS, and Network setup
– Start tuning, playing, tuning, playing….
./iozone –azcR –U /mnt/testdir –f /mnt/testdir/testfile \ -b exceloutput.xls > logfile
Page 22 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Rewrite Graph
64
128
256
512
1024
2048
4096
8192
1638
4
3276
8
6553
6
1310
72
2621
44
5242
88
1048
576
4
32
256
2048
16384
0
30000
60000
90000
120000
150000
180000
210000
240000
270000
300000
330000
360000
390000
420000
450000
Kbytes/sec
File size in Kbytes
Record size in Kbytes
Re-write performance
420000-450000
390000-420000
360000-390000
330000-360000
300000-330000
270000-300000
240000-270000
210000-240000
180000-210000
150000-180000
120000-150000
90000-120000
60000-90000
30000-60000
0-30000
CPU Cache effect Memory /buffer cache
Not measured
Physical disk I/O
Page 23 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Reread Graph
64
128
256
512
1024
2048
4096
8192
1638
4
3276
8
6553
6
1310
72
2621
44
5242
88
1048
576
4
32
256
2048
16384
0
50000
100000
150000
200000
250000
300000
350000
400000
450000
500000
550000
600000
650000
700000
750000
Kbytes/sec
File size in Kbytes
Record size in Kbytes
Re-read performance
700000-750000
650000-700000
600000-650000
550000-600000
500000-550000
450000-500000
400000-450000
350000-400000
300000-350000
250000-300000
200000-250000
150000-200000
100000-150000
50000-100000
0-50000
CPU Cache effect
Memory /buffer cache
Not measured
Physical disk I/O
Page 24 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
Random Read Graph
64
128
256
512
1024
2048
4096
8192
1638
4
3276
8
6553
6
1310
72
2621
44
5242
88
1048
576
4
32
256
2048
16384
0
40000
80000
120000
160000
200000
240000
280000
320000
360000
400000
440000
480000
520000
560000
600000
Kbytes/sec
File size in Kbytes
Record size in Kbytes
Random Read Performance
560000-600000
520000-560000
480000-520000
440000-480000
400000-440000
360000-400000
320000-360000
280000-320000
240000-280000
200000-240000
160000-200000
120000-160000
80000-120000
40000-80000
0-40000
CPU Cache effect
Memory /buffer cache
Not measured
Physical disk I/O
Page 25 of NFS Industry ConferenceOctober 22-23, 2002
NFS
INDUSTRY
CONFERENCE
References• http://www.iozone.org
• http://www.iozone.org/src/current– Contains 8K vs. 32K Transfer Size graphs
• http://www.sourceforge.net/projects/nfstestmatrix– Includes functional tests, destructive tests,
and benchmarks for Linux systems
• http://www.mclx.com/orph– Linux Server performance review (late 2001)