IBM Spectrum Scale 4.2: Administration and Programming Reference€¦ · Administra tion and...

IBM Spectrum ScaleVersion 4 Release 2.0

Administration and ProgrammingReference

SA23-1452-06

IBM

NoteBefore using this information and the product it supports, read the information in Notices on page 891.

This edition applies to version 4 release 2 of the following products, and to all subsequent releases andmodifications until otherwise indicated in new editions:v IBM Spectrum Scale ordered through Passport Advantage (product number 5725-Q01)v IBM Spectrum Scale ordered through AAS/eConfig (product number 5641-GPF)v IBM Spectrum Scale for Linux on z Systems (product number 5725-S28)

Significant changes or additions to the text and illustrations are indicated by a vertical line (|) to the left of thechange.

IBM welcomes your comments; see the topic How to send your comments on page xiv. When you sendinformation to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believesappropriate without incurring any obligation to you.

Copyright IBM Corporation 2014, 2016.US Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.

Contents

Tables . . . . . . . . . . . . . . . ix

About this information . . . . . . . . xiPrerequisite and related information . . . . . . xiiiConventions used in this information . . . . . xiiiHow to send your comments . . . . . . . . xiv

Summary of changes . . . . . . . . xv

Chapter 1. Performing GPFSadministration tasks . . . . . . . . . 1Requirements for administering a GPFS file system . 1

adminMode configuration attribute . . . . . . 2Common GPFS command principles . . . . . . 3

Specifying nodes as input to GPFS commands . . 3Stanza files . . . . . . . . . . . . . . 4Listing active GPFS commands . . . . . . . 5

Chapter 2. Managing the GPFS cluster . 7Creating your GPFS cluster . . . . . . . . . 7Displaying GPFS cluster configuration information . 7Adding nodes to a GPFS cluster . . . . . . . . 8Deleting nodes from a GPFS cluster . . . . . . 9Changing the GPFS cluster configuration data . . . 10Security mode . . . . . . . . . . . . . 20Running IBM Spectrum Scale without remote rootlogin . . . . . . . . . . . . . . . . 21

Configuring sudo . . . . . . . . . . . 21Configuring the cluster to use sudo wrapperscripts . . . . . . . . . . . . . . . 22Configuring a cluster to stop using sudo wrapperscripts . . . . . . . . . . . . . . . 23

Node quorum considerations . . . . . . . . 23Node quorum with tiebreaker considerations . . . 23Displaying and changing the file system managernode. . . . . . . . . . . . . . . . . 24Determining how long mmrestripefs takes tocomplete . . . . . . . . . . . . . . . 24Starting and stopping GPFS . . . . . . . . . 25

Chapter 3. Managing file systems . . . 27Mounting a file system . . . . . . . . . . 27

Mounting a file system on multiple nodes . . . 28GPFS-specific mount options . . . . . . . 28

Unmounting a file system . . . . . . . . . 29Unmounting a file system on multiple nodes . . 29

Deleting a file system . . . . . . . . . . . 29Determining which nodes have a file systemmounted . . . . . . . . . . . . . . . 30Checking and repairing a file system . . . . . . 30Dynamic validation of descriptors on disk . . . . 32Listing file system attributes . . . . . . . . . 33Modifying file system attributes . . . . . . . 34Querying and changing file replication attributes . . 34

Querying file replication . . . . . . . . . 34Changing file replication attributes . . . . . 34

Using Direct I/O on a file in a GPFS file system . . 35File compression. . . . . . . . . . . . . 35Setting the Quality of Service for I/O operations(QoS) . . . . . . . . . . . . . . . . 40Restriping a GPFS file system . . . . . . . . 42Querying file system space . . . . . . . . . 43Querying and reducing file system fragmentation . 44

Querying file system fragmentation . . . . . 45Reducing file system fragmentation . . . . . 45

Protecting data in a file system using backup . . . 46Protecting data in a file system using themmbackup command . . . . . . . . . . 46Backing up a file system using the GPFS policyengine . . . . . . . . . . . . . . . 52Backing up file system configuration information 52Using APIs to develop backup applications. . . 52

Scale Out Backup and Restore (SOBAR) . . . . . 53Scheduling backups using Tivoli Storage Managerscheduler . . . . . . . . . . . . . . . 54Configuration reference for using Tivoli StorageManager with IBM Spectrum Scale . . . . . . 54

Options in the Tivoli Storage Managerconfiguration file dsm.sys . . . . . . . . 54Options in the Tivoli Storage Managerconfiguration file dsm.opt . . . . . . . . 56Base Tivoli Storage Manager client configurationfiles for IBM Spectrum Scale usage . . . . . 57

Restoring a subset of files or directories from a localfile system snapshot . . . . . . . . . . . 58Restoring a subset of files or directories from a localfileset snapshot . . . . . . . . . . . . . 59Restoring a subset of files or directories from localsnapshots using the sample script . . . . . . . 60

Chapter 4. Configuring the CES andprotocol configuration . . . . . . . . 63Configuring Cluster Export Services . . . . . . 63

Setting up Cluster Export Services shared rootfile system. . . . . . . . . . . . . . 63Configuring Cluster Export Services nodes . . . 64Configuring CES protocol service IP addresses . 64Deploying Cluster Export Services packages onIBM Spectrum Scale existing 4.1.1 and above . . 65Verifying the final CES configurations . . . . 66

Creating and configuring file systems and filesetsfor exports. . . . . . . . . . . . . . . 66Managing protocol services . . . . . . . . . 66

Configuring and enabling SMB and NFS protocolservices. . . . . . . . . . . . . . . 66Configuring and enabling the Object protocolservice . . . . . . . . . . . . . . . 68Disabling protocol services . . . . . . . . 69

Managing protocol user authentication . . . . . 69

Copyright IBM Corp. 2014, 2016 iii

||

Setting up authentication servers to configureprotocol user access . . . . . . . . . . 69Configuring authentication and ID mapping forfile access . . . . . . . . . . . . . . 76Managing user-defined authentication . . . . 89Configuring authentication for object access . . 93Deleting the authentication and the ID mappingconfiguration . . . . . . . . . . . . 105Listing the authentication configuration . . . 107Verifying the authentication services configuredin the system . . . . . . . . . . . . 107Modifying the authentication method . . . . 108Authentication limitations . . . . . . . . 109

General authentication considerations . . . . . 112Configuring with the spectrumscale installationtoolkit . . . . . . . . . . . . . . . . 112

Chapter 5. Managing protocol dataexports . . . . . . . . . . . . . . 113Managing SMB shares . . . . . . . . . . 113

Creating SMB share . . . . . . . . . . 113Changing SMB share configuration . . . . . 114Creating SMB share ACLs . . . . . . . . 114Removing SMB shares . . . . . . . . . 114Listing SMB shares . . . . . . . . . . 114Managing SMB shares using MMC . . . . . 115SMB share limitations . . . . . . . . . 119

Managing NFS exports . . . . . . . . . . 120Creating NFS exports. . . . . . . . . . 120Changing NFS export configuration . . . . . 120Removing NFS exports . . . . . . . . . 121Listing NFS exports . . . . . . . . . . 121

Multiprotocol exports . . . . . . . . . . 121Multiprotocol export considerations . . . . . . 121

Chapter 6. Managing object storage 123Understanding and managing Object services . . 123Understanding the mapping of OpenStackcommands to IBM Spectrum Scale administratorcommands . . . . . . . . . . . . . . 125Changing Object configuration values . . . . . 125Changing the object base configuration to enableS3 API emulation . . . . . . . . . . . . 126Configuring OpenStack EC2 credentials . . . . 128Managing OpenStack access control lists using S3API emulation . . . . . . . . . . . . . 129Managing object capabilities . . . . . . . . 130Mapping of storage policies to filesets . . . . . 130Administering storage policies for object storage 131

Creating storage policy for object compression 132Adding a region in a multi-region objectdeployment . . . . . . . . . . . . . . 132Administering a multi-region object deploymentenvironment. . . . . . . . . . . . . . 134Unified file and object access in IBM SpectrumScale . . . . . . . . . . . . . . . . 135

Identity management modes for unified file andobject access . . . . . . . . . . . . . 135Authentication in unified file and object access 140The objectizer process . . . . . . . . . 140

File path in unified file and object access . . . 141Administering unified file and object access . . 142In-place analytics using unified file and objectaccess . . . . . . . . . . . . . . . 149Limitations of unified file and object access . . 150Constraints applicable to unified file and objectaccess . . . . . . . . . . . . . . . 151Data ingestion examples. . . . . . . . . 151curl commands for unified file and object accessrelated user tasks . . . . . . . . . . . 152

Configuration files for IBM Spectrum Scale forobject storage . . . . . . . . . . . . . 153

Chapter 7. Managing disks . . . . . 157Displaying disks in a GPFS cluster . . . . . . 157Adding disks to a file system . . . . . . . . 158Deleting disks from a file system . . . . . . . 158Replacing disks in a GPFS file system . . . . . 160Additional considerations for managing disks . . 161Displaying GPFS disk states . . . . . . . . 162

Disk availability . . . . . . . . . . . 162Disk status . . . . . . . . . . . . . 162

Changing GPFS disk states and parameters . . . 163Changing your NSD configuration . . . . . . 165Changing NSD server usage and failback . . . . 166Enabling and disabling Persistent Reserve . . . . 166

Chapter 8. Managing GPFS quotas 169Enabling and disabling GPFS quota management 169Default quotas . . . . . . . . . . . . . 170Implications of quotas for different protocols . . . 172Explicitly establishing and changing quotas . . . 173Setting quotas for users on a per-project basis . . 174Checking quotas . . . . . . . . . . . . 176Listing quotas . . . . . . . . . . . . . 177Activating quota limit checking . . . . . . . 178Deactivating quota limit checking . . . . . . 178Changing the scope of quota limit checking . . . 179Creating file system quota reports . . . . . . 179Restoring quota files . . . . . . . . . . . 180

Chapter 9. Managing GUIadministrators . . . . . . . . . . . 183

Chapter 10. Managing GPFS accesscontrol lists . . . . . . . . . . . . 187Traditional GPFS ACL administration . . . . . 187

Setting traditional GPFS access control lists . . 188Displaying traditional GPFS access control lists 189Applying an existing traditional GPFS accesscontrol list . . . . . . . . . . . . . 189Changing traditional GPFS access control lists 190Deleting traditional GPFS access control lists 190

NFS V4 ACL administration . . . . . . . . 191NFS V4 ACL Syntax . . . . . . . . . . 191NFS V4 ACL translation . . . . . . . . . 193Setting NFS V4 access control lists . . . . . 194Displaying NFS V4 access control lists . . . . 194Applying an existing NFS V4 access control list 194Changing NFS V4 access control lists . . . . 194

iv IBM Spectrum Scale 4.2: Administration and Programming Reference

Deleting NFS V4 access control lists . . . . . 195Considerations when using GPFS with NFS V4ACLs . . . . . . . . . . . . . . . 195

NFS and GPFS . . . . . . . . . . . . . 195Exporting a GPFS file system using NFS . . . 195NFS usage of GPFS cache . . . . . . . . 198Synchronous writing using NFS . . . . . . 199Unmounting a file system after NFS export . . 199NFS automount considerations . . . . . . 199Clustered NFS and GPFS on Linux . . . . . 199

Authorizing protocol users . . . . . . . . . 200Authorizing file protocol users . . . . . . 200Authorizing object users. . . . . . . . . 209Authorization limitations . . . . . . . . 214

Chapter 11. GPFS commands . . . . 217gpfs.snap command . . . . . . . . . . . 221mmaddcallback command . . . . . . . . . 226mmadddisk command . . . . . . . . . . 239mmaddnode command . . . . . . . . . . 245mmafmconfig command. . . . . . . . . . 248mmafmctl command . . . . . . . . . . . 251mmafmlocal command . . . . . . . . . . 264mmapplypolicy command . . . . . . . . . 266mmauth command . . . . . . . . . . . 276mmbackup command . . . . . . . . . . 281mmbackupconfig command . . . . . . . . 290mmbuildgpl command . . . . . . . . . . 292mmcallhome command . . . . . . . . . . 293mmces command . . . . . . . . . . . . 304mmcesdr command . . . . . . . . . . . 313mmchattr command . . . . . . . . . . . 321mmchcluster command . . . . . . . . . . 327mmchconfig command . . . . . . . . . . 331mmchdisk command . . . . . . . . . . . 354mmcheckquota command . . . . . . . . . 361mmchfileset command . . . . . . . . . . 365mmchfs command. . . . . . . . . . . . 371mmchlicense command . . . . . . . . . . 377mmchmgr command . . . . . . . . . . . 379mmchnode command . . . . . . . . . . 381mmchnodeclass command . . . . . . . . . 385mmchnsd command . . . . . . . . . . . 388mmchpolicy command . . . . . . . . . . 391mmchpool command . . . . . . . . . . . 394mmchqos command . . . . . . . . . . . 396mmclone command . . . . . . . . . . . 400mmcrcluster command . . . . . . . . . . 403mmcrfileset command . . . . . . . . . . 408mmcrfs command . . . . . . . . . . . . 414mmcrnodeclass command . . . . . . . . . 424mmcrnsd command . . . . . . . . . . . 426mmcrsnapshot command . . . . . . . . . 431mmdefedquota command . . . . . . . . . 434mmdefquotaoff command . . . . . . . . . 437mmdefquotaon command . . . . . . . . . 440mmdefragfs command . . . . . . . . . . 443mmdelacl command . . . . . . . . . . . 446mmdelcallback command . . . . . . . . . 448mmdeldisk command . . . . . . . . . . 449mmdelfileset command . . . . . . . . . . 455

mmdelfs command . . . . . . . . . . . 458mmdelnode command . . . . . . . . . . 460mmdelnodeclass command . . . . . . . . . 463mmdelnsd command . . . . . . . . . . . 465mmdelsnapshot command . . . . . . . . . 467mmdf command . . . . . . . . . . . . 470mmdiag command . . . . . . . . . . . 473mmeditacl command . . . . . . . . . . . 478mmedquota command . . . . . . . . . . 481mmexportfs command . . . . . . . . . . 485mmfsck command. . . . . . . . . . . . 487mmfsctl command. . . . . . . . . . . . 496mmgetacl command . . . . . . . . . . . 500mmgetstate command . . . . . . . . . . 503mmhadoopctl command. . . . . . . . . . 506mmimgbackup command . . . . . . . . . 508mmimgrestore command . . . . . . . . . 511mmimportfs command . . . . . . . . . . 513mmlinkfileset command . . . . . . . . . . 517mmlsattr command . . . . . . . . . . . 519mmlscallback command . . . . . . . . . . 522mmlscluster command . . . . . . . . . . 524mmlsconfig command . . . . . . . . . . 526mmlsdisk command . . . . . . . . . . . 528mmlsfileset command . . . . . . . . . . 532mmlsfs command . . . . . . . . . . . . 536mmlslicense command . . . . . . . . . . 540mmlsmgr command . . . . . . . . . . . 542mmlsmount command . . . . . . . . . . 544mmlsnodeclass command . . . . . . . . . 546mmlsnsd command . . . . . . . . . . . 549mmlspolicy command . . . . . . . . . . 552mmlspool command . . . . . . . . . . . 554mmlsqos command . . . . . . . . . . . 556mmlsquota command . . . . . . . . . . 559mmlssnapshot command . . . . . . . . . 563mmmigratefs command . . . . . . . . . . 566mmmount command . . . . . . . . . . . 568mmnfs command . . . . . . . . . . . . 570mmnsddiscover command . . . . . . . . . 579mmobj command . . . . . . . . . . . . 581mmperfmon command . . . . . . . . . . 592mmpmon command . . . . . . . . . . . 602mmprotocoltrace command. . . . . . . . . 607mmpsnap command . . . . . . . . . . . 611mmputacl command . . . . . . . . . . . 614mmquotaoff command . . . . . . . . . . 617mmquotaon command . . . . . . . . . . 619mmremotecluster command . . . . . . . . 621mmremotefs command . . . . . . . . . . 624mmrepquota command . . . . . . . . . . 627mmrestoreconfig command. . . . . . . . . 631mmrestorefs command . . . . . . . . . . 635mmrestripefile command . . . . . . . . . 639mmrestripefs command . . . . . . . . . . 642mmrpldisk command. . . . . . . . . . . 648mmsdrrestore command. . . . . . . . . . 655mmsetquota command . . . . . . . . . . 657mmshutdown command. . . . . . . . . . 661mmsmb command. . . . . . . . . . . . 663mmsnapdir command . . . . . . . . . . 674

Contents v

mmstartup command. . . . . . . . . . . 678mmtracectl command. . . . . . . . . . . 680mmumount command . . . . . . . . . . 684mmunlinkfileset command . . . . . . . . . 687mmuserauth command . . . . . . . . . . 690mmwinservctl command . . . . . . . . . 709spectrumscale command. . . . . . . . . . 711

Chapter 12. GPFS programminginterfaces . . . . . . . . . . . . . 723gpfs_acl_t structure . . . . . . . . . . . 726gpfs_clone_copy() subroutine . . . . . . . . 727gpfs_clone_snap() subroutine . . . . . . . . 729gpfs_clone_split() subroutine . . . . . . . . 731gpfs_clone_unsnap() subroutine . . . . . . . 733gpfs_close_inodescan() subroutine . . . . . . 735gpfs_cmp_fssnapid() subroutine . . . . . . . 736gpfs_declone() subroutine . . . . . . . . . 738gpfs_direntx_t structure . . . . . . . . . . 740gpfs_direntx64_t structure . . . . . . . . . 742gpfs_fcntl() subroutine . . . . . . . . . . 744gpfs_fgetattrs() subroutine . . . . . . . . . 747gpfs_fputattrs() subroutine . . . . . . . . . 749gpfs_fputattrswithpathname() subroutine . . . . 751gpfs_free_fssnaphandle() subroutine . . . . . . 753gpfs_fssnap_handle_t structure . . . . . . . 754gpfs_fssnap_id_t structure . . . . . . . . . 755gpfs_fstat() subroutine . . . . . . . . . . 756gpfs_get_fsname_from_fssnaphandle() subroutine 758gpfs_get_fssnaphandle_by_fssnapid() subroutine 759gpfs_get_fssnaphandle_by_name() subroutine . . 761gpfs_get_fssnaphandle_by_path() subroutine . . . 763gpfs_get_fssnapid_from_fssnaphandle() subroutine 765gpfs_get_pathname_from_fssnaphandle()subroutine . . . . . . . . . . . . . . 767gpfs_get_snapdirname() subroutine . . . . . . 769gpfs_get_snapname_from_fssnaphandle()subroutine . . . . . . . . . . . . . . 771gpfs_getacl() subroutine . . . . . . . . . . 773gpfs_iattr_t structure . . . . . . . . . . . 775gpfs_iattr64_t structure . . . . . . . . . . 778gpfs_iclose() subroutine . . . . . . . . . . 782gpfs_ifile_t structure . . . . . . . . . . . 784gpfs_igetattrs() subroutine . . . . . . . . . 785gpfs_igetattrsx() subroutine. . . . . . . . . 787gpfs_igetfilesetname() subroutine . . . . . . . 789gpfs_igetstoragepool() subroutine. . . . . . . 791gpfs_iopen() subroutine . . . . . . . . . . 793gpfs_iopen64() subroutine . . . . . . . . . 795gpfs_iputattrsx() subroutine . . . . . . . . 797gpfs_iread() subroutine . . . . . . . . . . 800gpfs_ireaddir() subroutine . . . . . . . . . 802gpfs_ireaddir64() subroutine . . . . . . . . 804gpfs_ireadlink() subroutine . . . . . . . . . 806gpfs_ireadlink64() subroutine . . . . . . . . 808gpfs_ireadx() subroutine . . . . . . . . . . 810gpfs_iscan_t structure . . . . . . . . . . 813gpfs_lib_init() subroutine . . . . . . . . . 814gpfs_lib_term() subroutine . . . . . . . . . 815gpfs_next_inode() subroutine . . . . . . . . 816gpfs_next_inode64() subroutine . . . . . . . 818

gpfs_next_inode_with_xattrs() subroutine . . . . 820gpfs_next_inode_with_xattrs64() subroutine . . . 822gpfs_next_xattr() subroutine . . . . . . . . 824gpfs_opaque_acl_t structure . . . . . . . . 826gpfs_open_inodescan() subroutine . . . . . . 827gpfs_open_inodescan64() subroutine. . . . . . 830gpfs_open_inodescan_with_xattrs() subroutine . . 833gpfs_open_inodescan_with_xattrs64() subroutine 836gpfs_prealloc() subroutine . . . . . . . . . 839gpfs_putacl() subroutine . . . . . . . . . . 841gpfs_quotactl() subroutine . . . . . . . . . 843gpfs_quotaInfo_t structure . . . . . . . . . 846gpfs_seek_inode() subroutine . . . . . . . . 848gpfs_seek_inode64() subroutine . . . . . . . 850gpfs_stat() subroutine . . . . . . . . . . 852gpfs_stat_inode() subroutine . . . . . . . . 854gpfs_stat_inode64() subroutine . . . . . . . 856gpfs_stat_inode_with_xattrs() subroutine . . . . 858gpfs_stat_inode_with_xattrs64() subroutine . . . 860gpfsFcntlHeader_t structure . . . . . . . . 862gpfsGetFilesetName_t structure . . . . . . . 863gpfsGetReplication_t structure . . . . . . . . 864gpfsGetSetXAttr_t structure . . . . . . . . 866gpfsGetSnapshotName_t structure . . . . . . 868gpfsGetStoragePool_t structure . . . . . . . 869gpfsListXAttr_t structure . . . . . . . . . 870gpfsRestripeData_t structure . . . . . . . . 871gpfsSetReplication_t structure . . . . . . . . 873gpfsSetStoragePool_t structure. . . . . . . . 875

Chapter 13. GPFS user exits . . . . . 877mmsdrbackup user exit . . . . . . . . . . 878nsddevices user exit . . . . . . . . . . . 879syncfsconfig user exit. . . . . . . . . . . 880

Chapter 14. Considerations for GPFSapplications . . . . . . . . . . . . 881Exceptions to Open Group technical standards . . 881Determining if a file system is controlled by GPFS 881GPFS exceptions and limitations to NFS V4 ACLs 882

Linux ACLs and extended attributes. . . . . 882General NFS V4 Linux exceptions and limitations 883Considerations for the use of direct I/O(O_DIRECT). . . . . . . . . . . . . . 883

Chapter 15. File system formatchanges between versions of GPFS . 885

Accessibility features for IBMSpectrum Scale . . . . . . . . . . 889Accessibility features . . . . . . . . . . . 889Keyboard navigation . . . . . . . . . . . 889IBM and accessibility . . . . . . . . . . . 889

Notices . . . . . . . . . . . . . . 891Trademarks . . . . . . . . . . . . . . 893Terms and conditions for product documentation 893IBM Online Privacy Statement. . . . . . . . 894

vi IBM Spectrum Scale 4.2: Administration and Programming Reference

Glossary . . . . . . . . . . . . . 895

Index . . . . . . . . . . . . . . . 901

Contents vii

viii IBM Spectrum Scale 4.2: Administration and Programming Reference

Tables

1. IBM Spectrum Scale library information units xi2. Conventions . . . . . . . . . . . . xiv3. Configuration attributes on the mmchconfig

command . . . . . . . . . . . . . 124. COMPRESSED and illCompressed indicators 385. Set QoS classes to unlimited . . . . . . . 416. Allocate the available IOPS . . . . . . . 417. Authentication requirements for each file

access protocol. . . . . . . . . . . . 918. Object services and object protocol nodes 1249. Object input behavior in unified_mode 137

10. Configuration options for [swift-constraints]in swift.conf . . . . . . . . . . . . 151

11. Configurable options for [DEFAULT] inobject-server-sof.conf . . . . . . . . . 153

12. Configurable options for [capabilities] inspectrum-scale-object.conf . . . . . . . 155

13. Configuration options for [DEFAULT] inspectrum-scale-objectizer.conf . . . . . . 155

14. Configuration options for[IBMOBJECTIZER-LOGGER] inspectrum-scale-objectizer.conf . . . . . . 155

15. Removal of a file with ACL entries DELETEand DELETE_CHILD . . . . . . . . . 193

16. ACL permissions required to work on filesand directories, while using SMB protocol(table 1 of 2) . . . . . . . . . . . . 203

17. ACL permissions required to work on filesand directories, while using SMB protocol(table 2 of 2) . . . . . . . . . . . . 204

18. ACL permissions required to work on filesand directories, while using NFS protocol(table 1 of 2) . . . . . . . . . . . . 205

19. ACL permissions required to work on filesand directories, while using NFS protocol(table 2 of 2) . . . . . . . . . . . . 205

20. Commands and reference to manage ACLtasks . . . . . . . . . . . . . . 208

21. ACL options that are available to manipulateobject read ACLs . . . . . . . . . . 212

22. GPFS commands . . . . . . . . . . 21723. Global events and supported parameters 23124. Local events and supported parameters 23225. key-value . . . . . . . . . . . . . 29626. key-value . . . . . . . . . . . . . 29727. Restoring a global snapshot . . . . . . . 63628. Restoring a fileset snapshot . . . . . . . 63629. GPFS programming interfaces . . . . . . 72330. GPFS user exits . . . . . . . . . . . 877

Copyright IBM Corp. 2014, 2016 ix

||||

x IBM Spectrum Scale 4.2: Administration and Programming Reference

About this information

This edition applies to IBM Spectrum Scale version 4.2 for AIX, Linux, and Windows.

IBM Spectrum Scale is a file management infrastructure, based on IBM General Parallel File System(GPFS) technology, that provides unmatched performance and reliability with scalable access to criticalfile data.

To find out which version of IBM Spectrum Scale is running on a particular AIX node, enter:lslpp -l gpfs\*

To find out which version of IBM Spectrum Scale is running on a particular Linux node, enter:rpm -qa | grep gpfs

To find out which version of IBM Spectrum Scale is running on a particular Windows node, open thePrograms and Features control panel. The IBM Spectrum Scale installed program name includes theversion number.

Which IBM Spectrum Scale information unit provides the information you need?

The IBM Spectrum Scale library consists of the information units listed in Table 1.

To use these information units effectively, you must be familiar with IBM Spectrum Scale and the AIX,Linux, or Windows operating system, or all of them, depending on which operating systems are in use atyour installation. Where necessary, these information units provide some background information relatingto AIX, Linux, or Windows; however, more commonly they refer to the appropriate operating systemdocumentation.

Note: Throughout this documentation, the term Linux refers to all supported distributions of Linux,unless otherwise specified.

Table 1. IBM Spectrum Scale library information units

Information unit Type of information Intended users

IBM Spectrum Scale:Administration and ProgrammingReference

This information unit explains how todo the following:

v Use the commands, programminginterfaces, and user exits unique toGPFS

v Manage clusters, file systems, disks,and quotas

v Export a GPFS file system using theNetwork File System (NFS) protocol

System administrators or programmersof GPFS systems

Copyright IBM Corp. 2014, 2016 xi

Table 1. IBM Spectrum Scale library information units (continued)


IBM Spectrum Scale: AdvancedAdministration Guide

This information unit explains how touse the following advanced features ofGPFS:

v Accessing GPFS file systems fromother GPFS clusters

v Policy-based data management forGPFS

v Creating and maintaining snapshotsof GPFS file systems

v Establishing disaster recovery foryour GPFS cluster

v Monitoring GPFS I/O performancewith the mmpmon command

v Miscellaneous advancedadministration topics

System administrators or programmersseeking to understand and use theadvanced features of GPFS

IBM Spectrum Scale: Concepts,Planning, and Installation Guide

This information unit providesinformation about the following topics:

v Introducing GPFSv GPFS architecturev Planning concepts for GPFSv Installing GPFSv Migration, coexistence and

compatibility

v Applying maintenancev Configuration and tuningv Uninstalling GPFS

System administrators, analysts,installers, planners, and programmers ofGPFS clusters who are very experiencedwith the operating systems on whicheach GPFS cluster is based

xii IBM Spectrum Scale 4.2: Administration and Programming Reference

Table 1. IBM Spectrum Scale library information units (continued)


IBM Spectrum Scale: DataManagement API Guide

This information unit describes the DataManagement Application ProgrammingInterface (DMAPI) for GPFS.

This implementation is based on TheOpen Group's System Management:Data Storage Management (XDSM) APICommon Applications Environment(CAE) Specification C429, The OpenGroup, ISBN 1-85912-190-Xspecification. The implementation iscompliant with the standard. Someoptional features are not implemented.

The XDSM DMAPI model is intendedmainly for a single-node environment.Some of the key concepts, such assessions, event delivery, and recovery,required enhancements for amultiple-node environment such asGPFS.

Use this information if you intend towrite application programs to do thefollowing:

v Monitor events associated with aGPFS file system or with anindividual file

v Manage and maintain GPFS filesystem data

Application programmers who areexperienced with GPFS systems andfamiliar with the terminology andconcepts in the XDSM standard

IBM Spectrum Scale: ProblemDetermination Guide

This information unit containsexplanations of GPFS error messagesand explains how to handle problemsyou may encounter with GPFS.

System administrators of GPFS systemswho are experienced with thesubsystems used to manage disks andwho are familiar with the conceptspresented in the IBM Spectrum Scale:Concepts, Planning, and Installation Guide

Prerequisite and related informationFor updates to this information, see IBM Spectrum Scale in IBM Knowledge Center (www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html).

For the latest support information, see the IBM Spectrum Scale FAQ in IBM Knowledge Center(www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.html).

Conventions used in this informationTable 2 on page xiv describes the typographic conventions used in this information. UNIX file nameconventions are used throughout this information.

Note: Users of IBM Spectrum Scale for Windows must be aware that on Windows, UNIX-style filenames need to be converted appropriately. For example, the GPFS cluster configuration data is stored inthe /var/mmfs/gen/mmsdrfs file. On Windows, the UNIX namespace starts under the %SystemDrive%\cygwin64 directory, so the GPFS cluster configuration data is stored in the C:\cygwin64\var\mmfs\gen\mmsdrfs file.

About this information xiii

http://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.htmlhttp://www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.html

Table 2. Conventions

Convention Usage

bold Bold words or characters represent system elements that you must use literally, such ascommands, flags, values, and selected menu options.

Depending on the context, bold typeface sometimes represents path names, directories, or filenames.

bold underlined bold underlined keywords are defaults. These take effect if you do not specify a differentkeyword.

constant width Examples and information that the system displays appear in constant-width typeface.

Depending on the context, constant-width typeface sometimes represents path names,directories, or file names.

italic Italic words or characters represent variable values that you must supply.

Italics are also used for information unit titles, for the first use of a glossary term, and forgeneral emphasis in text.

Angle brackets (less-than and greater-than) enclose the name of a key on the keyboard. Forexample, refers to the key on your terminal or workstation that is labeled with theword Enter.

\ In command examples, a backslash indicates that the command or coding example continueson the next line. For example:

mkcondition -r IBM.FileSystem -e "PercentTotUsed > 90" \-E "PercentTotUsed < 85" -m p "FileSystem space used"

{item} Braces enclose a list from which you must choose an item in format and syntax descriptions.

[item] Brackets enclose optional items in format and syntax descriptions.

The notation indicates a control character sequence. For example, meansthat you hold down the control key while pressing .

item... Ellipses indicate that you can repeat the preceding item one or more times.

| In synopsis statements, vertical lines separate a list of choices. In other words, a vertical linemeans Or.

In the left margin of the document, vertical lines indicate technical changes to theinformation.

How to send your commentsYour feedback is important in helping us to produce accurate, high-quality information. If you have anycomments about this information or any other IBM Spectrum Scale documentation, send your commentsto the following e-mail address:

[email protected]

Include the publication title and order number, and, if applicable, the specific location of the informationabout which you have comments (for example, a page number or a table number).

To contact the IBM Spectrum Scale development organization, send your comments to the followinge-mail address:

[email protected]

xiv IBM Spectrum Scale 4.2: Administration and Programming Reference

Summary of changes

This topic summarizes changes to the IBM Spectrum Scale licensed program and the IBM Spectrum Scalelibrary. Within each information unit in the library, a vertical line (|) to the left of text and illustrationsindicates technical changes or additions made to the previous edition of the information.

Summary of changesfor IBM Spectrum Scale version 4 release 2as updated, November 2015

Changes to this release of the IBM Spectrum Scale licensed program and the IBM Spectrum Scale libraryinclude the following:

Cluster Configuration Repository (CCR): Backup and restoreYou can backup and restore a cluster that has Cluster Configuration Repository (CCR) enabled. Inthe mmsdrbackup user exit, the type of backup that is created depends on the configuration ofthe cluster. If the Cluster Configuration Repository (CCR) is enabled, then a CCR backup iscreated. Otherwise, a mmsdrfs backup is created. In the mmsdrrestore command, if theconfiguration file is a Cluster Configuration Repository (CCR) backup file, then you must specifythe -a option. All the nodes in the cluster are restored.

Changes in IBM Spectrum Scale for object storage

Object capabilitiesObject capabilities describe the object protocol features that are configured in the IBMSpectrum Scale cluster such as unified file and object access, multi-region objectdeployment, and S3 API emulation. For more information, see the following topics:v Object capabilities in IBM Spectrum Scale: Concepts, Planning, and Installation Guidev Managing object capabilities in IBM Spectrum Scale: Administration and Programming

Reference

Storage policies for object storageStorage policies enable segmenting of the object storage within a single cluster for varioususe cases. Currently, OpenStack Swift supports storage polices that allow you to definethe replication settings and location of objects in a cluster. IBM Spectrum Scale enhancesstorage policies to add compression and unified file and object access functions for objectstorage. For more information, see the following topics:v Storage policies for object storage in IBM Spectrum Scale: Concepts, Planning, and Installation

Guide

v Mapping of storage policies to filesets in IBM Spectrum Scale: Administration andProgramming Reference

v Administering storage policies for object storage in IBM Spectrum Scale: Administration andProgramming Reference

Multi-region object deploymentThe main purpose of the object protocol is to enable the upload and download of objectdata. When clients have a fast connection to the cluster, the network delay is minimal.However, when client access to object data is over a WAN or a high-latency network, thenetwork can introduce an unacceptable delay and affect quality-of-service metrics. Toimprove that response time, you can create a replica of the data in a cluster closer to theclients using the active-active multi-region replication support in OpenStack Swift.Multi-region can also be used to distribute the object load over several clusters to reducecontention in the file system. For more information, see the following topics:

Copyright IBM Corp. 2014, 2016 xv

v Overview of multi-region object deployment in IBM Spectrum Scale: Concepts, Planning, andInstallation Guide

v Planning for multi-region object deployment in IBM Spectrum Scale: Concepts, Planning, andInstallation Guide

v Enabling multi-region object deployment initially in IBM Spectrum Scale: Concepts, Planning,and Installation Guide

v Adding a region in a multi-region object deployment in IBM Spectrum Scale: Administrationand Programming Reference

v Administering a multi-region object deployment environment in IBM Spectrum Scale:Administration and Programming Reference

Unified file and object accessUnified file and object access allows users to access the same data as an object and as afile. Data can be stored and retrieved through IBM Spectrum Scale for object storage or asfiles from POSIX, NFS, and SMB interfaces. For more information, see the followingtopics:v Unified file and object access overview in IBM Spectrum Scale: Concepts, Planning, and

Installation Guide

v Planning for unified file and object access in IBM Spectrum Scale: Concepts, Planning, andInstallation Guide

v Installing and using unified file and object access in IBM Spectrum Scale: Concepts, Planning,and Installation Guide

v Unified file and object access in IBM Spectrum Scale in IBM Spectrum Scale: Administrationand Programming Reference

S3 access control lists (ACLs) supportIBM Spectrum Scale for object storage supports S3 access control lists (ACLs) on bucketsand objects. For more information, see Managing OpenStack access control lists using S3 APIemulation in IBM Spectrum Scale: Administration and Programming Reference.

Changes in IBM Spectrum Scale for Linux on z Systems

v Compression supportv AFM-based Async Disaster Recovery (AFM DR) supportv IBM Spectrum Protect Backup-Archive and Space Management client supportv Support for all editions:

Express

Standard Advanced (without encryption)

For more information about current requirements and limitations of IBM Spectrum Scale forLinux on z Systems, see Q2.25 of IBM Spectrum Scale FAQ.

Change in AFM-based Async Disaster Recovery (AFM DR)

v Support for IBM Spectrum Scale for Linux on z Systems

File compressionWith file compression, you can reclaim some of the storage space occupied by infrequentlyaccessed files. Run the mmchattr command or the mmapplypolicy command to identify andcompress a few files or many files. Run file compression synchronously or defer it. If you defer it,you can run the mmrestripefile or mmrestripefs to complete the compression. You candecompress files with the same commands used to compress files. When a compressed file is readit is decompressed on the fly and remains compressed on disk. When a compressed file isoverwritten, the parts of the file that overlap with the changed data are decompressed on disksynchronously in the granularity of ten data blocks. File compression in this release is designed to

xvi IBM Spectrum Scale 4.2: Administration and Programming Reference

be used only for compressing cold data or write-once objects and files. Compressing other typesof data can result in performance degradation. File compression uses the zlib data compressionlibrary and favors saving space over speed.

GUI serversThe IBM Spectrum Scale system provides a GUI that can be used for managing and monitoringthe system. Any server that provides this GUI service is referred to as a GUI server. If you needGUI service in the system, designate at least two nodes as GUI servers in the cluster. A maximumof three nodes can be designated as GUI servers. For more information on installing IBMSpectrum Scale using the GUI, see IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

IBM Spectrum Scale management GUIThe management GUI helps to manage and monitor the IBM Spectrum Scale system. You canperform the following tasks through management GUI:v Monitoring the performance of the system based on various aspectsv Monitoring system healthv Managing file systemsv Creating filesets and snapshotsv Managing Objects and NFS and SMB data exportsv Creating administrative users and defining roles for the usersv Creating object users and defining roles for themv Defining default, user, group, and fileset quotasv Monitoring the capacity details at various levels such as file system, pools, filesets, users, and

user groups

Hadoop Support for IBM Spectrum Scale IBM Spectrum Scale has been extended to work seamlessly in the Hadoop ecosystem and isavailable through a feature called File Placement Optimizer (FPO). Storing your Hadoop datausing FPO allows you to gain advanced functions and the high I/O performance required formany big data operations. FPO provides Hadoop compatibility extensions to replace HDFS in aHadoop ecosystem, with no changes required to Hadoop applications.

You can deploy a IBM Spectrum Scale using FPO as a file system platform for big data analytics.The topics in this guide covers a variety of Hadoop deployment architectures, including IBMBigInsights, Platform Symphony, or with a Hadoop distribution from another vendor to workwith IBM Spectrum Scale.

IBM Spectrum Scale offers two kinds of interfaces for Hadoop applications to access File Systemdata. One is IBM Spectrum Scale connector, which aligns with Hadoop Compatible File Systemarchitecture and APIs. The other is HDFS protocol, which provides a HDFS compatible interfaces.

For more information, see the following sections in the IBM Spectrum Scale: AdvancedAdministration Guide:v Hadoop support for IBM Spectrum Scalev Configuring FPOv Hadoop connectorv HDFS protocol

IBM Spectrum Scale installation GUIYou can use the installation GUI to install the IBM Spectrum Scale system. For more informationon how to launch the GUI installer, see the Installing IBM Spectrum Scale using the graphical userinterface (GUI) section in IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

Performance Monitoring Tool using the Installation KitThe usage statement and optional arguments have changed during the installation of the toolkit.The new usage statement with options is as follows:spectrumscale config perfmon [-h] [-l] [-r {on,off}]

Summary of changes xvii

For more information, see IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

Protocols cluster disaster recovery (DR)You can use the mmcesdr command to perform DR setup, failover, failback, backup, and restoreactions. Protocols cluster DR uses the capabilities of Active File Management based AsyncDisaster Recovery (AFM DR) to provide a solution that allows an IBM Spectrum Scale cluster tofail over to another cluster and fail back, and backup and restore the protocol configurationinformation in cases where a secondary cluster is not available. For more information, seeProtocols cluster disaster recovery in IBM Spectrum Scale: Advanced Administration Guide.

Quality of Service for I/O operations (QoS)You can use the QoS capability to prevent I/O-intensive, long-running GPFS commands, calledmaintenance commands, from dominating file system performance and significantly delayingnormal tasks that also compete for I/O resources. Determine the maximum capacity of your filesystem in I/O operations per second (IOPS) with the new mmlsqos command. With the newmmchqos command, assign a smaller share of IOPS to the QoS maintenance class, whichincludes all the maintenance commands. Maintenance command instances that are running at thesame time compete for the IOPS allocated to the maintenance class, and are not allowed toexceed that limit.

Security mode for new clustersStarting with IBM Spectrum Scale V4.2, the default security mode for new clusters isAUTHONLY. The mmcrcluster command sets the security mode to AUTHONLY when it createsthe cluster and automatically generates a public/private key pair for authenticating the cluster. Inthe AUTHONLY security mode, the sending and receiving nodes authenticate each other with aTLS handshake and then close the TLS connection. Communication continues in the clear. Thenodes do not encrypt transmitted data and do not check data integrity.

In IBM Spectrum Scale V4.1 or earlier, the default security mode is EMPTY. If you update acluster from IBM Spectrum Scale V4.1 to V4.2 or later by running mmchconfig release=LATEST, thecommand checks the security mode. If the mode is EMPTY, the command issues a warningmessage but does not change the security mode of the cluster.

SnapshotsYou can display information about a global snapshot without displaying information about filesetsnapshots with the same name. You can display information about a fileset snapshot withoutdisplaying information about other snapshots that have the same name but are snapshots of otherfilesets.

spectrumscale OptionsThe spectrumscale command options for installing GPFS and deploying protocols have changedto remove config enable and to add config perf. For more information, see IBM Spectrum Scale:Concepts, Planning, and Installation Guide.

New options have been added to spectrumscale setup and spectrumscale deploy to disableprompting for the encryption/decryption secret. Note that if spectrumscale setup --storesecret isused, passwords will not be secure. New properties have been added to spectrumscale cofigobject for setting password data instead of doing so through enable object. For moreinformation, see IBM Spectrum Scale: Administration and Programming Reference.

The spectrumscale options for managing share ACLs have been added. For more information, seeIBM Spectrum Scale: Administration and Programming Reference.

ssh and scp wrapper scriptsStarting with IBM Spectrum Scale V4.2, a cluster can be configured to use ssh and scp wrappers.The wrappers allow GPFS to run on clusters where remote root login through ssh is disabled. Formore information, see the help topic "Running IBM Spectrum Scale without remote root login" inthe IBM Spectrum Scale: Administration and Programming Reference.

Documented commands, structures, and subroutinesThe following lists the modifications to the documented commands, structures, and subroutines:

xviii IBM Spectrum Scale 4.2: Administration and Programming Reference

New commandsThe following commands are new:v mmcallhomev mmcesdrv mmchqosv mmlsqos

New structuresThere are no new structures.

New subroutinesThere are no new subroutines.

Changed commandsThe following commands were changed:v mmadddiskv mmaddnodev mmapplypolicyv mmauthv mmbackupv mmcesv mmchattrv mmchclusterv mmchconfigv mmchdiskv mmcheckquotav mmchnodev mmcrclusterv mmdefragfsv mmdeldiskv mmdelfilesetv mmdelsnapshotv mmdfv mmfileidv mmfsckv mmlsattrv mmlsclusterv mmlsconfigv mmlssnapshotv mmnfsv mmobjv mmperfmonv mmprotocoltracev mmremotefsv mmrestripefilev mmrestripefsv mmrpldiskv mmsdrbackup

Summary of changes xix

v mmsdrrestorev mmsmbv mmuserauthv spectrumscale

Changed structuresThere are no changed structures.

Changed subroutinesThere are no changed subroutines.

Deleted commandsThere are no deleted commands.

Deleted structuresThere are no deleted structures.

Deleted subroutinesThere are no deleted subroutines.

MessagesThe following lists the new, changed, and deleted messages:

New messages6027-2354, 6027-2355, 6027-2356, 6027-2357, 6027-2358, 6027-2359, 6027-2360, 6027-2361,6027-2362, 6027-3913, 6027-3914, 6027-3107, 6027-4016, 6027-3317, 6027-3318, 6027-3319,6027-3320, 6027-3405, 6027-3406, 6027-3582, 6027-3583, 6027-3584, 6027-3585, 6027-3586,6027-3587, 6027-3588, 6027-3589, 6027-3590, 6027-3591, 6027-3592, 6027-3593

Changed messages6027-2299, 6027-887, 6027-888

Deleted messagesNone.

xx IBM Spectrum Scale 4.2: Administration and Programming Reference

Chapter 1. Performing GPFS administration tasks

Before you perform GPFS administration tasks, review topics such as getting started with GPFS,requirements for administering a GPFS file system, and common command principles.

For information on getting started with GPFS, see the IBM Spectrum Scale: Concepts, Planning, andInstallation Guide. This includes:1. Installing GPFS2. GPFS cluster creation considerations3. Configuring and tuning your system for GPFS4. Starting GPFS5. Network Shared Disk creation considerations6. File system creation considerations

This guide covers the administration and maintenance of GPFS and your file systems, and includes thefollowing topics:1. Requirements for administering a GPFS file system and Common GPFS command principles on

page 32. Chapter 2, Managing the GPFS cluster, on page 73. Chapter 3, Managing file systems, on page 274. Chapter 7, Managing disks, on page 1575. Chapter 8, Managing GPFS quotas, on page 1696. Chapter 10, Managing GPFS access control lists, on page 1877. Chapter 11, GPFS commands, on page 2178. Chapter 12, GPFS programming interfaces, on page 7239. Chapter 13, GPFS user exits, on page 877

10. Chapter 14, Considerations for GPFS applications, on page 88111. Chapter 15, File system format changes between versions of GPFS, on page 885

For more advanced GPFS administration topics, see the IBM Spectrum Scale: Advanced AdministrationGuide.

Requirements for administering a GPFS file systemRoot authority is required to perform all GPFS administration tasks except those with a function limitedto listing certain GPFS operating characteristics or modifying individual user file attributes.

On Windows, root authority normally means users in the Administrators group. However, for clusterswith both Windows and UNIX nodes, only the special Active Directory domain user root qualifies ashaving root authority for the purposes of administering GPFS. For more information on GPFSprerequisites, see the topic Installing GPFS prerequisites in the IBM Spectrum Scale: Concepts, Planning, andInstallation Guide.

The GPFS commands are designed to maintain the appropriate environment across all nodes in thecluster. To achieve this goal, the GPFS commands use the remote shell and remote file copy commandsthat you specify on either the mmcrcluster or the mmchcluster command.

The default remote commands are ssh and scp, but you can designate any other remote commandsprovided they have compatible syntax.

Copyright IBM Corporation IBM 2014, 2016 1

In principle, you can issue GPFS administration commands from any node in the cluster. The nodes thatyou plan to use for administering GPFS must be able to execute remote shell commands on themselvesand on any other node in the cluster. They must do so without the use of a password and withoutproducing any extraneous messages. Similarly, the nodes on which the GPFS commands are issued mustbe able to copy files to and from any other node in the cluster. And the nodes must do so without theuse of a password and without producing any extraneous messages.

The way the passwordless access is achieved depends on the particular remote execution program andauthentication mechanism that is used. For example, for rsh and rcp, you might need a properlyconfigured .rhosts file in the root user's home directory on each node in the GPFS cluster. If the remoteprogram is ssh, you can use private identity files that do not have a password. Or, if the identity file ispassword-protected, you can use the ssh-agent utility to establish an authorized session before you issuemm commands.

You can avoid configuring your GPFS nodes to allow remote access to the root user ID, by using sudowrapper scripts to run GPFS administrative commands. See Running IBM Spectrum Scale withoutremote root login on page 21.

GPFS does not need to know which nodes are being used for administration purposes. It is theadministrator's responsibility to issue mm commands only from nodes that are properly configured andcan access the rest of the nodes in the cluster.

Note: If your cluster includes Windows nodes, you must designate ssh and scp as the remotecommunication program.

adminMode configuration attributeGPFS recognizes the adminMode configuration attribute. It specifies whether all nodes in the cluster willbe used for issuing GPFS administration commands or just a subset of the nodes.

The adminMode attribute is set with the mmchconfig command and can have one of two values:

allToAllIndicates that all nodes in the cluster can be used for running GPFS administration commands andthat all nodes are able to execute remote commands on any other node in the cluster without theneed of a password.

The major advantage of this mode of operation is that GPFS can automatically recover missing orcorrupted configuration files in almost all circumstances. The major disadvantage is that all nodes inthe cluster must have root level access to all other nodes.

centralIndicates that only a subset of the nodes will be used for running GPFS commands and that onlythose nodes will be able to execute remote commands on the rest of the nodes in the cluster withoutthe need of a password.

The major advantage of this mode of administration is that the number of nodes that must have rootlevel access to the rest of the nodes is limited and can be as low as one. The disadvantage is thatGPFS may not be able to automatically recover from loss of certain configuration files. For example, ifthe SSL key files are not present on some of the nodes, the operator may have to intervene to recoverthe missing data. Similarly, it may be necessary to shut down GPFS when adding new quorumnodes. If an operator intervention is needed, you will see appropriate messages in the GPFS log or onthe screen.

Note List:

1. If the GPFS cluster is configured to support Clustered NFS (CNFS), all CNFS member nodes mustbelong to the subset of nodes that are able to execute remote commands without the need of apassword.

2 IBM Spectrum Scale 4.2: Administration and Programming Reference

2. If Tivoli Storage Manager (TSM) server is used to back up the GPFS file system data, the nodesthat are used as TSM clients must belong to the subset of nodes that are able to execute remotecommands without the need of a password.

3. Windows GPFS clusters typically use central mode. allToAll mode requires that the GPFSAdministrative service (mmwinserv) be configured to run as the special domain root account. Formore information, see the Installing GPFS on Windows nodes topic in the IBM Spectrum Scale:Concepts, Planning, and Installation Guide.

Clusters created with the GPFS 3.3 or later level of the code have adminMode set to central by default.Clusters migrated from GPFS 3.2 or earlier versions will continue to operate as before and will haveadminMode set to allToAll.

You can change the mode of operations at any time with the help of the mmchconfig command. Forexample, to switch the mode of administration from allToAll to central, issue:mmchconfig adminMode=central

Use the mmlsconfig adminMode command to display the mode of administration currently in effect forthe cluster.

Common GPFS command principlesThere are some common principles that you should keep in mind when you are running GPFScommands.

Those principles include:v Unless otherwise noted, GPFS commands can be run from any node in the cluster. Exceptions are

commands that are not supported in a particular operating system environment. Certain commandsmay additionally require the affected file system to be mounted.

v GPFS supports the "no" prefix on all Boolean type long (or dash-dash) options.

Specifying nodes as input to GPFS commandsMany GPFS commands accept a node or multiple nodes as part of their input, using the -N flag.

Nodes can be specified to GPFS commands in a variety of ways:

NodeA representation of an individual node, which can be any of these:v Short GPFS administration node interface name.v Long GPFS administration node interface name.v Short GPFS daemon node interface name.v Long GPFS daemon node interface name.v IP address corresponding to the GPFS daemon node interface.v GPFS node number.

Node - NodeA node range, indicated by specifying two node numbers separated by a hyphen (-), with the firstnode number being less than or equal to the second node number. For example, node range 3-8specifies the nodes with node numbers 3, 4, 5, 6, 7, and 8.

NodeClassA set of nodes that are grouped into system-defined node classes or user-defined node classes. Thesystem-defined node classes that are known to GPFS are:

allAll of the nodes in the GPFS cluster.

Chapter 1. Performing GPFS administration tasks 3

clientnodesAll nodes that do not participate in file system administration activities.

localhostThe node on which the command is running.

managernodesAll nodes in the pool of nodes from which file system managers and token managers areselected.

mountFor commands involving a file system, all of the local nodes on which the file system is mounted(nodes in remote clusters are always excluded, even when they mount the file system inquestion).

nonquorumnodesAll of the non-quorum nodes in the GPFS cluster.

nsdnodesAll of the NSD server nodes in the GPFS cluster.

quorumnodesAll of the quorum nodes in the GPFS cluster.

User-defined node classes are created with the mmcrnodeclass command. After a node class iscreated, it can be specified as an argument on commands that accept the -N NodeClass option.User-defined node classes are managed with the mmchnodeclass, mmdelnodeclass, andmmlsnodeclass commands.

NodeFileA file that contains a list of nodes. A node file can contain individual nodes or node ranges.

For commands operating on a file system, the stripe group manager node is always implicitly included inthe node list. Not every GPFS command supports all of the node specification options described in thistopic. To learn what kinds of node specifications are supported by a particular GPFS command, see therelevant command description in Chapter 11, GPFS commands, on page 217.

Stanza filesThe input to a number of GPFS commands can be provided in a file organized in a stanza format.

A stanza is a series of whitespace-separated tokens that can span multiple lines. The beginning of astanza is indicated by the presence of a stanza identifier as the first token on a line. Stanza identifiersconsist of the % (percent sign) character, followed by a keyword, and ending with the : (colon) character.For example, %nsd: indicates the beginning of an NSD stanza.

A stanza identifier is followed by one or more stanza clauses describing different properties of the object.A stanza clause is defined as an Attribute=value pair.

Lines that start with the # (pound sign) character are considered comment lines and are ignored.Similarly, you can imbed inline comments following a stanza clause; all text after the # character isconsidered a comment.

The end of a stanza is indicated by one of the following:v a line that represents the beginning of a new stanzav a blank linev a non-comment line that does not contain the = character

GPFS recognizes a number of stanzas:


%nsd:NSD stanza

%pdisk:Physical disk stanza

%vdisk:Virtual disk stanza

%da:Declustered array stanza

%rg:Recovery group stanza

The details are documented under the corresponding commands.

For more information about the GPFS Native RAID commands that use stanzas, see IBM Spectrum ScaleRAID: Administration.

A stanza file can contain multiple types of stanzas. Commands that accept input in the form of stanzafiles expect the stanzas to be syntactically correct but will ignore stanzas that are not applicable to theparticular command. Similarly, if a particular stanza clause has no meaning for a given command, it isignored.

For backward compatibility, a stanza file may also contain traditional NSD descriptors, although their useis discouraged.

Here is what a stanza file may look like:# Sample file containing two NSD stanzas

# Example for an NSD stanza with imbedded comments%nsd: nsd=DATA5 # my name for this NSDdevice=/dev/hdisk5 # device name on node k145n05usage=dataOnly# List of server nodes for this diskservers=k145n05,k145n06failureGroup=2pool=dataPoolA

# Example for a directly attached disk; most values are allowed to default%nsd: nsd=DATA6 device=/dev/hdisk6 failureGroup=3

Listing active GPFS commandsYou can list the active GPFS commands that are running on the file system manager node.

Most GPFS commands run within the GPFS daemon on the file system manager node. Even if you start acommand on another node of the cluster, the node typically sends the command to the file systemmanager node to be executed. (Two exceptions are the mmdiag command and the mmfsadm dumpcommand, which run on the node where they were started.)

To list the active commands on the file system manager node, follow these steps:1. Enter the mmlsmgr command with no parameters to discover which node is the file system manager

node. In the following example, the mmlsmgr command reports that node05 is the file systemmanager node:

Chapter 1. Performing GPFS administration tasks 5

|

|

||||

|

|||

# mmlsmgrfile system manager node---------------- ------------------gpfs1 192.168.145.14 (node05)

Cluster manager node: 192.168.145.13 (node03)

2. Go to the command console on the file system manager node and enter mmdiag --commands:# mmdiag --commands=== mmdiag: commands ===CrHashTable 0x1167A28F0 n 2

cmd sock 24 cookie 2233688162 owner 38076509 id 0x3FE6046C2700000D(#13) uses 1type 76 start 1460415325.957724 flags 0x106 SG none line mmdiag --commands

cmd sock 12 cookie 521581069 owner 57606185 id 0x3FE6046C2700000C(#12) uses 1type 13 start 1460415323.336314 flags 0x117 SG gpfs1 line mmrestripefs /dev/business1 -m

The output indicates that two GPFS commands are running: the mmdiag --commands command thatyou just entered and the mmrestripefs command, which was started from another node.

Note: The output contains two lines about active commands. Each line begins with the term cmd andwraps to the next line. You might be interested in the following fields:

start The system time at which the command was received.

SG The name of the file system, or None.

line The command as received by the GPFS daemon.

The remaining input is detailed debugging data that is used for product support.


||||||

|

|||||||

||

||

||

||

||

|

Chapter 2. Managing the GPFS cluster

There are several tasks involved in managing your GPFS cluster. This topic points you to the informationyou need to get started.

GPFS cluster management tasks include the following.v Creating your GPFS clusterv Displaying GPFS cluster configuration informationv Specifying nodes as input to GPFS commands on page 3v Adding nodes to a GPFS cluster on page 8v Deleting nodes from a GPFS cluster on page 9v Changing the GPFS cluster configuration data on page 10v Node quorum considerations on page 23v Node quorum with tiebreaker considerations on page 23v Displaying and changing the file system manager node on page 24v Determining how long mmrestripefs takes to complete on page 24v Starting and stopping GPFS on page 25

Note: In IBM Spectrum Scale V4.1.1 and later, many of these tasks can also be handled by thespectrumscale installation toolkit configuration options. For more information on the installation toolkit,see Using the spectrumscale installation toolkit to perform installation tasks: Explanations and examples section inthe IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

For information on RAID administration, see IBM Spectrum Scale RAID: Administration.

Creating your GPFS clusterYou must first create a GPFS cluster by issuing the mmcrcluster command.

For more information, see mmcrcluster command in IBM Spectrum Scale: Administration and ProgrammingReference.

For details on how GPFS clusters are created and used, see GPFS cluster creation considerations topic inIBM Spectrum Scale: Concepts, Planning, and Installation Guide.

Displaying GPFS cluster configuration informationWhen managing your GPFS cluster, you can display the current configuration information for the clusterby issuing the mmlscluster command.

The command displays:v The cluster namev The cluster IDv GPFS UID domainv The remote shell command being usedv The remote file copy command being usedv The repository type (CCR or server-based)v The primary GPFS cluster configuration server (if server-based repository)

Copyright IBM Corp. 2014, 2016 7

|

v The secondary GPFS cluster configuration server (if server-based repository)v A list of nodes belonging the GPFS cluster

For each node, the command displays:v The node number assigned to the node by GPFSv Daemon node namev Network IP addressv Admin node namev Designation, such as whether the node is a quorum node, a manager node, or an snmp_collector node

or all of these

To display this information, enter:mmlscluster

The system displays information similar to:GPFS cluster information========================

GPFS cluster name: cluster1.kgn.ibm.comGPFS cluster id: 680681562214606028GPFS UID domain: cluster1.kgn.ibm.comRemote shell command: /usr/bin/sshRemote file copy command: /usr/bin/scpRepository type: CCR

Node Daemon node name IP address Admin node name Designation--------------------------------------------------------------------------------------

1 k164n04.kgn.ibm.com 198.117.68.68 k164n04.kgn.ibm.com quorum2 k164n05.kgn.ibm.com 198.117.68.69 k164n05.kgn.ibm.com quorum3 k164n06.kgn.ibm.com 198.117.68.70 k164sn06.kgn.ibm.com quorum-manager

For complete usage information, see the mmlscluster command on page 524.

Adding nodes to a GPFS clusterYou can add nodes to an existing GPFS cluster by issuing the mmaddnode command. The new nodes areavailable immediately after the successful completion of this command.

You must follow these rules when adding nodes to a GPFS cluster:v You may issue the command only from a node that already belongs to the GPFS cluster.v A node may belong to only one GPFS cluster at a time.v The nodes must be available for the command to be successful. If any of the nodes listed are not

available when the command is issued, a message listing those nodes is displayed. You must correctthe problem on each node and reissue the command to add those nodes.

v After the nodes are added to the cluster, you must use the mmchlicense command to designateappropriate GPFS licenses to the new nodes.

To add node k164n06 to the GPFS cluster, enter:mmaddnode -N k164n06

The system displays information similar to:Mon Aug 9 21:53:30 EDT 2004: 6027-1664 mmaddnode: Processing node k164n06.kgn.ibm.commmaddnode: Command successfully completedmmaddnode: 6027-1371 Propagating the cluster configuration data to all

affected nodes. This is an asynchronous process.

To confirm the addition of the nodes, enter:


mmlscluster

The system displays information similar to:GPFS cluster information========================GPFS cluster name: cluster1.kgn.ibm.comGPFS cluster id: 680681562214606028GPFS UID domain: cluster1.kgn.ibm.comRemote shell command: /usr/bin/sshRemote file copy command: /usr/bin/scpRepository type: server-based

GPFS cluster configuration servers:-----------------------------------Primary server: k164sn06.kgn.ibm.comSecondary server: k164n05.kgn.ibm.com

Node Daemon node name IP address Admin node name Designation-------------------------------------------------------------------------------------1 k164n04.kgn.ibm.com 198.117.68.68 k164n04.kgn.ibm.com quorum2 k164n05.kgn.ibm.com 198.117.68.69 k164n05.kgn.ibm.com quorum3 k164n06.kgn.ibm.com 198.117.68.70 k164sn06.kgn.ibm.com quorum-manager

For complete usage information, see mmaddnode command on page 245 and mmlscluster commandon page 524.

Deleting nodes from a GPFS clusterYou can delete nodes from a GPFS cluster by issuing the mmdelnode command.

You must follow these rules when deleting nodes:v A node being deleted cannot be the primary or secondary GPFS cluster configuration server unless you

intend to delete the entire cluster. Verify this by issuing the mmlscluster command. If a node to bedeleted is one of the servers and you intend to keep the cluster, issue the mmchcluster command toassign another node as a configuration server before deleting the node.

v A node that is being deleted cannot be designated as an NSD server for any disk in the GPFS cluster,unless you intend to delete the entire cluster. Verify this by issuing the mmlsnsd command. If a nodethat is to be deleted is an NSD server for one or more disks, move the disks to nodes that will remainin the cluster. Issue the mmchnsd command to assign new NSD servers for those disks.

v GPFS must be shut down on the nodes being deleted. Issue the mmshutdown command.

To delete the nodes listed in a file called nodes_to_delete, issue:mmdelnode -N /tmp/nodes_to_delete

where nodes_to_delete contains the nodes k164n01 and k164n02. The system displays informationsimilar to:Verifying GPFS is stopped on all affected nodes ...mmdelnode: Command successfully completedmmdelnode: 6027-1371 Propagating the cluster configuration data to all

affected nodes. This is an asynchronous process.

To confirm the deletion of the nodes, issue:mmlscluster


GPFS cluster name: cluster1.kgn.ibm.com

Chapter 2. Managing the GPFS cluster 9

GPFS cluster id: 680681562214606028GPFS UID domain: cluster1.kgn.ibm.comRemote shell command: /usr/bin/sshRemote file copy command: /usr/bin/scpRepository type: server-based

GPFS cluster configuration servers:-----------------------------------

Primary server: k164sn06.kgn.ibm.comSecondary server: k164n05.kgn.ibm.com

Node Daemon node name IP address Admin node name Designation-------------------------------------------------------------------------------------


For complete usage information, see mmdelnode command on page 460 and mmlscluster commandon page 524.

Exercise caution when shutting down GPFS on quorum nodes or deleting quorum nodes from the GPFScluster. If the number of remaining quorum nodes falls below the requirement for a quorum, you will beunable to perform file system operations. For more information on quorum, see the section on Quorum, inthe IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

Changing the GPFS cluster configuration dataYou can use the mmchcluster or mmchconfig commands to change the configuration attributes.

After you have configured the GPFS cluster, you can change configuration attributes with themmchcluster command or the mmchconfig command. For more information, see the following topics:v The topic mmchcluster command in the IBM Spectrum Scale: Administration and Programming Reference

guidev The topic mmchconfig command in the IBM Spectrum Scale: Administration and Programming Reference

guide

Use the mmchcluster command to do the following tasks:v Change the name of the cluster.v Change the remote shell and remote file copy programs to be used by the nodes in the cluster. These

commands must adhere to the syntax forms of the ssh and scp commands, but may implement analternate authentication mechanism.

v Enable or disable the cluster configuration repository (CCR). For more information, see the Clusterconfiguration data files topic in the IBM Spectrum Scale: Concepts, Planning, and Installation Guide.

If you are using the traditional server-based (non-CCR) configuration repository, you can also do thefollowing tasks:v Change the primary or secondary GPFS cluster configuration server nodes. The primary or secondary

server may be changed to another node in the GPFS cluster. That node must be available for thecommand to be successful.Attention: If during the change to a new primary or secondary GPFS cluster configuration server, oneor both of the old server nodes are down, it is imperative that you run the mmchcluster -p LATESTcommand as soon as the old servers are brought back online. Failure to do so may lead to disruptionin GPFS operations.

v Synchronize the primary GPFS cluster configuration server node. If an invocation of the mmchclustercommand fails, you will be prompted to reissue the command and specify LATEST on the -p option to


synchronize all of the nodes in the GPFS cluster. Synchronization instructs all nodes in the GPFScluster to use the most recently specified primary GPFS cluster configuration server.

For example, to change the primary server for the GPFS cluster data, enter:mmchcluster -p k164n06

The system displays information similar to:mmchcluster -p k164n06mmchcluster: Command successfully completed

To confirm the change, enter:mmlscluster


GPFS cluster name: cluster1.kgn.ibm.comGPFS cluster id: 680681562214606028GPFS UID domain: cluster1.kgn.ibm.comRemote shell command: /usr/bin/sshRemote file copy command: /usr/bin/scpRepository type: server-based

GPFS cluster configuration servers:-----------------------------------

Primary server: k164sn06.kgn.ibm.comSecondary server: k164n05.kgn.ibm.com

Node Daemon node name IP address Admin node name Designation-------------------------------------------------------------------------------------


Attention: The mmchcluster command, when issued with either the -p or -s option, is designed tooperate in an environment where the current primary and secondary GPFS cluster configuration serversare not available. As a result, the command can run without obtaining its regular serialization locks. Toassure smooth transition to a new cluster configuration server, no other GPFS commands (mm...commands) should be running when the command is issued nor should any other command be issueduntil the mmchcluster command has successfully completed.

For complete usage information, see mmchcluster command on page 327 and mmlscluster commandon page 524.

You may be able to tune your cluster for better performance by re-configuring one or more attributes.Before you change any attributes, consider how the changes will affect the operation of GPFS. For adetailed discussion see the IBM Spectrum Scale: Concepts, Planning, and Installation Guide and themmcrcluster command.

Table 3 on page 12 details the GPFS cluster configuration attributes which can be changed by issuing themmchconfig command. Variations under which these changes take effect are noted:1. Take effect immediately and are permanent (-i).2. Take effect immediately but do not persist when GPFS is restarted (-I).3. Require that the GPFS daemon be stopped on all nodes for the change to take effect.4. May be applied to only a subset of the nodes in the cluster.


Table 3. Configuration attributes on the mmchconfig command

Attribute name and Description -i optionallowed

-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

adminMode

Controls password-less access

yes no no no immediately

atimeDeferredSeconds

Update behavior of atime when relatime isenabled

yes yes no yes if notimmediately,on restart ofthe daemon

autoload

Starts GPFS automatically

no no no yes on reboot ofeach node

automountDir

Name of the automount directory

no no yes no on restart ofthe daemon

cesSharedRoot

A directory to be used by the CES subsystem.

yes no yes (on allCES nodes)

no immediately

cipherList

The security mode of the cluster. This valueindicates the level of security that the clusteruses for communications between nodes inthe cluster and also for communicationsbetween clusters.

no no only whenchangingfromAUTHONLYor a cipherto EMPTYmode

no for newconnections

cnfsGrace

The number of seconds a CNFS node willdeny new client requests after a node failoveror failback

yes no yes no immediately

cnfsMountdPort

The port number to be used for rpc.mountd

yes no no no immediately

cnfsNFSDprocs

The number of nfsd kernel threads

yes no no no if notimmediately,on restart ofthe daemon

cnfsReboot

Determines whether the node will rebootwhen CNFS monitoring detects anunrecoverable problem.

yes no no yes immediately

cnfsSharedRoot

Directory to be used by the clustered NFSsubsystem


cnfsVersions

List of protocol versions that CNFS shouldstart and monitor



Table 3. Configuration attributes on the mmchconfig command (continued)


-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

dataDiskWaitTimeForRecovery

Controls the suspension of dataOnly diskrecovery


dataStructureDump

Path for the storage of dumps

yes no no yes if notimmediately,on restart ofthe daemon

deadlockBreakupDelay

When to attempt breaking up a detecteddeadlock

yes yes no no immediatelywith -i or -I

deadlockDataCollectionDailyLimit

Maximum number of times to collect debugdata in 24 hours


deadlockDataCollectionMinInterval

Minimum interval between two consecutivecollections of debug data


deadlockDetectionThreshold

Threshold for detecting deadlocks


deadlockDetectionThresholdForShortWaiters

Threshold for detecting deadlocks from shortwaiters


deadlockDetectionThresholdIfOverloaded

Threshold for detecting deadlocks when acluster is overloaded


deadlockOverloadThreshold

Threshold for detecting cluster overload


defaultMountDir

Default parent directory for GPFS file systems

yes yes no no for new filesystems

disableInodeUpdateOnFdatasync

Controls inode update on fdatasync formtime and atime updates.

yes yes no yes immediatelywith -i or -I

dmapiDataEventRetry

DMAPI attribute


dmapiEventTimeout

DMAPI attribute





-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

dmapiMountEvent

DMAPI attribute


dmapiMountTimeout

DMAPI attribute


dmapiSessionFailureTimeout

DMAPI attribute


enableIPv6

Controls whether the GPFS daemon is tocommunicate through the IPv6 network.

no no only whenenableIPv6is set to yes

notapplicable

if notimmediately,on restart ofthe daemon

enforceFilesetQuotaOnRoot

Controls fileset quota settings for the rootuser

yes yes no no if notimmediately,on restart ofthe daemon

expelDataCollectionDailyLimit

Maximum number of times to collectexpel-related debug data in 24 hours


expelDataCollectionMinInterval

Minimum interval between two consecutivecollections of expel-related debug data


failureDetectionTime

Indicates the amount of time it will take todetect that a node has failed

no no yes no on restart ofthe daemon

fastestPolicyCmpThreshold

Indicates the disk comparison countthreshold, above which GPFS forces selectionof this disk as the preferred disk to read

yes yes no yes immediatelywith -i

fastestPolicyMaxValidPeriod

Indicates the time period after which thedisk's current evaluation is considered invalid


fastestPolicyMinDiffPercent

A percentage value indicating how GPFSselects the fastest between two disks


fastestPolicyNumReadSamples

Controls how many read samples taken toevaluate the disk's recent speed





-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

fileHeatLossPercent

Specifies the reduction rate of FILE_HEATvalue for every fileHeatPeriodMinutes of fileinactivity.


fileHeatPeriodMinutes

Specifies the inactivity time before a file startsto lose FILE_HEAT value.


FIPS1402mode

Controls whether GPFS operates in FIPS140-2 mode.

no no no notapplicable

on restart ofthe daemon

forceLogWriteOnFdatasync

Controls forcing log writes to disk.


lrocData

Controls whether user data will be populatedinto the local read-only cache.


lrocDataMaxFileSize

Limits the data that may be saved in the localread-only cache to only the data from smallfiles.


lrocDataStubFileSize

Limits the data that may be saved in the localread-only cache to only the data from the firstportion of all files.


lrocDirectories

Controls whether directory blocks will bepopulated into the local read-only cache.


lrocInodes

Controls whether inodes from open files willbe populated into the local read-only cache.


maxblocksize

Maximum file system block size allowed

no no no yes on restart ofthe daemon

maxDownDisksForRecovery

Maximum number of failed disks allowed forautomatic recovery to continue


maxFailedNodesForRecovery

Maximum number of unavailable nodesallowed before automatic disk recovery iscancelled





-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

maxFcntlRangesPerFile

Specifies the number of fcntl locks that areallowed per file


maxFilesToCache

Number of inodes to cache for recently usedfiles


maxMBpS

I/O throughput estimate


maxStatCache

Number of inodes to keep in stat cache


metadataDiskWaitTimeForRecovery

Controls the suspension of metadata diskrecovery


minDiskWaitTimeForRecovery

Controls the suspension of disk recovery


mmapRangeLock

Specifies POSIX or non-POSIX mmapbyte-range semantics

Note: The list of NodeNames is allowed, but itis not recommended.

yes yes no yes immediately

nistCompliance

Controls whether GPFS operates in NIST800-131A mode for security transportmechanisms.

no no no notapplicable

if notimmediately,on restart ofthe daemon

noSpaceEventInterval

Time interval between noDiskSpace events ofa file system


nsdBufSpace

Percentage of the pagepool reserved for thenetwork transfer of NSD requests


nsdRAIDBufferPoolSizePct

Percentage of the page pool that is used forthe GPFS Native RAID vdisk buffer pool


nsdRAIDTracks

Number of tracks in the GPFS Native RAIDbuffer pool





-I optionallowed

GPFS mustbe stopped

on allnodes

List ofNodeNames

allowed

Change takeseffect

nsdServerWaitTimeForMount

Number of seconds to wait for an NSD serverto come up


nsdServerWaitTimeWindowOnMount

Time window to determine if quorum is to beconsidered recently formed


numaMemoryInterleave no no no yes on restart ofthe daemon

pagepool

Size of buffer cache on each node


pagepoolMaxPhysMemPct

Percentage of physical memory that can beassigned to the page pool


pitWorkerThreadsPerNode

Maximum number of threads to be involvedin parallel processing on each node serving asa Parallel Inode Traversal (PIT) worker


prefetchThreads

Maximum number of threads dedicated toprefetching data


readReplicaPolicy

The disk read replica policy


release=LATEST

Complete the migration to a new release

yes no no no if notimmediately,on restart ofthe daemon

rpcPerfNumberDayIntervals

Number of days that aggregated RPC data issaved


rpcPerfNumberHourIntervals

Number of hours that aggregated RPC data issaved


rpcPerfNumberMinuteIntervals

Number of minutes that aggregated RPC datais saved


rpcPerfNumberSecondIntervals

Number of seconds that aggregated RPC datais saved

no no no yes on restart ofthe d

Date post:	04-May-2018
Category:	Documents
Upload:	dokhue
View:	296 times
Download:	9 times

IBM Spectrum Scale 4.2: Administration and Programming Reference€¦ · Administra tion and...

Documents