+ All Categories
Home > Documents > Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab:...

Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab:...

Date post: 23-Mar-2018
Category:
Upload: phamdieu
View: 314 times
Download: 14 times
Share this document with a friend
91
Hands-On Lab: NetApp StorageGRID Webscale 10.3 March 2017 | SL10303 Version 3.1
Transcript
Page 1: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRIDWebscale 10.3

March 2017 | SL10303 Version 3.1

Page 2: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.32 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

TABLE OF CONTENTS

1 Introduction to Object Storage...................................................................................................... 3

2 Introduction to this Lab..................................................................................................................6

2.1 Lab Objectives........................................................................................................................... 6

2.2 Prerequisites.............................................................................................................................. 6

3 Lab Environment............................................................................................................................. 7

4 Lab Activities................................................................................................................................... 9

4.1 Lab Tools Overview...................................................................................................................9

4.2 Connect to Admin GUI............................................................................................................10

4.3 S3 Account Management........................................................................................................14

4.4 Configure S3 Client................................................................................................................. 18

4.5 Create S3 Buckets...................................................................................................................21

4.6 Examine ILM Rules..................................................................................................................24

4.7 Create New ILM rules..............................................................................................................25

4.8 Activate ILM rules....................................................................................................................36

4.9 Ingest data and Validate Data Placement............................................................................. 39

4.10 Validate ILM policy has been applied..................................................................................42

4.11 ILM Re-evaluation..................................................................................................................46

4.12 S3 Versioning.........................................................................................................................52

4.13 Configure a Swift Client........................................................................................................63

4.14 Configure the NAS Bridge....................................................................................................68

4.15 Exploring S3 and Swift APIs................................................................................................ 82

4.15.1 Access to S3 via the AWS CLI........................................................................................................................ 83

4.15.2 Access to S3 through Python...........................................................................................................................86

4.15.3 Access to Swift through the CLI.......................................................................................................................87

4.15.4 Access to Swift Using Python.......................................................................................................................... 88

5 Version History.............................................................................................................................. 90

Page 3: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.33 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1 Introduction to Object StorageObject storage is a little different than the familiar block and file storage. It organizes data into flexibly sizeddata containers called objects. Objects are stored in a flat namespace that may span multiple locations. Eachobject has both data (an un-interpreted sequence of bytes), and metadata (a unique ID plus an extensible set ofattributes that describe the object). Data stored using object storage is typically accessed by a client applicationusing an object protocol or API, rather than at the user level (accessing a file). A simple way to think about thisis that object storage is like valet parking—you give the valet your ticket, and you get your car back without everneeding to know anything about where the car was parked. The advantage of this approach is that data can bereferenced and retrieved just by providing the unique ID.

Figure 1-1: Object Storage Concepts

Three trends are contributing to increasing interest in object storage:

• Continued growth in the amount of unstructured data necessitates a new approach to storing andprotecting data. Object-enabled data management facilitates intelligent data placement to meet a widerange of performance, durability, availability, location, and longevity requirements.

• Enterprises are increasingly coping with decentralized data creation and consumption. The "primarydata center plus DR data center" model is being replaced by a multisite approach in which users,workloads, and data are brought closer together.

• The hybrid cloud is providing new options to balance cost and performance, and IT teams are looking athow they can best leverage both on-premises and cloud-based storage.

The Opportunity:

• Storage growth to billions of objects.• Decentralized data production and global distribution is driving new data management approaches to

reduce operation costs and complexity.• Due to data sovereignity, performance, cost, or a combination of all three, a hybrid cloud with both on-

premises and clooud-based storage is a requirement.

Use Cases

StorageGRID Webscale is extremely well suited for web data repositories, data archives, and media repositories.Each of these use cases has its own distinct set of requirements, but StorageGRID Webscale adapts toaccommodate the wide variety of needs encompassed in this set of use cases.

Page 4: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.34 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

Figure 1-2: Use Cases for Object Storage

Web Data Repositories

Web data repositories are characterized by small object size, high object count, and high transactions. Because itcan handle up to 100 billion objects in a single repository distributed across many locations and with S3, Swift andCDMI API support, StorageGRID Webscale is well suited for this use case.

Data Archives

Increasingly, enterprises are storing massive amounts of data for extended periods to satisfy both corporategovernance and legal requirements. With data archives of this type, cost and management are typically the mostimportant concerns. Long access latency is tolerated in exchange for reduced cost. StorageGRID Webscalesatisfies this use case with tape integration, proven data durability, and flexible, policy based management.

Media Repositories

Media repositories are characterized by large object sizes, with the need for geographical distribution, a need fordata integrity, and a low time-to-first byte latency. The geographically distributed, durable design of StorageGRIDWebscale satisfies these requirements. It also supports “ranged reads”, so, for example, a video can be streamedfrom any point without having to download the entire object.

Introduction to StorageGRID Webscale

The StorageGRID Webscale system is a distributed object storage system that stores, protects, and preservesfixed-content data over long periods of time. By employing a grid architecture that distributes copies of objectdata throughout the system, a highly reliable system is created where data is continuously available. If onepart of the system goes down, another immediately takes over, which results in objects always being availablefor retrieval. To implement this architecture, the StorageGRID Webscale system employs a system of networkconnected servers hosting grid nodes. These grid nodes host a collection of one or more services, each providinga set of capabilities to the StorageGRID Webscale system. To manage objects ingested into the system, theStorageGRID Webscale system employs metadata based information lifecycle management (ILM) rules. TheseILM rules determine what happens to an object’s data after it is ingested—where it is stored, how it is protectedfrom loss, and how long it is stored. The StorageGRID Webscale system operates over wide area network (WAN)links, providing the system with off-site loss protection. Copies are made and distributed throughout the systemso that objects are continuously available. In systems with multiple sites, this distribution of copies means that if asite is lost, data is not lost, and clients can seamlessly retrieve it from other sites.

Advantages of the StorageGRID Webscale system include the following:

• Software-defined storage management for hardware independence• Support for standard object storage protocols and APIs:

Page 5: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.35 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

• Simple Storage Service (S3)• OpenStack Swift• Cloud Data Management Interface (CDMI)

• Support for standard file system storage protocols:

• Network File System (NFS)• Server Message Block (SMB)

• Configurable data protection using replicated copies, erasure coding or cloud tiering.• Ability to address hardware obsolescence through the transparent migration of data from previous to

current generations of hardware.• Policy-driven tiered storage that reduces overall storage costs:

• Data can be migrated to lower cost tier based on “last access” time or other value.• An object stored on a low cost tier can be copied to faster tier once accessed. Consider a

policy that stores data on tape or public cloud – a policy to copy frequently accessed data toa replica provides user driven cache effect.

• Virtualization of storage across sites and tiers, insulating applications from changes to underlyingstorage infrastructure.

• Ability to monitor and verify data integrity proactively as data is stored, updated, retrieved, and deleted.• Non-disruptive upgrade, expansion, and maintenance that maintains access to content

StorageGRID Webscale Partners

StorageGRID Webscale can work with client applications that support storing to S3 or Swift compatible storage.

Figure 1-3: Sample Partners and Client Applications in the StorageGRID Webscale ecosystem

Page 6: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.36 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2 Introduction to this LabThis lab provides a basic introduction to StorageGRID Webscale (SGWS). The intent is to provide an overview offunctionality and terminology for self-education purposes.

In this lab, you will learn the basic functionality of StorageGRID Webscale by ingesting and retrieving data usingthe S3 and Swift object RESTful protocols, and CIFS and NFS NAS protocols. You will be introduced to theInformation Lifecycle Management rules and policies, and learn how to configure and protect objects stored in theStorageGRID Webscale system. And finally, you learn how to access the S3 and Swift protocols using 3rd partycommand line and scripting tools, and how application developers can integrate and use StorageGRID Webscalewith their client applications.

2.1 Lab ObjectivesThis lab has the following objectives:

• Configure StorageGRID Webscale for access by an API client.• Create an S3 account and configure the S3 Browser tool.• Create a Swift account and configure the Cloudberry Explorer tool.• Create custom ILM rules based on S3 buckets.• Ingest data using S3 Browser and Swift Cloudberry tools and validate ILM policies have been applied.• Enable and test S3 Versioning.• Integrate the NAS Bridge, and ingest data using NFS and SMB.• Ingest data using command line and scripting tools with the S3 and Swift APIs

2.2 PrerequisitesThe following prerequisites are recommended for this lab:

• Basic Linux command line (shell) knowledge.• Basic Windows skills.• Complete NetAppU WBT course “StorageGRID Webscale Administration course ”, or equivalent.• Review the StorageGRID Webscale Primer document, SGWS Administration Guide, and the NAS

Bridge Administration Guide.

Page 7: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.37 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

3 Lab EnvironmentStorageGRID Webscale is a software defined object storage solution. All grid nodes are virtual appliances. Thefollowing illustration depicts the components of the Lab environment.

Figure 3-1:

Table 1: Lab Host Information

Server \Resource

Purpose IP Address Username Password

Jumphost Lab access 192.168.0.5 Administrator Netapp1!

NTP Time server, linuxshell

192.168.0.9 root Netapp1!

DC1-ADM1 StorageGRIDadmin node

192.168.0.10 root bycast

Page 8: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.38 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

Server \Resource

Purpose IP Address Username Password

DC1-S1 StorageGRIDstorage node

192.168.0.6 root bycast

DC1-S2 StorageGRIDstorage node

192.168.0.7 root bycast

DC1-S3 StorageGRIDstorage node

192.168.0.8 root bycast

DC1-G1 StorageGRID APIgateway node

192.168.0.11 root bycast

DC1-ARC1 StorageGRIDarchive node

192.168.0.12 root bycast

LINUX1 Linux host 192.168.0.13 root Netapp1!

NAS Bridge GUI NAS Bridgeadministrativeconsole

192.168.0.14 [email protected] Netapp1!

Admin GUI \ GMI StorageGRIDGrid ManagementInterface (GMI) -web GUI hostedfrom the adminnode

192.168.0.10 root netapp01

Audit log export Read only accessto audit log

\\192.168.0.10\audit-export

Administrator Netapp1!

Page 9: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.39 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

4 Lab ActivitiesIn this lab you configure the StorageGRID Webscale object storage system for access using the S3 and Swiftprotocols. You will create Integrated Lifecycle Management (ILM) rules to manage objects ingested through S3 orSwift, and apply policy based on bucket/container name.

Webscale’s support for the widely adopted S3 protocol ensures interoperability with many existing cloud basedapplications. Customers and Service Providers can create cloud solutions with on and off premises storage, andleverage Webscale’s granular ILM rules to ensure data is stored in the most cost effective manner, while stillhonoring locality and performance requirements.

To accomplish these objectives, the lab provides the following lab activities:

• Lab Tools Overview on page 9• Connect to Admin GUI on page 10• S3 Account Management on page 14• Configure S3 Client on page 18• Create S3 Buckets on page 21• Examine ILM Rules on page 24• Create New ILM rules on page 25• Activate ILM rules on page 36• Ingest data and Validate Data Placement on page 39• Discover StorageGRID Object ID for sample files• Validate ILM policy has been applied on page 42• ILM Re-evaluation on page 46• S3 Versioning on page 52• Configure a Swift Client on page 63• Configure the NAS Bridge on page 68• Exploring S3 and Swift APIs on page 82• Access to S3 through Python on page 86• Access to S3 via the AWS CLI on page 83• Access to Swift Using Python on page 88

4.1 Lab Tools OverviewNote: At several points in this lab you will use Copy and Paste operations. Please use browser Copy andPaste (right click, select) operations for this purpose. Ctrl+C and Ctrl+V may not work as expected.

To facilitate quick access to lab resources the following desktop short cuts have been created.

1. S3 Browser: Client used to perform storage actions (create buckets, ingest, retrieve) using the S3protocol.

2. Cloudberry Explorer: Client used to perform Swift storage actions (create containers, ingest andretrieve).

3. NetApp StorageGRID Webscale: shortcut to Admin Web GUI.4. NetApp StorageGRID Webscale NAS Bridge: shortcut to NAS Bridge web interface.5. PuTTY: SSH client – used to access a Linux client for command line, API, and NFS exercises6. Sample Files: to be used for ingest and ILM tests.7. Notepad++: use for copy and paste of text strings required for the demo.8. Webscale Audit Logs: Read only mapped drive to Webscale Audit logs.9. Downloads: short cut to download folder.

Note: The position of these icons may be different on your screen.

Page 10: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.310 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

12

3

45

6

7

8

9

Figure 4-1:

4.2 Connect to Admin GUIThe administrative interface to StorageGRID Webscale is called the “Grid Management Interface”, or GMI. It wasformerly called the Network Management System (NMS). Both names are still used, however going forward GMIis preferred. It provides a web- based interface.

1. Double click the shortcut for NetApp StorageGRID Webscale.

Page 11: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.311 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-2:

2. Your browser will launch, and open the StorageGRID Webscale Grid Management Interface (GMI)3. Login to the account with Username root, and password netapp01.

Page 12: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.312 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

3

Figure 4-3:

4. Review the NMS/GMI Dashboard.5. Click the Grid button to display the grid topology.

Page 13: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.313 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5

Figure 4-4:

6. Click the + next to Data Center 1 (under LOD Grid) to expand it. The LOD environment consists of asingle site SGWS deployment: 3 Storage Nodes, 1 Admin Node, 1 Gateway Node and 1 Archive Node.

Page 14: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.314 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6

Figure 4-5:

4.3 S3 Account ManagementIn this activity you will create an S3 account. Webscales’ native support for the S3 protocol allows interoperabilitywith Amazon S3 compatible applications. The Account Access Key Identifier and Account Secret Access Key willbe familiar to users of AWS S3.

1. Click the Tenants button

Page 15: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.315 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-6:

2. Click the + Create button to create a new account.

2

Figure 4-7:

3. The “Add Tenant Account” popup window will launch.4. Select the S3 protocol.

Page 16: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.316 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

4

Figure 4-8:

5. In the “Tenant Details, Name”, type LOD .6. Click Save.

56

Figure 4-9:

7. The “Save Keys” dialog appears8. Click Download.

Figure 4-10:

9. When the dialog box appears, select Save File.10. Click OK.

Page 17: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.317 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9

10

Figure 4-11:

11. A file with a name like “Secret Access Key - LOD - Y6DSCN7IV5TOZ5SSQM1B.csv” will bedownloaded to the Download folder (see shortcut on desktop).

You can also copy and paste the “Account Access Key Identifier” and “Account Secret Access Key” to anotepad. You will need this information to configure your S3 client.

Figure 4-12:

9

10

12. After the dialog box disappears, (back in the “Save Keys” dialog) click Finish.

Page 18: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.318 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9

Figure 4-13:

13. The new tenant account will appear on the Tenant Accounts screen.

Figure 4-14:

4.4 Configure S3 ClientThis lab activity uses the S3 Browser. This application provides an easy to use interface for the S3 lab exercises.

1. You will need the IP address of the API Gateway node: 192.168.0.11 (or optionally, the hostname: dc1-g1.demo.netapp.com).

2. You can also find this information by clicking on the Grid tab.

Page 19: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.319 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

Figure 4-15:

3. In the Grid Topology pane, select Data Center 1 > DC1-G1 > SSM > Resources, and scroll down to the“Network Addresses” section.

3

Figure 4-16:

Page 20: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.320 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

You will also need your S3 Account Access Key Identifier, and Account Secret Access key that weredownloaded in the previous section.

4. Open the Downloads folder used by the browser.5. Look for the Secret Access Key*.csv file.6. Open the credentials file with Notepad.7. Notice the Access Key ID and Secret Access Key (the longer value is the Secret Access Key).

5

Figure 4-17:

8. Double click the S3 Browser on the desktop.

8

Figure 4-18:

Page 21: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.321 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9. The “Add New Account” window will launch.10. Enter LOD for the Account Name.11. Select S3 compatible storage.12. Enter the IP address of the Gateway node with port 8082. In this lab, this will be 192.168.0.11:8082,

or dc1-g1.demo.netapp.com:8082.13. Fill in the Access Key ID and Secret Access Key.14. Click Add new account.

10

11

12

13

14

Figure 4-19:

4.5 Create S3 BucketsIn this lab activity you create three S3 buckets using the S3 Browser tool. Later, you will create ILM policies basedon these bucket names.

1. Click the New Bucket Icon.

Page 22: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.322 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-20:

2. Name the bucket 3-copy.3. Click Create new bucket.

Page 23: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.323 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

3

Figure 4-21:

4. Repeat the process twice more, to create a bucket named 1-copy, and another named ec-copy.5. You should now have three buckets: one named “1-copy”, one named “3-copy”, and one named “ec-

copy”.

Page 24: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.324 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5

Figure 4-22:

4.6 Examine ILM RulesAdvanced Information Lifecylcle Management policy is a key feature of StorageGRID Webscale. It allows granularevaluation of each object, and then applies storage actions to configure the following:

• Data protection – how many replicated copies or erasure coded objects the grid will create andmaintain.

• Location – where these copies will be maintained. In production grid environments, customers createStorage Pools to determine location.

In this example you manage objects based on the S3 bucket they are stored in, but administrators can createrules based on other metadata such as object size, or user specified metadata, and last access time.

1. Return to the StorageGRID NMS/GMI user interface.2. Click on the ILM drop-down button and click on Rules.

Page 25: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.325 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

Figure 4-23:

3. Examine the default “Make 2 Copies” rule. This default rule is created with a new Webscale installation,and protects objects that do not meet the criteria for any other rule.

Figure 4-24:

4.7 Create New ILM rulesIn this activity, you create a set of Information Lifecycle Management (ILM) rules. These ILM rules definehow objects stored to the StorageGRID Webscale system are protected. You will create rules that apply dataprotection based on the S3 bucket the object is stored into. You will also create a 3 copy replicate rule for the 3-copy bucket, a 1 copy replicate rule for the 1-copy bucket, and two rules for the ec-copy bucket: a erasure codedrule for objects larger than 1 MB, and a replicated rule for smaller objects.

Page 26: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.326 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

First, create a 3 copy rule.

1. Click the ILM drop-down button and select Rules (if you are not there already).2. Click the + Create button.

2

Figure 4-25:

3. Configure the new rule in the “Create ILM Rule” dialog box.4. Enter 3-copy for the “Rule Name”.5. Enter 3 copy rule as the “Description”.6. Leave “Ignore” as the “Tenant Account”.7. For Bucket Name, select Equals.8. Enter 3-copy.9. For Object Type select S3/Swift.10. Click Next.

Page 27: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.327 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

45

67 8

9

10

Figure 4-26:

11. Set “Reference Time” to Ingest Time.12. Set “Placements”

• From day 0 store forever.• Create 1 copies as replicated in All Storage Nodes.

13. Click +, and add two more Create 1 copies as replicated in All Storage Nodes lines to create 3copies.

14. Click Refresh on “Retention Diagram”.15. Verify 3 copies are created.16. Click Save.

Page 28: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.328 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

16

11

12 13

14

15

Figure 4-27:

Now, create the 1 copy rule.

1. Click the ILM drop-down button and select Rules.2. Click the + Create button.3. Configure the new rule in the “Create ILM Rule” dialog box.4. Enter 1-copy for the “Rule Name”.5. Enter 1 copy rule as the “Description”.6. Leave Ignore as the “Tenant Account.”7. For Bucket Name, select Equals.8. Enter 1-copy.9. For Object Type select S3/Swift.10. Click Next.

Page 29: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.329 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

45

67 8

9

10

Figure 4-28:

11. Set “Reference Time” to Ingest Time.12. “Placements”

• From day 0 store forever.• Create 1 copies as replicated in All Storage Nodes

13. Click Refresh on Retention Diagram.14. Verify 1 copy is created.15. Click Save.

Page 30: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.330 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1514

13

1211

Figure 4-29:

Now, create an erasure coding (EC) rule. First create the Erasure Coding Profile:

1. Click the ILM drop-down and select Erasure Coding.

1

Figure 4-30:

2. Under “Profile Name”, enter LOD 2 plus 1.3. For the “Storage Pool”, select All Storage Nodes.4. The EC Scheme is 2+1, as there are only three Storage Nodes in this deployment.5. Click Apply Changes.

Page 31: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.331 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

23

54

Figure 4-31:

Perform this activity to create a new EC Copy rule.

1. Click the ILM drop-down button, and select Rules.2. Click the + Create Button.3. Configure the new rule in the “Create ILM Rule” dialog box.4. Enter EC-copy for the “Rule Name”.5. Enter EC copy rule as the “Description”.6. Leave Ignore as the “Tenant Account”.7. For Bucket Name, select Equals.8. Enter ec-copy.9. For “Object Type” select S3/Swift.10. Click on Advanced filtering...

Page 32: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.332 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

109

876

54

Figure 4-32:

11. In the “Advanced Filtering” dialog box, select Object Size (MB), greater than and enter 1. See Figurefor example.

12. Click Save.

11

12

Figure 4-33:

13. Back in the “Create ILM Rule” dialog box, click Next.

Page 33: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.333 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

13

Figure 4-34:

14. Set reference time to Ingest Time.15. “Placements”

• From day 0 store forever.• Create 1 copies as erasure coded in All Storage Nodes (LOD 2 plus 1)

16. Click Refresh on Retention Diagram.17. Verify Erasure Coded copy is made (use of EC icon) using LOD 2 plus 1.18. Click Save

Page 34: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.334 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1817

16

1514

Figure 4-35:

Perform this activity to create a new ec-copy-small rule.

1. Click the ILM drop-down button and select Rules.2. Click the + Create Button.3. Configure the new rule in the “Create ILM Rule” dialog box.4. Enter EC-copy-small for the “Rule Name”.5. Enter EC copy small as the “Description”.6. Leave “Ignore” as the “Tenant Account”.7. For “Bucket Name”, select Equals.8. Enter ec-copy.9. For “Object Type” select S3/Swift.10. Click on Advanced filtering....

Page 35: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.335 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

45

67 8

9

10

Figure 4-36:

11. In the “Advanced Filtering” dialog box, select Object Size (MB), less than or equals and enter 1. SeeFigure for example.

12. Click Save.

11

12

Figure 4-37:

13. Back in the “Create ILM Rule” dialog box, click Next.14. Set reference time to Ingest Time .15. Set “Placements”.

• From day 0 store forever.• Create 1 copies as replicated in All Storage Nodes.

16. Click Refresh on “Retention Diagram”.

Page 36: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.336 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

17. Verify 1 copy is created.18. Add 1 more replicated copy (start by clicking +).19. Click Refresh on “Retention Diagram” again.20. Verify 2 copies will be kept.21. Click Save.

21

19

18

16

14 15

Figure 4-38:

4.8 Activate ILM rulesOnce the ILM rules are created, you need to add the rules to the ILM policy, and activate it.

1. From the ILM drop-down, select Policies.

Page 37: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.337 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-39:

2. Click Configure from the menu on the left.3. Check boxes to add the rules to the policy: 1-copy, 3 copy, EC-copy and ec-copy-small.4. ILM rules are evaluated top down. When a rule’s filter is met, it is executed and ILM is satisfied. Only one

rule is executed per object.5. Leave the 2 copy rule as default and at the bottom – this will be applied to any objects that do not meet

the criteria for other rules.6. Give the Proposed Policy a name: Webscale lab .7. Click Save, then click OK in the popup to verify. It may take up to a minute before the “Activate” button

becomes clickable.8. Click Activate, then click OK in the popup to verify.

Page 38: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.338 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

87

6

5

3

2

Figure 4-40:

Note: Notice the order in which the rules are applied. Rule are evaluated in a top down order tocreate a data management policy. Only 1 rule is executed per object. Also take note of the Re-evaluate Content button. This will re-evaluate data in the system, and apply the new rules.

9. Click Active from the menu on the left.10. Expand each of the “+” symbols.11. Validate that the new rules are active.

Page 39: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.339 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9

10

Figure 4-41:

12. Minimize the StorageGRID window for now.

4.9 Ingest data and Validate Data Placement1. On the desktop, open the Sample Files folder.2. Open the S3 Browser.3. Select the 1-copy bucket.4. Drag and drop the 1-copy.txt file into the upper right box of S3 Browser, as shown.

Page 40: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.340 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

3

4

Figure 4-42:

5. Repeat for the “3-copy” rule.6. Make sure you select the 3-copy bucket in the S3 browser.7. Drag and drop 3-copy.txt to S3 Browser like you did with “1-copy.txt”.

Page 41: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.341 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6

7

Figure 4-43:

8. Repeat for the “EC-copy” rule.9. Make sure you select the ec-copy bucket in the S3 browser.10. Drag and drop the ec-copy.txt and ec-copy-small.txt files into the EC-copy bucket in the S3 Browser.11. Minimize the “Sample Files” and “S3 Browser” windows.

Page 42: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.342 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9

10

Figure 4-44:

4.10 Validate ILM policy has been appliedThis activity uses the Object Lookup tool to validate that the ILM rules are applied correctly to an object.

1. Return to the StorageGRID Webscale GMI.2. Select Grid from the top menu bar.3. In the “Grid Topology” section, select Data Center 1, then DC1-ADM1.4. Select CMN > Object Lookup.5. Select the Configuration tab.6. If you know the Object Identifier you can enter it here. In this case, you do not yet know the Object

Identifier so enter the bucket/key identifier. The tool will look up the Object Identifier for you. Enter 1-copy/1-copy.txt into the Object Identifier box

Important: Object Identifier lookup by bucket/key values is case sensitve!

7. Click Apply Changes.

Page 43: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.343 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

8. Click the Overview tab to see the results.

3

2

4

5

6

7

8

Figure 4-45:

9. Scroll down to see all of the information. Note that the object is stored as “1 copy”.

9

Figure 4-46:

10. Repeat for 3-copy/3-copy.txt. Notice that the object is stored 3 times.

Page 44: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.344 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

10

Figure 4-47:

11. Repeat for ec-copy/ec-copy.txt. Notice that the object is stored with erasure coding in accordancewith the ILM rules you set.

Page 45: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.345 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

11

Figure 4-48:

12. Repeat for ec-copy/ec-copy-small.txt. Notice that the object is stored twice, in accordance with theILM rules you set.

Page 46: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.346 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

12

Figure 4-49:

4.11 ILM Re-evaluationAnother strength of StorageGRID Webscale is the ability to change ILM rules and apply them, not only to newobjects, but also to re-evaluate existing objects. If business or regulatory requirements change, customers mayneed to adjust how their objects are stored. The re-evaluation process takes place in the background to ensurethat performance is not impacted.

In this activity, you disable the existing “3-copy rule”, and create a new 3-copy rule that only makes 2 replicatedcopies. After you modify the ILM rules and activate the new ILM policy, run the ILM re-evaluation. Once the ILMre-evaluation is complete, you can verify that the object stored in the 3-copy bucket now has only 2 copies.

1. From the ILM drop-down, select Rules.

Page 47: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.347 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-50:

2. Press the + Create button.3. In the “Create ILM Rule” dialog window, enter the following:

• “Name”: new-3-copy.• “Description”: New rule: 3-copy bucket - 2 copies only.• “Tenant Account”: Ignore.• “Bucket Name”: Equals 3-copy.• “For Object Type”: S3/Swift.

4. Click Next.

4

3

Figure 4-51:

5. In the new window, enter the following:

• “Set Reference Time”: Ingest Time.• For Placements: From day 0 store forever.• Create 1 copies as replicated in All Storage Nodes.

Page 48: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.348 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

• Press the + button, to create a second copy: Create 1 copies as replicated in All StorageNodes (same as the last one).

6. Refresh the “Retention Diagram” to confirm Placement.7. Click Save.

5

6

7

Figure 4-52:

8. Verify that the “new-3-copy” rule is created and listed in the ILM Rules list. It should currently not be usedin the Active Policy (no checkmark).

9. From the ILM drop-down, select Policies.

Page 49: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.349 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

8

9

Figure 4-53:

10. From the left menu, select Configure.11. Give the Proposed Policy a new name. Enter Webscale Lab-v2 in the “Policy” field.12. Under “Saved Rules”, click on new-3-copy v1.0 under “Add to Policy”.13. Unselect the old rule: 3-copy v1.0.14. If needed, under “Proposed Policy”, sort the rules into the appropriate order by dragging the hand

icon under “Actions”. The “Make 2 Copies v1.0” default rule should always be at the bottom. Rules areevaluated top down.

15. The proposed policy should now contain the new-3-copy rule instead of the old 3-copy rule. Click Saveon the proposed policy.

16. Click OK to confirm.17. It takes some time to save. Once saved, activate the new policy: Press Activate.18. Click OK to confirm the activation.19. After about 1 minute, the Re-evaluate Content button will become clickable.20. Click Re-evaluate Content.

Page 50: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.350 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

20

10

11

12

13

14

15 17

Figure 4-54:

21. You will receive a verification pop up after clicking Re-evaluate Content.22. Click OK.

21

Figure 4-55:

23. Select Grid from the top menu.24. Navigate to LOD Grid > Data Center 1 > DC1-ADM1 > CMN > Grid Tasks.25. Verify the “ILM Re-evaluation” task is running. The time the task takes depends roughly on the amount

of content residing in the system. The re-evaluation of existing objects runs as a background task onthe grid.

26. The task will move to “Historical” once completed. Wait for this re-evaluation task to complete beforemoving to the next step.

Page 51: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.351 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

23

25

24

Figure 4-56:

After you modify the ILM rules and activate the new ILM policy, run the ILM re-evaluation. Once the ILM re-evaluation is complete, you can verify that the object stored in the 3-copy bucket now only has 2 copies.

1. Select Grid from the top menu.2. Navigate to Grid Topology > LOD Grid > Data Center 1 > DC1-ADM1.3. Expand DC1-ADM1, and select CMN > Object Lookup.4. Select the Configuration tab.5. Enter 3-copy/3-copy.txt into the Object identifier box.6. Click Apply Changes.

1

2

3

4

5

6

Figure 4-57:

7. Switch to the Overview tab.

Page 52: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.352 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

8. Verify that there are now only 2 copies of this object.

7

8

Figure 4-58:

4.12 S3 VersioningIn StorageGRID Webscale 10.3, S3 Versioning is supported, this enables managing and storing multiple versionsof an object. In this lab, you will exercise the S3 versioning capabilities in StorageGRID Webscale.

This lab activity uses the S3 Browser to enable and try S3 versioning.

1. Click and open the S3 Browser from the desktop.

1

Figure 4-59:

2. Create a new bucket called versioning.3. Leave “Bucket region” as Default.4. Click Create New Bucket.

Page 53: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.353 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

3

4

Figure 4-60:

5. Right click on the versioning bucket and select Edit Versioning Settings. Versioning is enabled on aper bucket basis.

Page 54: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.354 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5

Figure 4-61:

6. Click the Enable versioning for bucket check box.7. Click OK. Versioning is now enabled for the bucket.

Page 55: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.355 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6

7

Figure 4-62:

8. Now test S3 versioning. Create a new file called version.txt, and store it on the desktop. Inside the filetype version 1.

Figure 4-63:

9. Drop the version.txt file into the versioning bucket.

Page 56: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.356 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

9

Figure 4-64:

10. Click on the Versions tab in the S3 browser.11. Review the version information under the “Versions” tab.

10

11

Figure 4-65:

Page 57: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.357 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

12. Edit the “version.text” file located on the Desktop, and change the text to version 2. Drop the updatedfile into the bucket.

12

Figure 4-66:

13. The S3 browser now displays 2 versions of “version.txt” residing in the versioning bucket.

13

Figure 4-67:

14. Download the “version.txt” file. It will return you to version 2.

Page 58: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.358 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1414

Figure 4-68:

15. Restore version 1, and download the file: From the Versions tab, select revision #1.16. Click the Restore button.

Page 59: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.359 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1516

Figure 4-69:

17. The S3 Browser may display duplicates in the Versions pane. Press Refresh to get an updated view.

Page 60: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.360 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

17

Figure 4-70:

18. “Revision 3” should now be displayed.

Page 61: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.361 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

18

Figure 4-71:

19. Download and open the “versions.txt” file. Inside you will now find “version 1”.

Page 62: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.362 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1919

Figure 4-72:

20. You can now also delete old versions. Since revision 1 and 3 are duplicates, you can delete “version 1”by selecting it, and pressing Delete.

21. Click Yes.

Page 63: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.363 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

21

20

Figure 4-73:

4.13 Configure a Swift Client

In this exercise, you will create a container and ingest an object using the Swift protocol. The Cloudberry Explorerclient application will be used for this exercise.

4.13 Configure a Swift Account1. From the Grid Management Interface (GMI), select Tenants from the top menu.2. Press the + Create button.

1

2

Figure 4-74:

Page 64: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.364 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

3. Select Swift.4. Enter LOD Swift as the “Name under Tenant Details”.5. Record the username as swiftadmin for later use.6. Enter the password: Netapp1!.7. Click Save.

3

4

6

7

Figure 4-75:

8. From the “Tenants Accounts” page, record the ID for the LOD Swift account:

8

Figure 4-76:

4.13 Configure Cloudberry Explorer for Swift1. On the Windows desktop, click the Cloudberry Explorer icon.

Page 65: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.365 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

Figure 4-77:

2. When the “Select Cloud Storage” window opens, select OpenStack as the Cloud Storage.

Page 66: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.366 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

Figure 4-78:

3. When the “Add New OpenStack Storage Account” window opens, enter LOD Swift as the “DisplayName”.

4. Enter the Username:20157648367352802448:swiftadmin. The format of the username is“ID:swiftadmin”. The ID was created and recorded during the creation of the Swift account.

5. Enter the password created for the Swift account as the API Key. The password is Netapp1!.6. The Authentication service is the Gateway Node followed by /auth/v1.0. StorageGRID Webscale uses

port 8083 for the Swift protocol..

• Enter: https://dc1-g1.demo.netapp.com:8083/auth/v1.07. Select Do not use for the Keystone version.8. Click Test Connection. The security certificate will be reported as not issued by a trusted certificate

authority. Click Allow.

Page 67: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.367 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

3

45

67

8

Figure 4-79:

9. Wait for “Connection Success”. Click Close.10. Click OK.11. Close the “Registered Accounts” window.

2

1

Figure 4-80:

4.13 To Test Cloudberry Explorer with StorageGRID Webscale1. On the left pane, select “Source”: LOD Swift.2. On the right pane, select “Source”: My Computer.3. In the left pane, press the Cube Icon, to create a new container.4. Name the Container: swifttest.5. In the left pane, open the swifttest container.

Page 68: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.368 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1 2

35

Figure 4-81:

6. Copy some files from My Computer to LOD Swift (drag and drop). A sample “swifttest.txt” file isprovided in the Sample Files directory on the Desktop. Since there is no specific rule for the swifttestcontainer, the object will be protected with the default 2 copy rule.

Note: The objects ingested into the swifttest container will trigger the 2 copy default rule for ILM.

2

Figure 4-82:

4.14 Configure the NAS Bridge

In this exercise, you will integrate the NAS Bridge with the StorageGRID Webscale system. You will configureNFS and CIFS shares and test ingesting files into the NAS shares.

4.14 Configuring the NAS BridgePerform the following activity to configure the NAS bridge.

1. Connect to the NAS Bridge web interface by selecting the desktop icon NetApp SGWS – NAS Bridge2.0, or opening FireFox and entering http://nas.demo.netapp.com.

2. Login into the interface by entering [email protected] as the username, and Netapp1! as thepassword.

3. Agree to the End User License.

Page 69: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.369 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

Figure 4-83:

Configure the DNS

1. Select Bridge from the left menu bar.2. Click Add next to DNS Servers.3. Enter the IP address, 192.168.0.2534. Enter 1 as the Priority.5. Click Save.

1 34

5

Figure 4-84:

Page 70: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.370 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6. Wait until the Status reports “READY”.

6

Figure 4-85:

Configure NTP Servers

1. Select Bridge from the left menu bar.2. Click Add next to NTP Servers.3. Enter the host name, dc1.demo.netapp.com.4. Click Save.

1

3

4

Figure 4-86:

Page 71: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.371 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5. Wait until the Status reports “READY”.

5

Figure 4-87:

Configure SGWS Integration

1. Select StorageGRID from the left menu bar.2. Enter hostname: dc1-g1.demo.netapp.com.3. Enter port 8082.4. Enter your S3 credentials previously created, they can be re-used for the lab.5. Enter both the “Access Key” and the “Secret Key”.

Remember: You collected this information back in 3.3 of the lab guide, and it's in a .txt file inyour “Downloads” folder

6. Click Save.7. Wait unil Status reports “READY”.

Page 72: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.372 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

12

5

6

7

Figure 4-88:

Configure the Active Directory

1. Select Active Directory from the left menu bar.2. Click Add next to Active Directory Servers.

1

2

Figure 4-89:

3. Enter the “Domain”: demo.netapp.com4. Enter “Primary Controller”: dc1.demo.netapp.com5. Enter “NETBIOS Domain”: DEMO6. Enter “Computer name”: NAS7. Enter “Admin Username”: Administrator8. Enter “Password”: Netapp1!9. Click Save.

Page 73: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.373 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

34

56

78

9

Figure 4-90:

10. Wait until Status reports “Ready”.

10

Figure 4-91:

Configure the Cache Devices

1. Select Cache Devices from the left menu bar.2. Click Add next to “Cache Devices”.

Page 74: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.374 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

2

Figure 4-92:

3. Enter “Name”: SMB1,4. Select “Disk”: /dev/sdb.5. Click Save.

34

5

Figure 4-93:

6. Wait until SMB1 status is “READY”.

Page 75: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.375 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6

Figure 4-94:

7. Repeat these steps using name NFS1, and disk /dev/sdc.

7

Figure 4-95:

Configure an NFS Share

1. Select NFS Filesystems from the left menu bar.2. Click Add next to “NFS Filesystems”.

Page 76: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.376 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

1

2

Figure 4-96:

3. For StorageGRID, select dc1-g1.demo.netapp.com from the drop down list.4. Enter nfs1 (must be lowercase) as the Bucket name.5. Select NFS1 as the Cache Device.6. Enter /nfs1 (remember the forward slash) as the Export path.7. Click Add Permission.

34

56

7

Figure 4-97:

8. Enter * (wildcard) for the Client.9. Click Readwrite.10. Select No Root for the Squash .11. Leave Anon UID and Anon GID at default.12. Click Add.

Page 77: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.377 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

89

10

12

Figure 4-98:

13. Click Save.

13

Figure 4-99:

14. Wait until status is “READY”.

Page 78: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.378 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

14

Figure 4-100:

Configure an SMB Share

1. Select SMB Filesystems from the left menu bar.2. Click Add next to “SMB Filesystems”.

1

2

Figure 4-101:

3. For “StorageGRID”, select dc1-g1.demo.netapp.com.4. For “Bucket”, enter smb1 (all lower case).5. For “Cache Device”, select SMB1.6. For Active Directory, select demo.netapp.com.7. For Share name, enter smb1.8. For Owner, leave as the default Administrator.9. For Group, leave as the default Domain Users.10. Click Save.

Page 79: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.379 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

34

56

78

9

10

Figure 4-102:

11. Wait until Status returns “READY”.

11

Figure 4-103:

Test the NFS Filesystem

From the Linux Machine, perform the following tasks:

1. Using Putty, ssh to the Linux Machine: linux1.demo.netapp.com.2. Login: root, password Netapp1!.3. Enter the following commands:

mkdir /mnt/nfs1mount –t nfs –o vers=3,rsize=1048576,wsize=1048576,hard,intr nas.demo.netapp.com:/mnt/nfs1 /mnt/nfs1

4. Test by copying some files into the mount point: “/mnt/nfs1”.

Page 80: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.380 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5. Using S3 Browser, you can view the contents of the bucket: nfs1. You may need to wait 1 minute forobjects to be stored to SGWS.

Test the SMB Filesystem

From the Windows Jumphost, perform the following tasks:

1. Open Windows Explorer on the Windows desktop.2. Right-click This PC, and select Map network drive....

2

Figure 4-104:

3. Select a free Drive Letter.4. Enter folder: \\nas.demo.netpp.com\smb15. Click Finish.

Page 81: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.381 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

34

5

Figure 4-105:

6. Test by copying some files into the mounted share.

Page 82: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.382 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

6

Figure 4-106:

7. Using S3 Browser, you can view the contents of the bucket: “smb1”. Wait for a minute and refresh S3Browser's view.

7

Figure 4-107:

4.15 Exploring S3 and Swift APIs

In this section, you will learn how to access the S3 and Swift APIs using command line and scripting tools. Theseexercises demonstrate how client applications can be modified to store data into StorageGRID Webscale usingthe S3 or Swift APIs.

Page 83: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.383 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

4.15.1 Access to S3 via the AWS CLI

In this section you will learn how to directly ingest and retrieve objects to/from StorageGRID via the AWS S3Command Line Interface (CLI). This will demonstrate how users or application developers can programmaticallyintegrate with StorageGRID Webscale using the S3 protocol, to store data into the GRID, or get it back out.

4.15.1.1 Persisting Credentials1. Find the PuTTY icon on the desktop (or in the task bar) and use it to run the PuTTY program.

1

Figure 4-108:

2. In the PuTTY window, choose Linux Client3. Click Load.4. Click Open.

Page 84: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.384 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

2

3

4

Figure 4-109:

5. A window will open with a log in prompt. Log in with userid root, using Netapp1! as the password.6. After you log in, configure the S3 account using the command aws configure --profile webscale.

Specify the webscale profile. Specify the access key, and the secret access key. Use us-gov-west-1 as the default region and json as the output format.. This setting will default to use path styleaccess. Use the S3 Access Key and S3 Secret Access Key that you obtained back in S3 AccountManagement on page 14.

[root@Linux1 ~]# aws configure --profile webscaleAWS Access Key ID [****************FCX5]: GGJNYGCPXKWSSBQYR8ICAWS Secret Access Key [****************ZgWu]: Mh4Eax22r+lx/kSmj78/yMtheUXuNbbR37dLnGYxDefault region name [us-gov-west-1]: us-gov-west-1Default output format [json]: json[root@Linux1 ~]#

To verify the configuration, you can view the following files:

[root@Linux1 ~] cat ~/.aws/credentials

The credentials file contains contains S3 Access Key and Secret Access Key credentials, organized by “profiles”.

[webscale]aws_access_key_id = GGJNYGCPXKWSSBQYR8ICaws_secret_access_key = Mh4Eax22r+lx/kSmj78/yMtheUXuNbbR37dLnGYx

4.15.1.2 Using the AWS CLIIn this Lab the AWS CLI is already installed. In case you want to install it on your own machine, refer to https://aws.amazon.com/cli/.

Page 85: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.385 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

The SSL/TLS certificate for this StorageGRID instance has already been placed at ~/grid_lod.crt.

First, list the existing buckets for our S3 profile “webscale”:

aws s3 ls --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt

Enter this command into the PuTTY window and observe the output:

[root@Linux1 ~]# aws s3 ls --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt2016-09-22 11:37:30 1-copy2016-09-22 11:37:18 3-copy2016-09-22 11:37:35 ec-copy2016-09-22 13:50:36 nfs12016-09-22 13:51:50 smb12016-09-22 12:13:39 versioning[root@Linux1 ~]#

Enter this command into the PuTTY window to perform the same operation with SSL verification disabled.

[root@Linux1 ~]# aws s3 ls --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --no-verify-ssl/usr/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)2016-09-22 11:37:30 1-copy2016-09-22 11:37:18 3-copy2016-09-22 11:37:35 ec-copy2016-09-22 13:50:36 nfs12016-09-22 13:51:50 smb12016-09-22 12:13:39 versioning[root@Linux1 ~]#

Now you will create a new bucket:

[root@Linux1 ~]# aws s3 mb s3://new-bucket --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crtmake_bucket: s3://new-bucket/[root@Linux1 ~]#

Next, copy a file into this bucket:

aws s3 cp ~/files/1/test.txt s3://new-bucket/ --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt

A subsequent “ls” operation on the bucket will list the contents:

aws s3 ls s3://new-bucket/ --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt

You can use the cp command to copy the object back:

aws s3 cp s3://new-bucket/test.txt ~/copied.txt --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt

Now, if you want to sync a whole directory, use the sync command:

aws s3 sync ~/files/ s3://new-bucket/ --profile webscale --endpoint-url https://dc1-g1.demo.netapp.com:8082 --ca-bundle ~/grid_lod.crt

As you can see, this command did not sync the empty directory “3”. This is because S3 does not know aboutfolders as we know them from POSIX based file systems. Nearly all S3 clients only interpret “/” as a subdirectoryfor making it more human readable.

A full documentation of the supported commands for the AWS S3 CLI can be found at http://docs.aws.amazon.com/cli/latest/userguide/using-s3-commands.html. Among other interesting commands are:

Page 86: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.386 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

Remove an empty bucket:

aws s3 rb s3://bucket-name

Remove a non-empty bucket:

aws s3 rb s3://bucket-name --force

Copy a file to an S3 bucket:

aws s3 cp file.txt s3://my-bucket/

Sync a whole directory recursively to/from S3 (will re-copy all files with a different size or modified time):

aws s3 sync <source path> s3://my-bucket/<target-path>

Delete a file in S3:

aws s3 rm s3://my-bucket/<path to the file>

For the cp, mv or rm command, you can use the --recursive option to walk through all directories, includingsubdirectories.

Another useful tool is S3cmd (http://s3tools.org/s3cmd), which supports more features than the official AWS CLI.

4.15.2 Access to S3 through Python

In many cases, developers will integrate with StorageGRID Webscale using the S3 API via program code. AWSprovides many SDKs for accessing S3 programmatically. StorageGRID Webscale is 100% compatible with theS3 API, thus supports the various S3 SDKs. Some of the supported languages are Java, Ruby, Python, PHP, Go,etc.

In this exercise, you will briefly perform basic S3 operations in Python against StorageGRID Webscale.

First, configure the Python library (“boto3”) to only use path-style access. While StorageGRID also supports“virtual hostname access”, this is not enabled in the lab environment. Edit the following file (previously created byaws):

[root@Linux1 ~] vi ~/.aws/config

Add lines to the file to make it look like this:

[profile webscale] s3=addressing_style = pathoutput = jsonregion = us-gov-west-1

Now, create a new file s3.py and paste the following code to access StorageGRID:

import boto3import boto3.session

session = boto3.session.Session(profile_name='webscale')

endpoint = 'https://dc1-g1.demo.netapp.com:8082's3 = session.resource(service_name='s3', endpoint_url=endpoint, verify='/root/grid_lod.crt')

client = s3.meta.client

You can execute the Python script by running:

[root@Linux1 ~] python s3.py

As this script only connects to StorageGRID, there will not be any output.

Page 87: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.387 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

Next, you can create, list and delete buckets:

# Create new buckets3.Bucket('my-bucket').create()

# List all bucketsfor bucket in s3.buckets.all(): print(bucket.name)

#Delete buckets3.Bucket('my-bucket').delete()

Note that the code might throw errors if the bucket already exists.

A new object can be uploaded with the following code:

obj = s3.Object('my-bucket', 'my-key')

obj.upload_file('file/1/test.txt')

The object can be downloaded to a file via:

obj = s3.Object('my-bucket', 'my-key')obj.download_file('target.txt')

If desired, you can also add metadata or encryption to the file you upload:

obj = s3.Object('my-bucket', 'my-key')

obj.upload_file('input.txt', ExtraArgs={'Metadata': {'customer_id': '42'}, 'ServerSideEncryption': 'AES256'})

When you look up the object in GMI using the object lookup, you can see that the metadata has persisted.

Encryption has also been applied.

You can also directly return the metadata, size, and last modification time for an object:

obj = s3.Object('my-bucket', 'my-key')print obj.metadataprint obj.content_lengthprint obj.last_modified

And you can also copy objects directly within StorageGRID. This copy process happens within StorageGRID anddoes not flow through the client:

copied_obj = s3.Object('<source bucket>','<target key>')copied_obj.copy_from(CopySource='/<source bucket>/<source key>')

4.15.3 Access to Swift through the CLI

In this section you learn how to directly ingest and retrieve objects to/from StorageGRID Webscale through thePython Swift client from the Command Line Interface (CLI). This allows users and application developers toprogrammatically integrate with StorageGRID Webscale, to put data into the StorageGRID Webscale, or get itback out.

4.15.3.1 Using Python Swift Client on the CLIFirst, log into the “Linux Client” host with PuTTy.

In this Lab setup, the Python Swift client is already installed. In case you want to install it on your own machine,refer to https://github.com/openstack/python-swiftclient.

The SSL/TLS certificate for this StorageGRID instance has already been placed at ~/grid_lob.crt.

Page 88: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.388 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

Check the version of the Python Swift client:

[root@Linux1 ~] swift –version

Specify the auth url using the –A argument, the swift account using the –U argument, and the –K argument tospecify the password. The SSL/TLS certificate can be specified using the –os-cacert argument. The Swift accountID can be copied from the GMI, under “Tenants”. The stat command provides the status of the specified swiftaccount.

[root@Linux1 ~]# swift -A https://dc1-g1.demo.netapp.com:8083/auth/v1.0-U 20157648367352802448:swiftadmin -K Netapp1! --os-cacert grid_lod.crt stat

You can add the account information into your .bashrc profile:

# Swift Connection Settingsexport ST_AUTH=https://dc1-g1.demo.netapp.com:8083/auth/v1.0export ST_USER=20157648367352802448:swiftadminexport ST_KEY=Netapp1!

And run the swift command with fewer arguments:

[root@Linux1 ~]# swift --os-cacert grid_lod.crt stat

[root@Linux1 ~]# swift --os-cacert grid_lod.crt list

[root@Linux1 ~]# swift --os-cacert grid_lod.crt upload my-container upload.txt

[root@Linux1 ~]# swift --os-cacert grid_lod.crt download my-container upload.txt

4.15.4 Access to Swift Using Python

In this exercise, you will perform some basic operations using the Swift API and Python using StorageGRIDWebscale as the storage endpoint.

1. Create a new file called “swift.py”, and paste the following code to access StorageGRID Webscale:

The user is the ID for the Swift account created and specified from the NMS. The Authentication URLpoints to the Gateway Node. The SSL/TLS certificate for this StorageGRID instance has already beenplaced at ~/grid_lob.crt.

import swiftclientuser='20157648367352802448:swiftadmin'key='Netapp1!'authurl='https://dc1-g1.demo.netapp.com:8083/auth/v1.0'cacert='/root/grid_lod.crt'

conn=swiftclient.Connection( user=user, key=key, authurl=authurl, cacert=cacert )

2. Execute the Python script by issuing:

[root@Linux1 ~] python swift.py

As this script only connects to StorageGRID Webscale, there will not be any output.3. Next, create and list containers:

# Create a new containercontainer_name='my-container-swift'conn.put_container(container_name)

Page 89: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.389 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

#List containersfor container_list in conn.get_account()[1]: print container_list['name']

Upload a new object can with the following code, and include specific custom metadata:

# Create an object with metadatafile_name='swift.txt'

with open('swift.txt', 'r') as swift_file: conn.put_object(container_name, file_name, contents=swift_file.read(), content_type='type/plain', headers={'X-Object-Meta-CustomerID':'42','X-Object-Meta-Color':'red'})

4. List the objects in the container:

for data in conn.get_container(container_name)[1]: print data['name']

5. Retrieve the object:

# Get objectget_obj=conn.get_object(container_name, file_name)with open('new_swift.txt','w') as get_file: get_file.write(get_obj[1])

6. Delete the object:

conn.delete_object(container_name, file_name)

Page 90: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Hands-On Lab: NetApp StorageGRID Webscale 10.390 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

5 Version History

Version Date Document Version History

Version 0.9 July 2014 Initial Version

Version 1.0 October 2014 Insight 2014 Version

Version 1.0 December 2014 Lab on Demand Version

Version 2.0 October 2015 Insight 2015 Version

Version 3.0 February 2016 Updated for SGWS v10.2

Version 3.1 September 2016 Updated for SGWS v10.3 (Insight 2016 Version)

Version 3.1 November 2016 Fix minor typos

Page 91: Hands-On Lab: NetApp StorageGRID Webscale 10 · PDF file3 Lab Environment ... Hands-On Lab: NetApp StorageGRID Webscale 10.3 10

Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exactproduct and feature versions described in this document are supported for your specific environment.The NetApp IMT defines the product components and versions that can be used to constructconfigurations that are supported by NetApp. Specific results depend on each customer's installation inaccordance with published specifications.

NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of anyinformation or recommendations provided in this publication, or with respect to any results that may be obtainedby the use of the information or observance of any recommendations provided herein. The information in thisdocument is distributed AS IS, and the use of this information or the implementation of any recommendations ortechniques herein is a customer’s responsibility and depends on the customer’s ability to evaluate and integratethem into the customer’s operational environment. This document and the information contained herein may beused solely in connection with the NetApp products discussed in this document.

Go further, faster®

© 2017NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consentof NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, Data ONTAP®,ONTAP®, OnCommand®, SANtricity®, FlexPod®, SnapCenter®, and SolidFire® are trademarks or registeredtrademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks orregistered trademarks of their respective holders and should be treated as such.


Recommended