Download - ECS and Nginx LB and CloudArray - WordPress.com€¢ Check traffic distribution between ECS nodes • Traffic should be roughly similar (other load balancing technics should give better

ECS with Nginx load Balancer and CloudArray VasilyPantyukhin

http://oldhenhut.com

ECSpreparation

CreateReplicationGroup

• ReplicationGroupcombinestwoECSsU300.

AddnewBaseURL

• Domainsuffixecspoc.localshouldbeusedastheBaseURL

• UseBaseURLwithoutNamespaceoptionchecked

CreateS3user

• CreateObjectuser

• GenerateS3password

4

CreateBucketandmodifyDNS

• CreateBucketwiththenamecabucket

• AddwildcardDNSrecord*.ecspoc.local

• Pingthebucket’sFQDN

5

ConfigureCloudArraypart1

UpgradeCloudArraytothelatestversion

• NavigatetoAdministration>CheckForUpgrade

• Upgradetoanewversion.

• Ittakesseveralminutes.

6

CheckCloudProviderwithoutaloadbalancer

• NavigatetoCloudArray>CloudProviders

• CloudProvidertype=ViPR

• EnableEncryption

• FortestpurposesCompressionshouldbedisabled(enabledonthescreenshot)

• Key=S3userconfiguredonECS

• Secret=S3passwordconfiguredonECSforthespecificuser

• Node=DNDsuffixecspoc.local

• BucketName=cabucket

• DisableSSL(canbeenabledinproduction)

• Port=9020(noSSL)

7

• Configureperformanceoptimizer

• CloudProviderisconfigured

8

DeleteCloudProvider

WecheckedthatCloudProvidercanbeconfiguredwithoutloadbalancer.Nowwecandeletethis

CloudProvider,configureNginxandconfigureanewCloudProvidervialoadbalancer.

• DeleteCloudProvider

9

ConfigureNginx

CheckNginxinstallation

• CheckNginxversionsandinstalledmodules

$/usr/local/nginx/sbin/nginx-Vnginxversion:nginx/1.9.2builtbygcc4.8.2(Ubuntu4.8.2-19ubuntu1)configurearguments:--with-http_ssl_module--with-http_spdy_module--with-http_realip_module--with-http_stub_status_module

• Usingcurl,youshouldseesomeHTMLoutputifyoutrytoaccessport80onnginxLinux

host.

$curl127.0.0.1<!DOCTYPEhtml><html><head><title>Welcometonginx!</title>

• Modifynginxconfig

$vi/etc/nginx/nginx.confhttp{server{listen*:9020;server_name*.ecs-poc-lb1.ecspoc.local;}}

• Reloadnginxconfig

$servicenginxrestartor

$kill-HUP`cat/var/run/nginx.pid`or

$nginx–sreload

• Inawebbrowser,gotoyourserver'sdomainnameorIPaddress:

http://ecs-poc-lb1.ecspoc.local:9020

10

Configureloadbalancing

InthattestSSLterminationwasnotused.IncaseifSSLisneeded,checktheAppendix.

• Modifyconfig

• ECS1nodesconfiguredastheupstreamservers

• DefaultloadbalancingmethodisRoundRobin

• Anylisteningportcanbespecified.Inthetest9020wasusedasforupstreamserversaccess.

• Bucketnamewillbeaddedtothebeginningofdomainprefixecspoc.localasservername.

ThatoptionisspecifiedasanobjectbaseURLinECS.

• Theoptionproxy_set_headerHost$hostisveryimportant.Bydefaultthe"Host"headeris

re-writtentotheIPaddressornameandportnumberoftheupstream(definedby

proxy_pass).When$hostvariableissetasaHostheader,hostnamefromtherequestwillbe

returnedback.

$vi/etc/nginx/nginx.confworker_processesauto;events{worker_connections512;}http{upstreamECS{serverECS-POC-NODE1.ecspoc.local:9020;serverECS-POC-NODE2.ecspoc.local:9020;serverECS-POC-NODE3.ecspoc.local:9020;serverECS-POC-NODE4.ecspoc.local:9020;}server{listen*:9020;server_name*.ecspoc.local;location/{proxy_passhttp://ECS;proxy_set_headerHost$host;}}}

• Checkifconfigurationfileisvalid.$/usr/sbin/nginx-tnginx:theconfigurationfile/etc/nginx/nginx.confsyntaxisoknginx:configurationfile/etc/nginx/nginx.conftestissuccessful

• Reloadnginxconfig

$servicenginxrestart

MoreaboutnginxDirectives:http://wiki.nginx.org/Modules

11

CheckaccesstoECSvialoadbalancer

• Inabrowsertypeloadbalancernamewith9020port.

• Checkthe“NoNamespaceForAnanymousRequest”errormessage.Thatiscorrectbehavior.

• AccessviaS3browsercanbecheckedaswell.

ConfigureCloudArraypart2

CreateCloudProvider

• Configurenewbucket.cabucket4.ecspoc.localshouldhaveIPaddressogtheloadbalancer.

• ConfigureCloudProviderthesamewayasitwasontheprevioustest.cabucket4shouldbe

specifiedastheBucket.

• FortestpurposesCompressionshouldbedisabled(enabledonscreenshots).

12

• ConfigurePerformanceOptimizer

• Checktheresult

• ConfigureaccesstoECSwithS3browser.

• NavigatetoAccount>EditAccount

• StorageType=S3CompatibleStorage.

• RESTEndpoint=cabucket4.ecspoc.local

• AccessKeyID=S3userconfiguredonECS

• SecretAccessKey=S3userpasswordgeneratedforS3user

13

• NavigatetoTools>Options>Connection.

• Uncheckusesecuretransfer(HTTPS).SSLwasnotusedinourtests.

• CheckifsomeobjectswerecreatedontheBucketusedbytheCloudArray.

ConfigureCache

• AddasecondHardDisktotheCloudArrayvirtualmachine

• 50Gbdiskinourcase

14

• NavigatetoCacheManagement>CacheSources

• Scanforadditionalcachesources

• 49GBPool_1wasdiscovered

• Addcache

15

• SelectPool_1asasource

• ChangeCachePageSizeto256KB.

Important,1MBdefaultPageSize(presentedonthescreenshot)canleadtoproblemswith

cachedraininginVMwareenvironment.AccordingtotheClouArraybestpracticedoc:“Cache

pagesgreaterthan1MBrequireaninternalsectorsizeof4KBratherthanthetypical512bytes.

Thelargersectorsizeisnotrecommendedfortransactionalworkloadssuchasdatabaseor

typicalfilesystemapplications.”But4Ksectorsizesaren’tcurrentlysupportedonESXi5.

• CheckSelectOneBigCacheUsingAllAvailableCapacity

• Bytheway,cacheisexpandableifothercachesourcesareavailable(notinourcase).

16

ConfigurePolicy

• NavigatetoProvisioningPolicies

• SelectCacheandCloudProvider

17

CloudArraylogs

• Incaseofanytroubles,pleasenavigatetoAdministration>SupportCaptureUtility>

GenerateSupportCapture

• Checkca_support.tgz>log>CloudArray.log

ConfigureNFS

• NavigatetoShares>Settings

• EnableNFS

18

• CreatenewShare

• SelectPolicy

• SpecifyCapacity

19

• AddClients

20

• Makeavailable

• LoginwithsshtoaLinuxmachine.

• CheckthatNFSexportisaccessible.

$showmount-e10.10.201.63Exportlistfor10.10.201.63:/export*/export/1TBNFS

21

• MountandcheckNFSexport.

$mkdir/NFSmnt$mount-tnfs10.10.201.63/export/backup/NFSmnt$df-hFilesystemSizeUsedAvailUse%Mountedon/dev/sda20G948M19G5%/udev119M4.0K119M1%/devtmpfs49M208K49M1%/runnone5.0M05.0M0%/run/locknone122M0122M0%/run/shm172.20.8.169/export/1TBNFS1024G70M492G0.01%/NFSmnt

• WritedatatoNFSexport.$cptest.txt/NFSmnt$ls/NFSmnttest.txt

ConfigureCIFS

• NavigatetoShares>Settings

• EnableCIFS

• SpecifyNameandDescription

• CheckAllowGuestAccessForCIFSShare

22

• CreatenewCIFSShare

• SelectPolicy

• SpecifyCapacity

• MakeAvailable

• ReadOnly=No

• Browseable=Yes

• CheckRead-WriteGuest

23

• LogintoWindowshost

• CheckCIFSShareavailability

• Checkreadandwrite

TestCloudArraybehavior

• Startwritingbigfiles

• NavigatetoDashboard

• CheckUsedandDirtycachecharacteristics(smoothlittlebitwavyincreasewithlater

decreaseareexpected).

24

• CheckCloudTransferRate(saw-toothcurveisexpected)

TestreplicationbetweenECSs

• LoginwithS3browsertoECS1

• Checkthenumberofobject

• Checkthesizeofthebucket

• LoginwithS3browsertoECS2

• Checkthenumberofobjectinthesamebucket(shouldbethesamenumberasforECS1)

• Checkthesizeofthesamebucket(shouldbethesamesizeasforECS1)

25

• ComparecapacityonECS1andECS2

• CurveonECS2shouldbemoresmoothcauseofECS1toECS2replicationnature

26

CheckRoundrobinloadbalancing

• ChecktrafficdistributionbetweenECSnodes

• Trafficshouldberoughlysimilar(otherloadbalancingtechnicsshouldgivebetterbalancing)