Date post: | 05-Nov-2015 |
Category: |
Documents |
Upload: | manoj-kumar |
View: | 23 times |
Download: | 0 times |
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 1/17
VagrantisoneofthosetechnologieswhichIoncedidn'tthinkIneeded,butaftergivingitagoitquicklybecameanindispensablepartofmydailyworkflow.Withoutit,workingonServerObserverwouldhavebeenmuchmoredifficult.ThesamecaneasilybeappliedtomanyotherprojectsIhaveworkedonaspartofmydegree.
Vagrantitselfisonlypartofthesolution,ithandlesthegenerationandbasicimagingofvirtualmachinestoformisolateddevelopmentenvironments.However,Vagrantdoesn'thandletheprovisioningofthesevirtualmachines.It'sstilluptoustohandletheinstallationandconfigurationofthepackageswerequiretowork.Thankfully,Vagrantiseasilyintegrateswith
DevelopmentnvironmentwithVagrantandAnile
Pulihed:22Ma2014Tag:development,vagrant,pthon
HowtoueVagrantandAniletogethertouildapowerful,flexileandportaledevelopment
environment
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 2/17
existingprovisionersincludingChef,Puppet,andmyprovisionerofchoiceAnsible.
Onceavirtualmachinehasbeengeneratedandaprovisionerhasbeendevelopeditisveryfasttostartadevelopmentenvironmentonabrand-newcomputerwithouthavingtoworryaboutfinding,installing,andconfiguringmultipleprojectdependencies.Anydevelopercanpickupaprojectandstartworkingonitwithlittlemanualwork,inthesameenvironmentaseveryoneelse,usingtheirfavouritetext-editorontheirlocalcomputer.
ThisarticlewilltakeyouthroughthestagesofsettingupaVagrantconfigurationforanewvirtualmachine,andhowtoprovisionthiswithsoftwareusingtwocustomAnsibleplaybooks.Thiswillbeaveryhands-ontutorialwhereI'drecommendtryingthingsforyourself.ItisbackedbyarepositoryonGitHubwithacompletecopyofalloftheresourcesusedinthistutorial.
Inthistutorialwe'llcreateaDjangodevelopmentenvironment,usingaPostgreSQLbackend.Despitethis,thistutorialcaneasilybeadaptedforothersetups,suchastheNginx/MySQL/PHP-FPMsetupinthephpbranch.
PreparationIfyouhaven'tusedtheTerminalbeforeonyourcomputer,Iwouldsuggestyoufamiliariseyourselfwiththebasics.Thistutorialisveryterminal-heavy,itwouldbewiseforyoutoknowwhatyou'redoing.Ataveryminimumyou'llneedtoknowhowtonavigatethefilesystemonyouroperatingsystem.CommandsgivenhereshouldbethesameforMacandLinux.
Makesureyou'veinstalledthelatestversionofVagrantforyour
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 3/17
platform(version1.5.4attimeofwriting),aswellasAnsible(version1.6.1attimeofwriting).Ifyoudon'talreadyuseityou'llalsoneedtoinstallVirtualbox,althoughitispossibletouseVagrantwithVMwareFusionorHyper-V.Oncethisisdonewe'rereadytostartbuildingourfirstdevelopmentenvironment.
Vagrantisdesignedtobecommitteddirectlyintoyourprojectssourcecontrol,alongwithanyprovisionersused.Thisiscertainlysomethingthatisadvised,asitmakestheuseoftheenvironmentassimpleaspossible,andalsohelpstodocumentwhatthesystemdependenciesareandwhentheychange.Vagrantwillcreatesomefileswhichyoudonotwanttocommitthough,justastheuseofvirtualenvlaterinthistutorialwill.Thefollowing.gitignorefilewillstopanyofthesefilesfrombeingcommittedbyaccident.
#Vagrant.vagrant/
#Pythonbin/build/include/lib/local/.pyc
GettingtartedCreateadirectoryonyoulocalfilesystemtoworkfromandnavigateintothisfromtheterminal.Feelfreetoinitiatesomeversioncontrolsoftwarewithinthedirectory,beforerunningvagrantinit.Thiswillgenerateabasicconfigurationfileforusto
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 4/17
workfrom,includingsomeextensivecommentingwhichshouldhelpyouunderstandwhatisgoingon.Forthisusecasethevastmajorityofthisisnotneeded,sowe'llremovethisoncewe'vegainedabasicunderstandingofwhatisrequired.
Vagrantdoesn'tusethestandardimagesmostoperatingsystemdistributorsprovide.Insteaditusesa'vagrantbox',whichwillcontainmultiplepre-configuredpackagesaswellastheoperatingsystem.Anyonecangenerateabox,andVagrantCloudalreadyhasmanytochoosefromwithvariouspre-installedpackages.ItendtoworkfromastandardUbuntuPrecise64box,asthisistheoperatingsystemthatmostofmyserversrun.Tomakeuseofthis,we'llsetconfig.vm.boxto'hashicorp/precise64'.
Whilewe'readdingthistotheconfigurationwe'llforwardaportfromourlocalcomputertothePythondevelopmentserverandconfigureasharedfolderwhichwillallowustoworkonthecode-baseonourlocalcomputerandhaveVagrantsynchronisechangesintothedevelopmentenvironment.ChangeyourVagrantfileasfollows:
#*mode:ruby*#vi:setft=ruby:
#VagrantfileAPI/syntaxversion.Don'ttouchunlessyouknowwhatyou'redoing!VAGRANTFILE_API_VERSION="2"
Vagrant.configure(VAGRANTFILE_API_VERSION)do|config|#AllVagrantconfigurationisdonehere.Themostcommonconfiguration#optionsaredocumentedandcommentedbelow.Foracompletereference,#pleaseseetheonlinedocumentationatvagrantup.com.
#EveryVagrantvirtualenvironmentrequiresaboxtobuildoffof.
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 5/17
config.vm.box="hashicorp/precise64"
#Createaforwardedportmappingwhichallowsaccesstoaspecificport#withinthemachinefromaportonthehostmachine.Intheexamplebelow,#accessing"localhost:8080"willaccessport80ontheguestmachine.config.vm.network"forwarded_port",guest:8080,host:8080
#Createapublicnetwork,whichgenerallymatchedtobridgednetwork.#Bridgednetworksmakethemachineappearasanotherphysicaldeviceon#yournetwork.config.vm.network"public_network"
#Iftrue,thenanySSHconnectionsmadewillenableagentforwarding.#Defaultvalue:falseconfig.ssh.forward_agent=true
#ShareanadditionalfoldertotheguestVM.Thefirstargumentis#thepathonthehosttotheactualfolder.Thesecondargumentis#thepathontheguesttomountthefolder.Andtheoptionalthird#argumentisasetofnonrequiredoptions.config.vm.synced_folder"./app","/var/www/djangoapp"end
You'llnoticethattheconfig.vm.synced_folderlinemaps./apponourlocalfilesystemto/var/www/djangoappontheremotefile-system.You'llneedtomakethe./appdirectorywithinthesamedirectoryasyourVagrantfilenow,oradjustthispathtosuit.
Onthecommandline,runvagrantup.AtthispointVagrantwilldownloadtheUbuntuboxifitisn'talreadyonyoursystem,makeacloneforthespecificvirtualmachine,andbootthevirtualmachinewiththegivenconfiguration.Ifneededitispossibletochangethedefaultemulatedhardware(defaultof
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 6/17
512MBmemoryandasingle-coreCPU)orhaveoneVagrantfilelaunchmultiplevirtualmachines.
ProviioningVagrantAtthispointwehaveadevelopmentvirtualmachine,butitlacksthedependenciesofourproject;thisiswheretheprovisionercomesin.Inordertogetstartedweneedtoset-outabasicdirectorystructurecontainsthefilesandfoldersrequiredbyAnsible.Initially,thismayappeartobeoverkillbutgettingeverythingorganisednowmakeslifeeasieratalaterdate.Wecanusetheterminaltotakeafewshortcutsandquicklybuildtherequireddirectorystructure.
mkdirpprovision/{setup,deploy}/{tasks,vars,handlers}touchprovision/{setup,deploy}/{tasks,vars,handlers}/main.ymltouchprovision/vagrant.yml
Theresultingfile-structureshouldbeasfollows:
app/provision/setup/tasks/main.ymlvars/main.ymlhandlers/main.ymldeploy/tasks/main.ymlvars/main.yml
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 7/17
handlers/main.ymlvagrant.ymlVagrantfile.gitignore
TheideahereistospilttheAnsibleconfigurationupintomultipleplaybookstohelpkeepeverythingorganisedandasreusableaspossible.Thesetupdirectorywillhavetheconfigurationfortheoperatingsystem,whilethedeploywillcontaineverythingrequiredforthevirtualenv.
Now,let'stellVagrantabouttheprovisioners.AddthefollowingjustinsidetheclosingendstatementintheVagrantfile:
#Installrequiredsoftware,dependenciesandconfigurationsonthe#virtualmachineusingaprovisioner.config.vm.provision"ansible"do|ansible|ansible.playbook="provision/vagrant.yml"end
Then,addthefollowingtoprovision/vagrant.yml:
hosts:allroles:setupdeploy
Now,whenanewvirtualmachineisstartedwithVagrantitwillautomaticallyrunthevagrant.ymlwithAnsible,whichwillruneachoftheplaybooksinturn.
IntallingApplicationThissectionwillfocusonthesetupplaybook.Thefirstthingweneedtodoisinstallthesystemdependencieswerequire.Add
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 8/17
thefollowingtoprovision/setup/tasks/main.yml:
name:Installpackagessudo:yesapt:pkg={{item}}state=installedupdate_cache=yeswith_items:#Databasepostgresqllibpqdev#RequiredforAnsibletointeractwithpostgrespythonpsycopg2#RequiredforAnsibletointeractwithpostgres
#PythonDevpythondevpythonsetuptoolspythonvirtualenv
Thiswillupdatethepackagecachewithapt,andthenensureeachoftheprogramsareinstalledalongsidetheirdependencies.You'llnoticetwodependencieshaveacommentnexttothemdetailingthey'rerequiredbyAnsible.ThesetwoprogramsallowAnsibletointeractdirectlywithPostgreSQL,allowingustoautomateitssetup.
Earlieronwebootedourvirtualmachine,butitwasnotprovisionedasthishadn'tbeendevelopedatthetime.Ratherthanthrowingawayandrebuildingthevirtualmachinewecanforcevagranttoprovisionthevirtualmachinebyrunningvagrantprovision.Thefinallinesoftheoutputshouldlooklikethis:
PLAYRECAP********************************************************************default:ok=2changed=1unreachable=0failed=0
ConfiguringPotgreQLAtthispointwe'remakinggoodprogress,howeverweneedto
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 9/17
configurePostgreSQLsothatourDjangoprojectcanuseit.Thefirstthingthatneedsdoingistheauthenticationconfigurationsolocaluserscanusepasswordauthentication.Todothisafilecalledpg_hba.confneedsreplacing,Iwillnotexplainthisinanydetailhere,howeverthefileneededisintheGitHubrepository.Downloadacopyandadda'files'directorywithinthesetupplaybookandplacethefileinthere,namedpg_hba.conf.
Addthefollowingtotheprovision/setup/tasks/main.ymltocopythenewconfigurationontotheserver:
name:Allowpasswordauthenticationforlocalsocketuserssudo:yescopy:src=pg_hba.confdest=/etc/postgresql/9.1/main/pg_hba.confforce=yesnotify:RestartPostgres
You'llnoticethisstatementendswithanotifyaction.Thiscallsahandlerwiththegivennameintheeventthatthetaskchangesanythingonthesystem.Inthiscaseshouldthepg_hba.conffilechange,it'llrestartPostgreSQLtoloadthenewconfiguration.Wedoneedtowritethishandlerthough,soaddthefollowingtoprovision/setup/handlers/main.yml.
name:RestartPostgressudo:yesservice:name=postgresqlstate=restarted
NowwehavereconfiguredPostgreSQLtoallowlocalusertologinweneedtocreateauserandadatabase.First,we'lladdsomevariableswhichcontainthedetailswewanttouseforthenewuser.Addthefollowingtoprovision/setup/vars/main.yml:
db_name:django_appdb_user:django
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 10/17
db_password:sdjgh34iutwefhfgbqkj3
Now,wecanusetheAnsiblePostgreSQLmoduletoconfigureauserandadatabase.Addthefollowingtoprovision/setup/tasks/main.yml:
name:CreateDatabasesudo:yessudo_user:postgrespostgresql_db:name={{db_name}}
name:CreateUsersudo:yessudo_user:postgrespostgresql_user:name={{db_user}}password={{db_password}}state=presentrole_attr_flags=NOSUPERUSER,CREATEDB
name:ProvideuserwithDBpermissionssudo:yessudo_user:postgrespostgresql_user:user={{db_user}}db={{db_name}}priv=ALL
Thesudo_userlinestellAnsiblewhatusertorunacommandas,andsincePostgreSQLwillonlyallowuserstologinfromthepostgresaccountbydefaultwetellAnsibletorunthecommandsasthisuser.
Withthissectioncompletewehavenowsuccessfullyprovisionedavirtualmachinewithallofthesystemdependenciesfortheproject.Toforcevagranttore-provisionthevirtualmachineandsoloadthelatestchangesrunvagrantprovision.
WorkingwithVirtualenvNowwe'remakinggoodprogress,butit'stimetomoveonto
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 11/17
provisioningthesoftwaredeploymentdependencies.Fromhereonwe'llbeworkingwiththedeployplaybook.Thefirstthingtoconsideriswe'llbeusingpiptoinstallaseriesofpythonmoduleswithinaplaybook.Todothisweneedarequirementsfilewhichwillbepassedtopip.Let'sinstallthelatestversionsofDjangoandSouth,aswellaspsycopg2sothatDjangocantalktothePostgreSQLdatabase.Createafilecalledrequirements.txtwithintheappdirectory,andaddthefollowing:
DjangoSouthpsycopg2
Now,we'lltellAnsibletocreateavirtualenv,andthentoinstalltherequiredmodules.Addthefollowingtoprovision/deploy/tasks/main.yml:
name:SetupVirtualenvpip:virtualenv={{virtualenv_path}}requirements={{virtualenv_path}}/requirements.txt
You'llnoticetheuseofthevirtualenv_pathvariable,let'saddthisintheprovision/deploy/vars/main.ymlfile:
virtualenv_path:/var/www/djangoapp
Nowrunvagrantprovisionagaintosetupthepythonvirtualenvironment,thenwecanstartworkingwithDjango.
WorkingwithDjangoOncetheprovisionerhasfinishedrunningthat'sprettymuchitfortheinitialsetup,howeverwecouldtakethisfurther.Let'ssetupabasicDjangoapplicationwithSouthandthenuseansibletosetupthedatabaseonthevirtualmachinecreation.RunthefollowingtoSSHintothevirtualmachine,activatethevirtualenvironmentandcreateanewDjangoapplication.
DANILGROV HOWNAV
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 12/17
vagrantsshcd/var/www/djangoappsourcebin/activatedjangoadmin.pystartprojectvagranttest
Thiswillhavecreatedtheinitialapp,butnowweneedtoupdatethesettings.pyfilesoDjangocantalktoPostgreSQL.Findthefollowinglinesinapp/vagranttest/vagranttest/settings.py:
#Database#https://docs.djangoproject.com/en/1.6/ref/settings/#databases
DATABASES={'default':{'ENGINE':'django.db.backends.sqlite3','NAME':os.path.join(BASE_DIR,'db.sqlite3'),}}
Andreplacethemwith:
#Database#https://docs.djangoproject.com/en/1.6/ref/settings/#databases
DATABASES={'default':{'ENGINE':'django.db.backends.postgresql_psycopg2','NAME':'django_app','USER':'django','PASSWORD':'sdjgh34iutwefhfgbqkj3','HOST':'localhost',}}
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 13/17
AndaddSouthformigrationssupport:
#Applicationdefinition
INSTALLED_APPS=('django.contrib.admin','django.contrib.auth','django.contrib.contenttypes','django.contrib.sessions','django.contrib.messages','django.contrib.staticfiles','South',)
Now,let'smakeAnsiblesyncourdatabaseforus.
name:Djangosyncdbdjango_manage:command=syncdbapp_path={{virtualenv_path}}/vagranttestvirtualenv={{virtualenv_path}}name:Djangomigratedjango_manage:command=migrateapp_path={{virtualenv_path}}/vagranttestvirtualenv={{virtualenv_path}}
Nowprovisionthevirtualmachineonefinaltimetoensurethatitrunsthroughproperly.
PuttingverthingtoUeThat'sallthereistoit.Nowanydevelopersworkingontheprojectsimplyclonetherepositoryandrun:
vagrantupvagrantsshcd/var/www/djangoappsourcebin/activatepythonvagranttest/manage.pyrunserver0.0.0.0:8080
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 14/17
Oncethey'vedonethisthey'llhaveanidenticalenvironmenttoeveryoneelse,willbeabletocontinueworkingintheirfavouriteeditor,andwillbeabletoaccesstheprojectbysimplyaccessinghttp://localhost:8080inabrowserontheirlocalmachine.
ThereareafewVagrantcommandsworthknowing:vagrantup:Bootsavirtualmachinevagrantprovision:Forcetheprovisionerstoberunagainavirtualmachine.Usefulforupdatingtheconfigurationofexistingvirtualmachines.vagranthalt:Shutdownavirtualmachine.vagrantsuspend:Sleepavirtualmachine.vagrantdestroy:Deleteavirtualmachinefromyoursystem.
It'salsoworthnotingthatyoushouldalwaysadd.vagrant/totheignorefileforyoursourcecontrol.Otherwiseyouwillbecommittingentirevirtualmachinestoyourproject,whichisthelastthingyouwanttodo.
ummarVagrantprovidesaflexiblewayofallowingmultipledeveloperstoworkoncomplexprojectswithlittlemanualsetuptimeorcomplexity,andwithoutlitteringtheirsystemwithprojectdependencies.Itisawell-designedplatformwhichallowscomplexinfrastructuretobereplicatedonalocalcomputerwitheasebyasmanydevelopersasrequired.
Ansiblepresentsapowerfulwayofprovisioningserversandvirtualmachineswithaminimalleaningcurve.ThisarticlehasonlytouchedonthebasicsofwhatAnsiblecando,coveringthebasicsofkeepingyourconfigurationorganised,andhowtosetupandconfigurebasicsoftwarepackages.
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 15/17
BetweenVagrantandAnsibleyouhaveeverythingyouneedtocreateportabledevelopmentenvironmentwhichcaneasilybecommittedintosourcecontrolwiththerestofyourprojectforanydevelopertouse.
ThesourceforthisexampleprojectisavailableonGitHub.
Gotomethingou'dliketoa?FindmeonTwitter,ordropmeanemail.
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 16/17
MoreLikeThiroadFallWildCampAtripouttoroadFall,Dartmoor,tophotographthewaterfallwithadramaticackdrop
ikepackingatTeignheadFarmRuinAikepackingtripacroDartmoortowildcampandTeignheadFarmRuin.
PeterTavandFurTor
4Comments DanielGrovesWebDesign Login1
Share SortbyBest
Jointhediscussion
Reply
pageworthy 2monthsagoThankssomuchforcreatingthistutorial.IthelpedmeatonandmylocaldevelopmentisnowinVagrant!
Reply
DanielGroves 2monthsagoadmin >pageworthyGladyoufoundithelpful.Feelfreetoshoutifyou'vegotanyquestions.
Reply
Chris 9monthsagoGreatwriteup.Helpedmealot.Onethingthough,it'snotrecommendedtoaddyourpythoncodetotheWebserversdocumentroot(/var/www/),asthatopensupforotherstobeabletoreadit.
Reply
DanielGroves 9monthsagoadmin >ChrisThanksChris.
GenerallyforaproductionenvironmentIcreateanewuserfortheapplication,andruneverythingfortheparticularapplicationunderthatuser.ThereasonIdidn'tdothatinthisarticlewaspurelytokeepthingsassimpleaspossibleforthereader.
Thankyouforpickinguponthis,Ireallyappreciatetheinputofothers.
Recommend
Share
Share
Share
Share
6/27/2015 DevelopmentEnvironmentswithVagrantandAnsibleDanielGroves
https://danielgroves.net/notebook/2014/05/developmentenvironments/ 17/17
ackpackingtoFurToronDartmoorforthePhotograph.
ThendofPhoto365Attheendof2014IcommittedtomakinganattemptonaPhoto365challenge.Itdidn'tgotoplan,andhere'whitwan'tafailure.
lewhereontheWe
PoweredJekllandhotedonDigitalOcean.
Copright20102015DanielGrove.Forlicencedetail,eetheGitHurepoitor.