IXP Manager Workshop V4 Deep Dive · IXP Manager Workshop V4 Deep Dive Barry O’Donovan & Nick...

IXP Manager Workshop

V4 Deep Dive

BarryO’Donovan&NickHilliard27th Euro-IXForum

Berlin,Germany– October25th 2015

Introduction / House Keeping / Agenda

Whatdoyouwanttocoverhere?Suggestions/preparedstuff:– Currentwishlists/plans– AdeeplookatIXPManagerV4

• Mytoolstack forthoseinterested

– Longtermgoal– funding,sponsorship,etc.– Somepracticalscripting

CurrentWishLists/Plans

Current Wish Lists / Plans (and WIP)

HelpdeskIntegrationL2ACLsAdvancingP2PfunctionalityResellerportsviap-tagPatchpanelmanagementIntelligentProvisioning– Portconfigurationtemplatesandautomaticconfiguration

L2 ACLs

Dynamicportsecurityjustdoesnotwork inanIXP– MostofushavealreadymovedtostaticL2ACLs

NeedthisfunctionalitysupportedinIXPManager:– Preventdrivererror/PEBKAC– AllcustomerstoupdateL2ACLs->nomore2AMphonecalls

Usualconstraintsexist:– Security– Switchvendoragnosticwithreferenceimplementation(s)– Someknobsanddials

Advancing P2P Functionality

CurrentlyP2PfunctionalityisviaRRDs– Thislimitsfeaturedevelopmentforanythingbutsimplyp2pgraphs

EvaluatingOpenTSDB andGraphite/Carbon/Whisper– WhisperislikeRRDinthatitisafixedsizedatabase– However,OpenTSDB ismuchmorecomplexandisdistributed

DiskIOisanissue– butnotinsurmountableWithamoredatabaselikebackend,wecanpresentmoreinterestingdetailstoourcustomers:topnpeers,burstiest peers,presentationofthedatacanbecomealotmoreuserfriendly,…

Reseller Ports via p-tag

IXPManagersupportsthefan-outportmodelforresellersasisNewequipmentatINEXallowsthep-tagmodelThisbreaksalotofthings:– Port/trafficgraphsviamrtg /snmp– MACaddresslearningforp2pgraphs– WehavetwomemberswherewehavenovisibilityL

L2ACLs+advancesinP2P/sflow collectionshouldfixallthisJ

Patch Panel Management

Currentlywemanage~25datacentre patchpanelsviawikiLHavehopedtoimplementthisinIXPManagerfor~7yearsLAnumberoffalsestarts.Why?– We’vebeencompletelyoverthinkingthiswithwiz-bangfeatures– Wejustneedthewikiequivlent forcryingoutloud!– Nofancygraphing,nomanaginginternalinterrack panels…

Patch Panel Management

PatchPanelManagementto:– Recordpatchpanelreferenceandmediatype,numberofports– Onaperportbasis,assigntoacustomer– Recordthedatacentre assignedID– Connection(anddisconnection)date– Optionallyassigntoaswitchport

• Needtomanagecustomerportmovesasaprocessnow!– Oroptionallyassigntocustomerco-locatedequipment

Nomorebike-sheddingthis!(ourselvesincluded!)

Intelligent Provisioning

Anoftenrequestedfeature– eveninternallyatINEXCreateaprovisioningflowforcommontaskssuchas:– Joiningtheexchange(westilluseaWorddocument,yuck!L )– Provisioningaport->verymanual


Provisioningaport->verymanual– Assignport– Assignpatchpanelportandsendcrossconnectdetails– Configureport(quarantine)– ConfirmcrossconnectisinplaceandrecordID– Testportandensurespeed/duplexarematched– Quarantineport(checktraffic,checkBGProutesadvertised)– PutportliveinproductionpeeringLAN– Setuproutecollector,routeserverandAS112sessions– Sendannouncement


Provisioningaport->verymanual– Assignport– Assignpatchpanelportandsendcrossconnectdetails– Configureport(quarantine)– ConfirmcrossconnectisinplaceandrecordID– Testportandensurespeed/duplexarematched– Quarantineport(checktraffic,checkBGProutesadvertised)– PutportliveinproductionpeeringLAN– Setuproutecollector,routeserverandAS112sessions– Sendannouncement


Provisioningaport->verymanual– Assignportand patchpanelportandsendcrossconnectdetails– Configureport(quarantine)– ConfirmcrossconnectisinplaceandrecordID– Testportandensurespeed/duplexarematched– Quarantineport(checktraffic,checkBGProutesadvertised)– PutportliveinproductionpeeringLAN– Setuproutecollector,routeserverandAS112sessions– Sendannouncement


So,howdowedesignaprovisioningsystemforeveryIXP?Carefully.Intelligently.Dynamically.Eachstephascommonattributes:– ActionpendingcustomerorIXP– Testtodetermineifstepiscompletecorrectly/incorrectly– Actiontotakeoncompletion?– Messaging– Advancetonextstep

Createaframeworkusingcontractstobuildupaprocesswithsteps

Intelligent ProvisioningPort Configuration Templates

Standardised portconfigurationsat(obviously)essentialatanIXPINEXperformedaforkliftupgradefromBrocadetoExtremerecentlyUsedIXPManager’sdatabasetoconfigureallportsonnewswitches– Timeefficient,nooperatorerrors,ensuredstandardconfigs withoutmissing

elements

CouldbeeasilyintegratedintoIXPManagerforcopyandpaste,but:– Wouldpreferittoactually:shutdownport+wipe+configure+enableagain– WouldalsolikeittovalidateRANCID/Oxidizedconfigs forerrors/ommissions

Vendoragnostic!

IXPManagerV4

IXP Manager V4

Majorversionchangesusuallymeanmajorchanges– Thisistruehere->butmainlyonthebackend

IXPManagerisyourtypicalMVCstack– We’recompletelychangingtwoelementsofthis:

• Thecontroller• Theview

IXP Manager V4 – Doctrine ORM

Let’stalkaboutthemodelfirst:thedatabaseLayerbetweenMySQL/MariaDBandthePHPapplicationisDoctrineORMV3ofIXPManagerchangedfromDoctrine1.2toDoctrine2– ActivemodeltoORM(ObjectRelationalMapping)

Anexample:


<?php$c = new Entities\Customer;$c->setName( "Big ISP Ltd" );$c->setAutsys(64496);// ...D2EM::persist( $c );


$custRepo= D2EM->getRepository( “Entities\Customer” );

$customers = $custRepo->findAll();

foreach( $customers as $c ) {// do something

}


Stableproject– aroundsince2006It’sactuallyanumberoflibrariesthatstacktogether.Mainly:– DoctrineDBAL:DatabaseAbstractionLayer– DoctrineORM:ObjectRelationalMapper/ing

http://www.doctrine-project.org/


Keyconcepts:– Entities:representasingledatabaserowfromagiventable– Proxies:compiled entitiesthatarefullytransparenttoyourcode.

• Allowsforlazyloading,loadingincompletedetails,etc.

– Repositories:handlessetsofentities• Mostofourcomplexqueriesarehandledbywayofproxies:


class Customer extends EntityRepository {/** * Utility function to provide a array of * all active and current customers. */ public function getCurrentActive(

$asArray =false, $trafficing =false, $externalOnly = false, $ixp = false )

{…

} …

}

IXP Manager V4

HowdidwestarttalkingaboutDoctrine?

IXPManagerisyourtypicalMVCstack– We’recompletelychangingtwoelementsofthis:

• Thecontroller• Theview

Let’slookattheviewnext…

IXP Manager V4 – The View

What’saviewcomponent?Whydowehaveit/needit?– Separateslogicfrompresentation– Inlargerprojects,UIdesignersdon’tneedtobeabletocodetomanagethe

frontend– Eradicatesspaghetticode– Allowsfortemplating withlayouts– Allowsforskinning

Foryears,theonlygameintowninPHPwasSmarty– It’sgodawfulL Itstinkstohighhell.– Okay,that’sprettyharsh.It’sofits’ageandhasn’tmovedforward…


ContenderstoSmarty:Twig– modernOOPdesign– goodextensibility– wellsupportedandwidelyused

Blade– thebuilt-indefacto viewforLaravel

Problem:viewsadddeveloperoverhead:moresyntax,libraries,functions,etc.tolearnL


PHPPlates– NativePHPtemplates– nonewsyntaxtolearn– InspiredbyTwig– Supportslayoutsandinheritance– Easytoextendwithfucntions andextensions

– NB:Platesisafulltemplatesystem,notspagetti codebyanothername

Inreality,IXPManagerv4supportsSmarty,BladeandPlatesoutofthebox.

IXP Manager V4 – The Controller

ThebiggestpartofMVCisC->thecontrollerHandles:– Routingofrequests– Middleware– Inputvalidation– Controllers– Responses

OurnewcontrollerisLaravel.Notjustacontroller,aframework.


Whychangeframeworkatall?– Developerapathywhichleadsto:– Stagnationofthecodebase– Neworprospectivedevelopersareturnedoff– Newfeaturesremainunimplementedbecausetherearebetterways– Staymoderntoleveragenewtechniquesandservices


TheLaravel framework providesnewtechniquesandintegrations:– Serviceproviderframework– Events– Queues– Taskscheduling– Testing– Migrations– Packagemanagement


Wecan’tthrowawaytheexistingcodebasethough.OverthecourseofV4’slifetime,we’llmigratefromZend toLaravel– i.e.Zend/Smartywillco-existwithLaravel/Plates forquitesometime– Thismeansnewfeaturescanbeimplementedimmediatelyusingthelatest

technologies– Wedon’tneedtodisappearforsixmonthstorewritetheentirecodebase

Howwillthisbeachieved?– An(in)elegent solution!


Laravel isnowthedefaultframeworkandroutesrequestsIfarequesthitsLaravel foraroutethatdoesnotexist/isnotimplementinLaravel:– Itthrowsa404exception

Inapp/Http/Kernel.php wecatchthat404exception– AndspinuptheZend Framework

Zend willthenhandleifpossibleorthrowanother404– (handledasapagenotfoundinZend andpresentedtotheuser)


try { return $this->sendRequestThroughRouter($request);

} catch( \Symfony\Component\HttpKernel\Exception\NotFoundHttpException $e ) {

require_once 'Zend/Application.php';

$application = new \Zend_Application(APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini’

);

$application->bootstrap()->run();}

IXP Manager V4 – Events and Queues

Eventsprovideasimpleobserverimplementation– Youcansubscribeandlistenforeventsinpackages/extentions!– Eventscanbefired whensomethingsignificant happens– Eventlistenerscanqueuetheeventforofflineprocessing

Laravel queuessupportBeanstalkd,IronMQ,AmazonSQS,Redis andsynchronous(local,immediate)

Whatkindofthingscanwedowiththis..?


PhysicalinterfacechangedinIXPManager:firephysIntChangedEvent– MRTGlistenercancheckforportorportspeedchangeandregenerateMRTG

configurationandreloadthedaemon– Billingnotificationslistenercancheckforspeedchangeandemailaccounts for

billingpurposes– Switchconfigurationlistenercanrolloutconfigurationchangetoswitch(firesevent)

• Physicalinterfacestatuslistenercaninspectinterfaceformatchingspeed/duplex

– Patchpanellistenercouldtakesome actionifaportischangedthathasaconnectedcrossconnect


VLANinterfacechangedinIXPManager:firevlanIntChangedEvent– AS112listenercan(de)configureBGPsessionasnecessary– Routecollectorcan(de) configureBGPsessionasnecessary– Routeserverscan(de)configureBGPsessionasnecessary– Othereventlistenersmayinclude:regeneratingSmokeping&Nagios configurations,

DNSPTRentries.– EnablingIPv6couldadditionallysendemailwithdetailsorstarttheIPv6enable

processtowalkthecustomerthroughconfigurating sessionstoroutecollector,servers,as112,etc.

IXP Manager V4 – Service Providers

CentralpieceofLarabel’s applicationbootstrappingRegisters:– Controllersandroutes– Eventlisteners– Middleware

CanbeusedtoextendIXPManagerwithouthackingthemaincodebase

IXP Manager V4 – Contracts

InterfacesthatdefinecoreservicesprovidedbyLaravelAlsohowwewilldevelopextensionstoIXPManager– Designacontract– Developreferenceimplementation(s)tothatcontract

Example:Helpdeskintegration– Firstabitofhistory….

New Helpdesk - [email protected]

Up to 2008 - Shared IMAP Mailbox2009 - Cerberus

Served us well but extreme feature creep2013 - Realisation that we need something new

Helpdesk research => maximum pain

From an INEX Members’ Update

New Helpdesk - [email protected] an INEX Members’ Update


Up to 2008 - Shared IMAP Mailbox2009 - Cerberus

Server us well but extreme feature creep2013 - Realisation that we need something new

Helpdesk research => maximum painPain so great, we stuck with what we had

2015 - Try again, new methodology => pain killerz


New Helpdesk - [email protected] an INEX Members’ Update


Candidates included:Freshdesk, GrooveHQ, Zendesk, Kayako, Cerb5 and many more…

Contrary to my initial preconceptions, the winner was ZendeskExcellent API (essential as we needed to import old tickets)Easily configurable triggers and automationsSupports markdownNice UI plus iOS / Android appsPoor reporting (@ Zendesk Regular anyway


IXP Manager V4 – Helpdesk Integration

Needtobeableto:– Createcustomers(organisations)onthehelpdesksystem– Createusersonthehelpdesksystem– Findticketsbyorganisation– Createtickets– Update/closetickets

It’sworkinprogressbutmostoftheintegrationisdone


Contract:app/Contracts/Helpdesk.php– ticketsFindAll()– organisationNeedsUpdating($custLocal,$custHelpdesk )– organisationCreate($cust )– organisationUpdate($helpdeskId,$customer)– organisationFind($id)– contactNeedsUpdating(…)– userCreate()– userUpdate()– …


Zendesk ReferenceImplementation– app/Services/Helpdesk/Zendesk.php

ServiceProvider– app/Providers/HelpdeskServiceProvider.php– Thisfileneedsupdatingfornewimplementations

Instantiation:– $helpdesk=App::make('IXP\Contracts\Helpdesk');

Configuration:– config/helpdesk.php– EnvironmentconfigurationviaPHPDotEnv


Sample.env forZendesk:

HELPDESK_BACKEND=zendeskHELPDESK_ZENDESK_SUBDOMAIN=ixpHELPDESK_ZENDESK_TOKEN=yyyHELPDESK_ZENDESK_EMAIL=john.doe@example.com

IXP Manager V4 – Statistics Backend

IXPManagercurrentlyonlysupportsMRTG/logforportstatsFromearlier,wealsowanttosupportportstatsviasflow /p2p– WouldalsoliketosupportMRTG/rrd

DE-CIXhaveofferedabountyforthisworkWillbeimplementedinthesamewayasthehelpdesk– i.e.anybackendcouldbesubstitued onceitisimplementedagainstthe

providedcontract

Threereferenceimplementations:sflow/p2p,MRTG/log,MRTG/rrd

IXP Manager V4 – Installation Changes

ThePHPdevelopmenttoolchainhaschangedsincev3– Gitsubmodules nolongernecessary

PHPhasapackagemanagementsystemcalledcomposer– Allthirdpartydependancies nowinstalledviacomposer– Includes:ZF1,Laravel,Smarty,Plates,Zendesk API,Doctrine,etc.

Frontendassetshandledsimilarlyviabower– Includesjquery,Bootstrap,etc.

IXP Manager V4 – Vagrant



1. InstallVagrant(http://www.vagrantup.com/ )2. InstallVirtualBox (http://www.virtualbox.org/ )3. CloneIXPManager,checkoutv4andinstalldependancies:

git clone https://github.com/inex/IXP-Manager.git ixpmanagercd ixpmanagergit checkout v4

composer update


4. Start-upVagrant:vagrant up5. Thiswilltakeawhile– itexecutesbootstrap.sh whichwill:

– apt-getupdate,upgradeandinstallalldependancies forIXPManager’sLAMPenvironment

– ConfigureMySQLandphpMyAdmin– Installcomposerandbower– ConfigureandpopulatetheIXPManagerdatabasewithsampledata– ConfigureApacheandIXPManager


6. Onceit’scomplete,youcan:– AccessIXPManagerat:http://localhost:8088/

• Adminusernameandpassword:vagrant/vagrant1– SSHintothevirtualmachinewith:vagrant ssh– Yourixpmanager directoryismountedunder/vagrant– MySQLisavailablevia:mysql –uroot–ppassword ixp

• Orhttp://localhost:8088/phpmyadmin


ManagingyourVM:– Shutdowncleanlybylogginginand: sudo shutdown –h now– Tosuspend: vagrant suspend– Toforceshutdown: vagrant halt– Tobringup: vagrant up– Forstatus: vagrant status

IXP Manager V4 – Documentation

Remember:v4isabridgingversionfromZF1toLaravelAssuch,documentationisalsoahalfwayhouseExistingdocumentationavailableat:– https://github.com/inex/IXP-Manager/wiki

Newdocumentationwillbe:– https://ixp-manager.readthedocs.org/en/latest/– Source:https://github.com/inex/ixp-manager-docs

IXP Manager V4 – My Tool Stack

AppleOSXwithHomebrewfor:– php,bash,bgpq3,git,joe,mariadb,node,sshfs andmuchmore

Atomasatexteditor– Withlanguage-php,linter(sameforCSS,JS,etc)andDash

Vagrant(latestUbuntuLTS)Git,GitHub,TravisCISkipper(ORMGUI,http://www.skipper18.com/ )

QuickCodingExample?

Quick Coding Example

Let’smakeanewArtisancommand– ArtisanistheCLIcomponentofLaravel

./artisan make:console DemoListCustomers


Nowlet’sedittheresultantfile:– app/Console/Commands/DemoListCustomers.php

GivethecommandanameanddescriptionAndlet’sseeifitworks…


Andlet’sseeifitworks…nopeL

Extendingthewrongclass– needtouseandextend:– useIXP\Console\Commands\CommandasIXPCommand;– =>classDemoListCustomers extendsIXPCommand {

Nooptions/argumentsrequiredNeedtoregisterthecommandinapp/Console/Kernel:– protected$commands=[…]

Nowitworks!Butdoesnothing...


Let’sgetandlistallcustomers:Completethefire()method:

$customers = \D2EM::getRepository( 'Entities\Customer' )->getCurrentActive();

foreach( $customers as $c )$this->info( $c->getName() );

Thanks for listening!

[email protected]

https://github.com/inex/IXP-Manager

Mailing list:https://www.inex.ie/mailman/listinfo/ixpmanager

Date post:	27-May-2020
Category:	Documents
Upload:	others
View:	31 times
Download:	0 times

IXP Manager Workshop V4 Deep Dive · IXP Manager Workshop V4 Deep Dive Barry O’Donovan & Nick...

Documents