Reusable Platform InvestigationDan Sellars | Manager, Software Development | CANARIE

27

CANARIEisinvestigatingtheadoptionofareusableplatform:

>Whyarewegoinginthisdirection?

>Whocanbestbenefitfromthisplatform?

>Howareweapproachingtheinvestigation?

>Whichoptionsareweconsidering?

Introduction

28

ReusablePlatform

OurmandateistosupportCanadianresearchersby:

> Fundingsoftwaredevelopment

> Providingsoftwaretoacceleratetimetodiscovery

> EnablingeffectiveuseofCanadianDigitalInfrastructure

> Promotingre-useofexistingsoftware— sowecanfundmore!

> SupportingthecommunityofCanadianresearchsoftwaredevelopers

ResearchSoftwareatCANARIE

29

> IdentifyareusableplatformwhereCANARIEcandevelopexpertise

>Promoteitforre-useinthecommunity

>Determineifapermanentdevelopmentteamcouldassistresearchsoftwaredevelopers

>Usetheresultingplatformasaninteroperabilitytestbed

>Wewanttocomplement— notcompete

WhatisCANARIE’sgoal?

30

>Wouldsuchanapproachbeacceptedbythecommunity?• Whatabout“NotInventedHere”?

>CanCANARIEaddenoughvalue?

>Canwesupportenoughteamstobeofvalue?

>Woulditbeadaptableenough?

PossiblePitfalls

31

>Thosewhodonotyethaveaplatform

>Thosewhowanttofocusonresearch-facingsoftware• Avoidingframeworkdevelopment• Donotwanttorecreatewhathasbeendonebefore

>Non-software-expertsinneedofawayintodigitalinfrastructure• Theso-called,“long-tail”ofbigdataresearchers

Whocanbestbenefitfromthisplatform?

32

Whatisanidealreusableplatform?

33

>First,whatisaplatform?• “asoftwaresystemthatsupportsmostorallofthefunctionalityaresearcherusesintheirworkflow”

>Areusableplatformwouldmeetthefollowingneeds:• Community-friendly• Computeintegrations• Datamanagementcapability• Extendable

DataManagement

Compute

Community

CANARIE

ResearchSpecific

Extensions

>Wedonotexpectonesolutiontofitall• Multipleoptionscouldbeviable• Acombinationmayalsoprovideamorecompletesolution

>Researchsoftwareisanotonesizefitsall• Butwethinkagoodbasewouldbeofwidebenefit

>Reusableplatformsarenotforeverybody• Therewillalwaysbeaneedforcustomdevelopmenttobringnewtechnologiestothecommunity

Morethoughtsonreusableplatforms

34

Aphasedapproachtotheinvestigation:

> Phase1:Highlevelreviewof~6options• Assessbyinspectionagainstdefinedcriteria• Selectashortlistforthenextphase

> Phase2:Closeevaluationof2-3platforms• Narrowdowntoonecandidateforthenextphase• Refinephase3goals

> Phase3:Deploymentofaprototype• Deployandprovidesampleextensionstothisplatform

Solicitfeedbackfromthecommunity

WhatapproachisCANARIEtaking?

35

• Ease ofdeployment• Flexibility• Supportformultipleprojects• Easeofuse• Cost• Stability

Whatarethedefinedcriteria?

36

• Reliabilityandsupport• Researchdatamanagementsupport• Securityandprivacy• Sharingandcommunication• Suitability• Existingfeatureset

JustconcludedPhase1• Startedbroadinvestigationfirst

Projectsthatweinvestigated• HubZero https://hubzero.org/• Open ScienceFrameworkhttps://osf.io• JupyterNotebook/Jupyter Hubhttp://jupyter.org/• CBRAIN http://mcin-cnim.ca/neuroimagingtechnologies/cbrain/• Alces Flighthttp://alces-flight.com/• Airavatahttp://airavata.apache.org• GenApphttp://genapp.rocks/

Wherearewenow?

37

HUBzero:asoftwareplatformusedtocreatedynamicwebsitesthatsupportresearch,education,andscientificcollaboration.> 60+HUBzero-basedsitesservingdifferentcommunities• Interactivesimulationtoolsdeliveredthroughwebbrowser• Usergroupsforprivatecollaboration• Mechanismforuploadingandsharingresources• Ratingandcitations• Wikisandblogs• Usersupportarea• Statisticsaboutusersandusagepatterns

HUBzero

38

OpenScienceFramework:providesfreeandopensourceprojectmanagementsupportforresearchersacrosstheentireresearchlifecycle.

• Structuredprojects• Controlledaccess• Enhancedworkflow• Trackprojectchanges• Viewprojectanalytics

>HostedintheUS

OpenScienceFramework

39

JupyterNotebook:awebapplicationthatenablesuserstocreateandsharedocumentsthatincludelivecode,visualizations,equations,andtext.

> Jupyter Notebookcombinesthreecomponents:• Thewebapplication• Kernels• Notebookdocuments

>WeknowComputeCanadaareinvestigating• Sowillnotproceedwiththisasaprimaryoption

Jupyter Notebook

40

CBRAIN:aweb-enabledplatformtoaccessandprocessdistributeddatawithhigh-performancecomputingcenters(HPCs).

> ConsistsoftwoRuby-on-RailsApplications:

• BrainPortal:aweb-basedgraphicaluserinterface• Userscanupload,tag,andsearchtheirfiles• Launchcompute-intensiveprocessingjobsonremoteHPC

• Bourreau: thebackendoftheCBRAINarchitecture– Managesthebackendtasks

CBRAIN

41

AlcesFlight:afully-featured,scalableHighPerformanceComputing(HPC)environmentforresearchandscientificcomputing.

>Mainfeatures:• CanbeconfiguredwithAlces Gridware

• provides750+applicationsandlibraries• ChoiceofHPCjobscheduler• SSHandgraphical-desktopconnectivityforusers• Datamanagementforfile-basedstorageandobjectstorage

Alces Flight

42

ApacheAiravata:asoftwareframeworkforsupportingexecutionandmanagementofcomputationalscientificapplicationsandworkflows.• Mainfocusisingridbasedsystems

> Severalcomponents:• Airavata API• Orchestrator• WorkflowInterpreter• ApplicationFactory(Gfac)• Registry• Messaging

Airavata

43

GenApp:amodularframeworkformulti-scalesciencecomputations.

> BuildsGUI-basedandweb-basedapplicationsonacollectionofJSONwrappedmodules• Extendableandadaptabletonewtargetlanguagesandenvironments

>CanbeintegratedwithAiravata• Providesthecapabilitytoharnessdistributedcomputingresources

*NottobeconfusedwithGenAP!

GenApp*

44

> JustabouttostartPhase2>Gaincommunityfeedback>RefineactivitiesforinclusioninPhase3e.g.• DeployatestinstanceontheDAIRCloud?• Willrepackagingmakeiteasiertoinstall?• Adding/integratingservicesofusetoallusers i.e.fromscience.canarie.ca?• Bettertrainingmaterials?• Wouldconsultingservicesbebeneficial?• PossibleintegrationwithResearchDataManagement(RDM)tools

>Offerasareusableplatformsupportedbyusforuseinafuturecall

NextSteps

45

>Wouldthishavebeenahelpwhenyoustarteddevelopment?>Doyouhaveanyadvice?>WhatdoyouseeasthebestwaythatCANARIEcouldoffersupport?• Provideexpertiseonintegration?• Extendinginfrastructurefunctionality?• IntegratewithComputeCanadaservices?

>Wouldyoubeinterestedingettinginvolved?• Advisorycapacity?• Usecaseprovider?• Offeringresearchfacingsoftwareforintegration?

Whatcommunityinputwouldwelike?

46

>TalkwiththeCANARIESoftwareteamduringtheevent> Email:software@canarie.ca>Chooseitasadiscussiontopicfortomorrow>WewillaskagainintheclosingQ&Asession>Now,ifyouhavesome!Wehavesomequestions…

HowtoGiveFeedback

47

Thank you!Questions?

48