GODANActionWP1:datastandards:survey,gap
analysisandrecommendationsFocusonweatherdatastandards
ValeriaPesceGlobalForumonAgriculturalResearchandInnovation(GFAR)
LeighDodds,JeniTennison,PaulineL'HénaffOpenDataInstitute(ODI)
PanagiotisZervas(AgroKnow)
GlobalOpenDataforAgricultureandNutritioninitiative(GODAN)
TheGODANinitiativewasoneoftheoutputsoftheG8InternationalConferenceonOpenDataforAgricultureandwasannouncedattheOpenGovernmentPartnershipConferenceinOctober 2013.Theinitiativefocusesonbuildinghigh-levelsupportamonggovernments,policymakers,internationalorganizationsandbusiness.
Currentlyaround500partnersworldwidefromnationalgovernments,non-governmental,internationalandprivatesectororganizations.
www.godan.info
GODANAction
Three-and-a-half-yearprogramme launchedbytheUK’sDepartmentforInternationalDevelopment.
GODANActionbringstogetheragricultureandnutritionspecialistsandopendataexpertsandwillsupportGODANinitsmissionbybuildingpeople’scapacitytoengagewithopendata.
http://www.godan.info/godan-action
UndertheDFIDGODANfundingstream
GODANAction:Focalareas1) Standards - Enhancingdatastandardsandpromotingbestpracticein
agricultureandnutritiontoimproveinteroperability.1) Mapofagri-fooddatastandards2) Gapanalysisonuseandusabilityofdatastandards3) Recommendationstoaddressgaps4) à Specificationsà services,pilotimplementations
2) Research - Identifyingandimprovingtoolsandmethodsforevaluationoftheimpactofopendatausageininitiativesandinvestmentsinagricultureandnutrition.
3) Capacities - Buildingthecapacityanddiversityofopendatausers,leadingtomoreeffectiveuseofdataintacklingkeyagricultureandnutritionchallenges.
Purposeoftheglobalmapofdatastandards
• Themainpurposeofaglobalmapofdatastandardsinaspecificfieldistopromotethediscoveryandreuseofvocabularies andtheirproperties,classesandcontrolledvalues.Thereuseofexistingvocabulariespromotesgreaterinteroperabilitybetweenvocabulariesanddatasets.paraphrasingwhattheDublinCoreMetadataInitiativesaysabouttheirDCMIRegistry(http://dcmi.kc.tsukuba.ac.jp/dcregistry/ )
• Helpidentifyoverlaps,duplication,gaps andlimitstoadoption,>>encouragesnottoduplicateeffortsandtocollaboratetobothdevelopandusecommonstandards
Approachforbuildingmapofdatastandards
•Notduplicating,buildingonwhatexists(e.g.syncwithAgroPortal)
•Collaborativeeffort
•Broadcoverage
•Opendataangle
•Designedforgapanalysis (morelater)
Anybodycanaddorclaimastandard
Thesaurus CodelistOntology ISOspecification MessagingstandardTaxonomy
Agronomy Naturalresources Fisheries Valuechains
Allcontributorsacknowledged
Calltoactiontopartnersandexperts
Format APIs Mappings License
CategorizationofdatastandardsbycontentDomain-specific• Bysub-domain:Whichdomainclassification?Attemptedone
basedonFAO+USDAclassificationsHowfartogowithneighboringdisciplines?
• Bydatatype:alignmentwithGODAN“AgriculturalSectorPackage”fortheOpenDataCharter
Sub-domains• AgriculturalResearch,Technologyand
Engineering• Agro- Economics,Businessand
Industry• AnimalScienceandAnimalProducts• EducationandAgriculturalExtension• FarmsandFarmingSystems• FisheriesandAquaculture• FoodandHumanNutrition• ForestScienceandForestProducts• Government,AgriculturalLawand
Regulations• HealthandPathology• NaturalResources,Earthand
Environment• PlantScienceandPlantProducts• RuralandAgriculturalSociology
MapofstandardssofarVEST / AgroPortal MAP OF STANDARDSvest.agrisemantics.org
Numberofdatastandardsbydomain
Criteriaforgapanalysis
Toidentifygaps,needtoidentifyassessmentcriteria.Criteria developedbasedon:
• TheassessmentprocessusedbytheUKGovernment’sOpenStandardsBoard
• TheODIOpenDataCertificates criteria
Categoriesofassessment:Fitnesstopurpose Adoption Usability Openness
ExampleofadoptionassessmentcriteriaUsedinsoftware Isthisstandardusedinsoftwaretools(e.g.as
controlledvaluesforsomefields,orasexportformat,orasdatamodel)?
Yes,inmanytoolsthatareverypopular 5Yes,inafewtoolsthatareverymuchused 4Yes,inafewtools 3Yes,in1-2tools 2No 0Notclear- N/A 1
Usedindatasets Isthisstandardusedindatasets(e.g.asserializationformat,orasdatamodel/elementset,orascontrolledvaluesforsomecolumns/dimensions)?
Yes,inmanybymanyproviders 5Yes,inmany,byafewproviders 4Yes,inafew 3No 0Notclear- N/A 1
Endorsed Doesthestandardhaveastrongsupportfromdifferentinterestgroups?
Yes,verystrongly 3Yes,moderately 2No 0Notclear- N/A 1
Regulatory Isthestandardpublishedbyarecognizedstandardizationbodyorasagovernmentdirective?
Yes 3No 0Notclear- N/A
1
Long-term,sustainable
Isthemaintainingorganizationalong-standingandauthoritativebody?Isthemaintainercommittedtosustainandpreservethestandard?
Yes,highly 3Yes,reasonably 2No 0Notclear- N/A
1
Participatory,collaborative
Isparticipationinthecreationprocessofthestandardopentoallrelevantstakeholders?
Yes 3No 0Notclear- N/A 1
Exampleofusabilityassessmentcriteria
Versatile Isthestandardavailableindifferentformatsfordifferenttechnologies?(e.g.XML,JSON,RDF)?
Yes,manyformats 3Yes,2-3similarformats 2No 0Notclear- N/A 1
ServedbyAPIs ArethereAPIsandwebservicesthatallowapplicationstoworkwiththevocabulary?Chooseasmanyanswersasyouneed.
Yes,togetweb- oruser- friendlyresults3
Yes,tolookupterms/conceptsusingseveralparameters4
Yes,toautomaticallyannotatetextordata5
Yes,toperformcross-walksbetweenvocabularies6
Yes,toextract/lookupsubsetsofvocabularies7
No 0Notclear- N/A 1
Manageable Isthestandardmanagedinacollaborativeenvironment?Isitmanagedonaspecializedvocabularymanagementplatform?
Yes,onspecializedvocabularymanagementplatform3
Yes,onacollaborativeenvironment(e.g.Github)2
No 0Notclear- N/A 1
Exampleofopennessassessmentcriteria
Machine-readable
Isthestandardavailableinmachine-readableformats?(XML,CSV,RDF...)
Yes 3No 0Notclear- N/A 1
Meaningful Ifmachine-readable,isthestandardserializedusingtheappropriatevocabularylanguage(RDFs,OWL,SKOS,OBO)andsemanticallyappropriatevalues?
Yes 3No 0Notclear- N/A 1
Referenceable DoesituseURIsdereferenceableasURLsasidentifiersofclasses,propertiesandinstances?
Yes 3No 0Notclear- N/A 1
Linked IsthestandardavailableasLinkedData?I.e.serializedasRDFandabovealllinkingtoURIsinothervocabularies?
Yes 3No 0Notclear- N/A 1
Mapofstandards– topic1:weatherdata• Specificusecaseofweatherdatausedinfarmmanagementinformationsystems
• Coverage– Total of 65 weather and use-case related data standards
http://vest.agrisemantics.org/by-type-of-data/7623+7550+7626/7623/7550/7626
• Refined classification of weather and use-case data types
Weather / meteorological dataGeospatial data / objectsFarm management data
Weather / meteorological dataWeather observations (live or historical)Weather monitoring infrastructureClimate dataWeather forecasts
Farm management dataeBusiness data (inventory, sales, suppliers...)Crop management data from the farmCrop growth modelsFarm sensors infrastructureObserved field data from the farm
Examplesofdatastandardsrelevantfortopic1
Assessmentandgapanalysis• Weatherdatastandards• Specificusecaseofweatherdatausedinfarmmanagementinformationsystems
Assessmentmetadata+consultationswithexperts
Forgeospatialandweatherdata• BenSchaap(GODANSecretariat)• GiovanniL'Abate(CRAItaly)• SimonCox(CSIROAustralia,RDA)
Forweatherdataforfarmmanagement• SoonhoKim(IFPRI,ICASAstandards)• AndresFerreyra(AgGateway)• HugoBesemer(Wageningen UR)• FrancescoBenincasa(RDAWeatherIG,
BarcelonaSupercomputingCenter)• AllarddeWit(Alterra)• ChristopherBrewster(TNONetherlands)
Weatherdatastandards– Summary• Variety ofdatamodels,dataformatsandvocabulariesthatareusedto
exchangethesedata• Someolderstandardsarestillverymuchused,eitherforlegacy andcompliance
reasons(likeBUFRorGRIB)orbecauseoflong-termpracticeinresearch(NetCDF)
• Standardizationbodieshaveworkedongeospatialandobservationsmodelsandrelatedschemas(ISO/OGC,especiallytheISO19100series),startingtobeusedalsobythemeteorologicalcommunity(CSML, METCE,IWXXMschemas)
• RecentlyAPI-basedweatherdataservicesstartedservingdataondemand andinapplication-friendlyformatslikeJson anduser-friendlyformatslikeCSV
• Workonvariablenamingconventions• FMIS:subsetofweatherdatavariables;agreementwithweatherdata
providers;workonvariablenaming
Gapanalysisonweatherdatastandardsa) Thebiggestchallengeswithweatherdataarerelatedtoissuesofdataavailability,
discoverability,quality,coverageanddocumentation.
b) Datastandardsarenotwelldocumented,differencesbetweenoverlappingstandardsarenotclarified.
c) Bothforweatherdataandforfarmmanagementdata,standardizationofvariablenames acrossthedifferentcommunitiesandevenwithinthesamecommunityisanissue;moreingeneral,therearemanydifferentcodelists usedbydifferentauthorities,withlimitedalignment.
d) Fewdatastandardspublishedasmachine-readableandlinkedvocabularies.
à Intermediariesstillhavetodomostoftheworkconverting,processing,re-purposingthedatabetweenthedifferentstepsinthedatavaluechain.
Keyrecommendationsforbetteruseandusabilityofweatherdatastandards
• Addressdiscoveryissuesrelatingtoweatherdata(improvinguseofdiscoverymetadata tohelpcatalogueanddescribedata)• Improvethedocumentationandself-description ofexistingdatastandards(creatingdeveloperdocumentation;publishingexistingvocabulariesinnewways);offercommunitysupport,Q&Aservices• Identify2-3keycodelists thatshouldbepublishedinmorelinkable,versatileformats;linkkeycodelistsandpublishexistingalignments;providewebservicesforcross-walks.
Nextstepsondatastandards
• Datapublicationonlinehelpdesk• Q&Aservice;Facilitateauthors/expertsonstandardstoregisterandprovidetheirhelp• Facilitatedatapublisherstoregisterandaskforhelpfromtheregistered
authors/expertsonstandards• Sameiterationfornutritiondataandlanddataasforweatherdata
• Usecases>surveyofstandards>gapanalysis>recommendations• Specificationsforstandardinteroperabilityservices
• Basedontherecommendations,forall3typesofdata• Interoperabilityspecificationsthatmakestandards(forall3typesofdata)interoperable
andreusable• Standardinteroperabilityservicesforpilotinterventions
• Basedonrecommendationsandspecificationsofservices• Pilotsforprovidinginteroperabilityservicesforstandardsidentifiedtousecasesfromthe
thematictopics
Usefullinks• GODAN:http://godan.info• GODANActionmapofstandards:http://vest.agrisemantics.org• AgroPortal:http://agroportal.lirmm.fr• TheassessmentprocessusedbytheUKGovernment’sOpenStandardsBoard:Corequestions:https://standards.data.gov.uk/core-assessment-questions
• TheODIOpenDataCertificatescriteria:https://certificates.theodi.org/en/• AgSectorPackage:http://agpack.info/• DCKOSTypesvocabulary:http://wiki.dublincore.org/index.php/NKOS_Vocabularies
• Blogpostongapanalysisandrecommendations:http://www.gfar.net/news/gfar-and-odi-lead-work-gap-analysis-and-recommendations-weather-data-standards
Gapanalysisandrecommendationstobepublishedsoon.
