2019

IMS-GIS Data Centre: Report on secondment period Jan-July 2019

iLaria MarengoJuly 2019

1

Acknowledgements:

Sincere thank to those that made the six month secondment (Jan2019-Juky2019) possible,SAERI and Centre for Remote Environment (Elaine Shemilt and John Gardner), and those thatinvested their time and collaborated in the initial development and exploration of new dataservices, Andrew Brooks, the Satellite Receiving Station staff (Neil Loney, Paul Crawford, and thetechnicians), and the entire OME team (Jason Swedlow, Jean Marie Burel, Simon Li, Will Moore,Dominic Lindner, Peter Walkzysko, Mark Carrol, and June Matthew) which hosted the IMS-GISdata centre manager in their office at Life Science and who dedicated their time to providesupport to the development of OMERO scripts for environmental scientists.

Suggested citation:

Marengo I, 2019, IMS-GIS data centre: report on secondment period Jan-July 2019. Technicalreport, SAERI

For more information, please contact the South Atlantic Environmental Research Institute (SAERI) atinfo@saeri.ac.fk or visit http://south-atlantic-research.orgPO Box 609, Stanley Cottage StanleyFIQQ 1ZZFalkland IslandsTel: +500 27374www.south-atlantic-research.org

SAERI is a registered Charity in England and Wales (#1173105) and is also on the register of approved Charities in the Falkland Islands (C47). SAERI also has a wholly-owned trading subsidiary – SAERI (Falklands) Ltd – registered in the Falkland Islands.

2

1. Introduction

The first secondment concluded with the draft of a road map and strategy document forthe IMS-GIS data centre in which 5 main Work Packages (WP) were identified. Thework in the period between January and July 2019 was focussed on the delivery of theobjectives, within the 5 work packages, whose priority was determined by projects andcollaborations started already in the first secondment, namely the delivery of a dataportal and a webGIS tool for the Government of Montserrat and the collaboration withthe OME team (Univeristy of Dundee, School of Life Science) to make OMERO imagecatalogue a useful tool also for environmental science. The timeline of the secondsecondment, with the main events, is illustrated below.

Overall the effort was put on:

being innovative in the use and promotion of Open Source; broadening and strengthening scientific collaborations that can add further value to

the services and activities carried out by the centre; utilising the latest technology and realising tools/services that are manageable in the

long term, practical and user-friendly.

2. Work package 1

The deadline of end of March for the delivery of a data portal, based on CKAN1, and thepreparation of a webGIS tool for the Government of Montserrat (this work was framed inthe contract with the Joint Nature Conservation Committee signed in November 2018)occupied the majority of the time in the first half of the second secondment. The designand development phase of the data portal took place in the UK, as well as setting themain server which would have hosted the virtual machines with the data portal,postgresql database and the webGIS. The help provided by the Satellite Receiving

1 https://ckan.org/

3

Station (software developer Andrew Brooks) in this delivery was fundamental. Thedecision to have as much as the system and the tools ready before going to Montserratwas taken based on few practical reasons: better access to the internet and IT resourcesand more time for troubleshooting, unavailability of the IT support on the caribbeanisland for the entire period of the visit.

Once in Montserrat, the SAERI-UoD team (figure 1) worked intensively for two weeksand it was joined by the JNCC IT Service Manager, Mr Carl Cilenti, and by the FalklandIslands data manager Jorge Batista Echevarria. The work consisted in bringing theserver from Dundee (where it had been delivered by the JNCC) connecting it to the localnetwork and importing the CKAN data portal that had been developed in Dundee,documenting every step to make it replicable to other countries. Despite the describedwork seems straightforward, there is a simple rule in IT, which is basically “nothing iseasy”. The main impediments in a quick installation of the server and the data portalwhere all related to the lack of local IT support in the first week (due to people on leave)and difficulties in setting permissions and access to some components of the server.Nevertheless, at the end of the two weeks the data portal was released(https://gisdataportal.gov.ms/) and the service proved to be working efficiently andwithout issues.

In parallel, the Montserrat project required as well the development of a webGIS serviceand the design and delivery of the first webGIS project for the island. This is not a newdata service for the IMS-GIS data centre, rather a well oiled tool and its implementationwas much easier than that one of the data portal. The webGIS is hosted in its ownwebsite https://webgis.gov.ms/ and hopefully more webGIS projects will follow to the firstone delivered in March.

Figure 1: The SAERI-UoD team (iLaria Marengo and Andrew Brooks) with Mrs LavernRogers-Ryan in Montserrat.

4

With the achievement of the data portal service, time and attention went on thedevelomment of the second main objective of work package 1, and precisely theadaptation of OMERO image catalogue to environmental data. OMERO is a fullyfunctional UoD tool developed by the OME team(https://www.openmicroscopy.org/teams/ ) at the School of Life Science. With OMERO itis already possible to view, organise, analyse and share the data (images but also othertabular files) from anywhere the users have internet access. Thus, instead of building anew “OMERO” the aim is to add to the current tool functionalities that can be very usefulto and interesting for environmental scientists. As mentioned earlier, the starting point foradding new customised scripts to OMERO was the feedback collected in the Falklandsin January/February. A list of 24 new python scripts were added to the existent OMEROlist and have been shared with the OME community2. All the new scripts are currentlylooking at how to automatise users tasks such as for example adding tags for facilitatingthe search of the images, using controlled vocabulary for tags, storing metadata andmaking retrievable, and sharing images with the wider public.

However, OMERO has the enormous and appealing potential to link software for imageanalysis like FIJI and Orbit and to interact with R, a well-know statistical computinglanguage used by the majority of scientists nowadays. As a result, with OMERO theIMS-GIS data centre wants to provide the users not only with a proper image catalogue(database) but also with a support for running image analyses with the user’s preferredtools and extract information from data. Two are the main examples of image data analyses with OMERO: the imagesegmentation on drone images of peguin and albatross colonies (test images providedby Falklands Conservation, see figure 2) and the calculation of otoliths shapes and theirrelationship with the spatial location of the fish (R script provided by Brendon Lee –Falkland Islands Government – Fisheries Department, see figure 3).

2 https://github.com/saeri-ims/omero-docker-compose

5

Figure 2: Example of image segmentation with Orbit. The top image shows an albatrosscolony where the seabirds have been identified using their white head as main signature(there is a yellow line all around the heads of the albatross which correspond to asegment). The bottom image is a gentoo penguin colony and the object-base analysesidentified the majority of the penguins. These are in yellow.

6

Figure 3: The outline of the otoliths was calculated and imported in OMERO using R.The original script provided by Brendon Lee (FIG-Fisheries scientist) was adapted byDominic Lindner (OME team). Code available at this repositoryhttps://github.com/dominikl/shapeR

3. Work packages 2 and 4

While the preparation of the data portal was carried out in Dundee at the satellitereceiving station, the entire January and mid February were spent in the FalklandIslands trying to deliver some objectives listed in WP2 and WP4, specifically engagingwith the data users located on the islands, where the IMS-GIS data centre has beenbuilding a strong bond since 2013, and promoting new and improved data services bybroadening the network of users.

A session was held in SAERI and Falklands Conservation to demonstrate the potentialof OMERO image catalogue. At the end of the session, colleagues at SAERI were askedto write down feedback and a list of functionalities they considered useful to have. Thelist was taken back to Dundee and used as starting point for the development of pythonscripts (see work package 1 session).

Publicising the activity of the data centre and its capacity in delivering tools that canfacilitate use and access to data is paramount to gain some “popularity” in thecommunity of data users both at local and international scale. In fact, it is important togive practical demonstrations on how the new tools help people in using data andretrieving valuable information. With handy examples it is easier to explain the positiveimpact of data management in daily data related tasks, including data sharing andaccessibility.

With this in mind, a meeting with the Historic Dockyard Museum (Falkland Islandsmuseum) manager was held in early February and set the basis for a work collaboration.The plan was to show through a small pilot project, how the inventory of the heritagesites of the Falklands Islands could be easily hosted in a spatial database and publicisedto the wide public using a webGIS serivce. The pilot project turned out to be succesfuland as a result, the museum has submitted a proposal in which the support from theIMS-GIS data centre is costed and factored in.

In general a larger catchment of users should lead to have more projects and hence,more resources to sustain the data centre should become available. Similarly, theparticipation to the International QGIS users/developers conference was planned withthe intention of raising awareness of the data centre activities amongs the Open Sourcecommunity and create a valuable network with other specialists in the field.

The conference was hosted in A Coruna (Spain) and gave the opportunity to introduceMrs Lavern Rogers-Ryan (see figure 4), GIS manager at the Montserrat Governmet, tothe “GIS Open Source” community. A part networking and presenting the IMS-GIS datacentre tools, the conference time was used to work with Lavern on the Montserratproject, and mainly on the design and development of the webGIS service.

7

Figure 4: Mrs Lavern Rogers-Ryan and iLaria Marengo at the QGIS internationalconference in A Coruna, Spain.

Other objectives of WP4 included the launch of the data portal in the Falkland Islands bymid July. Unfortunately this did not occurred due to misunderstanding with the local ITsupport team. Similarly, neither OMERO nor the data portal were show cased amongacademics of UoD. The road map and strategy document mentioned of a workshop tobe held before the end of the secondment, however, it proved difficult to find a good timeto gather a good number of people for the event. As a result, a pdf with the main IMS-GIS data services and achievements reached with the collaboration with UoD wasprepared and circulated to representatives at the geography department.

4. Work package 3

The majority of the objectives listed under the “knowledge exchange and capacitybuilding” work package are focussed on transferring skills from the IMS-GIS data centrestaff to end users and between IMS-GIS data centre team members. Considering thatusers are the drivers of the data services, it is important to ensure that they gainconfidence and understanding on how to use these services. Otherwise, the likelihoodthat the tools are abandoned in the early stages of their implementation is very high, andas a result, the effort and investment in building data management systems and servicesbecomes worthless.

As mentioned in the work package 2 section of this document, the initial months of thesecondment were spent in the Falklands to demonstrate the new tools, retrievingfeedback from the users but also working with government officers, precisely thebiosecurity and human resources officers, on the design of a spatial database and a

8

webGIS service. The former tool was built with the intent of facilitating the entering andretrieving of data from a spatial database (PostgreSQL) which connects directly to QGISand allows the user to map directly the data and use QGIS also as interface for dataentering. The latter was designed with the aim at informing newly employed governmentofficers on the location and distribution of houses and services in Stanley.

Time in January/February was also spent in setting up on the IMS-GIS data centreserver a series of webGIS project which mirrored those already published from theremote server. The reason for creating a mirror webGIS service is simply explained bythe typology of data users, which are both local and overseas.

As far as documentation and training courses for Montserrat Government officers areconcerned, there was not enough time on the island to run a “multi-day” session on howto use the data portal and the webGIS tool. Unfortunately, a series of circumstances didnot allow to find time in the three months of support included in the contract (from April toJune). A lesson learned for future works is to account for more flexibility in the delivery oftraining courses after the end of the project.

5. Work package 5

The focus in WP5 was on finding the way to sustain the IMS-GIS data centre throughproject collaborations and partnerships after the conclusion of the second secondmentand the contract with the Joint Nature Conservation Committee (see WP1). As pointedout in the other sections of this document, without a doubt the strength of the IMS-GISdata centre is providing data services for the the local and also international communityof data users. Therefore, the utility of the services and the positive impact of them on thesuccess of a project should place the IMS-GIS data centre as “default” participant togrant proposal. With activities and time budgeted in each proposal, the long-termsustainability of the data centre would be at last ensured.

At the end of February 2019, the Project Steering Group meeting took place with thetarget of listing all available funding opportunities and planning how to approach thesponsors. The projects which came as more possible were:

Rutland Water: a long term discussion on developing an app for mobile devices tookplace in the past and never materilised in a form of contract,

Namibia Nature Foundation Ludertitz Centre of Excellence: SAERI had initialcontacts with the NGO for the development of data management system in supportof marine protected area,

Chilean Universities: SAERI run a workshop in Stanley which set the starting point offuture collaboration for the realisation of an Austral Earth Observation Centre ofExcellence. The IMS-GIS data centre potentially would have been included in thepartnership as provider of data service and spatial data infrastructure for datamanagement.

The road map and strategy document mentioned about submitting a bit to a relevantdonor during the period of the second secondment, however none of the potential

9

funding opportunities listed at the Project Steering Group meeting turned into realworking opportunities. Additionally, time for writing and submitting a grant was literallyshort due to the very demanding tasks set in the other work packages.

6. Challenges

The challenges highlighted in the first secondment are all confirmed through the workcarried out in the last six months. A short recap is provided below.

Due to current internet constraints (high costs and narrow bandwidth) local data userswill need to be served by a local system, while overseas data users can be supported bythe an off-site data centre. Even working remotely and accessing data from Dundee tothe local server has proved to be difficult (mainly very slow) and therefore inefficient. Another challenge in developing data services based on cutting-edge technology andopen source programs is the gap in skills, expertise and familiarity encoutered in theterritories where the data users live. Hence, while developing a new data service it isimportant to bear in mind how to transfer it to a country where IT capacity is limited andmainly based on Windows and where IT infastructure are less available (and it is costlyto buy hardware in first instance and maintain it). Training and a financial investment inIT capital asset need to be considered in any data management project which targetsmall and remote islands/territories. Building local capacity take time to be achieved andeven once this goal is reached there is the challenge to sustain the staff and the dataservices in the long term. Currently it has been noted that data, although everyonerecognises their importance, will not be considered priority for small and remoteterritories, hence it is unlikely that a government with a restricted budget is willing totsustain data management by itself. Most likely it will wait and ask for help from externalinvestors and stakeholders.

Another main challenge is related to the complexity of some of the open sourceprograms used by the IMS-GIS data centre. CKAN and in part OMERO are “alive” toolswhich means that they have active community in the background that provide supportand updates to the programs. Additionally both are based on python language, mostly“customised” to both tools. There are documentation on how to install the programs andthere are example of python scripts but in both cases these are accessible by everyone.Again, the ability of mastering IT skills become very much an important aspect of a datamanager and it will not be always available on the islands. Thus, also in this situation it isimportant to consider providing long term assistance to the data users in the territories.

Finally, the University has the opportunity through the library to issue Digital ObjectIdentifiers (DOIs) which are in wide use mainly to identify academic, professional, andgovernment information, such as journal articles, research reports and data sets, andofficial publications though they also have been used to identify other types ofinformation resources, such as commercial videos. However, DOIs are expensive in factthe University (library) paid approximately £3000 in the frst year and then it pays a yearlysubscription of £1500 (ex VAT). With the licence to mint DOIs, there is no a maximumlimit of DOIs which means that the library can mint as many as necessary. However,

10

there are strict guidelines on the metadata available and that accompany the DOIs. UoDlibrary can mint DOIS for a repository hosted in Dundee for free, this is the case of all thedata in OMERO, but they cannot provide this service for an external repository.