ANNUAL REPORT 2015-16
INFORMATION AND COMMUNICATIONS TECHNOLOGY
RESEARCH COMPUTINGEnabling Research Success
2 3
2015-2016ICT RESEARCH COMPUTING - Annual Report
Director’s MessageEnabling research success through professionalism, excellence, and passion.
Our MissionWe advance the University of Saskatchewan’s research mission
by supporting researchers with a team of highly skilled professionals
and a cohesive suite of research-focused IT services.
Our VisionWe will earn broad recognition as an essential partner
in support of the University’s digitally-enabled research.
I’m pleased to welcome you to this inaugural annual report for ICT Research Computing.
Beyond covering 2015/16 activity, the report introduces our team and the services we offer to the research community. Throughout, you’ll find stories of research success and our role in enabling that success.
More than just advanced software and large-scale infrastructure, Research Computing is a group of highly skilled professionals, dedicated to the realization of the university’s research goals. The team has grown this past
year, adding capacity in software development, research data management and support for the health sciences. Many existing team members have risen to the challenges of new or expanded roles.
Reflecting the team’s diversity of expertise, 2015/16 work has ranged from the creation of a new shared model of high performance computing, to tackling complex storage needs, to dramatic improvements in quantum computing simulation.
Of particular note is our leadership in developing a national research data management platform. Collaborating with partner institutions, under the stewardship of Compute Canada and the Canadian Association of Research Libraries, this platform will facilitate the discovery and sharing of data between researchers across Canada and around the world.
On the horizon for the coming year are new services, including electronic laboratory notebooks and secure data management for clinical research. Continued growth of the team will further our capabilities in supporting “big data”, including visualization, image processing, bioinformatics and phenomics.
I feel privileged to work with this dedicated, passionate team, as we continue to support and advance the success of research at the University of Saskatchewan.
Thank you for joining us, Chad CollerDirector, ICT Academic and Research Technology
Adam McKenzie BSc, MSc (Computer Science)
Research Data Management Analyst - Adam supports USask and Compute Canada
researchers on the management of their research data, including its preservation,
movement, and storage, and develops software for the NRDR project.
Sergiy Stepanenko BSc, MSc (Electrical Engineering)
Advanced Computing Coordinator / WestGrid Site Lead - Sergiy coordinates the
effort of Compute Canada funded staff at the U of S, manages services for Compute
Canada, and manages local advanced computing services.
Todd Trann BSc (Electrical Engineering), BSc (Computer Science)
Lead Software Developer NRDR - Todd coordinates software development effort
across Compute Canada and CARL on the National Research Data Repository (NRDR)
project.
Juan Carlos Zuniga-Anaya MSc (E. Engineering), PhD (Computer Science)
Advanced Research Computing Analyst - Juan supports USask and Compute Canada
researchers using HPC infrastructure, including scripting, algorithm design, parallel cod-
ing and porting, and collaborates and co-publishes with University researchers.
Mike Winter BSc, MSc (Computer Science)
Senior Software Developer NRDR - Mike develops software for the NRDR project,
including in-depth analysis and optimization of preservation pipeline software.
Keith Jeffrey MSc (Electrical Engineering)
Project Manager / Senior Analyst NRDR - Keith manages the NRDR project, liaises
with local and remote researchers and research centers, and provides advice on IT
policy and governance.
4 5
2015-2016ICT RESEARCH COMPUTING - Annual Report
Who We AreEnabling research success by providing services and technologies backed by dedicated experts.
Jason Hlady BSc, MSc (Physical Chemistry)
Manager - Jason has worked in research computing for more than 15 years at the U of S
and WestGrid/Compute Canada. In addition to managing the University’s Research Com-
puting group, Jason is the RDM lead on Compute Canada’s Science Leadership Council.
Song Hu BSc, MSc (Electrical Engineering), PEng
Analyst - Song supports researchers using Linux across campus, and provides research
support in targeted areas, including Geological Sciences and Engineering.
John Costa BSc (Physics), MSc (Biomedical Engineering)
Senior Analyst - John consults with senior leadership in Health and Medicine, analyzes
and develops solutions related to health research, including in the areas of Personal Health
Information and HIPA compliance, and provides input on IT policy and governance.
Research Computing staff accelerate the University’s research goals through a direct support model. The
group is highly capable and well educated—there are 2 PhDs and 7 MScs in the group presently. They assist
researchers in securing research funding through consultation and strengthening of proposals. They train the
next generation of HQP, provide expertise, maintain services and infrastructure, assist researchers in the use of
services and infrastructure, and create new services.
Ian MacPhedran BSc, MSc, PhD (Structural Engineering), PEng
Senior Analyst - Ian provides guidance and support to researchers across the breadth of
IT services, including research software, operating specialized research-enabling services
(e.g. GIS, NVIVO, DataShare), and grant application consultation.
Parallel Computing RDM ModelingImage Processing 3D PrintingGPU
Signal ProcessingBig Data SimulationPhysicsAssembly Language
OptimizationCompilers Health CanadaSecurity
Logic
Kernels
Electronics
Engineering
CAD
Numerical Linear Algebra Artificial Intelligence
GISFEA Linux Data Mining Visualization DatabasesSoftware Engineering
Algorithms Bioinformatics ProgrammingEngineering3DScientific Computing
Helping you do more, faster, better.
AccomplishmentsEnabling research success by contributing facilities and expertise to over 25 journal publications and 5 conference presentations.
1 40
42
25
7
ExcellenceIn 2015, Jason Hlady received
the highly coveted Compute
Canada Award of Excellence for
his dedication and commitment to
Compute Canada’s mission.
RoboticsSong Hu used his robotics expertise
to good effect by fixing Grizzly, a
research robot in Dr. Reza Fotouhi’s
lab. After disassembly, diagnosis,
and reassembly, this mechanical
four-wheeled Grizzly is back from
hibernation.
PublicationsResearch Computing has supported
or contributed to over 25 research
publications and 5 conference
papers in 2015, up from 19 in 2014.
Health SecurityJohn Costa and ICT Security
completed a Personal Health
Information environmental scan
for administrative units to assess
the University’s risks. Results will
inform new policy. Seven campus
units were surveyed.
SupercomputingJuan Zuniga and Sergiy Stepanenko
enabled John Tse (Physics) to
use one of the world’s biggest
supercomputers. Hosted in China,
Tianhe-1 was once the second
fastest supercomputer in the world
(2010-11).
TrainingAn advanced computing training
course (Introduction to Scientific
Computing) delivered by Juan
Zuniga was sold out — twice. The 40
attendees included faculty, students
and staff.
7
2015-2016ICT RESEARCH COMPUTING - Annual Report
6
ProfessionalismExcellencePassion
“...ICT support of my computer clusters has been invaluable. With-out their expert experience, it would not been able to keep the clusters working without major interruption. This is essential to my research.”
- Dr. John S. Tse, Professor, Department of Physics
“ Jason’s expertise and devotion were essential in being able to launch Globus so successfully as a national service in such a timely manner. ”- Lindsay Sill, Executive Director of WestGrid at Compute Canada
“ It's invaluable to have this expertise available to faculty within the [College of Medicine] because it not only expedites our productivity and accelerates our discovery rate, but ensures that our data (a.k.a, our blood, sweat and tears) remain secure from loss via hardware errors or external attacks.” - Julia Boughner, Professor, Department of Anatomy and Cell Biology
8 9
2015-2016ICT RESEARCH COMPUTING - Annual Report
What Can We Do For You?Enabling research success by delivering research infrastructure, software and consulting services.
Research Data ManagementManaging data is vital to every research effort. A data management plan
(DMP)— a plan for curation, access, discoverability, and preservation — ensures
your most expensive asset is fully protected and compliant with TriCouncil
funding policy.
Visualization & Image ProcessingApplying techniques to see data in novel ways using sophisticated visualization software
and hardware. From 2D and 3D digital models to 3D printed models, our specialists
can help transform your data.
Data Collection ToolsREDCap is a sophisticated, secure web application for building and managing
online surveys and databases, quickly and securely. Specifically tailored to the
needs of clinical research studies, REDCap is Research Ethics approved, and
ready to use.
We help you do more, faster, better.
Advanced Research Computing (ARC)ICT provides infrastructure and support staff to solve problems that are too big to
tackle on your desktop computer. The High Performance Computing Research Facility
(HPCRF) is a modern data centre that hosts supercomputer clusters for researchers,
as well as ICT’s advanced research computing systems. The WestGrid / Compute
Canada staff in Research Computing can help you access Canadian ARC resources,
design computer systems for research projects, and use the University’s ARC systems
effectively.
Consulting and AdvisingResearch Computing’s breadth of expertise across many domains makes light work
of heavy problems. Our experts help researchers design research solutions and
sharpen their grant applications. From recommending computational approaches
to directly collaborating with faculty on projects and tough problems, we enable
research success by providing expertise, saving you time—time that you can spend
on research, rather than technology.
Custom SoftwareResearch relies heavily on software to solve difficult problems. While many software
programs exist, other problems remain unserviced by commercial products. Research
Computing provides guidance and analysis for the development of custom software
programs.
10 11
2015-2016ICT RESEARCH COMPUTING - Annual Report
Dr. Artur Sowa
Artur Sowa and Juan Zuniga collaborated on
simulation of quantum computers. In 2015, Juan
designed and implemented novel algorithms that
improved the performance of a quantum annealing
simulation to world record levels. The initial simulation
algorithm could model an 8-qubit quantum computer
in a server’s main memory. Over the course of their
collaboration, Juan’s algorithm design improvements
made it so Artur could simulate a 38-qubit quantum
computer in about the same amount of space. Each
qubit represents a doubling of the compute space,
and so Juan’s algorithmic improvements represent
a billion-fold (!) performance improvement. The
original algorithm would have required a Zettabyte
of memory in one computer — one billion Terabytes,
about equal in size to ALL the traffic over the global
Internet in 2015. With Juan’s improvements, the
simulation was completed on a single computer at
the University this year!
A Billion Times Bigger
The cost of medical imaging depends crucially
on the trade-off between the signal acquisition
and processing time versus final image quality.
Any improvement in either the hard-wired signal
acquisition apparatus or the digital signal processing
algorithms translates directly into improved
diagnostics and reduced healthcare cost, e.g. in
recent years, significant progress was achieved via
an application of innovative compressed-sensing
techniques. Opportunities for further progress
are now emerging from new insights into signal
processing algorithms in conjunction with new
concepts for control of signal acquisition in a quantum
environments such as an MRI machine. Images to the
right contrast the effects of quantum denoising.
Quantum Image Processing
Consulting ComputingAwards and Grants
“ ...with Juan Carlos and others we have produced world-class results, challenging those of bigger, better funded teams. ”
- Artur Sowa, Professor, Department of Mathematics and Statistics
12 13
2015-2016ICT RESEARCH COMPUTING - Annual Report
Research Computing deploys some of the
most powerful scientific software programs
in the world in for the research community.
The most popular titles include ArcGIS
(mapping and spatial analysis), MATLAB
(numerical analysis and simulation) and
Mathematica (support for symbolic algebra).
Over the course of the year, hundreds of
students and faculty use these tools tens
of thousands of times.
With ICT, the research community leverages
these powerful scientific tools to advance
their research.
The assortment of software tools and
technologies empower and enable
researchers to achieve results that, with
only a few hours, would rival the life-time
productivity of researchers only a decade
ago.
Research SoftwareEnabling research success by providing specialized research software programs and the expertise to support them.
44 instruments
187 studies
21 users
58,417 uses
37,142 uses
8,968 uses
7,375 uses
5,920 uses
Do more with research software. ICT hosts software services and manages the deployment and licensing of essential research software packages.
REDCapArcGIS
MatLabMathematica
OriginMaple
NVIVO is an analytical tool designed to support qualitative research. Available
to researchers at no charge, ICT works in partnership with the Social Sciences
Research Laboratories (SSRL) to support NVIVO. Together, we provide services to
help researchers. ICT Research Computing supports NVIVO server for collaborative
projects — researchers work on the same data through a shared NVIVIO database.
Scientific Software
ICT recently added ORIGIN, a leading package for producing publication-ready
graphics, to our library of scientific tools for researchers. Origin offers plotting
features unavailable elsewhere, including digitization of pictures of graphs—
converting scanned images and extracting the data. Origin was used over 1500
times in May alone. Origin and OriginPro are available to the research community
at the U of S and CLS.
Software: By The Numbers
14 15
2015-2016ICT RESEARCH COMPUTING - Annual Report
Global Institute for Water Security
The Global Institute for Water Security (GIWS) has big
computing needs. Modeling water usage, weather
patterns, and climate takes serious horsepower. In
2015, ICT Research Computing and GIWS partnered
to develop a shared model of high performance
computing (HPC) that enables the GIWS to leverage
more computing power than they could purchase on
their own. The traditional way to use these servers
would be to purchase a rack to hold the computers,
storage, private network switches and cabling, and
get a system administrator to build and maintain a
standalone HPC cluster for the GIWS. A standalone
cluster wouldn’t allow GIWS researchers to use more
computers than they purchased, nor to share the
systems when they weren’t being used.
Instead, the GIWS bought 20 servers to add to
the Plato cluster, and used software to guarantee
GIWS researchers access to the same amount of
computational resources as a standalone system
would have provided. The whole is greater than the
sum of the parts. GIWS 1, Research Computing 1.
The Global Institute for Water Security
(GIWS) and its partners foster collaboration
to address regional and global challenges
of Water Security, including the sustainable
use and protection of water resources and
protection against water-related hazards
such as flooding and drought.
Key research themes include Climate
Change and Water Security. The Institute
works to ensure that society globally
has the understanding and the tools
to sustainably manage and protect the
world’s water resources, and that Canada
has the research and expertise needed
to understand and manage our water
systems in an era of rapid societal and
environmental change.
Ground Water Modeling
Win-Win Shared Computing
StorageConsultation
“ …ICT has been a great support for our researchers to help set up and run their analyses. ”
- Saman Razavi, Professor, SENS and GIWS
Computing
National Research Data Repository
16 17
2015-2016ICT RESEARCH COMPUTING - Annual Report
Enabling research success by providing convenient access to secure, backed up storage for research data.
Storage Services
Research data is the most important asset of the research enterprise. Keeping it safe, yet convenient to access and share, is of paramount importance to ICT. The Cabinet service provides fully redundant and backed up storage right to the desktop. Designed for day-to-day access to frequently changing data, it is perfect for business and academic documents, and small research data that needs to be at your fingertips. For convenient and secure sharing of files with colleagues, Cabinet is integrated with OwnCloud, a “DropBox-like” service that runs at the University.For larger research data, ICT provides DataShare, a shared network disk that provides space for your research group.
DataShare permits large files to be easily shared among many users on campus, straight to the desktop. Over 100 groups use DataShare—with their data securely stored and backed up, researchers need not worry about disk failures, loss or theft. For the biggest of datasets, ranging from tens to thousands of terabytes, Research Computing has the experts who can help researchers access Compute Canada’s national storage platform, from getting resource allocations to moving big data across the country. Whether your data is measured in megabytes or in petabytes, Research Computing can help you keep your data safe, and share it with your collaborators.
Data of all shapes and sizes
Enabling research success by building a framework for preserving and discovering Canadian research data.
A team at the University of Saskatchewan in Research Computing is at the centre of a project to develop the technology for a national research data repository. With funding from both CFI MSI and the Province of Saskatchewan, Compute Canada and the Canadian Association of Research Libraries are collaborating with the U of S team on a two year project to build a scalable national platform for research data management and discovery. Tri-Council funding agencies state that “Research data resulting from agency funding should normally be preserved in a publicly accessible, secure and curated repository or other platform for discovery and reuse by others.” This includes expectations for researchers
to create data management plans, and to provide the necessary metadata to facilitate understanding and reuse. The pan-Canadian platform that we are building will provide tools and services to support the curation, access, discoverability, and preservation of research data, allowing researchers across Canada in a range of disciplines to have access to publicly funded data, addressing a longstanding gap in Canada’s digital research infrastructure. This service will also provide a framework that allows existing and future data repositories to be federated within a coherent system.
Pan-Canadian Platform
1716
18 19
2015-2016ICT RESEARCH COMPUTING - Annual Report
Dr. Julia Boughner
Big problems require big solutions. Dr. Boughner’s
custom built Optical Projection Tomography (OPT)
machine is a purpose built system designed to capture
3D, high fidelity CT (computed tomography) images of
early stage embryonic development of various small
mammals.
The question for the Boughner lab is, how do you store
large amounts of data securely and reliably? The answer
to that question came from Research Computing.
Expecting to generate more than 10 TB of data over
the lifetime of the project, storage and the supports
were key.
Adding 15 TB of space to DataShare did the trick.
Now raw imaging data is seamlessly accessible from
the acquisition computer to the rendering computer,
without the need to manually transferring the data,
without the worry of data loss and with the comfort of
knowing support is just a phone call away.
My NSERC- and CFI-funded research
team is imaging, in 3D, the expression
patterns of genes that drive development
in the mouse face, with a focus on jaws
and teeth. Also using 3D image data
(micro-CT and OPT), we are deciphering
the developmental-genetics processes
that coordinate change among teeth as
well the bones and muscles of the head.
We use this new knowledge to clarify
evolutionary changes, and disease
etiologies, in primate including human
heads, jaws and dentitions.
Evolutionary developmental biology / anthropology
StorageConsulting
Big Storage For Big Data “ Having secure, efficient hardware and storage is a keystone of my research team’s productivity over the short- and long-term.”
“ I appreciated their consultation to identify my needs, as well as their advice about what would best advance my research approach. ”
- Julia Boughner, Professor, Department of Anatomy and Cell Biology
TFLOPS GB RAM
CORESNODES
USERS
Plato is an HPC cluster designed to support research projects, serving the entire University of Saskatchewan. Zeno
is a computing cluster with fast networking and specialized graphics processors to accelerate scientific computing.
Meton is a single computer with considerable CPU power and even more memory (RAM). These systems are
great for getting research done, and are the perfect place to develop projects before scaling up to more powerful
systems in Compute Canada. HPC Research Computing experts on campus are available to discuss your needs.
Compute Canada leads the acceleration of research and innovation by deploying state-of-the-art advanced research computing (ARC) systems, storage and software solutions. Research Computing can connect you to Compute Canada systems and experts.
20 21
2015-2016ICT RESEARCH COMPUTING - Annual Report
Specifications
PLATO
Advanced Research Computing: By The NumbersEnabling research success by providing Plato, Zeno, and Meton through the High Performance Computing Research Facility.
18563712
214116
77
96,101 jobs
45 groups
214 researchers
1103 yearsof computing in one year
10.6 TFLOPS
8 Nodes
92 Cores
256 GB RAM
ZENO
Specifications
250 kWHPCRF delivers 250 kW of electricity. Enough to
power 9 homes.
Power
855xPLATO is 855 times as powerful as a modern
business computer.
Performance
10HPCRF hosts 10 clusters from various groups on
campus.
Hosting
METON1.6 TFLOPS
48 Cores
2048 GB RAM
1 Gb Network
Specifications