EMMC-CSA On-line Facilitation Workshop: Introductions to ......Do you need income from software to...

The European Materials Modelling Council

EMMC-CSA On-line Facilitation

Workshop:

Introductions to Standards in

Software Development for

beginners

Alexandra Simperler, Goldbeck Consulting

18th January 2019


This facilitation workshop invites persons who

would like to write simple to medium complex

software and aims to provide some thought-

provoking impulses to get them started.

The participants are responsible for verifying the

quality of all listed sources and make their own

choice on how to proceed with their software

endeavours.

Disclaimer


• The Innovation Phase

• Scoping

• Documentation

• Ownership and Licensing

• Testing

• Version Control

• Project Management

• The Technical Debt

• Legacy

Outline


Innovation starts out of a situation

I need software/code, because …

– Too much data for a manual analysis

– Want to save time

– There is no software tool available

– Want to improve current situation

– Part of a PhD/postdoc challenge

– To provide people with software that they can use for a

purpose

The Innovation Phase


The Innovation Phase - check your inventory

• What do I know about programming?

• Whom do I know who could help me?

• What networks do I have access to?

• What do I need to keep my curiousity going

and ideas flowing?

• What resources do I have?

• Why is it worth doing?

• How to develop a can-do-attitude?


How to write software, best coding practises, programming

languages

• Supervisor/Mentor/colleagues

• On the job training (IT departments, national computing facilities)

• Collaboration

• Software Carpentry (https://software-carpentry.org/)

• Online course sites

– https://www.lynda.com/

– https://www.udemy.com

– https://www.coursera.org

– https://www.futurelearn.com/using-futurelearn

The Innovation Phase – enlarge your inventory


Note taking is very important:

• Record

• Documentation of the process (when did you do what)

• Reflection tool – get new ideas

• Failure log – what to avoid and why

How to?

Pen and paper, online diary, lab journal; format should suit

your personality

Different to documenting your software!

The Innovation Phase – document it!


• Nurture your curiosity (change of scenery, time

out, space)

• Bounce off ideas with other people

• Persistence

• Take action

• Tolerate failure

• Retrain

• Join a community

• Accept help

The Innovation Phase – keep the good work up!

Always consider your

personality!


• Existing software/code

• Programming Language

• Hardware

Scoping


This is a situation where you write an additional

functionality

• Get an overview of the scope of the software

– If it is set up modular you can contribute without

having to understand the entire software

– If not, your piece of code can affect the whole

software

• Are software libraries used?

– Your application may have to use it.

Scoping – Existing software/code


• Check if programming language is supported for the OS/hardware platforms

• If you add to existing software � use the same language or make sure you can embed your language of choice.

• Fortran (for many scientific applications), Python (very popular, UK is offering python clubs for school children), C++ (often for commercial programming), Java (client-server web applications), JavaScript (interactive web pages), PHP (server-side scripting language), R (statistical computing), Perl (scripting, regular expressions), and more …

• Important: learn and follow standard coding guidelines, whatever you write should be readable by 3rd parties, especially the developer community (coding workshop, dedicated books, forums, …)

Scoping – Programming Language


• Jupyter notebooks, http://jupyter.org/

• Comments in code

• Doxygen, https://www.stack.nl/~dimitri/doxygen/

• Sphinx, http://www.sphinx-doc.org/en/stable/

Scoping – Programming Language

Thou MUST

document

your code!


Hardware (shared memory, HPC, grid computing, graphics cards -GPU, etc.) and software (Windows, Mac OS, Linux, etc.). Your code needs to be fit for a particular operating system and may require compilers (translators between your code and the language of the actual computer it runs on); for some HPC systems professional training on how to deploy code, is highly recommended:

http://www.archer.ac.uk/training/

http://www.prace-ri.eu/training/

http://www.prace-ri.eu/best-practice-guides/

https://molssi.org/education/summer-schools/

http://mpitutorial.com/tutorials/

Scoping – Hardware and Operating System


Technical Documentation• Model underlying the software; for material modelling, see

https://emmc.info/emmc-cwa-on-materials-modelling-terminology-classification-and-metadata-is-published/

• Code architecture, algorithms, accuracy, limitations

• Version/release number/release date

• Information about software developer(s)

• Copyright and license information

• Information about the compiler used to create the respective executables and the corresponding compiler options

• Get credit for your software: https://software.ac.uk/blog/2018-11-26-credit-and-recognition-research-software-current-state-practice-and-outlook

Documentation


User Documentation

• How to install the software?

• How to use the software?

(Manual)

• Physical units for each of the input/output parameters (if

applicable)

• Tutorials (with inputs and outputs and ALL data needed,

which hardware the calculations were run on, software

version and author – if they are != developer)

• Marketing Materials

Documentation


If you are an employee (“work-for-hire”) your employer owns the rights to software you develop, unless differently agreed in a contract

In the US, UK and EU there are no patents for software, but there is a protection via copyright.

Copyright protection is granted from the sole fact that you have created the software, similar to works of literature.

Copyright protects your work as in the computer programme but not the functionality, programming language and data formats (MS Office vs LibreOffice)

To govern the use or redistribution of software you will use software licenses as legal instrument

Ownership and Licensing

IMPORTANT: check your

work contract and the laws of

your country


Licensing

Adapted from: A. Morin, J.

Urban, P. Sliz, A Quick Guide

to Software Licensing for the

Scientist-Programmer, PLoS

Comp. Biol. 8, e1002598

(2012).

https://en.wikipedia.org/wiki/

Software_license

Provides a good overview

on retainment of copyright,

and the right to

perform/display/copy/

modify/distribute/sublicenseIMPORTANT: do license your

software!


Free and open-source software (FOSS)

• Copyleft practice: grants the right to distribute and modify; all rights must be preserved in all copies and modified versions derived from the software (same benefits for all); GNU General Public License (GPL), GNU Lesser General Public License (LGPL), and Mozilla Public License (MPL)

• Permissive: guarantees the maximum of rights to the licensee, minimal requirements regarding distribution; Berkeley Software Distribution (BSD), Massachusetts Institute of Technology (MIT), Apache, and Educational Community License (ECL)

Proprietary software

• Commercial: bespoke licenses, usually right to perform and display

Licensing


Do you need income from software to fund its development? Commercialize your tools or make them accessible to commercial vendors. (permissive preferred)

For some developers licensing can be a philosophical question and they may have strong feelings towards open source

The Institute of Formal and Applied Linguistics at Charles University in Prague provide an open license selection tool: https://ufal.github.io/public-license-selector/

More information about licenses can be found https://choosealicense.com/licenses/

Licensing – How to choose?


https://opensource.org/licenses

https://help.github.com/articles/licensing-a-repository/

Example LAMMPS: https://github.com/lammps/lammps

Licensing – Open source


Might require legal aid, very important find information!

Will require infrastructure , “paperwork”

EULA – End User License Agreements

annual, perpetual, token, paper licenses

generation of licence keys

license servers for floating licenses

License management Solutions, such as FlexLM

https://www.flexera.com/products/software-monetization/flexnet-licensing.html

Academics need to check with department/faculty/university

University College London is running

https://xip.uclb.com/ct/software/ and using a commercial provider: https://www.e-lucid.com/

Licensing – Commercial


• Unit testing: hereby, the functionality of a specific section of

code is tested.

• Integration testing: hereby, interfaces between and

interaction of modules are tested.

• Component testing: the handling of data between

components are tested, usually a range of data and data

types are tested.

• System Testing: an input will be created and edited, a

calculation/operation will be performed and the output will be

checked.

• Operational Acceptance tests/ User Acceptance tests: will

be performed to check if operations can work and if users

would accept the actual software product.

Testing in a commercial level


• … that all parts of the software have been executed and function as planned

• …. that software behaves similar on different platforms

• … that software gives consistent results when there are updates to OS or libraries

• … consistent results between different releases

Unit tests and functional tests are important

Unit tests comprise testing the input and output of each ffunction and module

Functional tests comprise testing input and output of the entire software

Testing the underlying model (validation): Comparison of calculated results with experimental data or data created by other software

Testing on a small level to ensure …


More information can be found here:

• https://www.guru99.com/unit-testing-guide.html, a guide to test driven development

with links to different unit testing software libraries

• https://software.ac.uk/resources/guides/testing-your-software

• https://gitlab.kitware.com/cmake/community/wikis/home

• https://www.atlassian.com/continuous-delivery/different-types-of-software-testing

Testing


Version Control

Keep track of different stages of code development and enable

developing teams

There are version control systems (VCS) available

• Recording of all changes to the code (by whom and when)

• Recovery of a previous state of the software

• Archiving of all states of the software

• Coordination of parallel editing of the same piece of code by several developers.

• Simultaneous work on different branches of the code.

• Merging of different branches of the code


Version Control

More information can be found here:

• https://git-scm.com/docs/gittutorial is a tutorial on how to use git, which is the mostly used version control system. Of interest could be also:

• https://www.software.ac.uk/blog/2016-10-06-top-tips-version-control

• https://molssi.org/education/best-practices/

• http://swcarpentry.github.io/git-novice/

• https://github.com/trein/dev-best-practices/wiki/Git-Commit-Best-Practices

• https://www.atlassian.com/git/tutorials/comparing-workflows


Project Management

Agile methods break projects into smaller/iterative periods

Suitable for the developing cycle of software

Kanban: framework to implement agile methodology,

visual system for managing work and improving workflows

Scrum: iterative and incremental framework for managing

software development

http://agilemanifesto.org

https://www.atlassian.com/agile

https://www.atlassian.com/agile/kanban/kanban-vs-scrum


Project Management - Kanban

https://www.atlassian.com/agile/kanban

For individuals

and academic

groups


Project Management - Scrum

https://commons.wikimedia.org/wiki/File:Scrum_process.svg

ScrumMaster

max

For commercial software owners and solution architects/ software consultants


Project Management – Acceptance Criteria

“Conditions that a software product must satisfy to be accepted by a user, customer or other stakeholder.” (Microsoft Press)“Pre-established standards or requirements a product or project must meet.” (Google)

MoSCoW prioritization technique: SoftwareMust have, Should have, Could have, and Won't have (this time)).

Some further reading can be found here:https://www.softwaretestinghelp.com/what-are-the-quality-attributes/https://hackernoon.com/quality-attributes-in-software-architecture-3844ea482732http://prince2.wiki/Quality


Project Management - Risk

• Should be managed proactively �

optimising success by minimising threats

and maximising opportunities.

• All software projects are inherently risky:

novel, constrained, often based on

assumptions, performed by people and

subject to external influences

• individual risk � effect on project

objectives; overall risk = accumulation of

individual risks � can cause ‘exposure of

stakeholders to the consequences of

variation in outcome’

• Risk Management aims to exploit or enhance positive risks (opportunities)

while avoiding or mitigate negative risks (threats)

http://kanzconsulting.net/


Project Management – Change Management

http://kanzconsulting.net/

• Be aware of all changes that will

impact cost, time, quality and

scope

• Manage expectations in line with

such changes

• Don’t miss dependencies (scope

creep)

• Log imposed or requests for

changes

• analyse impact and obtain suitable

approval (your Governance!) �

create understanding and shared

expectation of the new situation

Expectati

ons

Cost

Quality Time

Scope

“project management's triple/quadruple

constraint”

EXPECTATIONS


Why all these efforts, these “Best Practices”?

“Shipping first-time code is like going into debt.

A little debt speeds development so long as it is

paid back promptly with refactoring. The danger occurs when the debt is not repaid. Every minute spent on code that is not quite right for the programming task of the moment counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unfactored implementation, object-oriented or otherwise.” Ward Cunningham, 1992 OOPLSA conference.

https://www.agilealliance.org/introduction-to-the-technical-debt-concept/

The Technical Debt


Technical Debt can naturally occur � using short cuts to

create a viable but not necessarily a perfect piece of

software that still delivers business value

The issues may be related to coding practices and style,

test coverage, missing documentation, potential bugs, etc.

Solution:

Monitor where technical debt will occur and then be

prepared to gradually pay it back

The Technical Debt


What happens if you do not

want/cannot develop

software anymore?

Your software could turns

into legacy software – old,

still useful, no support, no

development

Your Legacy


• Commercialisation (Company, Trust, …)

• Forming a network of contributors

• Pass it on to an academic successor

• Put it into repository

• Plan the software architecture beyond the first stage from the beginning and envision how future extension can be implemented

• Make your software sustainable

• Also on the small scale: teach your colleagues, pass it on

Your Legacy – The future is bright!

You can use this tool to evaluate the sustainability of your software https://www.software.ac.uk/resources/online-sustainability-evaluation


• Software Sustainability Institute (SSI): https://www.software.ac.uk/resources/top-tips

• Guide to software development and projects at the Netherlands eScience Center:

– https://legacy.gitbook.com/book/nlesc/guide/details

• The Molecular Science Software Institute (MolSSI)– https://molssi.org/education/best-practices/

• a good book on scientific software development in general:– https://www.amazon.com/Introduction-Performance-Computing-Scientists-

Computational/dp/143981192X

• Some Blog Sites and Forums– http://best-practice-software-engineering.blogspot.com/

– https://softwareengineering.stackexchange.com/

Next steps:


This project has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No 723867

Date post:	27-Sep-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

EMMC-CSA On-line Facilitation Workshop: Introductions to ......Do you need income from software to...

Documents