Git -> Which Git?
After evaluation phase: GitHub was selected for hosting the new C2SMrepositories
C2SMCenter for Climate Systems Modeling
github.com/C2SM-RCM/
is our central GitHub organization
All the code in the cosmo.cscs.ch and hpcforge SVN repositories willbe migrated to GitHubWe have designed a workflow for using GitHub for our C2SM-RCMusers
2 / 44
Central repositories on Github
Central repositories are located at: github.com/C2SM-RCM
Users need to email [email protected] and sign the C2SMlicense agreement to gain access to the repositories
Most of the repositories are private and need to stay that way!
The git repositories will contain:
the full history of the development from the SVN repositories
the master branch and trunk tags only (no development branches!)
the vendor codes, which contain the code directly from the code owner
4 / 44
Mapping the SVN repos to git
The structure of the two SVN repositories has been analyzed andmapped to approximately 30 git repositories
The names of the git repositories are lowercase and correspond to thecode within
The full mapping of the SVN repositories to git repositories can befound at the end of the migration document
Example for the cosmo.cscs.ch repository:
Directory Sub-directories Sub-sub-directories Git repository
cosmo trunk cclm_c2sm cclmcosmo_c2sm cosmo
vendor cclm cclm-vendorcosmo cosmo-vendor
5 / 44
Permissions
There will be three different levels of permissions for accessing theC2SM-RCM repositories:
1 OwnersTwo or three people only ([email protected])Have complete control over the C2SM-RCM organizationCan create or delete repositories, add users, and write to everyrepository
2 AdminsOne or two people for each repository (admin-codename)Have write access to their assigned repositoryAdd new versions of code and incorporate new features and bug fixes
3 UsersEveryone who is not an owner or adminHave read access to all of the central repositoriesMust sign a license agreement before they are granted access
6 / 44
Git Terminology
1 fork : personal copy of a repository on Github
2 branch : an independent line of development
3 master : the default branch
4 commit : a snapshot of your project at a certain time
5 remote : a repository linked to the local repository
6 pull request : request on web interface to merge code into a centrallocation
8 / 44
Start Using github.com/C2SM-RCM
github.com/C2SM-RCM/cosmo
Is the central COSMO repository, everyone in the organization canclone it!!
#git clone [email protected]:C2SM�RCM/cosmo.git
You candevelop localcommit (local action)
However the central github.com:C2SM-RCM
Will not contain branchesMaster does not allow you to push your commits.
So how do I develop my code?
9 / 44
Use Case I: User Develops NewFunctionality
1 Create a fork of the parent repository through github web.
2 Clone the repository into your workspace:
#git clone [email protected]:/cosunae/fieldextra.git
10 / 44
[email protected]:C2SM-
RCM/
fieldextra.git
[email protected]:cosuna/
fieldextra.git
Use Case I: User Develops NewFunctionality
3 Create a branch into your personal fork for development:#git branch �a <feature>
4 Develop:#git commit .
5 push repository into personal fork in order to save work#git push origin <feature>
C2SM-RCM/
cosmo.git
cosunae/cosmo.git
11 / 44
Use Case I: User develops newfunctionality
Your fork is your fork. You are free to choose your working flow.
Some Good Practices:
Develop local and perform (at least) daily push to save your work.Develop in branches.
Your code will be reviewed if you ever want to merge your developments inthe central repository, follow stricter policies:
1 Functionality <==> 1 Branch. Avoid building "monster" brancheslike "PhD_branch".Name of branch should resemble the functionality.Keep a clean history and meaningful commit messages. Avoidcommits like "apply fix".Git convention: first line of the commit message should be a summary.
12 / 44
Use Case II: Update your fork fromupstream repository
C2SM-RCM/
cosmo.git
cosunae/cosmo.git
1 Add the upstream repository as a remote#git remote add upstream https://github.com/C2SM�RCM/cosmo
2 Update the cloned code of the upstream#git fetch upstream
3 merge the upstream branch/commit/tag with your fork:#git merge <branch>
13 / 44
Use Case III: Collaborative Development
We will develop projects in different github organizations
Choose a workflow at the beginning of the project.Do not apply direct merges/push into master, always code reviewusing Pull Requests
15 / 44
Create/Manage Sub-Organizations
Project forks will be hosted in Sub-Organizations,
Your project manages this organization. You can decide
who has admin rightswho has write access (accept pull requests)Do not forget to protect your master
16 / 44
Communication Between Forks
How do we synchronize/communicate the different developments beingcarried out in the multiple forks??
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
17 / 44
Communication Between Forks
Assume some development was done in cosunae/cosmo.git
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
18 / 44
Communication Between Forks
Pull Request into parent fork...
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
P
u
l
l
R
e
q
u
e
s
t
19 / 44
Pull Request
Use for
Merge developments into a different branch or fork/branch.
Make sure the code passes the quality metrics of the repository (tests,documentation, etc.)Create a Pull Request from your branch into C2SM-RCM/<repo>Fix in your branches recommendations from reviewersGroup <repo>_admins group will do the merge when approved.
20 / 44
Communication Between Forks
Other children forks from MCH-APN can merge from upstream repositoriesin order to update their repositories...
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
f
e
t
c
h
&
m
e
r
g
e
21 / 44
Communication Between Forks
Technically possible, but not recommended
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
P
u
l
l
R
e
q
u
e
s
t
22 / 44
Communication Between Forks
Instead... Pull Request to central repository will make the developmentsvisible to whole organization.
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
P
u
l
l
R
e
q
u
e
s
t
23 / 44
Communication Between Forks
After any merge into master at C2SM-RCM, any fork can merge (updateits fork) from it.
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
f
e
t
c
h
&
m
e
r
g
e
f
e
t
c
h
&
m
e
r
g
e
24 / 44
Issues
Publisher Mode
You can use Issues as a tracking system.Easy way to organize your tasks and link them to the code.
25 / 44
Issues
Publisher Mode
But Issues are also very efficient way to communicate with children forks inthe organization.
Use labels for different categories:Report on open/close bugs: status, use cases affected...Report ongoing developments functionalities.Follow developers assigned to an issue allows direct contact for furtherinfo
bug new feature question
26 / 44
Issues
Leave a message in Issues of your parents
Issues and PR get an ID, for example ]158Reference Issue ID in git commits will be reflected in github Issuespagegit commit -m"This is a fix for Issue ]158" .https://help.github.com/articles/closing-issues-via-commit-messages/
27 / 44
Communication Between Forks
Cherry Picking
git cherry-pick <commit>
A B C D E
F G D’
Head
git cherry-pick D
28 / 44
Cherry Picking Other Forks
We can cherry pick from any other fork.A commit is a unique hash, no need to specify the remote (fork)
C2SM-RCM/
cosmo.git
MCH-APN/cosmo.git
cosunae/cosmo.gitminusinf/
cosmo.git
IAC/cosmo.git
kosterreid/cosmo.git
g
i
t
c
h
e
r
r
y
-
p
i
c
k
D
29 / 44
Policies
For central repository admins
Do not create branches in C2SM-RCM/<repo>
For project fork admins
Delete branches that are no longer usedAlways delete the fork once work is finished or merged into parentrepositoryAlways protect the master branch against direct push
For user forks
Do not invite external collaborators to your fork that did not sign theC2SM agreement.
30 / 44
Public Repositories
Public Repositories on C2SM-RCM have certain advantages
public repositories do not increase the costadditional tools offered for free for public repositories
Go Public!!
In C2SM-RCM organization if you can
31 / 44
Migration details
The git migration will take place in December
At the time of the migration, the SVN repositories will be frozen andshould not be used anymore
The central repositories will be created and placed atgithub.com/C2SM-RCM
As soon as the central repositories are ready, users will be notified byemail
The SVN repositories will be kept intact in a read-only state
Users must migrate their own branches
34 / 44
Migrating your personal branches
1 Sign up for an account on Github (github.com). Gain access to theC2SM-RCM organization by emailing [email protected] yourGithub user name.
35 / 44
Migrating your personal branches
2 Fork the desired code into your personal webspace. This is done bynavigating to the desired repository and clicking the Fork button.
36 / 44
Migrating your personal branches
3 Download the migration materials by going togithub.com/C2SM-RCM/migration and clicking the DOWNLOAD ZIPbutton. Save and unpack the zip file where you would like to developyour code.
!! Be sure to download the migration materials instead of cloning them asa git repository. This will ensure that you do not generate nested git
repositories !!37 / 44
About the migration script
4 Run the migration script, following the instructions in the READMEfile and the migration document.
c2sm_svn2git.sh: bash script to migrate user’s branchesScript requires Ruby gem svn2git- installed on all machines at IAC andon KeschAlso an authors file which should be placed in same location as thec2sm_svn2git fileREADME file with detailed instructionsUser must fill in 4 variables at beginning of script
38 / 44
Preparing the migration script
User must define 4 variables in the USER INPUT section of the script:
1 REPO:name of the git repository for which you would like to migrate branches.Example: for the cosmo repository, this line should read:REPO="cosmo"
2 BRANCHES:Name of the branches that should be migratedSame name as the folder in the SVN repository containing the branchFor one branch, use BRANCHES=(feature1), two branches useBRANCHES=(feature1 feature2)Example: To migrate the branch located athttps://cosmo.cscs.ch/cosmo/branch/iac/ksilver/calibrationBRANCHES=(calibration)
39 / 44
Preparing the migration script
User must define 4 variables in the USER INPUT section of the script:
3 USERYour user name for the SVN repositoryExample: USER="ksilver"
4 FORKPath to your fork of the code on GithubIf using https protocol, should include user name in addressExample (https protocol):FORK="https://[email protected]/kosterried/cosmo"Example (ssh protocol): FORK="[email protected]:kosterried/cosmo"
40 / 44
Running the migration script
Shell script should be run on the development computer where youwould like the git repository
Simply run as any other shell script: ./c2sm_svn2git.sh fromcommand line
May take between several minutes up to a couple of hours to run
Provides output file migration.out for troubleshooting
Will generate a git repository locally and push all the commits to yourfork on Github
Will also synchronize your local repository with the fork if there havebeen commits in the central repositories
May prompt you for your SVN password and Github username andpassword. Be careful which is which.
41 / 44
Git resources
For help with:1 git
git help: displays general information about git and man pages forspecific commandshttp://git-scm.com/doc : Comprehensive git reference manualhttp://gitref.org: Quick reference of commonly used git commands
2 Githubhttps://help.github.com: Help pages for Githubhttps://guides.github.com: Step by step guides to Github features
3 C2SM-RCM repositories : For help with the C2SM-RCM repositoriesplease contact the organization owners at [email protected]
4 The migration : For help with the git migration contact either KatieOsterried ([email protected]) or Carlos Osuna([email protected])
43 / 44