© 2016 Flatiron Health, Inc.
Engineering Ladders as Your Cultural Manifesto
Code Driven, Feb 18th 2016
Gil ShklarskiVP of Technology [email protected]
Maayan RothEngineering [email protected]
© 2016 Flatiron Health, Inc.
who am i
Used to optimize and scale
Now I optimize and scale
© 2016 Flatiron Health, Inc.
Flatiron Health
250+ employees including:14 Medical oncologists and
oncology nurses6 Practice administrators3 Clinical oncology pharmacists
80+ Engineers
Key stats:Founded: 2012Fundraising to date: $313MM
Our Mission: To serve cancer patients and our customers by dramatically improving
treatment and accelerating research
Our client Network: 230 Cancer Clinics2,000 Clinicians 1M Active cancer patients
© 2016 Flatiron Health, Inc.
Flatiron Health
Largest real-world oncology data source
© 2016 Flatiron Health, Inc.
Today
High growth startupsEngineering cultureCareer ladders
- Mechanics- Lessons from Flatiron Health
© 2016 Flatiron Health, Inc.
High growth startups
What your friends
think you do
What your mother
thinks you do
What society
thinks you do
What your clients
think you do
What your friends
think you do
What I think I do
© 2016 Flatiron Health, Inc.
Engineering @ High growth startups
- (a lot of) opportunities for individuals to step up- opportunities to decide how engineers do things- opportunities to decide what we value in
engineering as a team- opportunities to decide what we value about an
engineer’s contribution
Now, realistically, when you grow quickly: ‘s/opportunities to decide/lack of clarity about/g’
© 2016 Flatiron Health, Inc.
Engineering @ High growth startups
t
n_hired_engineers
© 2016 Flatiron Health, Inc.
Engineering @ High growth startups
t
n_hired_engineers
© 2016 Flatiron Health, Inc.
Engineering @ High growth startups
t
n_engineers
© 2016 Flatiron Health, Inc.
Engineering Culture
How engineers do things:
- How we make things- How we operate things- How we function as a
team
“How we engineer”
© 2016 Flatiron Health, Inc.
Formalizing Engineering Culture
Source: First Round Review, “How I Structured Engineering Teams at LinkedIn and AdMob for Success”, Kevin Scott (SVP engineering and operations at LinkedIn)
© 2016 Flatiron Health, Inc.
Formalizing Engineering Culture
Formalizing an engineering culture is important beyond the value it provides to the engineering team.
It helps engineers explain how we are going to help the company win.
Otherwise, - Non leverageable feature work - Unsustainable architectures. - Inevitable team burnout.
© 2016 Flatiron Health, Inc.
Formalizing Engineering Culture
2.5 years and 30 software engineers in...
Time to talk about the careers of our engineers- Changing the world, business path, technological
curiosity is a lot- Not enough, though - need to define what is a
promotion
Career Ladder
© 2016 Flatiron Health, Inc.
A career ladder
- Articulation of a career path- Career path is divided into levels you climb
(‘ladder’)- Specification of a set of desired engineering
behaviors we expect at each level - The same behaviors (‘swim lanes’) are present at
each level, with progression by levels- Separate ladders exist for individual contributors
and people managers
© 2016 Flatiron Health, Inc.
A career ladder
Level 1 2 3 4 5 6
Swim Lane 1
Swim Lane 2
Swim Lane 3
© 2016 Flatiron Health, Inc.
We haven’t invented the concept
Our ladder is inspired by ideas our engineers learned from:
but we customized the contents to our engineering challenges.
Special thanks to Camille Fournier, sharing ‘s ladder in their blog. It is inspired by ‘s ladder...
© 2016 Flatiron Health, Inc.
This sounds so formal, do we need this?
A definition for career growth:- Personal guide- Meaningful discussions / feedback - Clear expectations setting
Clarity → Accountability
Consistency, fairness and clarity in:- Performance standard- Compensation (formulaic compensation FTW)- Titles if exist
© 2016 Flatiron Health, Inc.
Flatiron’s Engineering IC Career Ladder
Specification of a set of desired engineering behaviors we expect at each level
Ladder competencies are split into 3 swimlanes:- TS: Technical Skill, Knowledge, and Expertise- GSD: Impact, Reliability, and Execution- C: Contributions to Eng and Flatiron Culture
This covers an important chunk of ‘how we engineer’.
© 2016 Flatiron Health, Inc.
Ladder sub-categories:TS1: Coding and testing
TS2: Architecture and design
TS3: Production and deployment
Manifesto items covered:- How we make things: SCM, Testing, Code Review, ...- How we operate things: On-call, deployment, access to
production- How we Function as a team: Collaboration, high bus factor
Technical Skill, Knowledge, and Expertise (TS)Technical Skill, Knowledge, and Expertise (TS)
© 2016 Flatiron Health, Inc.
TS / Coding and Testing: Sample Rubrics
E3 E4TS1: Writes correct, well-tested, and clean code with minimal guidance.
Consistently follows best practices within minimal prompting through code reviews. Can design and execute on more complex testing independently, such as integration testing.
TS1: Leaves code and tests in substantially better shape than before. Contributes to overall quality and stability of Flatiron codebase.
Enforces best practices in code reviews. May contribute to creating or updating style guides and establishing best practices.
TS2: [Design and architecture] TS2: [Design and architecture]
TS3: [Production and deployment] TS3: [Production and deployment]
© 2016 Flatiron Health, Inc.
TS/ Design and Architecture: Sample Rubrics
E3 E4TS1: [Coding and testing] TS1: [Coding and testing]
TS2: Makes small design decisions independently, making appropriate tradeoffs between simplicity/readability and performance.
TS2: Code and designs avoid premature optimizations, and are extensible and reusable. Provides detailed technical feedback on the designs of others. Has a comprehensive understanding of a substantial component of the Flatiron technology ecosystem.
TS3: [Production and deployment] TS3: [Production and deployment]
© 2016 Flatiron Health, Inc.
TS / Production and Deployment: Sample Rubrics
E3 E4TS1: [Coding and testing] TS1: [Coding and testing]
TS2: [Design and architecture] TS2: [Design and architecture]
TS3: Trusted to participate in on-call/Tier 3 support rotations and/or deploy code to production with minimal guidance and supervision. Capable of debugging production bugs with help.
TS3: Trusted to participate in on-call/Tier 3 support rotations and/or deploy code to production, or modify production databases, independently.
© 2016 Flatiron Health, Inc.
Ladder sub-categories:GSD1: Execution and delivery, independence
GSD2: Planning and estimation
GSD3: Technical communication
Manifesto items covered:- How we make things: Deliver incremental value, ...- How we operate things: Access to production, ...- How we Function as a team: Documentation, ...
Impact, Reliability, and Execution (GSD)
© 2016 Flatiron Health, Inc.
GSD / Execution and Delivery, Independence: Sample Rubrics
E3 E4GSD1: Delivers on larger features within an existing product or pipeline, which may be vaguely scoped or designed, with minimal guidance. Knows when and who to ask for help in order to get themselves unblocked.
GSD1: Drives, executes, and delivers on work that spans a complete product component or subsystem, from design through successful deployment. Able to unblock themselves independently in most cases.
GSD2: [Planning and estimation] GSD2: [Planning and estimation]
GSD3: [Technical communication] GSD3: [Technical communication]
© 2016 Flatiron Health, Inc.
GSD / Planning and Estimation: Sample Rubrics
E3 E4GSD1: [Execution and delivery, independence]
GSD1: [Execution and delivery, independence]
GSD2: Breaks commits or tickets into appropriately-sized pieces with guidance. Makes and communicates accurate time estimates for individual tickets.
GSD2: Breaks stories into appropriately-sized commits, to facilitate incremental progress and effective code review. Makes and communicates accurate time estimates for own work, potentially spanning multiple sprints. Contributes to roadmap, planning and estimation for their team.
GSD3: [Technical communication] GSD3: [Technical communication]
© 2016 Flatiron Health, Inc.
GSD / Technical Communication: Sample Rubrics
E3 E4GSD1: [Execution and delivery, independence]
GSD1: [Execution and delivery, independence]
GSD2: [Planning and estimation] GSD2: [Planning and estimation]
GSD3: Writes first design documents and solicits and integrates technical feedback from peers. Contributes to documentation and processes that make the team more effective (e.g. on-call playbooks).
GSD3: Consistently communicates technical decisions through design docs, tech talks, and the wiki. Ensures high bus factors through documentation and training of apprentices.
© 2016 Flatiron Health, Inc.
Ladder sub-categories:C1: Cross-functional or domain expertise
C2: Recruiting
C3: Mentoring and onboarding
Manifesto items covered:- How we make things: - How we operate things: - How we Function as a team: A bunch here
Contributions to Eng and Flatiron Culture
© 2016 Flatiron Health, Inc.
C / Cross-Functional or Domain Expertise: Sample Rubrics
E3 E4C1: Understands the clinical or customer facing implication of their features. Effectively accepts guidance and information from team members in other disciplines (e.g. Product, Accounts) or members of other engineering teams.
C1: Able to explain domain or customer-facing implications of their project to other engineers and provide context to new team members. Communicates effectively across functions, is able to work well with Product, Clinical, Accounts, Medical Informatics, etc., as necessary.
C2: [Recruiting] C2: [Recruiting]
C3: [Mentoring and onboarding] C3: [Mentoring and onboarding]
© 2016 Flatiron Health, Inc.
C / Recruiting: Sample Rubrics
E3 E4C1: [Cross-functional or domain expertise]
C1: [Cross-functional or domain expertise]
C2: Contributes to hiring “sell” efforts (e.g. reverse interviews, attending recruiting events or meetups).
C2: Trained and certified as an interviewer (pending enough time in company/industry). Actively participates in software engineering interview process.
C3: [Mentoring and onboarding] C3: [Mentoring and onboarding]
© 2016 Flatiron Health, Inc.
C / Mentoring and Onboarding: Sample Rubrics
E3 E4C1: [Cross-functional or domain expertise]
C1: [Cross-functional or domain expertise]
C2: [Recruiting] C2: [Recruiting]
C3: N/A C3: Successfully mentors a new hire or hosts an intern. Contributes to onboarding through e.g. owning a bootcamp session, creating bootcamp tickets, or improve onboarding documentation. Increases engineering team knowledge through presenting at Engineering or company Knowledge Sharings, or participating in intergrouplets.
© 2016 Flatiron Health, Inc.
Engineering Ladder as a living document
- Deep retro after each performance cycle- We didn’t get it fully right in the first review cycle- Two examples:
- Initial version adopted the term ‘system ownership’ - everyone wanted to own a system (contradicts our bus-factor-value…)
- Initial version focused too much on career-growth by increasing systems breadth. Second version added acknowledgement for going deep, still doesn’t capture complexity and friction of some of our systems. → 3rd generation coming
© 2016 Flatiron Health, Inc.
Engineering Ladder as a living document
- Ladders are a very “big hammer” - An amazing tool for keeping people accountable - Need to be careful not to incentivize the wrong
behaviors- As a performance standard - living document is ok,
but need to converge quickly- Build your ladder not too early, but not too late- Running a Perf process using ladders:
- Ladders proved to be an advocacy tool for managers- Helped provide feedback that is specific and actionable
© 2016 Flatiron Health, Inc.
Key Takeaways
(I hope that)- You understand what is a career ladder- You care more about career ladders and about the
performance process in your team- You see the connection between formalizing
engineering cultures and a career ladders- You care more about documenting your
engineering culture...
© 2016 Flatiron Health, Inc.
Questions?
© 2016 Flatiron Health, Inc.
KEEP CALMAND
FIGHT CANCER
Shameless plug: We are hiring! Drop me a note at [email protected] or visit http://flatiron.com