04 September 2007 Kaiser: COMS W4156 Fall 2007 1

COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

Kaiser+4156@cs.columbia.edu

http://york.cs.columbia.edu/classes/cs4156/

04 September 2007 Kaiser: COMS W4156 Fall 2007 2

What is Software Engineering?

• NOT just programming

• NOT just programming [part of] a large software system

• NOT just programming as a member of a large team

04 September 2007 Kaiser: COMS W4156 Fall 2007 3

What is Software Engineering?“Software engineering is the application of a systematic,

disciplined, quantifiable approach to the development, operation, and maintenance of software.[1] The term software engineering was popularized during the 1968 NATO Software Engineering Conference (held in Garmisch, Germany) by its chairman F.L. Bauer, and has been in widespread use since. The discipline of software engineering encompasses knowledge, tools, and methods for defining software requirements, and performing software design, software construction, software testing, and software maintenance tasks.[2] Software engineering also draws on knowledge from fields such as computer engineering, computer science, management, mathematics, project management, quality management, software ergonomics, and systems engineering.[2]“ (Wikipedia)

04 September 2007 Kaiser: COMS W4156 Fall 2007 4

What do Software Engineers do?

• Anyone can call themselves a software engineer, but not all programmers are software engineers (and not all software engineers are programmers)

• Newly hired software engineers typically assigned to test or maintain someone else’s pre-existing code

• Maintenance = Everything after initial deployment– When was the last time you used a commercial or

open source program labeled “Version 1.0”?– 50-90+% of total lifecycle costs

04 September 2007 Kaiser: COMS W4156 Fall 2007 5

What do Software Engineers do?

• Categories of maintenance– Corrective - reactive modification to correct

discovered problems– Adaptive - modification to keep it usable in a changed

or changing environment– Perfective - improve performance or maintainability– Preventive - modification to detect and correct latent

faults Note: 4156 will focus primarily on developing

and testing your own very small new software system, for practical reasons

04 September 2007 Kaiser: COMS W4156 Fall 2007 6

Why Study Software Engineering?• Writing a program is easy

– Program ~= code (possibly with comments)• Developing a software system is harder

– System ~= program plus technical documentation sufficient such that someone other than original developers can maintain, typically involving environmental interoperation (beyond just UI and file system)

• Developing a software product is very hard– Product ~= system plus customers, fulfilling the

business needs of those customers, with customer-oriented documentation and support

04 September 2007 Kaiser: COMS W4156 Fall 2007 7

Why Study Software Engineering?• Software Engineering aims at supporting the

development of high-quality software products– High-quality software products are more robust,

efficient and effective– High-quality software products are easier to use,

understand, modify, and compose with other high-quality software products

Note: 4156 will focus primarily on developing and testing your very small software system, with no customers, for practical reasons

04 September 2007 Kaiser: COMS W4156 Fall 2007 8

Why Study Software Engineering?• To attain high-quality software, software engineers must

be skilled professionals who follow defined processes and “best practices”

• Some schools offer entire degree programs in software engineering, not just a few CS courses (e.g., CMU, U. Texas at Austin)

• At present there is no licensing of software engineers in the US, only specialty “certifications” typically offered by vendors (e.g., IBM, Microsoft, Sun, Red Hat) but also by vendor-neutral organizations (e.g., IEEE, Linux Professional Institute)

• Licensing of software engineers is very controversial, e.g., what would the Professional Engineer exam test?

• Guide to the Software Engineering Body of Knowledge (SWEBOK)

04 September 2007 Kaiser: COMS W4156 Fall 2007 9

Why is this course called Advanced Software Engineering?

• Until ~10 years ago, undergraduate CS majors were required to take COMS W3156 Introduction to Software Engineering, oriented towards sophomores and juniors, whereas 4156 was an equivalent elective oriented towards seniors and graduate students

• Both courses presented a traditional view of the software lifecycle focused on detailed documentation, independent of any particular technology, and stepped through the full lifecycle (through delivery to “customers”)

• 4156 now concentrates on component model framework technology and emphasizes the software testing subset of a shortcut lifecycle

• This course does not teach programming, and covers only a tiny smattering of analysis and design

04 September 2007 Kaiser: COMS W4156 Fall 2007 10

Course Outline

1. Component-based software development– Component model frameworks (COM+, .NET, EJB,

CCM, Web Services)

2. Quality assurance– Code inspection, unit and integration testing, stress

testing, test planning

3. Other software engineering topics as time permits

– Pair programming, software process, requirements, UML, software architecture, design patterns, refactoring, …

04 September 2007 Kaiser: COMS W4156 Fall 2007 11

Who am I?

• Gail Kaiser• kaiser+4156@cs.columbia.edu • Professor at Columbia for >20 years (!)• BS from MIT, MS and PhD from CMU• Worked summers and/or consulted for startups, defense

contractors, industry research labs, Software Engineering Institute, Educational Testing Service

• Research in self-managing systems ("autonomic computing"), software testing, publish/subscribe event systems, computer and network security, applied AI, Web technologies, collaborative work, information management, distributed systems, and software development environments and tools

04 September 2007 Kaiser: COMS W4156 Fall 2007 12

Who are the TAs?

• Mr. Aaron Fernandes, apf2114@columbia.edu

• Mr. Jayesh Kataria, jvk2104@columbia.edu

• Both 3rd/last semester MS students

04 September 2007 Kaiser: COMS W4156 Fall 2007 13

Who are you?

• Graduate and upper level undergraduate students in Computer Science, Computer Engineering, Biomedical Informatics, Electrical Engineering, maybe others

• Sincerely interested (hopefully) in learning to apply Software Engineering methodologies and technologies – not just programming skills - to building and maintaining useful and usable software products

• 4156 is required for the CS MS Software Systems track (unless you “import” a prior upper-level software engineering course from another institution)

04 September 2007 Kaiser: COMS W4156 Fall 2007 14

Textbooks

• Ron Patton, Software Testing, 2nd edition, Sams Publishing, ISBN 0672327988 (required)

• Clemens Szyperski, Component Software: Beyond Object Oriented Programming, 2nd edition, Addison-Wesley, ISBN 0201745720 (recommended)

• Ordered for this class through the Columbia University Bookstore

04 September 2007 Kaiser: COMS W4156 Fall 2007 15

Final Grade Breakdown

• 50% Team Project– 2 iterations at 25% each

• 35% Individual Assessments [exams]– 15% midterm– 20% final

• 15% Individual Development Assignments – Three small homeworks to develop and demonstrate

proficiency on component-based technologies– To acquire skills to be used in the team project

04 September 2007 Kaiser: COMS W4156 Fall 2007 16

Team Project

• Students work primarily in pairs (2 people)• Grouped into 2-pair teams (4 people)• Each team will invent, design, code and test a

very small software system using some component model framework

• 2 rapid iterations of project - each 3-4 weeks long• All major system functionality should be (more or

less) operational by the end of the 1st iteration• 2nd iteration will focus on quality assurance issues

04 September 2007 Kaiser: COMS W4156 Fall 2007 17

Pairs and Teams

• Pairs ideally self-chosen – find a partner (if odd enrollment, one triplet permitted but no onesies)

• Meet someone in class or advertise on the 4156 discussion forum on CourseWorks (http://courseworks.columbia.edu)

• 2 pairs may request each other for a team• Pairs within same team may be permitted to “switch”

(discuss with instructor in advance!)• Unattached individuals and pairs will be assigned by the

teaching staff• Each team will submit a proposal for the very small

system they want to build

04 September 2007 Kaiser: COMS W4156 Fall 2007 18

Do the Team Project AS A TEAM

1. Develop and test your project code together with your programming pair partner

2. Integrate together with your team’s other pair3. Co-author project documentation4. Demonstrate your project to the teaching staff

All members of the same team must use the same programming language (e.g., Java, C++, C#), the same IDE (e.g., Netbeans, Eclipse, VisualStudio), the same platform (e.g., Windows, Mac, Linux, Solaris), and the same component model framework (e.g., EJB, COM+, CCM)

04 September 2007 Kaiser: COMS W4156 Fall 2007 19

Frequently Asked Questions• Do I have to join a pair/team?

YES!• Can I do the project alone?

NO!• I’m a CVN student, I work full time, I have a heavy class schedule, I

have a long commute, I don’t like working in groups, I can only program in an obscure scripting language that no one else knows, etc. - Can I do the project alone?

NO!

04 September 2007 Kaiser: COMS W4156 Fall 2007 20

Individual Assessments

• Midterm and Final • Take-home examinations that must be

completed by each student working individually• Demonstrate your understanding of software

engineering principles and technologies, and how they apply to your team project (graded)

• Evaluate your own, your partner’s and your team’s work on the project thus far (not graded, but contributes to project grade differentials)

• No other exams

04 September 2007 Kaiser: COMS W4156 Fall 2007 21

Individual Development Assignments

• Small design and programming homeworks early in semester

• Must be done individually (not with pair or team, nor anyone else)

• Goal is to quickly come up to speed on components and component model framework technology

• Graded high-pass/pass/low-pass/fail (meaning fail the course – or drop the course – not just lose 5% of the grade!)

04 September 2007 Kaiser: COMS W4156 Fall 2007 22

First Assignments Due Next Week!

• Tuesday 11 September, 10am

• Assignments posted on course website

• Submit via CourseWorks

• Individual Development Assignment #1

• Pair Formation

04 September 2007 Kaiser: COMS W4156 Fall 2007 23

Course Outline

1. Component-based software development– Component model frameworks (COM+, .NET, EJB,

CCM, Web Services)

2. Quality assurance– Code inspection, unit and integration testing, stress

testing, test planning

3. Other software engineering topics as time permits

– Pair programming, software process, requirements, UML, software architecture, design patterns, refactoring, …

04 September 2007 Kaiser: COMS W4156 Fall 2007 24

What Is Pair Programming?

• Two software engineers work side-by-side at one computer, sliding keyboard and mouse back and forth (or simulated via desktop sharing technologies)

• Continuously collaborate on same design, algorithm, code, test, etc.

• Continuous informal review

04 September 2007 Kaiser: COMS W4156 Fall 2007 25

What Is NOT Pair Programming?

• Splitting up the work

• Taking turns doing the work

• One person doing all the workNote: 4156 requires pairs, but pair

programming is only encouraged, not required

04 September 2007 Kaiser: COMS W4156 Fall 2007 26

What About Productivity?• Very controversial – common practice in some

software development organizations and unthinkable in others

• Demonstrated in some studies to improve productivity and quality of software products under some circumstances– Particularly novice-novice pairs (vs. novice solos) or

expert-expert pairs (vs. expert solos)– Two programmers work together nearly twice as fast and

think of more solutions to a problem as two working alone– While attaining higher defect prevention and defect

removal

04 September 2007 Kaiser: COMS W4156 Fall 2007 27

Benefits

• Increased discipline. Pairing partners are more likely to "do the right thing" and are less likely to take long breaks.

• Better code. Pairing partners are less likely to produce a bad design due to their immersion, and tend to come up with higher quality designs.

• Resilient flow. Pairing leads to a different kind of flow than programming alone. Pairing flow happens more quickly: one programmer asks the other, "What were we working on?"

04 September 2007 Kaiser: COMS W4156 Fall 2007 28

Benefits

• Multiple developers contributing to design. If pairs are rotated frequently, several people will be involved in developing a particular feature. This can help create better solutions, particularly when a pair gets stuck on a particularly tricky problem

• Improved morale. Pair programming can be more enjoyable for some engineers than programming alone.

• Collective code ownership. When everyone on a project is pair programming, and pairs rotate frequently, everybody gains a working knowledge of the entire codebase.

04 September 2007 Kaiser: COMS W4156 Fall 2007 29

Benefits

• Mentoring. Everyone, even junior programmers, possess knowledge that others don't. Pair programming is a painless way of spreading that knowledge.

• Team cohesion. People get to know each other more quickly when pair programming. Pair programming may encourage team gelling.

• Fewer interruptions. People are more reluctant to interrupt a pair than they are to interrupt someone working alone.

04 September 2007 Kaiser: COMS W4156 Fall 2007 30

Drawbacks

• Experienced developers may find it tedious to tutor a less experienced developer in a paired environment.

• A less experienced developer may feel intimidated pairing with a more experienced developer, which may result in less participation.

• Many engineers prefer to work alone, and may find the paired environment cumbersome.

• Experienced engineers quite likely produce code that is very accurate, and the additional theoretical gain from pairing is not worth the cost of an additional engineer.

04 September 2007 Kaiser: COMS W4156 Fall 2007 31

Drawbacks

• In the case where the team has slightly different work schedules, which is common in an environment that values work-life balance, the pair is only available during the overlap of their schedules. Therefore, not only does it require more man-hours to complete a task, a typical day has fewer pair-hours available, which further increases the overall task completion time.

• Where a company values telecommuting (working from home) or when an employee must work from outside the office for whatever reasons, pair programming can be difficult.

04 September 2007 Kaiser: COMS W4156 Fall 2007 32

Drawbacks

• Differences in coding style may result in conflict.

• Personality conflicts can result in one or both developers feeling awkward or uncomfortable.

• Some developers can sit in front of a computer for many hours straight, others like to get up and walk around often.

04 September 2007 Kaiser: COMS W4156 Fall 2007 33

All I Really Need to Know about Pair Programming I Learned in

Kindergarten. Laurie Williams and Robert R. Kessler.

Communications of the ACM, 43(5):108-114, May 2000.

http://collaboration.csc.ncsu.edu/laurie/Papers/Kindergarten.PDF

04 September 2007 Kaiser: COMS W4156 Fall 2007 34

All I Really Need to Know I Learned in Kindergarten : Uncommon Thoughts on Common Things

Robert Fulghum, 1988

04 September 2007 Kaiser: COMS W4156 Fall 2007 35

Share everything

• Two programmers are assigned to jointly produce one artifact

• One person typing or writing, the other continuously reviewing

• Both equal participants

• Both partners own everything

04 September 2007 Kaiser: COMS W4156 Fall 2007 36

Play fair

• One person “drives” while the other continuously reviews

• Important to take turns “driving”, even if one programmer much more experienced than other

• Observer is active and engaged, not passive

• Continuous analysis, design and code reviews

04 September 2007 Kaiser: COMS W4156 Fall 2007 37

Don’t hit your partner

• But make sure your partner stays focused and on-task

• Less time wasted (reading email, IMing, surfing web, etc.) than when working alone, since partner awaiting continuous contribution and input

• Each partner expects the other to follow prescribed development practices

04 September 2007 Kaiser: COMS W4156 Fall 2007 38

Put things back where they belong• Put negative thoughts in the trash can• Very difficult to work with someone with great

insecurity or anxiety about their programming skills• Should view pair programming as an opportunity

to improve skills by watching and obtaining feedback

• Also difficult to work with big egos• No one is infallible and above the input of another

04 September 2007 Kaiser: COMS W4156 Fall 2007 39

Clean up your mess

• Many obvious but unnoticed defects become noticed by another person watching over your shoulder

• Defects removed without the animosity that might develop during a formal inspection meeting

04 September 2007 Kaiser: COMS W4156 Fall 2007 40

Don’t take things too seriously

• Ego-less programming

• Don’t be defensive with receiving criticism

• But don’t always just agree with your partner

• Initial adjustment period to pair programming in general and partner in particular

04 September 2007 Kaiser: COMS W4156 Fall 2007 41

Say you’re sorry when you hurt somebody while moving furniture

• Appropriate workspace layout critical to success

• Slide the keyboard, don’t move the chairs

• Effective communication, within pairs and among pairs, is paramount

• Programmers need to see each other, ask each other questions, and make decisions on things such as integration issues

04 September 2007 Kaiser: COMS W4156 Fall 2007 42

Wash your hands of skepticism before you start

• Partner buy-in critical to success

• A jelled team is a group of people so strongly knit that the whole is greater than the sum of the parts

04 September 2007 Kaiser: COMS W4156 Fall 2007 43

Flush

• When pair programmers work on something independently, need to throw away and then rewrite jointly

• Or at least review jointly – but more likely then to still have defects

04 September 2007 Kaiser: COMS W4156 Fall 2007 44

Warm cookies and cold milk are good for you

• Pair programming can be very intense and mentally exhausting

• Periodically take a break

04 September 2007 Kaiser: COMS W4156 Fall 2007 45

Live a balanced life – learn some and think some and draw and paint and sign and dance and play and

work every day some

• Communicate with others on a regular basis• Most programmers would probably say they

preferred to work alone in a place where they wouldn’t be disturbed by other people

• But informal discussions allow for effective idea exchange and efficient transfer of information

04 September 2007 Kaiser: COMS W4156 Fall 2007 46

Take a break from working together every afternoon

• Might do experimental prototyping, tough deep concentration problems, and logical thinking alone

• Simple, well-defined, rote coding more efficiently done alone and then reviewed with a partner

04 September 2007 Kaiser: COMS W4156 Fall 2007 47

When you go out into the world, watch out for traffic, hold hands and

stick together

• No competition between the paired programmers

• Both work for a singular purpose• Do not place blame for defects or

problems on either partner• Pair should trust each other’s judgment

and loyalty to the team

04 September 2007 Kaiser: COMS W4156 Fall 2007 48

Be aware of the power of two brains

• When two people working together, each has own set of knowledge and skills

• Large common subset allows to interact effectively

• Pool resources to accomplish tasks

04 September 2007 Kaiser: COMS W4156 Fall 2007 49

04 September 2007 Kaiser: COMS W4156 Fall 2007 50

Reminder: First Assignments Due Next Week!

• Tuesday 11 September, 10am

• Assignments posted on course website

• Submit via CourseWorks

• Individual Development Assignment #1

• Pair Formation

04 September 2007 Kaiser: COMS W4156 Fall 2007 51

Upcoming Deadlines

• Individual development assignment #2 due September 18th

• Teams announced September 18th • Team project concept due September 25th • Individual development assignment #3 due

October 2nd • Revised project concept due October 9th – first

iteration begins

04 September 2007 Kaiser: COMS W4156 Fall 2007 52

COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

Kaiser+4156@cs.columbia.edu

http://york.cs.columbia.edu/classes/cs4156/