Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | imogene-fitzgerald |
View: | 219 times |
Download: | 0 times |
Andreas Kuehlmann
Software Integrity Group, Synopsys
July 19, 2015
Making a Practical Impact in SW Verification
© 2015 Synopsys, Inc. 3
A Typical SW Project
Complex Software
More Bugs
More Complex SW
More Bugs
Fix Bugs
Fix Bugs
New Features
New Features
The Grand Challenge:• There seems to be only a way down• Most projects don’t know where they
are on that path
© 2015 Synopsys, Inc. 4 Automotive
PrivacyFSIHealthCare
Apollo 11: 145 kLOC (lines of code)
Microsoft Windows: 2.3 MLOC (1992) 40 MLOC (2009)
Automobile: 50 kLOC (1981) 100 MLOC (2014)
Size of SW applications continues to grow Number of SW developers increases steadily
SW compliance regulations start emergingSoftware tools market grows
Security testing growing fastest Dynamic ready for disruption
The Challenges and Opportunities
© 2015 Synopsys, Inc. 6
Challenge 1:
• Can’t train them to become experts in temporal logic– This was tried in the HW world with little success
Developers don't think functionally, they thinkoperationally
© 2015 Synopsys, Inc. 7
Challenge 1 (cont.):
• Can’t expect them to understand global loop invariants and be able to maintain them
• Can’t explain bugs in such terms
• Developers understand gdb– Explain bugs in gdb’s terms!
http://homepages.ius.edu/rwisman/C455/html/notes/Chapter2/Loops/lpinv.htm
Developers don't think functionally, they thinkoperationally
© 2015 Synopsys, Inc. 8
Challenge 2:
Developers (like all humans) have a very limited memory and context switch is expensive!
In 1885, Herman Ebbinghaus did an experiment where subjects memorized a list of meaningless three letter words and tracked how quickly his subjects forgot the words.
Source: Gerald Weinberg: Quality Software Management: Systems Thinking
© 2015 Synopsys, Inc. 9
Challenge 2 (cont):
• Test the code while it is fresh in your mind– TDD
– Real-time code analysis
– …
Developers (like all humans) have a very limited memory and context switch is expensive!
Applied Software Measurement, Capers Jones, 1996
© 2015 Synopsys, Inc. 10
Challenge 3:
• Developers have a low tolerance for false bug alarms–1st false bug report
“Well, it didn’t get this right”
–2nd false bug report
“Annoying”
–3rd false bug report
“This tool is useless”
–4th false bug report
“Boss, why are you wasting my time, get rid of that tool!”
Don’t get in the developer’s way!
© 2015 Synopsys, Inc. 11
Challenge 4:
“My code is right! – I don’t need any tools!”
Yet
“What idiot broke the build again!”
Developers have big egos!
© 2015 Synopsys, Inc. 12
Challenge 5:
• Need to enable the tail end and pull them to the left– “Guard-rails” for developers
Not all developers are A or B grade
© 2015 Synopsys, Inc. 13
Challenge 6:
Competing factors for project delivery of a SW product:
1. Number and complexity of features
2. Cost to develop them
3. Time needed for developing them
4. Quality and security of the delivered product
1, 2, and 3 have clearly measurable metrics
W/o metrics, 4 is the weak player and often sacrificed
Management has often little insight into the quality of the SW code developed by their teams
© 2015 Synopsys, Inc. 14
From my Personal Point of View….
If you want to make a broader practical impact…
• Don’t start from a solution and then chase a problem (“the worm and the elephant”)
– This leading to success is the exception
– Instead, pick an urgent and broad problem
– Even at the expense of “academic beauty”
• Try to get ahead of the train
– Instead of following which I have seen too often in the SW verification research community
– Security, IoT, Self-driving cars, ….
• Stay connected with reality
– There are millions of SW developers out there in the “stone age”
– There are a huge amount of low-hanging fruits
• There is a lot of luck involved – use your instinct!
– Don’t be afraid for change!