Software Aspects of Software Aspects of Strategic Defense SystemsStrategic Defense Systems

Team TurkeyTeam TurkeyJoeJoe

Kim Kim SenthilSenthilSmithaSmitha

President Reagan’s President Reagan’s SDISDI

In March 1983, President Reagan In March 1983, President Reagan called for a “Strategic Defense called for a “Strategic Defense Initiative” (SDI).Initiative” (SDI).

““I call upon the scientific community I call upon the scientific community … to give us the means of … to give us the means of

rendering these nuclear weapons rendering these nuclear weapons impotent and obsolete.”impotent and obsolete.”

The SDI program came to be The SDI program came to be popularly called “Star Wars.”popularly called “Star Wars.”

Parnas’ BackgroundParnas’ Background• Doesn’t object to weapons Doesn’t object to weapons

development in generaldevelopment in general• 8yrs experience working on military 8yrs experience working on military

aircraftaircraft• 20yrs in the software engineering field20yrs in the software engineering field• June 28, 1985 - Parnas resigns from June 28, 1985 - Parnas resigns from

the the $1,000 / day$1,000 / day panel panel

Reasons for resignationReasons for resignation• Software is unreliableSoftware is unreliable• Unattainable goal due to SDI propertiesUnattainable goal due to SDI properties• Software techniques inadequateSoftware techniques inadequate• SE improvements will be insufficientSE improvements will be insufficient• AI and Automatic Programming won’t helpAI and Automatic Programming won’t help• Problems with ProofsProblems with Proofs• Research is inefficient and ineffectiveResearch is inefficient and ineffective

Software is UnreliableSoftware is Unreliable• Software often produced with “bugs”Software often produced with “bugs”• Problems persist for several versions and sometimes Problems persist for several versions and sometimes

worsen with upgradesworsen with upgrades• Digital computers have large # of states but made Digital computers have large # of states but made

from redundant subsystems (which can be from redundant subsystems (which can be exhaustively testedexhaustively tested but the but the whole system whole system cacann’’t)t)

• # of p# of possible states too high in Softwareossible states too high in Software• Functions describing their behavior not continuous and Functions describing their behavior not continuous and

can’t be mathematically verifiedcan’t be mathematically verified• LLogical expressions often harder to understand than ogical expressions often harder to understand than

the program itselfthe program itself• Most Programmers don’tMost Programmers don’t know the tools of the tradeknow the tools of the trade

Can’t Trust that SDI CharacterCan’t Trust that SDI Character• Target and decoys have unknown characteristics, (need Target and decoys have unknown characteristics, (need

to identify, track, and direct weapons towards them)to identify, track, and direct weapons towards them)– Fatal errors Fatal errors will occur will occur if developed without knowledge of if developed without knowledge of

characteristics or with characteristics that can be changed by characteristics or with characteristics that can be changed by an attacker on day of battlean attacker on day of battle

• Attackers countermeasures make network of sensors Attackers countermeasures make network of sensors and weapons unreliableand weapons unreliable– Fail-soft only successful when: failures predicted from past Fail-soft only successful when: failures predicted from past

history, component failures unlikely and statically independent, history, component failures unlikely and statically independent, system has excess capacity, real-time deadlines can be missedsystem has excess capacity, real-time deadlines can be missed

– None true for None true for SDISDI system system• Impossible to test under real conditionsImpossible to test under real conditions

– No faith without extensive testsNo faith without extensive tests

Most massive, costly software Most massive, costly software ever attemptedever attempted

• Service period too short for humans to debug and Service period too short for humans to debug and modify programsmodify programs– Debugger’s notes on army truck in Vietnam – not possible Debugger’s notes on army truck in Vietnam – not possible

in 30-90 minute warin 30-90 minute war• Real-time computation deadlinesReal-time computation deadlines – – worst case worst case

amount of resources can’t be predictedamount of resources can’t be predicted– Efficiency and predictability require some preruntime Efficiency and predictability require some preruntime

scheduling, need worse case real-time schedule scheduling, need worse case real-time schedule • Large variety of sensors and weapons each Large variety of sensors and weapons each

requiring complex software, suite will grow during requiring complex software, suite will grow during development and after deployment (subject to development and after deployment (subject to independent modification)independent modification)– Difficulties increase with: size of the system, # of Difficulties increase with: size of the system, # of

independent subsystems, and # of interfacesindependent subsystems, and # of interfaces

One Shot at the TitleOne Shot at the Title• Flow chart Flow chart approach approach –– “think like a computer” “think like a computer”

– IImproved with larger stepsmproved with larger steps• Leads to confusion as data has different meaning Leads to confusion as data has different meaning

under different circumstancesunder different circumstances• Concurrency – program appear to be doing more Concurrency – program appear to be doing more

than one thing at a timethan one thing at a time• Multiprocessing – program DOES more than one Multiprocessing – program DOES more than one

thing at a timething at a time• Yes, Professional Programmers use this Yes, Professional Programmers use this

conventional conventional approachapproach• Trial and Error – software released when rate of Trial and Error – software released when rate of

finding new errors slows downfinding new errors slows down

New SE TechniquesNew SE Techniques• Research aimed at reducing amount of Research aimed at reducing amount of

information needed to test and maintaininformation needed to test and maintain– Structured programming and formal program Structured programming and formal program

semanticssemantics– Use of formally specified abstract interfaces Use of formally specified abstract interfaces

(information hiding)(information hiding)– Use of cooperating sequential processesUse of cooperating sequential processes

• Gap between theory and practiceGap between theory and practice– Good software engineering can be done, it’s just Good software engineering can be done, it’s just

far from easyfar from easy– It reduces, NOT ELIMINATES, errors thus It reduces, NOT ELIMINATES, errors thus there is there is

still still a a need testingneed testing

Improvements in SEImprovements in SE• New languages and environments will help but New languages and environments will help but

they are not a major impediment to our work.they are not a major impediment to our work.• AI makes big claims but can offer no helpAI makes big claims but can offer no help• Automatic Programming is just a euphemism Automatic Programming is just a euphemism

for programming in a higher-level language.for programming in a higher-level language.– Still need to specify an algorithmStill need to specify an algorithm

• NNo breakthroughso breakthroughs– The fault lies not in our tools but in ourselves and in The fault lies not in our tools but in ourselves and in

the nature of our product.the nature of our product.

Artificial Flowers and IntelligenceArtificial Flowers and Intelligence

• AI-1 - AI-1 - SSolving problemsolving problems which previously could only which previously could only be done with human intelligencebe done with human intelligence– This definition changes over timeThis definition changes over time– Best work in this area makes no attempt to mimic people’s Best work in this area makes no attempt to mimic people’s

problem solving techniquesproblem solving techniques– Mostly problem specific, requires abstraction and creativity Mostly problem specific, requires abstraction and creativity

to transfer the workto transfer the work• AI-2 - Heuristic or Rule Based Programming/Expert AI-2 - Heuristic or Rule Based Programming/Expert

SystemsSystems– Approach is dangerous and misleadingApproach is dangerous and misleading– Rules obtained are inconsistent, incomplete, and inaccurateRules obtained are inconsistent, incomplete, and inaccurate– Evolutionary approach results in poorly understood behavior Evolutionary approach results in poorly understood behavior

which is hard to predictwhich is hard to predict– Spectacular behavior on small # of obvious casesSpectacular behavior on small # of obvious cases

Prove itProve it

• Can’t use eCan’t use exhaustive case analysisxhaustive case analysis• No pNo prolonged, realistic, testing rolonged, realistic, testing • Use Use Mathematical analysisMathematical analysis

– Don’t have exact specifications to Don’t have exact specifications to which one can apply a proofwhich one can apply a proof

– Proofs themselves may contain errorsProofs themselves may contain errors– Concurrency adds difficulty to proofsConcurrency adds difficulty to proofs– No techniques to prove programs No techniques to prove programs

robust enough to operate with robust enough to operate with unknown hardware failures or input unknown hardware failures or input errors errors

Getting what you’ve paid forGetting what you’ve paid for• Those who make purchasing Those who make purchasing

decisions don’t know what they’re decisions don’t know what they’re buyingbuying

• Most difficult and crucial step in Most difficult and crucial step in research is to identify and define the research is to identify and define the problem problem

• Practical considerations restrict Practical considerations restrict important theoretical problemsimportant theoretical problems

• Research should be judge by teams Research should be judge by teams of successful researchers and of successful researchers and experienced system engineersexperienced system engineers

• These people considered to valuable These people considered to valuable to spend time reviewing proposalsto spend time reviewing proposals

Some other perspectives on Some other perspectives on SDISDI

•A Debate on the feasibility of SDI was sponsored by A Debate on the feasibility of SDI was sponsored by CSPR & MIT in 1985CSPR & MIT in 1985•The debate was moderated by Michael L. Dertouzos The debate was moderated by Michael L. Dertouzos PhD '64 of MIT PhD '64 of MIT •Parnas and Joseph Weizenbaum of MITParnas and Joseph Weizenbaum of MIT against SDIagainst SDI•Charles L. Seitz '65 of Caltech and Danny Cohen of Charles L. Seitz '65 of Caltech and Danny Cohen of the University of Southern California USC spoke in for the University of Southern California USC spoke in for SDI. SDI.

•Parnas presented his argument based on the papers Parnas presented his argument based on the papers he has submitted to SDIO at the time of is resignation he has submitted to SDIO at the time of is resignation from the panel.from the panel.

Parnas’ ArgumentParnas’ ArgumentSince:Since:• Specifications not known in advance, Specifications not known in advance, • Realistic testing is not possible,Realistic testing is not possible,• No chance to fix software during No chance to fix software during

use,use,• No foreseeable technology changes No foreseeable technology changes

this,this,

Therefore – Therefore – It is not possible to construct SDIIt is not possible to construct SDIsoftware that you could trust to work.software that you could trust to work.

Seitz’ Seitz’ ArgumentArgumentSinceSince• A hierarchical architecture seems best, A hierarchical architecture seems best, (because more natural, used in nature, (because more natural, used in nature,

understood by military, allows abstraction understood by military, allows abstraction up levels …) up levels …)

• Physical organization should follow logical Physical organization should follow logical organization, (simplest choice, natural) organization, (simplest choice, natural)

• Tradeoffs to make software problem Tradeoffs to make software problem tractable are in the choice of system tractable are in the choice of system architecture (not in new / radical architecture (not in new / radical methods) this makes software problems methods) this makes software problems tractable.tractable.

Seitz’ Seitz’ ArgumentArgument

• Loose coordination allows us to Loose coordination allows us to infer system performanceinfer system performance

(assume stat. independence, …) (assume stat. independence, …) allows system reliability estimates.allows system reliability estimates.

Therefore it is possible to create Therefore it is possible to create reliable SDI battle management reliable SDI battle management software.software.

From the debate……From the debate……• Parnas says “We can’t test it”Parnas says “We can’t test it”

• Seitz then replies “We can build it.” Seitz then replies “We can build it.”

• Cohen mentions the space shuttle as an example Cohen mentions the space shuttle as an example of a system requiring large and complex software.of a system requiring large and complex software.

• Parnas’ response is that whereas NASA can Parnas’ response is that whereas NASA can delay a launch up until the last second, the delay a launch up until the last second, the president cannot call up the USSR to delay a president cannot call up the USSR to delay a nuclear war.nuclear war.

From the debate……From the debate……• Seitz argues that SDI will be much better than Seitz argues that SDI will be much better than

the existing ABM systems. In essencnce he the existing ABM systems. In essencnce he says something useful could be built but says something useful could be built but doesnt really address the issue of testing it.doesnt really address the issue of testing it.

• Parnas argues that it doesn’t make any Parnas argues that it doesn’t make any difference what is built or how it is built, difference what is built or how it is built, because there won’t be any means of testing because there won’t be any means of testing that it meets requirements.that it meets requirements.

• While people for SDI keep coming with While people for SDI keep coming with arguments to support SDI, they fail to provide arguments to support SDI, they fail to provide anwers to the specific issues raised by Parnasanwers to the specific issues raised by Parnas

Patriot Missile performance in Patriot Missile performance in the Gulf warthe Gulf war

• The Patriot system has 7.4 ft long missile The Patriot system has 7.4 ft long missile powered by a single stage solid propellant powered by a single stage solid propellant rocket motor that runs at mach 3 speedsrocket motor that runs at mach 3 speeds

• The missile weighs 2200 lbs and its range is 43 The missile weighs 2200 lbs and its range is 43 milesmiles

• The patriot is armed with a 200 lb high The patriot is armed with a 200 lb high explosive war head detonated by a proximity explosive war head detonated by a proximity fuse that causes shrapnel to destroy the fuse that causes shrapnel to destroy the intended targetintended target

• The system is built around radar and fast The system is built around radar and fast computers computers

Operation:Operation:• The missile is launched and guided to the The missile is launched and guided to the

target in three phases:target in three phases:– First, the missiles guidance system turns the First, the missiles guidance system turns the

patriot toward the incoming missile as the patriot toward the incoming missile as the missile flies into the Patriot’s radar beammissile flies into the Patriot’s radar beam

– Then the Patriot’s computer guides the missile Then the Patriot’s computer guides the missile toward the incoming scud missiletoward the incoming scud missile

– Finally, the patriot Missile’s internal radar Finally, the patriot Missile’s internal radar receiver guides it toward the interception of receiver guides it toward the interception of the incoming missilethe incoming missile

• During the Gulf war the Patriot was During the Gulf war the Patriot was assigned to shoot down incoming Iraqi assigned to shoot down incoming Iraqi Scud or Al-Hussein missiles launched at Scud or Al-Hussein missiles launched at Israel and Saudi ArabiaIsrael and Saudi Arabia

Statistical analysis of the Patriot’s Statistical analysis of the Patriot’s performance during the Gulf war:performance during the Gulf war:

• The U.S. Army which was in charge of the The U.S. Army which was in charge of the Patriot claimed an initial success rate of 80% in Patriot claimed an initial success rate of 80% in Saudi Arabia and 50% in IsraelSaudi Arabia and 50% in Israel

• Those claims were scaled back to 70% and Those claims were scaled back to 70% and 40% respectively40% respectively

• Part of the reason the success rate was 30% Part of the reason the success rate was 30% higher in Saudi Arabia than in Israel is that in higher in Saudi Arabia than in Israel is that in Saudi Arabia the patriots merely had to push Saudi Arabia the patriots merely had to push the incoming scud missile away from military the incoming scud missile away from military targets in the desert or disable the war headtargets in the desert or disable the war head

• In Israel the scuds were aimed directly at cities In Israel the scuds were aimed directly at cities and civilian populations (Lager targets)and civilian populations (Lager targets)

……Analysis continuedAnalysis continued• The Patriot’s success rate in Israel was The Patriot’s success rate in Israel was

examined by the Israel Defense Forces examined by the Israel Defense Forces (I.D.F)(I.D.F)

• The IDF counted any scud that exploded on The IDF counted any scud that exploded on the ground (regardless of whether or not it the ground (regardless of whether or not it was diverted) as a failure of the patriotwas diverted) as a failure of the patriot

• A 10 month investigation by the House A 10 month investigation by the House Government Operations subcommittee on Government Operations subcommittee on legislation and national security concluded legislation and national security concluded that there was little evidence to prove that that there was little evidence to prove that the Patriot hit more than a few Scudsthe Patriot hit more than a few Scuds

Patriot missile software Patriot missile software problemproblem

• As reported by the U.S. General Accounting office, As reported by the U.S. General Accounting office, On 02/25/1991, a Patriot failed to track and On 02/25/1991, a Patriot failed to track and intercept a Scud missile because of a software intercept a Scud missile because of a software problem in the system’s weapons control problem in the system’s weapons control computer, the scud subsequently hit an Army computer, the scud subsequently hit an Army barracks, killing 28 Americans barracks, killing 28 Americans

• This problem led to inaccurate tracking calculation This problem led to inaccurate tracking calculation that became worse the longer the system that became worse the longer the system operatedoperated

• The patriot had never before been used to defend The patriot had never before been used to defend against Scud missiles nor was it expected to against Scud missiles nor was it expected to operate continuously for long periods of timeoperate continuously for long periods of time

A look at current missile defense A look at current missile defense scenarioscenario

• Some dreams never die. Do they?Some dreams never die. Do they?• SDI , which was envisioned by President SDI , which was envisioned by President

Regan continues to live.Regan continues to live.• The concept of missile defense remains The concept of missile defense remains

the same but the bounds of the dream the same but the bounds of the dream keeps changing.keeps changing.

• This can be attributed to the change in the This can be attributed to the change in the sophistication and the geographical sophistication and the geographical location of the hypothesized enemy.location of the hypothesized enemy.

A look at current missile A look at current missile defense scenariodefense scenario

• In the early 90’s SDI gets reincarnated, but In the early 90’s SDI gets reincarnated, but this time with a new name “ BMD”this time with a new name “ BMD”

• BMDO unlike SDIO has a string of projects BMDO unlike SDIO has a string of projects with relatively smaller goals.with relatively smaller goals.

• The projects under BMDO can be classified The projects under BMDO can be classified broadly under these categories,broadly under these categories,

  Terminal DefenseTerminal Defense Midcourse Defense SegmentMidcourse Defense Segment   Boost Defense SegmentBoost Defense Segment

A look at current missile A look at current missile defense scenariodefense scenario

A look at current missile A look at current missile defense scenariodefense scenario

• Further classification of these Further classification of these categoriescategories

• Terminal Defense SegmentTerminal Defense Segment THAAD, NTMD, PATRIOT – PAC3, etc..THAAD, NTMD, PATRIOT – PAC3, etc..• Midcourse Defense SegmentMidcourse Defense Segment

NMD – GMD, SMD, etc…NMD – GMD, SMD, etc…• Boost Defense SegmentBoost Defense Segment

Airborne Laser, Space Based Laser, Airborne Laser, Space Based Laser, etc ..etc ..

A look at current missile A look at current missile defense scenariodefense scenario

• Well how is the “BMD” Doing?Well how is the “BMD” Doing?• An estimated amount of 100 billion dollars An estimated amount of 100 billion dollars

have been spent on Missile defense.have been spent on Missile defense.• The goals of each of the subsystem is small The goals of each of the subsystem is small

compared to SDI due to the current compared to SDI due to the current scenariosscenarios

• ““This is a sharp change from the Reagan years, This is a sharp change from the Reagan years, perhaps because the technology used is closer at perhaps because the technology used is closer at hand and the threats are smaller.”hand and the threats are smaller.”

(Mosher, page 39, IEEE Spectrum, 1997)(Mosher, page 39, IEEE Spectrum, 1997)

A look at current missile A look at current missile defense scenariodefense scenario

– Smaller anticipated mission:Smaller anticipated mission:

““protect the U.S. … against an attack by a protect the U.S. … against an attack by a rogue state using a handful of warheads rogue state using a handful of warheads outfitted with … simple countermeasures.”outfitted with … simple countermeasures.”““also provide protection against an accidental also provide protection against an accidental launch of a few warheads by Russia or China.”launch of a few warheads by Russia or China.”“… “… no more than 100 hit-to-kill interceptors no more than 100 hit-to-kill interceptors based at old ABM site near Grand Forks, ND.”based at old ABM site near Grand Forks, ND.”

(Mosher, page 37, IEEE Spectrum, 1997)(Mosher, page 37, IEEE Spectrum, 1997)

A look at current missile A look at current missile defense scenariodefense scenario

• How do these smaller anticipated How do these smaller anticipated missions affect Parnas’s argument about missions affect Parnas’s argument about SDI wont be able to produce a trustworthy SDI wont be able to produce a trustworthy missile defense software?missile defense software?

• Fundamentally not as you can see from Fundamentally not as you can see from the “Test” facts below,the “Test” facts below,

• ““In the last 15 years, the U.S. has In the last 15 years, the U.S. has conducted 20 hit-to-kill intercepts, …. Six conducted 20 hit-to-kill intercepts, …. Six intercepts were successful; 13 of those intercepts were successful; 13 of those test were done in the last five years, and test were done in the last five years, and among them three succeeded.”among them three succeeded.”

Test Facts ……Test Facts ……

• ““No real attempts have been made to No real attempts have been made to intercept uncooperative targets – those that intercept uncooperative targets – those that make use of clutter, decoys, maneuver, anti-make use of clutter, decoys, maneuver, anti-simulation, and other countermeasures.”simulation, and other countermeasures.”

(Mosher, page 39, IEEE Spectrum, 1997)(Mosher, page 39, IEEE Spectrum, 1997)• In 1996, ex TRW engineer Nira Schwartz filed In 1996, ex TRW engineer Nira Schwartz filed

a “False Claims Act” suit, alleging that a “False Claims Act” suit, alleging that results of tests to distinguish warheads and results of tests to distinguish warheads and decoys were falsified by TRW.decoys were falsified by TRW.

(featured on “60 Minutes II” in January 2001)(featured on “60 Minutes II” in January 2001)

Test Facts ……Test Facts ……

• Lt. General Kadish – “Right now, from what I Lt. General Kadish – “Right now, from what I see, there is no reason to believe that we see, there is no reason to believe that we can’t make this work. But there’s a lot can’t make this work. But there’s a lot more testing to be done.”more testing to be done.”

• Secretary of Defense Donald Rumsfeld said,Secretary of Defense Donald Rumsfeld said, ““We are going to deploy a minimal Missile We are going to deploy a minimal Missile

Defense System, in the near future even if Defense System, in the near future even if the system has not been tested the system has not been tested completely.”completely.”

ConclusionsConclusions

• A trustworthy SDI Software seems highly A trustworthy SDI Software seems highly impossible.impossible.

• The arguments by others supporting SDI doesn’t The arguments by others supporting SDI doesn’t seem to answer the issues raised by Parnas.seem to answer the issues raised by Parnas.

• The newer scenarios of missile defense does not The newer scenarios of missile defense does not change Parnas’s argument fundamentallychange Parnas’s argument fundamentally

• The systems for limited mission seems to be The systems for limited mission seems to be more tractable than SDImore tractable than SDI

References:References:• http://www.cse.nd.edu/~kwb/nsf-ufe/star-wars/http://www.cse.nd.edu/~kwb/nsf-ufe/star-wars/• Broad, W.J., "Scientist at work: Philip E. Coyle III; Broad, W.J., "Scientist at work: Philip E. Coyle III;

words of caution on missile defense", words of caution on missile defense", New York New York Times,Times, January 16, 2001. January 16, 2001.

• DOD Ballistic Missile Defense Organization DOD Ballistic Missile Defense Organization (BMDO). Web site (BMDO). Web site http://www.acq.osd.mil/bmdo/http://www.acq.osd.mil/bmdo/

• http://www.clw.org/nmd/bmdfuzzylogic.htmlhttp://www.clw.org/nmd/bmdfuzzylogic.html