Post on 09-Apr-2018
transcript
8/7/2019 UNit-vii MIS
1/44
Acquisition of SW and HW (software and hardware)
Acquisition process improvement has been defined as the analysis of current softwareacquisition processes for deficiencies and implementing new/modified processes tocorrect those deficiencies. The software acquisition process improvement, defining it as:
A documented process for software acquisition planning, requirements
development/management, project management/oversight and risk management
Efforts to develop appropriate metrics for performance measurement and
continual process improvement
A process to ensure that key program personnel have an appropriate level of
experience/training in software acquisition
A process to ensure that each military department and defense agency select,
implement and adhere to established processes and requirements relating to software
acquisition
One of the better descriptions of the characteristics of a software acquisition process thatis successfully being used by leading software acquirers/developers is contained in theStronger Management Practices. Three fundamental management strategies are: Evolutionary Environment
Disciplined Development Processes
Collecting/Analyzing Meaningful Metrics
These strategies limit software development efforts to what can be managed and result indecisions based on sufficient systems engineering knowledge to adequately assess risks.In addition, business decisions are consciously made to invest time/resources in achievinghigh software process maturity levels. Strong upper-level management support is astrong enabler for ensuring that these combined strategies are successful.
Essential conditions in acquisition of SW AND HW
Organizations that have a strong, consistent evolutionary environment and practices forsetting product requirements, maintaining a disciplined development process and usingmetrics to oversee development progress achieve favorable cost, schedule and qualityoutcomes.
1
8/7/2019 UNit-vii MIS
2/44
Acquisition
The conceptualization, initiation, design, development, test, contracting, production,deployment, Logistics Support (LS), modification, and disposal of software systems,
Acquisition environment
Internal and external factors that impact on, and help shape, every software acquisitionprogram. These factors include political forces, policies, regulations, reactions tounanticipated requirements and emergencies.
Acquisition life-cycle
The life of a software acquisition program consists of phases, each preceded by amilestone or other decision point, during which a system goes through Research,
Development, Test and Evaluation (RDT&E) and production.Acquisition program
A directed, funded effort over the software acquisition life-cycle that provides a new,improved, or continuing materiel, weapon or information system or service capability inresponse to an approved need.
Acquisition process
The steps involved, and practices employed, in the planning, contracting,implementation, acceptance and follow-on activities of a software acquisition. Theacquirer will bear most of the responsibility for the results from the planning, contracting,and follow-on phases of the acquisition, while the developer is most accountable for theresults coming out of the implementation and acceptance phases of the acquisition.
Development process
The process of working out and extending the theoretical, practical, and usefulapplications of a basic design, idea, or scientific discovery, characterized by the design,coding, modification or improvement of software.
2
8/7/2019 UNit-vii MIS
3/44
The SW&HW Acquisition Process
It is necessary to first understand what a software acquisition process is before steps canbe taken to improve In following figure a generic nine-step process for softwareacquisition, summarized in Figure 1.
3
8/7/2019 UNit-vii MIS
4/44
4
8/7/2019 UNit-vii MIS
5/44
Figure 1: IEEE Std 1062, 1998 Edition S\W AND HW Acquisition Process StepsThere are three steps that comprise the planning phase of a software acquisition. Duringthis phase, the acquisition strategy should be planned based on a review of the acquirers
objectives. A formal Acquisition Strategy document is developed to guide theacquisition.
Implementing the acquisition strategy into the organizations process is the next step.Appropriate contracting practices need to be included in the process. The final step of theplanning process encompasses the determination of software requirements. In addition todefining the software being acquired, this step includes the preparation of the quality andmaintenance plans for accepting the suppliers/developers software. The release of aRequest for Proposal (RFP) is the completion milestone for the planning phase of thesoftware acquisition life-cycle.
The contracting phase of the software acquisition is also comprised of three steps. Thefirst step is the identification of potential suppliers. Suppliers should be selected whowill provide documentation for their software, demonstrate its performance, and submitformal proposals. Failure to perform any of these actions may serve as a basis forrejecting a potential supplier.
Past performance data from previous contracts should be reviewed for all potentialsuppliers. The next step is to actually prepare the contract requirements. The quality ofwork (acceptable performance and acceptance criteria) should be described, and contractprovisions that tie payments to deliverables should be prepared. Legal counsel shouldalso be sought to review and approve contractual language/requirements. Finally,proposals will be evaluated and a qualified supplier(s) will be selected. If appropriate, analternate supplier should be negotiated with, primarily as a risk reduction measure.
Supplier selection completes the contracting phase of the software acquisition life-cycle,which started with the release of the RFP and ends with the contract signing.Once the contract is signed, software development can begin and the productimplementation phase of the software acquisition life-cycle is initiated. During thisphase, the acquirer must manage supplier performance. The suppliers progress ismonitored to ensure all milestones are met. The acquirer also must approve allappropriate work products. Note that, during this phase, the acquirer has theresponsibility to provide all acquirer deliverables to the supplier when required. Theproduct implementation phase of the software acquisition is completed when the softwareproduct is delivered to the acquirer (pre-approval).
The next phase of the software acquisition life-cycle is product acceptance. Adequatetesting needs to be performed, and a process for certifying that all discrepancies havebeen corrected and all acceptance criteria have been satisfied must be established.Acceptance of the software product by the acquirer based on mutually agreed-to pre-defined criteria signals the end of the product acceptance phase.
5
8/7/2019 UNit-vii MIS
6/44
The follow-on phase of software acquisition covers the period between softwareacceptance and software retirement, when it is no longer in use. A follow-up of thesoftware acquisition contract should typically be performed to evaluate contractingpractices, record lessons learned, and evaluate user satisfaction with the software. During
this final phase of the software acquisition life-cycle, the acquirer should record andretain supplier performance data to use for the acquisition of future software products.More recently, the Software Engineering Institute (SEI) has released the SoftwareAcquisition Capability Maturity Model to represent the important phases and elements ofthe software acquisition processes.
Key areas necessary in SW &HW acquisition
The areas necessary to be considered while planning for acquisition of SW &HW orhardware includes many sub-components as mentioned in table -
Table 1: Key Process Areas
LEVEL FOCUS KEY PROCESS AREA5Optimizing
Continuous ProcessImprovement
Acquisition Innovation Management Continuous Process Improvement
4Quantitative
Quantitative Management Quantitative Acquisition Management Quantitative Process Management
3Defined
Process Standardization Training Program Acquisition Risk Management
Contract Performance Management Project Performance Management User Requirements Process Definition and Maintenance
2Repeatable
Basic ProjectManagement
Transition to Support (Acceptance/Follow-On) Evaluation (Implementation/Acceptance) Contract Tracking and Oversight(Implementation) Project Management (Implementation) Requirements Development and Management(Planning/Contracting)
Solicitation (Planning/Contracting) Software Acquisition Planning (Planning)
1Initial
Competent People and Heroics
6
8/7/2019 UNit-vii MIS
7/44
What is Software Acquisition Process Improvement?
Acquisition process improvement has been defined as the analysis of current softwareacquisition processes for deficiencies and implementing new/modified processes to
correct those deficiencies. The most comprehensive definition of software acquisitionprocess improvement, defining it as: A documented process for software acquisition planning, requirementsdevelopment/management, project management/oversight and risk management
Efforts to develop appropriate metrics for performance measurement andcontinual process improvement
A process to ensure that key program personnel have an appropriate level ofexperience/training in software acquisition
A process to ensure that each military department and defense agency select,implement and adhere to established processes and requirements relating to softwareacquisition
SW AND HW acquisition/development life-cycle:
Acquisition planning Requirements development/management Configuration management
Risk management Project management/oversight Test & evaluation Integrated team management Solicitation/source selection Performance measurement
Factors supporting software acquisition process improvement
System/software objectives and constraints are adequately defined /validated
System/software acquisition strategies are appropriate /compatible
Success-critical stakeholders have committed adequate softwareCapability / resources to perform their software-related tasks
Software product/process plans are feasible/ compatible
7
8/7/2019 UNit-vii MIS
8/44
Feasibility Rationale provides convincing evidence that software progress issatisfactory
Specified improvements include
(1) Documenting agreements between the software developer and acquirer that containbaseline requirements based on systems-engineering knowledge,
(2) Gated reviews during the software development process
(3) obtaining meaningful metrics from the developer for managing the program,
(4) attaining knowledge about the technology that is planned early in the developmentprocess,
(5) ensuring an appropriate balance between requirements and available resources,
(6) ensuring that the software design is mature before it is released, and
(7) having all processes under control prior to release. In general, the more thatacquisition managers know about software development processes and metrics, the betterequipped they are to improve the processes for software acquisition.(8) provide applicable improvement program administration and compliance guidance
and ensure that secretaries of the departments and selected agencies comply with thatguidance and
(9) assist departments/agencies with their respective improvement programs by ensuring
the use of applicable source-selection criteria and access to information regardingsoftware acquisition/development best practices in both the public and private sectors.
Three fundamental management strategies for SW AND HW acquisition are
Evolutionary Environment
Disciplined Development Processes
Collecting/Analyzing Meaningful Metrics]
These strategies limit software development efforts to what can be managed, and theyresult in informed decisions based on sufficient systems engineering knowledge such thatrisk can be adequately assessed. In addition, business decisions are consciously made toinvest sufficient time and resources in order to achieve high software process maturitylevels (high quality, on-time, within budget). Committed upper-level managementsupport is an effective enabler for ensuring that these combined strategies are successful.
8
8/7/2019 UNit-vii MIS
9/44
Suggestions for strengthening acquisition plans included specific
criteria are
Requirements baselines based on systems engineering are documented/agreed toby both the acquirer/developer before a program is initiated
Cost/benefits analyses are required when new requirements are proposed
Software acquirers/developers make efforts to continually improve practices overtime Gated reviews and deliverables are integrated into development processes
Requiring software developers to collect/analyze metrics (including earned value)to obtain knowledge of progress and to manage risk
Evolutionary Environment:
An evolutionary software acquisition environment is characterized by incrementalimprovements to software performance, as opposed to succumbing to programmaticpressures to set unrealistic expectations, in order to improve software processes on acontinual basis. An evolutionary environment limits software development to what ispossible to manage by adhering to well-understood, well-defined, manageablerequirements while simultaneously encouraging continuous process improvement. Anevolutionary product development is one of the fundamental elements of a manageableacquisition environment. The concept of continuous improvement is a critical part of theevolutionary environment, and must be supported by both the environment and theorganizational culture (including senior management) in order to be successful. Ad hoc
processes make it impossible to understand exactly when and how defects in the processoccur.
Disciplined Development Processes:Developers must follow disciplined, structured management and development processesin order for the software acquisition process to improve. Each phase of the softwareacquisition should end in a management review/gate to ensure that the project is ontrack. To pass these gated reviews, developers must demonstrate that acquirersexpectations and quality standards are met before proceeding to the next phase. Peerreviews to check each others work and remove defects, beginning at the earliest stages
of software development, are necessary to prevent costly rework and schedule delaysresulting from downstream defect detection. Disciplined software development processesthat are characterized by their reliance on configuration management, peer reviews andquality assurance significantly increase the probability of a successful acquisition, andhave been proven to assist leading organizations in identifying opportunities for softwareprocess improvement. As a result, areas of risk can be identified early and actions can betaken to control them.
9
8/7/2019 UNit-vii MIS
10/44
Figure 4 highlights a disciplined knowledge-based software development process that canserve as a basis for software acquisition process improvement, describing the informationneeded, the relevant activities and the deliverables for each phase of the process.
Figure 4: Highlights of the Knowledge-Based Software Development Process [GAO,2004]Requirements need to be managed and controlled before design work begins and alllower-level design elements should be adequately defined before the start of coding. It iscritical that the software acquirer and developer work closely together and have open andhonest discussions. Requirements management should be viewed as one of the mostcritical development tasks for ensuring a successful acquisition.
Requirements should be sufficiently documented and validated prior to preliminarydesign. In addition, acquirer/developer stakeholders must use sound systems engineeringtechniques to establish the software requirements, and aggressively manage and controlrequirements changes.
To enhance software coding and testing quality, requirements should be well-written andachievable. By this phase of the acquisition, designs should be sufficiently detailed.Other processes that are critical to achieving high quality software include peer reviews,
10
https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]8/7/2019 UNit-vii MIS
11/44
documented coding standards, frequent unit testing, access to a reuse library, and the useof program languages that enable code documentation to facilitate future understanding.Test plans should not be developed until after requirements are considered stable, andthere should be a minimum of one test to verify/validate each requirement.
One of the most widely-known and best-structured software acquisition and developmentprocesses is the SEIs CMMI series of processes. A very recent addition to this suite oftools is the CMMI Acquisition Module (CMMI-AM.
The focus of the CMMI is on what should be done, not how it should be done, so itallows a great deal of latitude in terms of implementation of the CMMI Key ProcessAreas. As with the other CMMI documents, a primary emphasis of the AcquisitionModule is on Integrated Product and Process Development (IPPD) concepts. Theseinclude the effective use of cross-functional or multidisciplinary teams; leadership
commitment (particularly of senior management); appropriate allocation/delegation ofdecision-making authority; definition of organizational structures that reward teamperformance; the design of downstream processes (transition to Operations and Support O&S) during the acquisition; a strong focus on customer needs throughout the softwarelife-cycle; timely and effective collaboration between all relevant stakeholders; thecontinuous and proactive identification and management of risk; and a focus on themeasurement and improvement of processes to develop/deliver a software product.
A graphical representation of the CMMI-AM process is shown in Figure 5. Followingthat figure is Table 2, which includes checklist-type questions that can be used to self-assess coverage of each defined CMMI-AM process area. The reader is referred to theCMMI-AM document to get a more in-depth discussion of each of the process areasdefined in Figure 5 and Table 2.
11
8/7/2019 UNit-vii MIS
12/44
Figure 5: Graphical Representation of the CMMI-AM ProcessTable 2: CMMI-AM Acquisition Process Areas [CMMI-AM, 2004]Process Area QuestionsOrganizational Environment
for Integration
Is infrastructure provided to establish fully functional teams fromamong all stakeholders?
Does the acquisition project have two focuses on integration technicaland organizational? Are technical capabilities/requirements examined/understood as theyrelate to overall project goals/objectives? Do organizational project entities operate with a single vision andcooperative purpose? Do project members understand the interrelationships of all projectaspects and how they relate to specific goals/objectives?
12
https://goldpractice.thedacs.com/practices/api/#_[CMMI-AM,_2004]%23_[CMMI-AM,_2004]https://goldpractice.thedacs.com/practices/api/#_[CMMI-AM,_2004]%23_[CMMI-AM,_2004]8/7/2019 UNit-vii MIS
13/44
IntegratedAcquisitionProjectManagement
Are project management processes consistent with/tailored from theorganizations standard processes? If a standard set of processes does not exist, has the project defined itsown processes to meet project needs? Is there an integrated management process that incorporates/involves all
stakeholders? Is the project management process defined in an overall ProjectManagement Plan, or equivalent document? Are formal interfaces used among stakeholders, in the form ofMemorandums of Understanding (MOUs), Memorandums of Agreement(MOAs), contractual commitments, associate contractor agreements, andsimilar documents, depending on the nature of the interfaces and involvedstakeholders? Does the projects defined process include all life-cycle processes(including IPPD) that are applied by the project? Do the projects defined processes include (1) selecting the team
structure, (2) allocating personnel resources, (3) implementing cross-integrated team communication, and (4) conducting issue-resolutionprocesses? Does development of a comprehensive project plan or the nature of theprojects defined process require an iterative effort due to a complex,multilayered, integrated team structure?
AcquisitionProjectPlanning
Does planning start by setting the acquisition project strategy, thenplanning the acquisition process in increasing levels of detail? Does the acquisition project review potential suppliers planningprocesses for adequacy and compliance? Are potential suppliers plans reviewed for consistency with system
acquisition plans? If the acquisition project involves replacement of an existing system,has operation/disposal of the existing system been considered as part of theacquisition planning for the new system? Are transition activities included in the acquisition project plan? For integrated teams: Does acquisition project data include that developed/used solely by aparticular team, as well as data applicable across integrated teamboundaries? Does acquisition project resource planning consider staffing of theintegrated teams?
Is stakeholder involvement planned down to the integrated team level? Is special attention paid to resource commitments? Do integrated team plans get buy-in from team members, interfacingteams, the acquisition project, and the acquisition process owners whosestandard processes are selected for tailoring?
Solicitation &ContractMonitoring
Does the project comply with applicable federal, departmental andservice acquisition regulations and policies? Does the solicitation address issues appropriate to the system domain or
13
8/7/2019 UNit-vii MIS
14/44
acquisition environment (e.g., supplier process evaluations, operationalsafety suitability/effectiveness, safety, certifications, architecturalevaluations and interoperability)? Are representatives responsible for functional disciplines within theacquisition project or stakeholder organizations consulted for appropriate
inclusion of those disciplines into the solicitation and contract monitoringprocess? When integrated teams are formed, is team membership negotiated withsuppliers and incorporated into an agreement? Do teaming agreements identify integrated decision-making, reportingrequirements (business and technical) and trade studies requiring supplierinvolvement? Are supplier efforts orchestrated to support the acquisition projectsIPPD efforts?
RequirementsManagement
Does requirements management include the direct management ofacquirer-controlled requirements and oversight of supplier requirements
management? Are requirements managed/maintained such that changes are notimplemented without assessing the impact on the project? Does the requirements management process define approvedrequirements providers and an approved path by which requirements areprovided to the supplier? Are suppliers prevented from receiving requirements changes fromunauthorized sources that are outside the flow of the acquirers establishedrequirements management process? Are commitments to requirements by acquisition project participantsdemonstrated by having coordinated/approved documents that define
requirements? Is each change to a controlled requirement assessed for impact onacquisition project performance, cost and schedule baselines, and projectrisk? Are existing cost, schedule and performance baselines changed, asrequired, to accommodate a requirements change?
RequirementsDevelopment
Are customer requirements grouped/coordinated into a set ofrequirements that will define the scope/direction of the acquisition? Do customer requirements and additional acquirer requirements becomethe basis for the processes used by the suppliers organization? Do requirements flow from the stakeholders/customer to the system
level, to multiple subsystem levels and, eventually, to software componentlevels? Is the responsibility for requirements development/flowdown splitbetween the acquirer and developer, and who is responsible for whichlevels? Is there an iterative process of requirements allocation, high-leveldesign and requirements definition for the next lower level? Does the acquirer exercise responsibility for defining/baselining the
14
8/7/2019 UNit-vii MIS
15/44
requirements levels under their control? Does the acquirer exercise responsibility for monitoring the suppliersdefinition of lower-level requirements, and reviewing/ensuring thatrequired work products are being developed.? Does the acquirer exercise responsibility for monitoring the lower-level
requirements development process to ensure that requirements produced ateach level result in a system that satisfies customer/operationalrequirements? In addition to functional/performance requirements, are interfacerequirements also covered? Do requirements also include non-functional requirements such as datarights, delivery dates, milestone exit criteria, and attributes such asevolvability, maintainability and reusability? Is the same requirements process used for acquiring services instead ofproducts?
Integrated
Teaming
Does integrated teaming consider the overall scope of/requirement for
participation of stakeholders from users; acquisition executives; acquisitionorganizations; developers (primes, subcontractors, suppliers, vendors); testorganizations; and other support organizations? Should the team adopt a common process for team operation or rely oneach team member to use his/her own organizations processes? Are various team member processes compatible at the interface pointsof the processes? Do life-cycle support considerations drive the selection of processesacross the team members (tools, support data commonality/compatibility)? Are integrated information infrastructures needed tofacilitate/coordinate within/external to the team, especially for
geographically dispersed members/stakeholders?Validation Is validation performed early and continuously throughout theacquisition life-cycle? Are validation processes used to demonstrate that acquisition processwork products fulfill the acquisition strategy, and that the processes willsuccessfully acquire products/services? Are validation processes used to ensure that received products/servicesfulfill their intended use? Is the test community treated as a major stakeholder in the validationprocess, beginning with up-front planning through final system validation? Has the acquisition project defined the degree to which validation is
required, both early in the acquisition project definition and whenproducts/systems are received? Do acquisition project plans identify adequate resources to executevalidation activities?
Verification Is verification performed early and continuously throughout theacquisition life-cycle? Does the acquisition project ensure that selected work products meetproject requirements?
15
8/7/2019 UNit-vii MIS
16/44
Does the acquisition project ensure that the evolving acquired productssatisfy contractual requirements?
Transition toOperations &Support
Does acquisition planning for transition include establishing supportstrategies through organic support infrastructures, contractor logisticssupport, or other sources?
Are the roles/responsibilities of the acquirer, supporter and user definedin the life-cycle support of the system? Has the acquirer determined whether it will execute the transitionfunction directly, or as a result of the acquisition itself? Are responsibilities for capability enhancements during the supportphase defined, taking into account the magnitude/complexity of theenvisioned change? Is the organization responsible for support (i.e., level 1 maintenance)and enhancements (i.e., level 2 maintenance) explicitly identified? Has the acquisition project assigned responsibility for implementationof process improvement practices?
Is the acquisition project working with operational units to plan forproduct transition into operational use and eventual disposal of the product(or technical/functional obsolescence of the software)?
ProjectMonitoring &Control
Are monitoring/control functions implemented in the acquisition projectas acquisition planning is performed and the acquisition strategy isdefined? Does the acquisition project have internal processes, plans and workproducts that should be monitored for progress/satisfactory completion? Do the monitored work products include specifications, plans, Requestfor Proposal components, etc.? Do monitored items include staffing levels/qualifications, system
performance objectives/thresholds, infrastructure readiness (tools,networks, etc.) and other project planning activities/products? Is acquisition project risk actively identified and mitigated? Is corrective action applied when execution does not match acquisitionproject planning (e.g., internal staffing, project plan completion dates,draft/final solicitation & contract award milestone slips)? Are corrective actions required to resolve variances from acquisitionproject plans tracked to closure? After supplier selection/contract award, is monitoring and control stillapplied to the internal acquisition project After supplier selection/contract award, does monitoring and control
also cover supplier performance against the suppliers project plan?AcquisitionProcess &ProductQualityAssurance
Have the acquisition project products and processes (e.g., solicitationpackages, etc.) been evaluated? Has the solicitation package been developed per the standard/formatagreed to by the project team? Does the solicitation package conform to all applicable policies andlaws? Does the acquisition projects risk management process conform to that
16
8/7/2019 UNit-vii MIS
17/44
called out for in the risk management plan?RiskManagement
Is the acquisition strategy influenced by risk identification andestimation of risk probability/impact? Does the acquirer assess/manage overall acquisition project risks overthe duration of the project?
Does the acquirer assess/manage risks associated with supplierperformance? For acquisitions using integrated teams, are risks associated with loss ofinter- or intra-team coordination considered?
ConfigurationManagement
Are acquisition work products (e.g., solicitation packages) placed underinternal CM control? Are interim/final primary/subordinate supplier work products monitoredto ensure that project goals are met? Are provisions made for conducting CM of supplierproducts/documentation? Are methods established/maintained to ensure that acquisition project
data is complete/consistent? Has the acquisition project determined which work products should beCM-controlled?
DecisionAnalysis &Resolution
Is there a repeatable, criteria-based decision-making process used formaking critical decisions that define/guide the acquisition process itself? Is there a repeatable, criteria-based decision-making process used formaking critical decisions with the selected supplier? Does the process for decision making document the acquisition projectdecision rationale? Can criteria for critical decisions be revisited when changes/technologyinsertion decisions impacting essential requirements are considered?
Measurement& Analysis Does the acquisition project have the information it needs fordetermining status of its activities throughout the acquisition life-cycle, thesuppliers activities per contractual requirements and the status of theevolving products acquired? In acquisition projects requiring multiple products or teamingrelationships, is additional information needed/available to ensureprogrammatic, technical and operational interoperability objectives areidentified/measured/achieved?
The use of checklists such as the one above helps maintain discipline and structure in thesoftware acquisition process, and can cover any phase of a software acquisition life-cycle-Table 3: IEEE Std 1062, 1998 Edition Software Acquisition Checklists [IEEE, 1998]
17
https://goldpractice.thedacs.com/practices/api/#_[IEEE,_1998]%23_[IEEE,_1998]https://goldpractice.thedacs.com/practices/api/#_[IEEE,_1998]%23_[IEEE,_1998]8/7/2019 UNit-vii MIS
18/44
No.
Category Subcategories
1 Organizational Strategy2 Define the Software3 Supplier Evaluation Financial soundness
Experience/capability Development/controlprocesses Technical assistance Quality practices
Maintenance service
Product usage Product warranty Costs Contracts
4 Supplier/AcquirerObligations
5 Quality/Maintenance Plans Contents of a qualityplan
Contents of amaintenance plan
6 User Survey Operation Reliability
Maintenance service Performance Flexibility
Installation Costs
Security Documentation Miscellaneous
7 Supplier PerformanceStandards
Performance criteria Evaluation/test
Correction ofdiscrepancies Acceptance criteria
8 Contract Payments9 Monitor Supplier Progress10 Software Evaluation Functionality
Performance Reliability Availability Ease of modification
Serviceability Ease of installation Ease of use Adequacy ofdocumentation Cost to acquire/use
11 Software Test12 Software Acceptance Rate actions for
certification Rate remedies ifsupplier fails
Finally, the use of checklists can benefit specific monitoring and oversight activities ofsoftware acquisition such as risk management. Gallagher[Gallagher, 1997] defines asoftware acquisition risk management Key Process Area (KPA) that can be put into ahigh-level checklist form (see Table 5) in order to assess whether Goals, Activities,Commitments, etc., associated with risk management have been covered in the softwareacquisition process.Table 5. Software Acquisition Risk Management KPA [Gallagher, 1997]
18
https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]https://goldpractice.thedacs.com/practices/api/#_[Gallagher,_1997]%23_[Gallagher,_1997]8/7/2019 UNit-vii MIS
19/44
Category Goal Description Check Goals 1 SA risk management is an integral part of the projects
defined SA process ______ 2 The project identifies/deals with risk in a positive manner,
such that identification is recognized/rewarded, and results
in effective risk handling ______ Activities 1 SA risk management activities are integrated into SA
planning ______ 2 The Software Acquisition Risk Management Plan is
developed in accordance with the projects defined SAprocess ______
3 The project team performs its SA risk managementactivities in accordance with documented plans ______
4 Risk management is conducted as an integral part of thesolicitation, project performance management, andcontract performance management processes ______
5 SA risk handling actions are tracked and controlled untilthe risks are mitigated ______
Commitments 1 The acquisition organization has a written policy for themanagement of SA risk ______
2 Responsibility for SA risk management activities isdesignated ______
Abilities 1 A group that is responsible for coordinating SA riskmanagement activities exists ______
2 Adequate resources are provided for SA risk managementactivities ______
3 Individuals performing SA risk management activitieshave experience or receive required training ______
Measurement 1 Measurements are made/used to determine status of theacquisition risk management activities and resultantproducts ______
Verification 1 Acquisition risk management activities are reviewed byacquisition organization management on a periodic basis ______
2 Acquisition risk management activities are reviewed bythe project manager on both a periodic and event-drivenbasis ______
Collecting/Analyzing Meaningful Metrics:Meaningful metrics are collected and analyzed by both software acquirers and developersin order to track progress, confirm knowledge, manage risk, improve processes andensure that all stakeholders are well-informed. Meaningful and suitably tailored metricsshould cover cost, schedule, software size, requirements, tests performed/completed,number of defects detected/corrected, and quality attributes. Acquirers can apply some ofthese same metrics to assess whether the developer will be able to deliver software withincost, schedule, performance and quality constraints. The use of earned value analysis
19
8/7/2019 UNit-vii MIS
20/44
tools to track cost/schedule and help mitigate risk is an important driver in determiningmeaningful metrics, supporting the ability of a program to maintain consistent,predictable practices and acceptable process outcomes. Sharing of metrics betweendevelopers and acquirers is an important aspect of a successful software acquisitionprocess.
Leading software developers have been described as relentless in their efforts to collectmetrics to improve project processes and results[GAO, 2004]. They typically alsoestablish goals and track metrics for company-wide initiatives, such as cost reductionefforts and customer satisfaction. These leaders have continually emphasized the criticalnature of measuring processes, collecting metrics and using them to improve performancein their workforce through training. In addition, a good part of their success can beattributed to their use of an Earned Value Management System (EVMS) and a projecttime-tracking system (to record time spent on cost of quality and cost of poorquality). The cost of poor quality is a direct reflection of the relative level ofeffectiveness of an organizations software acquisition and/or development processes.Typical metrics used by leading software developers are indicated in Table 6.
Software and hardware evaluation criteria
1)
Table 6: Metrics Used by Leading Software Developers [GAO, 2004]MajorMetric
Examples Usefulness
Cost Cost per phase Effort per phase Planned vs. actual cost
Cost performance index
Products of an earned valuemanagement system, indicatingprogress towards completing software
development against the plan Poor cost performance indexindicates problems meeting cost goals May need to consider project scopereduction to meet release dates, orcanceling the program
Schedule Planned vs. actual delivery dates Schedule estimation accuracy Percentage of project on time Schedule performance index
Products of an earned valuemanagement system, indicatingachieved schedule progress againstthe plan Used over the development life-
cycle to gauge progress towarddeveloping key products or meetingcritical milestones Close attention to scheduledeviations identifies the ability tomeet project goals, and if/whenadditional resources are needed
Size Amount of new, modified and Used by management to compare
20
https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]https://goldpractice.thedacs.com/practices/api/#_[GAO,_2004]%23_[GAO,_2004]8/7/2019 UNit-vii MIS
21/44
reused code Size estimation accuracy
amount of code produced vs.estimated Changes to size estimates indicatepotential cost/schedule problems
Requirements Total requirements/features
committed for delivery Percentage of requirementscompleted Number of requirements changes byphase
Used to assess progress towards
meeting acquirers performancedemands Large number of changes, or latechanges, can impact cost/schedulecommitments Large number of changes, or latechanges, can result in software withhigher defect levels
Tests Number if tests planned, completed,passed
Percent of plan tests completed Percent of planned tests passed
Determine extent to which plannedsoftware tests have been successfully
accomplished Deviation from planned number oftests may indicate inadequate testingand subsequent quality problems,resulting in costly rework in laterproject phases
Defects Number of defects per phase Phase defect originated vs. phasefound Cost to fix defect
Severity of defect Total unresolved defects
Track software problems to (1) thephase where they were found, (2) thephase where they should have beenfound and (3) determine the cost to
fix Defects found after the phase inwhich they were created indicateperformance problems that mayincrease cost/schedule due to reworkof software and correction ofdevelopment processesToo few defects found may indicateinadequate test coverage during thetest phase, or insufficient formaltechnical review of documents in the
design phaseQuality Cost of quality efforts Cost of poor quality (rework) Number of quality goalsmissed/achieved Customer satisfaction survey results
Provides information on potentialreliability of delivered software Indicates the amount ofmoney/time invested in developmentto assure a given quality level Defects found/fixed in the phasethat they occur indicates good process
21
8/7/2019 UNit-vii MIS
22/44
quality Defects found/fixed in downstreamphases are costly/time-consuming,and indicate weaknesses in thedevelopment process that will require
corrective action
2)Table 8: Recommended COTS-Based Software System AcquisitionBest PracticesPractice CommentsBest Practices for Establishing a Program BaselinePerform Software Architecture TradeStudies
Perform with system architecture trades Include major COTS and legacy components Supports Government software architecture
baseline selection Include user in all trades
Determine Realistic, IndependentBaseline Software Estimates
Size, effort, cost and schedule COTS, reuse and newly-developed Include tasks not reflected in cost models Include COTS refresh through both developmentand sustainment
Include Software in SystemsPerformance Requirements
Prioritized requirements COTS software support requirements Specialty engineering (reliability, maintainability& availability (RMA), security, safety)
Key Performance Parameters (KPP) Open system architecture
Best Practices for Obtaining Contractual InsightRequire Key Software Technical andManagement Deliverables
Highest risk reduction potential is in plans;requirements and architecture; test plans,procedures and reports; metrics reports; delivery,installation and O&M documentation Use electronic delivery
Require Timely Electronic Access toAll Software Products
COTS evaluation trade studies Intermediate and final products (requirements,architecture and design; implementation (including
code); integration and verification testing)Require Software-Level Technicaland Management Reviews
In addition to system reviews Include COTS software experts in reviews
Best Practices for Obtaining Contractual CommitmentMandate Compliance with RobustCommercial Standards
E.g., EIA/IEEE J-STD-016 (commercializedversion of MIL-STD-498) Tailor standard for COTS-based software system
22
8/7/2019 UNit-vii MIS
23/44
developmentRequire Contractor Commitment toSoftware Development Plan (SDP)
Require SDP to include processes for COTSsoftware Require Integrated Management Plan (IMP) tohave adequate system engineering and sustainment
for COTS Include commitment to SDP in IMPBest Practices for Providing Tools for Contract ManagementIncentivize Software Quality (NotJust Cost/Schedule)
Use award and incentive fee plans Reward adherence to defined software processesand software process improvement Reward timely/acceptable response toGovernment comments Reward low rework rates Reward meeting performance requirements (e.g.,RMA) post delivery/launch
Reward architecture development that supportsCOTS-based software system evolutionMandate Periodic Team SoftwareCapability Appraisals
Relate results and improvement actions directlyto award fees Explicitly include COTS processes in appraisals
Best Practices for Selecting a Capable Software Contractor TeamEvaluate SoftwareCapability/Processes of OfferorTeams
Individual team member evaluation isinsufficient Evaluate software capability as a separatesubfactor under the Mission Capability factor Weight according to software risk
Evaluate Teams Proposed Softwareand Related Processes
Corporate and past project process evaluation isinsufficient Include COTS software, systems engineering andlogistics processes
Evaluate Realism of Cost andSchedule Bids
Be suspicious of productivity extremes, COTS,reuse, low lines of code, and short integrationtimes Ensure that all COTS software tasks are includedin the cost and schedule bid Ensure bid contains sufficient cost and schedulemargin
Evaluate Software and HardwareArchitecture with System Design
Best Practices for Performing Technical Product ReviewFocus Technical Review Resourceson Areas of Highest Risk
IPTs, Technical Interchange Meetings (TIMs),working groups, per reviews, etc. Software-level technical reviews High risk/critical software products (includingCOTS software)
23
8/7/2019 UNit-vii MIS
24/44
Key software technical deliverablesInclude Users/Operators in AllTechnical Review Activities
Ensure users/operators understand the evolvingdesign, including COTS software capabilities andimpacts on O&M
Monitor Software Integration and
Verification Adequacy
Begin at the build level
Focus on areas of highest risk Focus on early performance analysis results andmeeting KPPs Ensure that COTS software performance ismeasured Ensure requirements allocated to COTS softwareare verified
Best Practices for Performing Software Process ReviewReview Effectiveness of TeamsDefined Software and RelatedProcesses
Identify process deficiencies (especially acrossteam boundaries and with COTS products) Assist with process improvement
Individual Level 2 and 3 CMMI/CMMcompliance may not be sufficient
Perform Periodic Team SoftwareCapability Appraisals
During contract performance Support for significant program or award feemilestones Explicitly include COTS processes
Review Teams Adherence toDefined Software and RelatedProcesses
Identify adherence deficiencies Assist in deficiency correction
Best Practices for Managing the ContractUse Incentive/Award Fees
Aggressively
Motivate good software and related practices
Focus on quality an d architectureApply Proactive QuantitativeManagement
Ensure a comprehensive software/systemsmetrics program balanced across informationcategories (include leading quality indicators, e.g.,rework) Perform cross-metric analysis Earned value alone is insufficient
Ensure Adherence to Software-Inclusive Requirements
Especially RMA, safety and security Especially COTS software supportability
Perform Periodic IndependentAssessments
Support for significant program or award feemilestones Act aggressively on findings
Best Practices That Span the Life-cycleSoftware Acquisition RiskManagement
Continuous software acquisition riskmanagement across all acquisition organizationlevels Program-level risk management and contractordevelopment risk management are necessary, but
24
8/7/2019 UNit-vii MIS
25/44
not sufficient Establish management reserves consistent withsoftware risks (including/especially COTSsoftware risks)
Software Systems Acquisition Integrate software acquisition with the system
acquisition process (from mission needsidentification through system retirement, especiallyduring pre-contract activities)
Figure 7: Use Quality Function Deployment to Translate Successful Acquisition
3) Phase Containment Matrix:A Phase Containment Matrix is a well-known tool for identifying and reducing softwaredefects across the various phases of a software life-cycle. It is constructed by identifying
the phases during which a defect is inserted as the column headings, and the phaseswhere the defect was detected as the row headings (see Figure 8).
Figure 8: Phase Containment Matrix for Tracking Software Development QualityWithin each cell, the number of defects that are detected at that phase in the software life-cycle are entered. There is a basic, yet critical, assumption that root-cause defect analysishas been performed in order to accurately determine in which phase of the life-cycle thedefect was actually inserted.In-phase defects are often not counted as rework, as these defects are assumed to be anatural part of the process. For example, the rework required to correct the 25 defects
25
8/7/2019 UNit-vii MIS
26/44
that were introduced during the requirements phase, but were also detected andeliminated in the requirements phase, would not be counted, even though there is stillpotential cost and schedule impact associated with the rework. Ignoring the cost of in-phase rework deludes both the customer and the organization, and deprives theorganization of an opportunity to improve its in-phase processes. Analyzing the cost of
performing rework on 25 in-phase requirements defects might provide significantjustification for improving an acquisition or development process to reduce the numberof introduced defects.
Out-of-phase defects are generally classified as rework and, just as generally, are going tocost more to fix than in-phase defects. The larger the span (in project phases) betweenthe insertion point and the detection point, the more expensive it is going to be, on a perdefect basis, to rework and fix it. This more expensive factor can be an order ofmagnitude of cost between each phase.The concept of a Phase Containment Matrix could, with some adaptation, also be appliedto improve software acquisition processes by defining an appropriate set of processes
within which software acquisition defects can be detected. Figure 9 provides a simpleexample of how this might be accomplished.
26
8/7/2019 UNit-vii MIS
27/44
ANTICIPATED BENEFITS OF IMPLEMENTATION: of SW and HWSuccessful implementation of software acquisition process improvement can result in:
Realistic, incremental and continuous improvements to software acquisitions Aprocess of evolutionary software acquisition avoids the temptation to succumb tounrealistic expectations in an attempt to force revolutionary changes in softwaredevelopment. Adherence to well-understood, well-defined, manageable requirements Anevolutionary software acquisition process ensures that software development is limited tothings that are possible to manage. Only requirements that are well-defined and well-understood can be adequately managed. Following of disciplined, structured software management and developmentprocesses that provide predictable results Disciplined, structured software managementand development process phases that end in a management review/gate ensure that the
acquisition will remain on track, and that project course corrections can be identified andimplemented effectively. Well-informed, well-disciplined and knowledgeable software acquisition teams The suitably tailored collection and use of process metrics ensures that the softwareacquisition is adequately tracked; that project knowledge is confirmed and shared amongstakeholders; that acquisition risk is effectively managed; and that software acquirers arefirmly integrated into the continuous improvement process.
27
8/7/2019 UNit-vii MIS
28/44
Increased probability of software acquisition successes and best value awards Heavy reliance on configuration management, risk identification/management, peerreviews and quality assurance, coupled with earned value management techniques, helpsensure a successful and improving acquisition process.
DETAILED CHARACTERISTICSKey Characteristics of the Acquisition Process Improvement Gold PracticeCharacteristic CommentsManageableDevelopmentEnvironment isDefined andImplemented
Business decisions must be made to invest suitabletime/resources in achieving high levels of software process maturity Honest relationships between the acquirer and developer must beestablished There must be a mutual understanding of software requirementsbetween the acquirer and developer in order to optimize
setting/managing requirements There must be a match between available resources andrequirements, supported by systems engineering analysis and trade-off analyses that consider the performance, cost and scheduleimpacts of major changes to software requirements Technical and administrative knowledge must be obtainedearly in the software development process Sufficient requirements knowledge must be demanded by PMsand other stakeholders at key process points In order to ensure that software requirements are appropriatelyset/managed, it is necessary to document that software requirements
are achievable based on systems engineering knowledgeDisciplinedDevelopmentProcesses,Supported by GatedReviews
Processes that meet program needs must be established andadhered to Acquirers should develop a list of tailored systems engineeringdeliverables that include software based on the results of requiredengineering activities at the appropriate stages/phases of systemdevelopment A sufficient amount of software development time should bedevoted to requirements-setting activities Well-written and achievable requirements must be developedand managed/controlled before actual software design begins
Lower-level software design elements should be defined beforeany coding begins The number and timing of requirements changes should beaggressively managed Test plans that are developed should be based on a stable set ofrequirements Ensure that the software design is mature before approvingrelease/production
28
8/7/2019 UNit-vii MIS
29/44
All production processes must be under control beforeproduction begins Provide for gated reviews of systems engineering baselines onan event-driven basis
Established and
Leveraged Metricsare Defined andUtilized
The collection and reporting of metrics related to software
acquisition and development should be required internally, and fromcontractors, in order to ensure adequate oversight knowledge ofsoftware-intensive acquisitions There should be relentless pursuit of tailored metrics that arederived from effective processes Require software contractors to collect/report cost, schedule,size, requirements, test, defect and quality metrics on a per-monthbasis, if appropriate, and before program milestones Ensure that contractors are using an earned value system thatreports work at a suitably detailed level Software acquisition should be enforced with suitable controls
and performance incentives in DoD acquisition policy, softwareacquisition improvement plans and software development contractsUse of AppropriateTools andTechniques toSupport AcquisitionActivities
Formal processes such as SA-CMM and CMMI-AM provide aframework for implementing a structured software acquisitionimprovement process Tools and techniques to be considered include documenttemplates, documentation standards, checklists, QFD, AHP, PhaseContainment Matrices, earned value tracking, knowledge-sharingsystems, lessons learned repositories, past performance databases,centralized acquisition resources, software tools that support riskmanagement, configuration management, source selection, decision-
making, etc.Use of IntegratedProcess Teams(IPTs) That InvolveAll Stakeholders(including EndUsers)
Integrated Process Teams conduct process analyses and identifybottlenecks and non-value-added process structure and flow, with aconstant focus on measurement/improvement of software acquisitionprocesses. IPTs keep the focus on the customers needs and requirements Effective IPTs contribute to collaborative requirementsdevelopment, an important attribute of the Air Force AgileAcquisition initiative Teams should be cross-functional and multi-disciplinary, andthere must be leadership/senior management commitment to support
the team (with appropriate allocation/delegation of decision-making) Organizational structures should be defined such that teamperformance is suitably rewarded
RELATIONSHIPS TO OTHER PRACTICES:
29
8/7/2019 UNit-vii MIS
30/44
The Figure below represents a high-level process architecture for the subject practice,depicting relationships among this practice and the nature of the influences on thepractice (describing how other practices might relate to this practice). These relationshipstatements are based on definitions of specific best practices found in the literature andthe notion that the successful implementation of practices may influence (or be
influenced by) the ability to successfully implement other practices. A brief descriptionof these influences is included in the table below.
Process Architecture for the "Acquisition Process Improvement" Gold Practice
30
8/7/2019 UNit-vii MIS
31/44
Other general criteria for evaluation of SW and HWLen Bass
Feb 2, 2010 2010 Carnegie Mellon University Externally visible properties Relationships among elements
Multiple different structures
Uses Guide to construction Artifact for analysisQuality attributes are properties of a system such as Functionality Reliability Usability Efficient 2010 Carnegie Mellon University Maintainability
Portability
Acquisition perspective
Given the importance of software architecture to software development,there are three portions of the life cycle where architectural knowledgeis important Requirements Proposal evaluation Architecture evaluation of system during developmentRequirements information must include quality attribute requirements
Business/mission goals must be available to proposers/designers
During Development
Acquirers should evaluate architecture during development To ensure designed system satisfies mission goals To ensure conformance To enable adaptation to changes in mission
Evaluating integrity and quality f SW and HW
Requirements analysis
Risk assessment
Development analysis
Code review
31
8/7/2019 UNit-vii MIS
32/44
Independent testing
Contract verification
Algorithm analysis
Software metrics
32
8/7/2019 UNit-vii MIS
33/44
Essentials In system Implementation
Adaptive tailoring to the scope of the target program
Integrated working closely with development teams
Self-improving
Essentials in System Implementation
Depending on the unique circumstances of the implementation process, the status of data
compilation, and the organizational climate, increased productivity is normally reached
between the second and fifth year of implementation. This is identified by the threshold
point. Again, this is dependent on a variety of factors including:
the skills and experience of the staff involved;
the priority and commitment by the organization;
33
8/7/2019 UNit-vii MIS
34/44
the implementation strategy; and
the status of data compilation.
The Implementation Plan
Implementation can be seen as a six phase process. They are:
Creating an awareness GIS needs to besoldwithin an organization. Theeducation of staff is very important. Depending on the way in which GIStechnology is being introduced to the organization the process for creating anawareness may differ. Technical workshops are often appropriate when a top-down approach exists, while management workshops are often more relevantwhen a bottoms-up approach exists. Education of the new technology should
focus on identifying existing problems within an organization. These often helpjustify a GIS acquisition. They include :
spatial information is poorly maintained or out ofdate;
spatial data is not recorded or stored in a standardway;
spatial data may not be defined in a consistentmanner, e.g. different classifications for timberinformation;
data is not shared between departments within anorganization;
data retrieval and manipulation capabilities areinadequate to meet existing needs; and
new demands are made on the organization thatcannot be met with existing information systems.
Identifying System Requirements
The definition of system requirements is usually done in a user needs analysis. A userneeds analysis identifies users of a system and all information products required by those
users. Often a prioritization of the information products and the data requirements of
those products is also undertaken. A proper user needs analysis is crucial to the
successful evaluation of GIS software alternatives.
34
8/7/2019 UNit-vii MIS
35/44
After user needs have been identified and prioritized they must be translated into
functional requirements. Ideally, the functional requirements definition will result in a set
of processing functions, system capabilities, and hardware requirements, e.g. data
storage, performance. Experienced GIS consultants often play a major role in this phase.
System Evaluations
Evaluating alternative hardware and software solutions is normally conducted in several
stages. Initially a number of candidate systems are identified. Information to support this
process is acquired through demonstrations, vendor literature, etc. A short listing of
candidates normally occurs based on a low level assessment. This followed by a high
level assessment based on the functional requirements identified in the previous phase.
This often results in a rating matrix or template. The assessment should take into account
production priorities and their appropriate functional translation. After systems have beenevaluated based on functional requirements a short list is prepared for those vendors
deemed suitable. A standard benchmark, as discussed earlier, is then used to determine
the system of choice.
Justifying the System Acquisition
The proper justification of the chosen system requires consideration of several factors.
Typically a cost-benefit analysis is undertaken to analyze the expected costs and benefitsof acquiring a system. To proceed further with acquisition the GIS should provide
considerable benefits over expected costs. It is important that the identification of
intangible benefits also be considered.
The justification process should also include an evaluation of other requirements. These
include data base development requirements, e.g. existing data versus new data needs and
associated costs; technological needs, e.g. maintenance, training, and organizational
requirements, e.g. new staff, reclassification of existing job descriptions for those staff
who will use the GIS.
System Acquisition and Start Up
After the system, e.g. hardware, software, and data, is acquired the start up phase begins.
This phase should includepilot projects. Pilot projects are a valuable means of assessing
progress and identifying problems early, before significant resources have been wasted.
35
8/7/2019 UNit-vii MIS
36/44
8/7/2019 UNit-vii MIS
37/44
Smith-son, 1988). Further Hirschheim & Smithson means that this can have majornegative conse-quences in terms of decreased user satisfaction but also broaderorganizational consequences in terms of system value.
We agree with the criticism of Hirschheim & Smithson, but when analysing goal-based
evaluation in an ideal typical way there is no imperative relation be-tween a focus ontechnical and economical aspects and goal-based evaluation. Of course, the stated goalscan be of a human or organisational character. However, the traditional way ofunderstanding goal-based evaluation is often related to harder measurable goals.
Further, there is no imperative relation between a goal-based approach, and a quantitativeprocess. A judgement of, if the goals have been fulfilled can be evaluated with aqualitative process. As we see it, the differences between a quantitative and qualitativestrategy is that the quantitative strategy aims to decide if the goals are fulfilled and whichgoals that are ful-filled. The fulfilment of the goals will be expressed in quantitative
numbers.
There are also goals of more social or human character. The fulfilment of this types ofgoals is preferably expressed in qualitative terms. The qualitative process has also,besides the if- and which questions, a better possibility to describe how the goals arefulfilled. This means that the qualitative approach aims at achieving richer descriptions.
The goals that are used for evalua-tion are derived from an organisational context. Thatmeans that they are situationally appli-cable, which means that they act like specificbusiness goals.
The basic strategy of this approach is to measure if predefined goals are fulfilled or not;to what extent and in what ways. The approach is deductive. What is measured dependson the
37
8/7/2019 UNit-vii MIS
38/44
character of the goals and a quantitative approach as well as qualitative approach couldbe used. In this paper we adopt the concept of goal-based evaluation from Patton (1990)in order to identify this approach.
2.2 Goal-free evaluation
The second identified approach is a more interpretative approach (e.g. Remenyi, 1999;Wal-sham, 1993). The interpretative perspective views IT-systems as social systems thathave in-formation technology embedded into it (Goldkuhl & Lyytinen, 1982). The aim ofinterpretive evaluation is to gain a deeper understanding of the nature of what is to beevaluated and to generate motivation and commitment (Hirschheim & Smithson, 1988).
The involvement of a wide range of stakeholder groups is essential to this approach ofevaluation. This can also be a practical obstacle where time or resources for theevaluation are short. Patton (1990) uses the term goal-free evaluation. Goal-freeevaluation is defined as gathering data on a broad array of actual effects and evaluating
the importance of these effects in meeting demonstrated needs (Patton, 1990, Scriven,1972). The evaluator makes a deliberate attempt to avoid all rhetoric related to programgoals; no discussion about goals is held with staff; no program brochures or proposals areread; only the programs outcomes and measurable effects are studied. The aim of goal-free evaluation is to (Patton, 1990):
1) avoid the risk of narrowly studying stated program objectives and thereby missingimportant unanticipated outcomes
2) remove the negative connotations attached to discovery of unanticipated effect:The hole language of side-effected or secondary effect or even unanticipatedeffect tended to be a put-down of what might well be a crucial achievement,especially in terms of new priorities.
3) eliminate the perceptual biases introduced into an evaluation by knowledge ofgoals; and
4) maintain evaluator objectivity and independence through goal-free conditions
2.3 Criteria-based evaluation
The third identified approach is a criteria-based approach. There are lot of criteria-basedap-proaches around such as checklists, heuristics, principles or quality ideals. In the areaof Human-Computer Interaction you can find different checklists or heuristics (e.g.Nielsen, 1994; Nielsen, 1993, Shneiderman, 1998). What is typical for these approachesis that the IT-systems interface and/or the interaction between users and IT-systems actsas a basis for the evaluation together with a set of predefined criteria. More actionoriented quality ideals and principles for evaluation can be found in Cronholm &
38
8/7/2019 UNit-vii MIS
39/44
Goldkuhl (2002) and in gerfalk et al (2002). The basis for these action oriented ideals isto understand if and how the IT-system support the actions performed in the business (seediscussion of IT-systems in section 3.1)
The criteria used are grounded in and derived from one or more specific perspectives ortheo-ries. For example, the criteria in Nielsens (1994) checklist are derived fromcognitive and computer science. The action oriented ideals are mainly derived fromlanguage action theory but also inspired by usability issues. Using criteria means to setfocus on certain qualities that according to the perspective is important to evaluate. At thesame time the attention accord-
39
8/7/2019 UNit-vii MIS
40/44
ing to the criteria also de-emphasize other qualities. The criteria chosen governs theevalua-tors attention and thereby the kind of knowledge the evaluator achieves.
Another difference in comparison to goal-based evaluation is that the criteria thatare used are not derived from a specific organisational context. That means that they aremore general ap-plicable (see section 2.1). Ideal typically, the basic strategy of criteria-
based evaluation is pure deductive. The word criteria is often used in relation topreordinate designs, and the use of this term has a hard scientific feel which supportsthe tendency to prioritize technical and quantitative data
40
8/7/2019 UNit-vii MIS
41/44
Different Units topics
a) MIS Approach
MIS is generally used by medium and larger scale organizations. However, smallorganizations are yet to understand its application. There is dire need to build upcomputer culture by properly disseminating information about computer applications andits benefits.
Implementation of MIS can be achieved by using any of the methods such as direct,parallel, modular or phase in.
Direct Approach
Direct installation of the new system with immediate discontinuance of the old existingsystem is reffered as cold turnkey approach. This approach becomes useful when thesefactors are considered.
1. The new system does no replace the existing system.2. Old system is regarded absolutely of no value3. New system is compact and simple.4. The design of the new system is inexpensive with more advantages and less riskinvolved.
Parallel Approach
The selected new system is installed and operated with current system. This method isexpensive because of duplicating facilities and personal to maintain both the systems. Inthis approach a target date must be fixed when the operations of old system cease andnew one will operate on its own.
Modular Approach
This is generally recognized as Pilot approach, means the implementation of a systemin the Organization on a piece-meal basis.
This has few advantages / merits
1. The risk of systems failure is localized2. The major problem can be easily identified and corrected before furtherimplementation.3. Operating personal can be trained before system is installed in a location.
41
8/7/2019 UNit-vii MIS
42/44
Phase-in-Implementation
This approach is similar to modular method but it differs because of segmentation ofsystem, however, not the organization. It has advantages that the rate of changes in agiven Organization can be totally minimized and the data processing resource can be
acquired gradually over a period of time. System exhibits certain disadvantages such aslimited applicability, more costs incurred to develop interface with old system and afeeling in the Organization that system is never completed.
b) Implementation Procedures
Planning the Implementation
After designing the MIS it is essential that the organization should plan carefully forimplementation. The planning stage should invariably include the following:
1. Identification of tasks of Implementation
Planning the implementation activities, acquisition of facilities, procedures development,generating files and forms, testing the system and evaluating and maintenance of thesystem.
2. Relationship establishment among the activity
Network diagram must be prepared to correlate concurrent and sequential activities.
3. Establishing of MIS
For monitoring the progress of implementation and for proper control of activities,efficient information system should be developed.
4. Acquisition of Facilities
For installation of new system or to replace current system the manager should prepare aproposal for approval from the management by considering space requirement movementof personal and location for utility outlets and controls.
5. Procedure Development
This is an important stop for implementation of the system including various activitiessuch as evaluation selection of hardware, purchase or development of software, testing
42
8/7/2019 UNit-vii MIS
43/44
and implementation strategies.
6. Generating Files and Forms
The MIS manager should generate files and formats for storing actual date. This requires
checklist data, format date storage forms and other remarks in data base.
7. Testing of the System
Test should be performed in accordance with the specifications at the implementationstage consisting of component test sub system test and total system test. Evaluation and maintenance of system
The performance should e evaluated in order to find out cost effectiveness and efficacy ofthe system with minimum errors due to designs environmental changes or services.
c) Software Maintenances
The proper maintenance is the enigma of the system development and it holds softwareindustry captive lying up programming resources. There are some problems inmaintenance such as regarding it as non rewarding non availability of technicians andtools no cognizance of users about maintenance problem and cost lack of standardprocedures and guidelines. Most programmers feel maintenance as low level drudgery. Ifproper attentions is paid over a period of time eventually less maintenance is required.
Types of Maintenance
The maintenance of system are classified into corrective/adaptive/perfective. Correctivemaintenance means repairing process or performance failures. Adaptive maintenancemeans changing the programming function whereas perfective maintenance deals withenhancing the performance or modifying the program.
Primary Activities of a Maintenance Procedure
Documentation is major part of maintenance in system development. Maintenance staffreceives requests from the authorized user. Programming library should be maintained.
Reduction in Maintenance Costs
Several organizations having MIS generally go in for reducing maintenance costs and itconsists of three major phases.
1. Maintenance management audit through questionnaires and interviews.2. Software system audit.3. Software modification.
43
8/7/2019 UNit-vii MIS
44/44
Evaluation Methods
Evaluation of the MIS in an organization is integral part of the control processes. Thereare several evaluation approaches such as quality assurance review compliance of auditsbudget performance review computer personal productivity assessment computer
performance evaluation service level monitoring user audit survey post installationreview and cost benefit analysis.
Evaluation performance measurement can be classified into two classes as effectivenessand efficiency. The relationship between effectiveness and efficiency is that the format isa measure of goodness of out put and the latter is a measure of the resources required toachieve the output.