International Conference OnSoftware Testing Analysis & Review

May 16-20, 2005Orlando, FL USA

F1

5/20/2005 10:00 AM

LEGAL COMPLIANCE IN QUALITYASSURANCE

Elle RinghamFidelity National Financial

BIOPRESENTATIONPAPER

Elle Ringham, J.D. Elle Ringham has been involved in Quality Assurance and Quality Management since 1990. Ms. Ringham graduated Law School in 2002, and has since incorporated compliance, auditability, SLA enforcement/measurement, etc. into her Quality Assurance practice. Elle considers education of all groups involved, coupled with a structured process improvement, to be the most effective way to introduce true Quality Assurance/Quality Management. Her approach ensures buy-in and support from everyone… stakeholders, executives, corporate counsel, developers, and QA resources.

Welcome!Welcome!

Legal Compliance in Quality Assurance

AgendaAgenda

What this lecture covers… what it doesn’tWhat is Legal ComplianceHow QA Fits InWhere Do You StartWhat Do You AskHow Do You Facilitate Compliance and AuditabilityTemplates and ArtifactsContact Information

What This Lecture Covers… What it Doesn’tWhat This Lecture Covers… What it Doesn’t

Will CoverDetermine if compliance issues applyAsking the right questionsHow to capture and measure auditability

Won’t CoverHOW to test various legal issuesIF a legal issue applies to your applicationSpecific legal adviceSpecific compliance issues

What is Legal Compliance?What is Legal Compliance?

Legal issues: State and FederalAccountabilityAuditabilityLegal CounselDue DiligenceContracts, standards, expectations

Standards Requirements

Process Other SLA’s

Client needs

AuditBudget

Federal Statutes

How It Fits InHow It Fits In

User Acceptance Testing

Testing Lab, Multiple platform

Standards, Process

Improvement

Functional and Negative testing

System Integration Testing

Requirements/Use Cases to Test

Cases

Defect TrackingAutomation and Regression

Load/PerformanceTesting

Test Planning 1. Bringing QA from a testing group into (true) Quality Assurance

2. Quality Management

3. Higher skill set required

4. Requires education with stakeholder and early introduction into project

Where Do You StartWhere Do You Start

Assess your business needAssess how your application addresses the needReview information with PMO and StakeholdersResearch legal issuesDiscuss findings with counselResearch audit guidelinesAssess appropriate QA efforts

Industry, GuidelinesBusiness

Data, Communication, Commerce

Ensure coverage and

UnderstandingState,

Federal, Etc.

Elements of audit

Merge Technology with Audit

Process, Metrics, Reporting

What Do You AskWhat Do You Ask

StakeholdersWhat are the known

compliance concerns?ExpectationsHow are these issues

addressed in other function?

Define known and foreseeable risks

Mitigation plan for risk(s)Define resources,

locations, tasks and utilization

PMOWhat other functional teams work

with complianceAdd tasks into project planEnsure time added to project plan for

researchDeviations expressed as impacts and

risks; also noted within SQA Test Plan and Testing Report

Ensure time added to project plan for corporate counsel

OtherHow will I add this to the Test PlanHow will I audit the elements of the statues

(guidelines, laws, etc.)What type/form of results will I need to compileWhere must the information be storedMust the information be publishedIs anyone required to review the results; who?Keep Risks and Issues open for upcoming or similar

projects

Corporate CounselWhat are the known compliance concerns?Do we have SLA’s or other contracts to

audit?What are the elements of the statute, law,

etc. that we need to audit?Explain some current case precedent of

these compliance issuesWhat do you require from other areas of

the company?Are you familiar with how technology

handles data?

Developers/DBAsHow does the design handle

process/business flowHow is data capturedWhat standards are used for securityAsk about design patternsAsk to see all modelsThere will be specific questions

associated with your compliance issues too!

QA GroupWhere in the Process does this fit?Who owns this area?How to we capture metrics?Note impacts, risks, mitigation steps taken

How Do You Facilitate Compliance and Auditability

How Do You Facilitate Compliance and Auditability

45%

25%

20%

10%Test Cases to Elements ofAudit

UAT

Development Testing

Reporting

Templates and ArtifactsTemplates and Artifacts

Mapped areas of coverage

Metrics of coverage per release (functional)

Load/Performance

Data pools, and negative efforts

Standards, Best Practice and Due Diligence

Templates (Cont.)Templates (Cont.)

Sarbanes-Oxley Template

Microsoft Excel Worksheet

Artifacts (Cont.)Artifacts (Cont.)

Example of Sarbanes-Oxley Document

Microsoft Word Document

Contact InformationContact Information

www.SANS.comhttp://www.developer.com/java/ent/print.php/3320861http://www.softlanding.com/sox/docs/workingguide.pdfhttp://www.gain2.org/sox404toolsum.htmwww.FindLaw.com

Elle.Ringham@fnf.com

Questions?Questions?

Thank you for your time!I cannot answer your legal questions. Please seek counsel for your specific needs.

Elle Ringham, J.D.

Quality Assurance Office

Legal Compliance in Quality Assurance Elle Ringham, J.D. Spring 2005 Biography: Elle Ringham has been involved in Quality Assurance and Quality Management since 1990. Ms. Ringham graduated Law School in 2002, and has since incorporated compliance, auditability, SLA enforcement/measurement, etc. into her Quality Assurance practice. Elle considers education of all groups involved, coupled with a structured process improvement, to be the most effective way to introduce true Quality Assurance/Quality Management. Her approach ensures buy-in and support from everyone… stakeholders, executives, corporate counsel, developers, and QA resources. The law and Quality Assurance has been a misunderstood marriage. Using the definitions and practices of law within the detailed, methodical, approached of Quality Assurance, organizations can increase effectiveness of production. It takes a holistic approach to understanding expectations in order to increase the actual (and perceived) level of quality. This is especially true when you marry technology and the law. In the last few decades, The Department of Defense and the Department of Justice have understood the need for this marriage. However, their approach was to find technology issues (be it in the form of risks or dependencies) and adjust our legal system (and responses) accordingly. We in the civilian field aren’t blessed with such a luxury; thus, we educated ourselves on the legal issues and add this information to our process. The following follows the “Who, What, Where, When, How” approach. When one is entering into an unknown domain, “where to start” is often the most difficult question to answer. Use the screen shots as a reference to the class taken. Additional information follows each slide.

Determining whether legal issues apply to your development efforts isn’t always simple. There may be obvious factors: Your efforts are within a well regulated industry, you are aware of Service Level Agreements, you are aware of state or federal agencies which oversee an aspect of your industry… etc. However, it may not be so obvious… you may have an eCommerce site, your portal collects information, you produce propriety software only, etc. Asking the right questions will certainly help, but what you do what the answers is equally important. The QA group will now take these answers and create templates for measurement and metrics, auditability metrics, and reports. Only your corporate counsel will know for sure whether a particular legal issue applies to your organization. Detailed legal advice needs to come from within, not a class or lecture like this. Although your research should be thorough, and your incorporation of legal elements into the QA process well defined, the actual legal elements are determined by legal counsel and state/federal agencies.

Legal compliance is the taking of a law, statute, etc. and mapping the elements of that law to areas of our technology. This can include, but is not limited to, individual elements of development, as well as the overall architecture, data acquisition, data repositories, security, and archiving. You will be mapping the accountability of various functions to their legal counterparts. These mapped elements allow for one level of auditability. It is with “first pass” that you begin to add depth to the auditability of compliance and software engineering. Planning with your company’s legal counsel involved will become standard practice. Due diligence will mean more than a phase within vendor selection! Legalese is your new second language (third, fourth… fifth). Although this may be new to your group, it will become second nature.

Adding Compliance and Audit brings QA from a Testing group to a true Software Quality Assurance/Quality Management group. It allows a greater degree of mapping and coverage from the standard Requirements Based approach. Your team will require a different skill set than you may have required before the introduction of audit. Understanding the law, dissecting elements, and mapping to the technology aren’t easy. Like most detailed tasks, they require experience and education to perform them effectively. The initial skills you should look for include an advanced reading and comprehension level, advanced degree preferred, and patience. Team members involved in this task should enjoy research, reading, writing, and multiple conversations about the same subject. Although with some internal education, the QA team will also need to educate external functional groups (including stakeholders). Process change is necessary for compliance efforts to be added to the SDLC, and education of all parties makes buy-in and support easier. Begin with the basics: your QA Process, what compliance issues pertain to your development efforts or product, how you can measure compliance, how will you report (metrics), and what their involvement is. Prepare to take the “steps” approach. Small, manageable steps… then onto to larger steps.

Your business (Industry) has areas of compliance or audit in which they must conduct their efforts. Find what these are. Once known, research and discussion with counsel will help identify how this pertains to the development efforts.

Record the answers given. They need to be objective measurable results. Look for “pass/fail” values.

Like all requirements, you will map your test cases to the elements of the law. As an example, the elements of a valid contract include offer and acceptance, consideration, competent parties, proper subject matter, mutual right to remedy, and mutual obligation to perform. Once you know what the elements are, you map you test cases and determine pass/fail criteria. Users, which may include corporate counsel, will perform the User Acceptance Testing aspect. They should validate expectations and auditability. Development efforts may be required in some areas of testing (SOX and Security is one example). Ensure that the testing is documented, including expected and actual results. Finally, it is the reporting aspect that allows for audit control and the element of true compliance. It’s not what you do, or say that you do, but what you can prove you did!

Remember that compliance may reach beyond just a requirements validation. Your Load and Stress efforts may be required; negative testing may be required. This relates to the practice of Due Diligence. Due diligence is used most often in connection with the performance of a professional or fiduciary duty, or with regard to proceeding with a court action. Due care is used more often in connection with general tort actions.

Such diligence as a reasonable person under the same circumstances would use. Use of reasonable but not necessarily exhaustive efforts called also reasonable diligence

The care that a prudent person might be expected to exercise in the examination and evaluation of risks affecting a business transaction

The process of investigation carried on usually by a disinterested third party (as an accounting or law firm) on behalf of a party contemplating a business transaction (as a corporate acquisition or merger, loan of finances, or esp. purchase of securities) for the purpose of providing information with which to evaluate the advantages and risks involved (the greatest exposure…for failure to conduct adequate due diligence arises in the context of public offerings of securities - G. M. Lawrence)

The defense (as to a lawsuit) that due diligence was conducted