+ All Categories
Home > Documents > Software testing techniques Software testing techniques Software Testability Presentation on the...

Software testing techniques Software testing techniques Software Testability Presentation on the...

Date post: 18-Jan-2016
Category:
Upload: erick-barber
View: 234 times
Download: 0 times
Share this document with a friend
29
Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology
Transcript
Page 1: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Software testing techniquesSoftware testing techniques

Software Testability

Presentation on the seminar

Kaunas University of Technology

Page 2: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Motivation

Poor testability

Ineffective testing

Severe penalties

Page 3: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

What is Software Testability?

•  degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context.

Page 4: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Software Testability …

• ... is not an intrinsic property of a software artifact • …can not be measured directly (such as software size). • … is an extrinsic property which results from

interdependency of the software to be tested and the test goals, test methods used, and test resources (i.e., the test context)

Page 5: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Lack of testability is bad. Why?

• A lower degree of testability results in increased test effort.

• In extreme cases a lack of testability may hinder testing parts of the software or software requirements at all.

Page 6: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Background

• properties of the software requirements• properties of the software itself (such as size, complexity

and testability)• properties of the test methods used• properties of the development- and testing processes• qualification and motivation of the persons involved in

the test process

Page 7: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Testability of Software Components

• Controllability• Observability• Isolateability• Separation of concerns• Understandability• Automatability• Heterogeneity

Page 8: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Controllability

•  The degree to which it is possible to control the state of the component under test (CUT) as required for testing.

Page 9: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Observability

• The degree to which it is possible to observe (intermediate and final) test results.

Page 10: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Isolateability

• The degree to which the component under test (CUT) can be tested in isolation.

Page 11: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Separation of concerns

• The degree to which the component under test has a single, well defined responsibility.

Page 12: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Understandability

• The degree to which the component under test is documented or self-explaining.

Page 13: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Automatability

• The degree to which it is possible to automate testing of the component under test.

Page 14: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Heterogeneity

• The degree to which the use of diverse technologies requires to use diverse test methods and tools in parallel.

Page 15: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Testability of Requirements

Requirements need to fulfill the following criteria in order to be testable:– consistent– complete– unambiguous– quantitative – verifiable in practice 

Page 16: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Improving software testability

• Test-driven development• design for testability (similar to design for test in

the hardware domain)

Page 17: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

So which system is “testable system”?

Roy Osherove: “For each logical part of the system, a unit test can be written relatively easily and quickly that satisfies all the following PC-COF rules at the same time:

 – Partial runs are possible– Consistent results on every test run– Configuration is unneeded before run– Order of tests does not matter– Fast run time”

Page 18: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Consistent results on every test run

• The test always has to fail or pass, but never switches between modes until a bug is fixed.

Page 19: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

What’s the problem with this code?

  public bool CanPurchase(Person p){            if (!(PersonValidator.IsValid(p))){                return false;            }             if (p.SSID != null &&                p.SubscriptionType != "Canceled"

&& p.CreditOnFile > 0){

                return true;            }             return false;        }  

Page 20: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Interfaces make code more testable

IValidator m_validator;public void SetValidator(IValidator validator) {      m_validator = validator; }               public bool CanPurchase(Person p) {     if (!(m_validator.IsValid(p))) {            return false;     }      if (p.SSID != null &&  p.SubscriptionType !=

"Canceled" &&  p.CreditOnFile > 0) {            return true;      }             return false;}

MyFakeValidator val = new MyFakeValidator();

val.whatToReturn = true; PersonLogic logic = new

PersonLogic();logic.SetValidator(val); Person p = new Person();bool result =

logic.CanPurchase(p);Assert.IsFalse(result);

Page 21: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Configuration is not needed, because…

• … ability to configure a class at runtime is an important one for unit tests.

• … if code requires external configuration before it is tested, it will take more time to create tests for that code.

• … it can also make the tests less manageable and not as easy to write.

Page 22: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

What’s the problem here?

public bool IsConnectionStringValid() {   string connString =         ConfigurationSettings.AppSettings["conString

"].ToString();         //do some stuff         return true; }

Page 23: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Use virtual method

Class ConnectionHelper{public bool IsConnectionStringValid() {         string connString = getConnectionString();            //do some stuff            //...            return true;} protected virtual string getConnectionString(){            return

ConfigurationSettings.AppSettings["conString"].ToString(); }

}

Page 24: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Derive and override it in testable class

TestableConfigBasedClass myClass = new TestableConfigBasedClass();myClass.mConnectString = "bad string";Assert.AreEqual(false,myClass.IsConnectionStringValid())

public class TestableConfigBasedClass:ConnectionHelper {     public string mConnectString;

            protected override string getConnectionString(){                return mConnectString;            }}

Page 25: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Fast run time

• In a non-testable system you would find it really hard, or really time consuming to write some of the tests against objects and make them run fast.

• That’s because you’d most likely have objects that do some sort of time consuming activity using their external dependencies.

Page 26: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

What if the IsValid method…

if (!(PersonValidator.IsValid(p))){                return false;}• Calls a web service to do its bidding? • reads rules from the database?

To make it testable :• Refactor the code to use interfaces,

OR• use a virtual method to check the validation and return a true/false

result, then override it in your “testable” class.

Page 27: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Order of tests does not matter (Data layer problem)

bool Insert(Person p){//insert person to database} bool Delete(Person p){//Delete person from database}most unit test frameworks cannot guarantee the order in which unit tests will be

executed

Page 28: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

Partial Runs of tests are possible

• When you have external state you’re likely to have a problem if you don’t run all the tests in your suite, since the external state between specific tests may need to be adjusted for the next test.

• This problem is solvable, but it’s not as easy as writing simple unit tests without external state

• The whole problem of rolling back external state is that of controlling the before and after states and making sure they are the same for each test run.

• If you can’t do that easily you’re either not using a testable system design, or you’re not writing a unit test.

Page 29: Software testing techniques Software testing techniques Software Testability Presentation on the seminar Kaunas University of Technology.

References

• http://www.sce.carleton.ca/faculty/yee/Improving_Testability.pdf [žiūrėta 2011.04.26]

• http://en.wikipedia.org/wiki/Software_testability [žiūrėta 2011.04.26]

• http://replay.web.archive.org/20060424002143/http://weblogs.asp.net/rosherove/articles/Design4Tesatbility1.aspx [žiūrėta 2011.04.26]

• http://www.etestinghub.com/testing_requirements.php [žiūrėta 2011.04.26]


Recommended