Date post: | 02-Jan-2016 |
Category: |
Documents |
Upload: | harry-greer |
View: | 214 times |
Download: | 0 times |
1
What is Unit Testing?
“To write test cases for every non-trivial function or method in the module so that each test case is [as] separate from the others [as] possible.”www.discovercomputers.info/ComputerSecurity/glossary.html
2
Unit Testing (Definition)
public bool isPrime( int num){ /*implementation omitted*/}
Definition of a prime number:◦ “A prime number (or a prime) is a
natural number which has exactly two distinct natural number divisors: 1 and itself.” (Wikipedia) “The number 1 is by definition not a prime number”
3
Example Function To Test
public bool isPrime( int num){ /*implementation omitted*/}
Test with:◦ small primes (2, 3, 5, 7, 11)◦ large primes◦ non-primes (1, 4, 9, 12, 11*17)◦ zero◦ negative numbers
4
Example Test Cases
Overview of Using NUnit
5
A free, open-source framework to help people create their own unit tests for any .Net language◦ We’ll use it for C#
Related to (“inspired by”) JUnit◦ JUnit is for Java testing
6
What is NUnit?
Make it quick and easy
to create unit tests,
and to run those tests,
and to get a list of which tests passed/failed.
7
NUnit’s Goal(s):
For this class, you will NOT be required to create your own Unit Tests.
You ARE required to write software that passes tests. The tests are provided to you, by the teacher
8
How do I Use NUnit?
You will download a starter project, which includes all the NUnit materials
A quick example of how this all will work will follow this slide
9
How do I Use NUnit?
Quick walkthrough of how to use this◦ Run the tests in the NUnit GUI◦ Generate a gradesheet (including
grade)◦ Run the program in normal console
mode
10
Demo:
11
Demo:How to run the tests in the NUnit GUI
12
Demo:How to Generate a gradesheet
(including grade)
13
Demo:How to run the program in normal console mode
Things you’ll need / find useful:◦ To select which project will be run:
Right-click on a PROJECT, then select "Set Startup Project"
◦ EditFind And ReplaceFind In Files This will let you search ALL files for a particular string
◦ How to get VS to comment/uncomment a block
14
Demo:
WorkflowHow will I use NUnit as part of my work in this particular course?
15
Student Workflow:◦ Download starter project◦ Examine first exercise, in Word .DOC
Figure out which tests are used in the exercise Run tests, figure out which ones have failed Write code to make one (or more) tests pass
Repeat until all tests pass Repeat until all exercises done
◦ Hand in Student_Answers.cs
16
Workflow
.ZIP a folder that contains your work, even if it’s only 1 file
◦ Student_Answers.cs◦ < Feedback file, possibly >
17
Submitting Your Work
Instructor workflow: ◦ Run tests to get basic grade◦ Double-check code◦ Grade for feedback/ stuff that’s not auto-
graded◦ Finalize grade & email gradesheets
18
Workflow
19
Solution Setup
2020
There are three projects inside the starter solution:◦ 01_PCE_Test_Runner
This contains all the (instructor-provided) code that handles running the tests.
This includes running the test in the GUI, and running the auto-grading logic
◦ 02_PCE_ForTests This is where the units tests themselves are located
◦ 03_PCE_StudentCode This is where YOUR CODE must be put
Details: Solution Setup
Mostly, you will NOT need to know about anything in here
01_PCE_Test_Runner
21
The only exception is RunTests.cs
You set the ‘mode’ from GUI to auto-grade by adjusting the doThis variable in Main
◦ (For a console application, you’ll need to set 03_PCE_Student_Code as the startup project)
01_PCE_Test_Runner
22
This project contains all the tests
TestHelpers.cs contains stuff that multiple tests, across multiple assignments, make use of◦ Input/output capturing code ◦ ‘Fuzzy’ comparisons
02_PCE_ForTests
23
PCE_Tests_To_Run.cs contains the actual tests – you’ll want to look in here for the details of a particular test.
02_PCE_ForTests
24
This is where YOUR CODE will be put.◦ There may be just one file (Student_Answers.cs),
or two (Student_Answers.cs and Program.cs)
◦ Your code (that you will hand in) always goes into Student_Answers.cs
03_PCE_Student_Code
25
If you want to experiment with a normal, console application, you do so by setting this project as the startup project, and then adding code to Main.
03_PCE_Student_Code
26
27
Detailed Walkthroughs
Details: GUI Test Runner
28
Select “01_PCE_Test_Runner” as startup project◦ Make sure that in RunTests.cs, the linedoThis = RUN_TEST_IN_GUI; is uncommented
In VS: DebugStart Without Debugging
In the NUnit test runner: click ‘Run’ button◦ Be careful about which test(s) you’ve selected
Details: GUI Test Runner
29
NUnit should auto-reload your code when you recompile◦ If it doesn’t, you can force it to reload:
File Reload Project orFile Reload Tests
You can then leave the test runner open, switch back to Visual Studio,change your program, and re-run your tests◦ Without having to quit-and-restart NUnit
Details: GUI Test Runner
30
Run the test(s) in the GUI Get the name of a test that is failing
1. Edit Find And Replace Find In Files
2. In 02_PCE_ForTests, in PCE_Tests_To_Run.csfind the test by hand
How to find a failing test
31
Note the tabs at the bottom: Errors And Failures:
◦ Lists out the exact error message when the test failed
◦ Tries to tell you where the failure occurred Tests Not Run:
◦ This should never happen (unless there’s an error with the test)
Test Output:◦ Anything you (or the test) Console.Writes will be
displayed here
Details: GUI Test Runner
32
Details: Autograder
33
Select “01_PCE_Test_Runner” as startup project◦ Make sure that in RunTests.cs, the linedoThis = PRODUCE_GRADESHEET;is uncommented
In VS: DebugStart Without Debugging You’ll see a lot of text in a console window,
and then the gradesheet should appear in a web browser
Details: Autograder
34
Not all tests that you can see in the GUI are graded
Failed tests are big, but passed tests are kinda small & on the bottom
Details: Autograder
35
Note that compromising the system in any way will get you a zero ◦ Cracking/changing/disabling tests
◦ WRITING CODE TO PASS A TEST, DESPITE NOT ACCOMPLISHING IT’S REAL GOAL Ex: A ‘FindInArray’ method that just asks “Did I get
asked to find the value 8? If so, return true”, so that it passes the test which asks “Is 8 in the array?”
Details: Autograder
36
Details: Using The Debugger
37
Select “01_PCE_Test_Runner” as startup project◦ Make sure that in RunTests.cs, the linedoThis = RUN_TESTS_UNDER_DEBUGGER;is uncommented
In VS: DebugStart Debugging◦ AFTER you’ve set some breakpoints
Using The Debugger
38
Put a breakpoint in the test (or your code)◦ Put the cursor on the line, then Debug Toggle
Breakpoint
Useful features: Step, Step Into, Step Out Watch windows
Using The Debugger
39
Details: The Unit Tests
40
Located in the 02_PCE_ForTests project, inside the PCE_Test.cs file
You should never need to change or modify these◦ We’ll go through them now, in order to make sure
you’ve understand how the NUnit tests work.◦ If you need to disable a test, you can just
comment the whole method out, and the rest of the system *should* just work
Code Details: NUnit Tests
41
[Test]
◦ Specifies that test will be automatically called by NUnit.
◦ The GUI automatically finds this test, and gets the name to display in the GUI
42
[Test] Attribute
[Category]◦ Used by the instructor to put a test into a grading
category [TestFixture], [TimeOut], [Description]
◦ Used to tell Nunit about tests
43
Attributes to ignore:
NUnit ‘magically’ finds the tests to run◦ Once the test starts, it runs just like normal code
We can tell NUnit to check that certain things are true at certain points, and fail if they’re not true◦ This is done using the Assert.That command
◦ If the function crashes (or throws exception), then the crash will be caught, prevented (normally), and the test will fail
44
Execution of a test
Assert.That( false, “msg” );
What this does:◦ If the first parameter (‘false’) is false,
then the test will fail, with the message “msg”
Test: Basic_Fail in the Nunit_Demo project
45
Basic (Failing) Test
bool correctAnswer = false;Assert.That(correctAnswer, “msg” );
This separates out the ‘is it correct?’ logic from the Assert statement
Notice that we can (and often will) use this pattern
◦ Test: Basic_Fail_2, Basic_Fail_3
46
More typical pattern
“Message” can work like Console.WriteLine:
int num = 10; Assert.That( num == 12, “num={0} isn’t what we expected”, num);
Test: Basic_Console_WriteLine
47
Using Assert Like Console.Write*
Typically, a test will call into your code, check the result, and then Assert that your answer is the same as the expected answer
Because the test code is in another project, you MUST make your classes PUBLIC in the StudentCode project
This will either have been done for you, or else we’ll cover how to create your own classes in C#
48
Calling Student Code
This is the first ‘real’ unit test Call into your codeCheck the answerPass (or fail) the test
Test: Basic_Unit_Test
49
Calling Student Code
Certain tests may redirect input, output◦ This means that they can ‘feed’ input to your
code, and ‘capture’ the output that you produce
Capturing Console.Out in the Test◦ Importance of the ‘fuzzy’ string comparison
Test-Supplied Console.In
50
Redirecting I/O
Using the TestHelpers software, the tests can capture output.◦ This is in the TestHelpers.cs file ; you’re welcome to
look at it, if you’re curious about how it works
Anything that you print (using Console.Write) can be captured
Test: Unit_Test_Console_WriteLine
51
Output (captured by the test)
The TestHelpers software also provides for a number of helpful routines, including ‘fuzzy’ comparisons◦ ‘Fuzzy’ for strings means case-insensitive, and
without (too much) regard for blank spaces
Test: Unit_Test_Console_WriteLine_Fuzzy
52
Fuzzy String Comparisons
The test can also assemble a string, which will be used as the input to your code.
Whenever you call Console.Readline, you will read everything up to the next newline◦ Newline in C#: \n
Example of input: ◦ “2\n3\nquit\n” is the same as typing
2<enter>3<enter>quit<enter>
Test: Unit_Test_Console_IO_Fuzzy53
Test-Supplied Input
public void Unit_Test_Values( [Values(-1, 0, 1, 10)]int valueToTry)
Tells NUnit to call this function with valueToTry set to -1, then set to 0, then 1, then 10
Test: Unit_Test_Values
54
[Values] Attribute
public void Unit_Test_Values_2([Values(1, 10)]int a, [Values(2, 4)]int b)
Tells NUnit to call this function with ALL POSSIBLE COMBINATIONS OF a and b:◦ a = 1, b = 2◦ a = 1, b = 4◦ a = 10, b = 2◦ a = 10, b = 4
Test: Unit_Test_Values_2
55
[Values] Attribute - Combo
[TestCase( 1, 2)][TestCase( 10, 4)]public void Unit_Test_TestCase(int a, int b)
Tells NUnit to call this function with EACH TestCase: ◦ a = 1, b = 2◦ a = 10, b = 4
Test: Unit_Test_TestCase
56
[TestCase] Attribute
Notice that before each test, the method marked with [Setup] will be run
In the example test, we’ll use this to initialize x to be 10, y to be 20, and ex5 to refer to an object
Test: Unit_Test_Setup
57
Tests that use the [Setup] Attribute
If you’re having trouble getting a test to pass,
you can try adding print statements to your code,
so that as the program runs,
it tells you about what it’s doing
58
Debugging Strategy: Printing