MT_01_unittest_python.pdf

Python Assertion and Unit Testing

In Debug > Settings Wheel > Configurations you can review your
config and edit the launch.json file. For example add arguments.
Python – VS Code debug

Python – VS Code debug
• launch.json with custom args
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit:
// https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"cwd": "${fileDirname}",
"args": [
"--movie=flowers_google/flowers.mp4"
]
},
{
"name": "Listen for XDebug",
"type": "php",
"port": 9000
},
{
"name": "Launch currently open script",
"type": "php",
"program": "${file}",
"cwd": "${fileDirname}",
"port": 9000
},
{
"name": "PowerShell: Launch Current File",
"type": "PowerShell",
"script": "${file}",
"cwd": "${file}"
}
]
}

Essential VS Code extensions
and keep your Python up to date
• VS Code: https://code.visualstudio.com/docs/python/python-tutorial
• To enable VS Code linting (a tool that analyzes source code to flag programming
errors, bugs, stylistic errors, suspicious constructs, etc.)

$ pip install pylint | To force a specific package version: $ pip install numpy==1.19.0
 To list currently installed packages and versions $ pip list
• To upgrade all local packages use pip-review (from elevated console)

$ pip install pip-review

$ pip-review --local --interactive

VS Code Settings (JSON)

To make various things work as it should in VS Code you may need to edit the View >
Command Palatte > Preferences: Open Settings (JSON) > settings.json file

For example to get Python to work correct with intellisense and pylint (in this case the
package cv2)

1. In VScode: CTRL + Shift + P

2. Choose "Preferences: Open Settings (JSON)"

3. Add the settings below to the settings JSON file

To make other essential adjustments search for: vscode python ”Open Settings
(JSON)” and your topic, program language etc.

More info at: https://code.visualstudio.com/docs/getstarted/tips-and-tricks
{
"python.linting.pylintEnabled": true,
"python.linting.enabled": true,
"python.linting.pylintArgs": ["--extension-pkg-whitelist=cv2"],
...
}

Python Assert statements
• Assertions are statements that assert or
state a fact confidently in your program
• Assertions are simply boolean
expressions that checks if the
conditions return true or false
• If it is true, the program does nothing
and move to the next line of code. If it's
false, the program stops and throws an
error
• It is also a debugging tool as it brings
the program to halt as soon as any error
have occurred and shows the location
where in the program the error occurred

Python assert statement example
• The condition is supposed to be always true. If the condition is false assert halts
the program and gives an AssertionError: assert <condition>, <error_message>
def calc_factorial(num):
if isinstance(num, str):
print("Sorry, factorial does not exist for string input")
return False
elif num < 0:
print("Sorry, factorial does not exist for negative numbers")
return False
elif int(num) != num:
print("Sorry, factorial does not exist for real numbers")
return False
elif num == 0:
print("The factorial of 0 is 1")
return 1
else:
factorial = 1
for i in range(1, num + 1):
factorial = factorial * i
print("The factorial of", num ,"is", factorial)
return factorial
def checkcalc_factorial(num, expected_value, assert_error):
'''Test code below this line'''
ret_val = calc_factorial(num)
# assert <condition>, <error message>
# if condition is not satisfied (true) program will stop and throw an AssertionError
assert ret_val == expected_value, assert_error
print(f"{assert_error}: {ret_val == expected_value} ... OK")
def test_negative_numbers_return_false():
checkcalc_factorial(-1, False, "test_negative_numbers_return_false")
def test_non_integer_return_false():
checkcalc_factorial(0.5, False, "test_non_integer_return_false")
def test_when_input_is_zero_return_one():
checkcalc_factorial(0, 1, "test_when_input_is_zero_return_one")
def test_when_input_is_three_teturn_six():
checkcalc_factorial(3, 6, "test_when_input_is_three_teturn_six")
def test_string_input_return_false():
checkcalc_factorial('t', False, "test_string_input_return_false")
if __name__ == '__main__':
try:
"""
# change the value for a different result
num = 7
# uncomment to take input from the user
num = int(input("Enter a number: "))
calc_factorial(num):
"""
# test code
test_negative_numbers_return_false()
test_non_integer_return_false()
test_when_input_is_zero_return_one()
test_when_input_is_three_teturn_six()
test_string_input_return_false()
except AssertionError as ex:
print(f"AssertionError: {ex}")
assert_factorial.py

Unit Testing with Python 1
• Python have several of testing frameworks available
– One is unittest, others are doctest, Nose and pytest
• unittest is inspired from JUnit and has a similar flavor as major unit testing
frameworks in other languages
– It is organized around test case classes which contain test case methods
– Naming follows Java camelCase in contrast to Pythons snake_case

PEP 8 style guide: https://www.python.org/dev/peps/pep-0008/
• unittest supports
– test automation,
– sharing of setup and shutdown code for tests,
– aggregation of tests into collections,
– and independence of the tests from the reporting framework
• Documentation
– https://docs.python.org/3/library/unittest.html

Unit Testing with Python 2
• unittest supports some important concepts in an object-oriented way
• test fixture
– A test fixture represents the preparation needed to perform one or more tests, and any
associate cleanup actions. This may involve, for example, creating temporary or proxy
databases, directories, or starting a server process
• test case
– A test case is the individual unit of testing. It checks for a specific response to a particular
set of inputs. unittest provides a base class, unittest.TestCase, which may be used to
create new test cases
• test suite
– A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests
that should be executed together
• test runner (or test driver)
– A test runner is a component which orchestrates the execution of tests and provides the
outcome to the user. The runner may use a graphical interface, a textual interface, or
return a special value to indicate the results of executing the tests

App testing (black box) vs. unit testing (white box)

Unit testing general
• Test methods recommended naming convention

(Test)_MethodToTest_ScenarioWeTest_ExpectedBehaviour

In the test method the pattern we use is ”tripple A”

Arrange, Act and Assert # Python
class RoundtripCheck(unittest.TestCase):
def test_to_roman_roundtrip_return_equal(self):
# The arrangement below is called tripple A
# Arrange - here we initialize our objects
integer = known_values[1]
# Act - here we act on the objects
numeral = to_roman(integer)
result = from_roman(numeral)
# Assert - here we verify the result
self.assertEqual(integer, result)
if __name__ == '__main__':
unittest.main(argv=['first-arg-is-ignored'],
exit=False)
// In C# with MSTest, Nunit, xUnit
[TestClass], [TestFixture],
public class ReservationsTests
{
[TestMethod], [Test], [Fact]
public void CanBeCancelledBy_AdminCancelling_ReturnsTrue()
{
// The arrangement below is called tripple A
// Arrange - here we initialize our objects
var reservation = new Reservation();
// Act - here we act on the objects
var result = reservation.CanBeCancelledBy(
new User { IsAdmin = true });
// Assert - here we verify the result
Assert.IsTrue(result);
}

unittest most common assert methods
https://docs.python.org/3/library/unittest.html#assert-methods
• Usually all assertions also take an optional message argument
• Template: self.assert***(first_arg, second_arg, msg=None) # msg is for error info etc.

A very limited Python unittest example
import sys
# make ZeroDivisionError pass
class MyZeroDivisionError(ValueError):
pass
def fact(n):
""" Factorial function, arg n: Number
returns: factorial of n """
if n == 0:
return 1
return n * fact(n - 1)
def div(n):
""" Just divide """
if n == 0:
raise MyZeroDivisionError('division by zero!')
res = 10 / n
return res
def main(n):
print(fact(n))
print(div(n))
if __name__ == '__main__':
if len(sys.argv) > 1:
main(int(sys.argv[1]))
import unittest
import factorial
# derivation from base class is necessary -> unittest.TestCase
# to run via console: python -m unittest --v
class TestFactorial(unittest.TestCase):
test_fact_inp = (0, 1, 2, 4, 5, 6, 10,)
test_fact_out = (1, 1, 2, 24, 120, 720, 3628800,)
def test_factorial_return_equal(self):
""" Testing fact as res = fact(5)
self.assertEqual(res, 120) """
for inp, out in zip(self.test_fact_inp, self.test_fact_out):
result = factorial.fact(inp)
message = f'factorial inp: {inp} and out: {out} gives an error message!'
self.assertEqual(out, result, message)
class TestDiv(unittest.TestCase):
@classmethod
def setUpClass(cls):
print('setupClass')
@classmethod
def tearDownClass(cls):
print('nteardownClass')
def test_div_return_ZeroDivisionError(self):
""" Test exception raise due to run time error in the div function """
self.assertRaises(factorial.MyZeroDivisionError, factorial.div, 0)
if __name__ == '__main__':
unittest.main(argv=['first-arg-is-ignored'], exit=False)
test_factorial.py
factorial.py

Python exceptions
BaseException
Exception
ArithmeticError
FloatingPointError
OverflowError
ZeroDivisionError
AssertionError
AttributeError
BufferError
EOFError
ImportError
ModuleNotFoundError
LookupError
IndexError
KeyError
MemoryError
NameError
UnboundLocalError
OSError
BlockingIOError
ChildProcessError
ConnectionError
BrokenPipeError
ConnectionAbortedError
ConnectionRefusedError
ConnectionResetError
FileExistsError
FileNotFoundError
InterruptedError
IsADirectoryError
NotADirectoryError
PermissionError
PermissionError
ProcessLookupError
TimeoutError
ReferenceError
RuntimeError
NotImplementedError
RecursionError
StopIteration
StopAsyncIteration
SyntaxError
IndentationError
TabError
SystemError
TypeError
ValueError
UnicodeError
UnicodeDecodeError
UnicodeEncodeError
UnicodeTranslateError
Warning
BytesWarning
DeprecationWarning
FutureWarning
ImportWarning
PendingDeprecationWarning
ResourceWarning
RuntimeWarning
SyntaxWarning
UnicodeWarning
UserWarning
GeneratorExit
KeyboardInterrupt
SystemExit
Read More!
https://
julien.danjou.info/
python-exceptions-
guide/
https://airbrake.io/
blog/python-
exception-handling/
class-hierarchy

• It is also possible to check the production of exceptions, warnings, and log
messages using the following assert methods
• Template: self.assert***(exception, callable/fun, *args, **keywords)
AssertRaises etc.
https://docs.python.org/3/library/unittest.html#assert-methods
self.assertRaises(factorial.MyZeroDivisionError, factorial.div, 0)

•
• assertRises() test that a specific exception is raised when callable/fun is called with
any extra keyword arguments

The test

Passes if the specific exception is raised

Is an error if another exception is raised

Fails if no exception is raised
• AssertRises() can also return a context manager by using the with statement

Context managers are a way of allocating and releasing some sort of resource
exactly when/where you need it. Example using with and file access
assertRaises and keywords with 1
self.assertRaises(basic1.MyTypeError, basic1.my_split, 'hello world', 2)
with open('file_to_write.txt', 'w') as fh_cm:
fh_cm.write('file contents')

• assertRaises(exception, *, msg=None) using keyword with
• If a callable/fun argument is not provided assertRaises returns an optional context
manager which can be used to access exception details
• Used as a context manager, assertRaises fails if associated with body does not raise
• When used as a context manager, assertRaises() accepts the additional keyword
argument msg
• The context manager will store the caught exception object in its exception attribute
• This can be useful if the intention is to perform additional checks on the exception
raised
assertRaises with keywords with 2
# alternative call with an optional contex manager
with self.assertRaises(basic1.MyTypeError) as cm:
basic1.my_split('hello world', 2)
# (cm) that also can perform additional checks on the exception raised
self.assertEqual('Input separator must be a string', cm.exception.args[0])

• assertRaisesRegex can match a string representation of a raised exception

Template: self.assertRaisesRegex(exception, regex, callable/fun, *args, **keywords)

regex may be a regular expression object or a string - equivalent to a regex.search()
• assertWarns(warning, callable, *args, **kwds), assertWarns(warning, *,
msg=None) and assertWarnsRegex(***) works in similar ways
• assertLogs(logger=None, level=None)

A context manager to test that at least one message is logged on the logger or one
of its children, with at least the given level
AssertRaisesRegex etc.
'''split should raise error with non-string input separator and if regex does not match'''
self.assertRaisesRegex(basic1.MyTypeError, 'Input', basic1.my_split, 'hello world', 2)
with self.assertRaisesRegex(basic1.TypeError, 'Input') as cm:
self.assertEqual('Input', cm.expected_regex.pattern)

unittest more specific assert methods
• More specific and type specific
asserts methods – all take at least
an optional message argument

A Python unittest example 2
'''basic1.py reworked example From:
https://docs.python.org/3/library/unittest.html
To run it: python basic1.py'''
import sys
# my class definition of the TypeError exception
# Calling reserved word pass does nothing
class MyTypeError(ValueError):
pass
def my_upper(my_str):
return my_str.upper()
def my_isupper(my_str):
return my_str.isupper()
def my_split(my_str, sep):
# check if separator is a string
if not isinstance(sep, str):
raise MyTypeError('''Input separator must be a string''')
return my_str.split(sep)
try:
ss = input("Enter a string: ")
print(f"my_upper: {my_upper(ss)}")
print(f"my_isupper: {my_isupper(ss)}")
print(f"my_split: {my_split(ss, ' ')}")
print(f"my_split: {my_split(ss, 2)}")
except BaseException as ex:
print(f"TypeError: {ex}")
sys.exit(0)
import basic1, unittest
'''test_basic1.py reworked example from: https://docs.python.org/3/library/unittest.html
To run it: python -m unittest --v Here is a short script to test three string methods:'''
class TestStringMethods(unittest.TestCase):
def test_upper(self): # test case methods must start with test
self.assertEqual(basic1.my_upper('foo'), 'FOO')
def test_isupper(self):
self.assertTrue(basic1.my_isupper('FOO'))
self.assertFalse(basic1.my_isupper('Foo'))
def test_split(self):
ss = 'hello world'
self.assertEqual(basic1.my_split(ss, ' '), ['hello', 'world'])
def test_non_string_split(self):
self.assertRaises(basic1.MyTypeError, basic1.my_split, 'hello world', 2)
with self.assertRaises(basic1.MyTypeError) as cm:
self.assertEqual('Input separator must be a string', cm.exception.args[0])
def test_non_string_split_regex(self):'
self.assertRaisesRegex(basic1.MyTypeError, 'Input', basic1.my_split, 'hello world', 2)
with self.assertRaisesRegex(basic1.MyTypeError, 'Input') as cm:
self.assertEqual('Input', cm.expected_regex.pattern)
if __name__ == '__main__':
unittest.main(argv=['first-arg-is-ignored'], exit=False)
test_basic1.py
basic1.py

Unit Testing in practice 1
• Good unit tests
– Are fully automated, i.e. write code to test code
– Offer good coverage of the code under test, including boundary cases and error handling
paths
– Are easy to read and maintain – acts like some documentation for the source code
– Express the intent of the code under test – test cases do more than just check it
– Enables refactoring and updates of the code with confidence
• Not so good unit tests
– Monolitic tests: all test cases in a single function
• Ex. test_foo()
– Ad hoc tests: test cases are scattered across test functions
• Ex. test_1(), test_2(), ...
– Procedural tests: test cases bundled into a test method that directly correspond to a target
method (as in the basic1 code example with isupper())
• Ex. test_foo() → (is testing the function) foo()

Unit Testing in practice 2
• A test is not a unit test if ...
– It talks to the database
– It communicates across the network
– It touches the file system
– It cannot run at the same time as any of your other unit tests
– You have to do special things to your environment (such as editing config files)
to run it
– https://www.artima.com/weblogs/viewpost.jsp?thread=126923
• Tests that do these things aren't bad. Often they are worth writing, and they
can be written in a unit test harness (as part of a specially prepared test
environment needed to execute the test)
• However, it is important to be able to separate them from true unit tests so
that we can keep a set of tests that we can run fast whenever we make our
changes

Code coverage
• Coverage measurement is typically used to gauge the effectiveness of tests.
It can show which parts of your code are being executed (covered) by the
test suite, and which are not
• Function coverage – Has each function (or subroutine) in the program been called?
• Statement coverage – Has each statement in the program been executed?
• Edge coverage – has every edge (arrow) in the Control Flow Graph been executed?

Branch coverage – Has each branch (also called Decision-to-Decision path) of
each control structure (such as in if and case statements) been executed? For
example, given an if statement, have both the true and false branches been
executed? Notice that this is a subset of Edge coverage
• Condition coverage – Has each Boolean sub-expression
evaluated both to true and false?
https://en.wikipedia.org/wiki/Code_coverage
https://en.wikipedia.org/wiki/Control-flow_graph

Coverage.py
• Coverage.py is a tool for measuring code
coverage of Python programs. It monitors your
program, noting which parts of the code have been
executed, then analyzes the source to identify
code that could have been executed but was not
• Note that high coverage numbers does not mean
that your code is clean from bugs!
# install
$ pip install coverage
$ coverage help
usage: coverage <command> [options] [args]
# run your test code (.coverage is created)
$ coverage run --branch test_factorial.py
# report to console (from .coverage file)
$ coverage report -m
# report to html (htmlcov folder is created)
$ coverage html
PS C:python_unittesting> coverage report -m
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------------------
factorial.py 19 8 8 2 56% 29-30, 33-36, 39-40, 27->29, 38->39
test_factorial.py 14 0 4 1 94% 23->exit
---------------------------------------------------------------
TOTAL 33 8 12 3 71%

Recommended viewing and reading
• Python Tutorial: Unit Testing Your Code with the unittest Module

https://www.youtube.com/watch?v=6tNS--WetLI
• Dive into Python 3

https://github.com/diveintomark/diveintopython3
• Python 3.x unittest documentation

https://docs.python.org/3/library/unittest.html
• Code Coverage

https://coverage.readthedocs.io

https://en.wikipedia.org/wiki/Code_coverage
• Martin Fowler on TestCoverage

https://martinfowler.com/bliki/TestCoverage.html

MT_01_unittest_python.pdf

More Related Content

Similar to MT_01_unittest_python.pdf (20)

Recently uploaded (20)

MT_01_unittest_python.pdf