Testing

Reference

Basics of Testing

• Tests compare that the result observed from running code is the same as what was expected ahead of time.
• Tests should be written at the same time as the code they are testing is written.
• Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional bahavior.
• Unit tests try to test the smallest pieces of code possible, usually functions and methods.
• Integration tests make sure that code units work together properly.
• Regression tests ensure that everything works the same today as it did yesterday.

Assertions

• Assertions are one line tests embedded in code.
• The assert keyword is used to set an assertion.
• Assertions halt execution if the argument is false.
• Assertions do nothing if the argument is true.
• The numpy.testing module provides tools numeric testing.
• Assertions are the building blocks of tests.

Exceptions

• Exceptions are effectively specialized runtime tests
• Exceptions can be caught and handled with a try-except block
• Many built-in Exception types are available

Unit Tests

• Functions are the atomistic unit of software.
• Simpler units are easier to test than complex ones.
• A single unit test is a function containing assertions.
• Such a unit test is run just like any other function.
• Running tests one at a time is pretty tedious. (let’s use a framework instead)

Running Tests with Pytest

• The py.test command collects and runs tests starting with Test or test_.
• . means the test passed
• F means the test failed or erred
• x is a known failure
• s is a purposefully skipped test

Edge and Corner Cases

• Functions often fail at the edge of their range of validity
• Edge case tests query the limits of a function’s behavior
• Corner cases are where two edge cases meet

Integration and Regression Tests

• Integration tests interrogate the coopration of pieces of the software
• Regression tests use past behavior as the expected result

Continuous Integration

• Servers exist for automatically running your tests
• Running the tests can be triggered by a GitHub pull request
• CI allows cross platform build testing
• A .travis.yml file configures a build on the travis-ci servers
• Many free CI servers are available

Test Driven Development

• Test driven development is a common software development technique
• By writing the tests first, the function requirements are very explicit
• TDD is not for everyone
• TDD requires vigilance for success (cheating leads to failure)

Fixtures

• It may be necessary to set up “fixtures” composing the test environment.

Glossary

• assert is a keyword that halts code execution when its argument is false
• Exceptions are customizeable cousin of assertions.
• try is a keyword that guards a piece of code which may throw an exception.
• except is a keyword used to catch and carefully handle that exception
• pytest is a python package with testing utilities
• py.test is a command-line program that collects and runs unit tests.
• Unit Tests investigate the behavior of units of code (such as functions, classes, or data structures).
• Regression Tests defend against new bugs, or regressions, which might appear due to new software and updates.
• Integration Tests check that various pieces of the software work together as expected.
• Continuous Integration automates the building and testing process accross platforms.
• Test-driven Development is a software development strategy in which the tests are written before the code.