Testing

Reference

Basics of Testing

  • Tests compare that the result observed from running code is the same as what was expected ahead of time.
  • Tests should be written at the same time as the code they are testing is written.
  • Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional bahavior.
  • Unit tests try to test the smallest pieces of code possible, usually functions and methods.
  • Integration tests make sure that code units work together properly.
  • Regression tests ensure that everything works the same today as it did yesterday.

Assertions

  • Assertions are one line tests embedded in code.
  • The assert keyword is used to set an assertion.
  • Assertions halt execution if the argument is false.
  • Assertions do nothing if the argument is true.
  • The numpy.testing module provides tools numeric testing.
  • Assertions are the building blocks of tests.

Exceptions

  • Exceptions are effectively specialized runtime tests
  • Exceptions can be caught and handled with a try-except block
  • Many built-in Exception types are available

Unit Tests

  • Functions are the atomistic unit of software.
  • Simpler units are easier to test than complex ones.
  • A single unit test is a function containing assertions.
  • Such a unit test is run just like any other function.
  • Running tests one at a time is pretty tedious. (let’s use a framework instead)

Running Tests with Pytest

  • The py.test command collects and runs tests starting with Test or test_.
  • . means the test passed
  • F means the test failed or erred
  • x is a known failure
  • s is a purposefully skipped test

Edge and Corner Cases

  • Functions often fail at the edge of their range of validity
  • Edge case tests query the limits of a function’s behavior
  • Corner cases are where two edge cases meet

Integration and Regression Tests

  • Integration tests interrogate the coopration of pieces of the software
  • Regression tests use past behavior as the expected result

Continuous Integration

  • Servers exist for automatically running your tests
  • Running the tests can be triggered by a GitHub pull request
  • CI allows cross platform build testing
  • A .travis.yml file configures a build on the travis-ci servers
  • Many free CI servers are available

Test Driven Development

  • Test driven development is a common software development technique
  • By writing the tests first, the function requirements are very explicit
  • TDD is not for everyone
  • TDD requires vigilance for success (cheating leads to failure)

Fixtures

  • It may be necessary to set up “fixtures” composing the test environment.

Glossary

  • assert is a keyword that halts code execution when its argument is false
  • Exceptions are customizeable cousin of assertions.
  • try is a keyword that guards a piece of code which may throw an exception.
  • except is a keyword used to catch and carefully handle that exception
  • pytest is a python package with testing utilities
  • py.test is a command-line program that collects and runs unit tests.
  • Unit Tests investigate the behavior of units of code (such as functions, classes, or data structures).
  • Regression Tests defend against new bugs, or regressions, which might appear due to new software and updates.
  • Integration Tests check that various pieces of the software work together as expected.
  • Continuous Integration automates the building and testing process accross platforms.
  • Test-driven Development is a software development strategy in which the tests are written before the code.