OverviewTeaching: 10 min
Exercises: 0 minQuestions
How can I automate running the tests on more platforms than my own?Objectives
Understand how continuous integration speeds software development
Understand the benefits of continuous integration
Implement a continuous integration server
Identify a few options for hosting a continuous integration server
To make running the tests as easy as possible, many software development teams implement a strategy called continuous integration. As its name implies, continuous integration integrates the test suite into the development process. Every time a change is made to the repository, the continuous integration system builds and checks that code.
Thought Experiment: Does Your Software Work on Your Colleague’s Computer?
Imagine you developed software on a MacOSX computer. Last week, you helped your office mate build and run it on their Linux computer. You’ve made some changes since then.
- How can you be sure it will still work if they update their repository when they come back from vacation?
- How long will that process take?
The typical story in a research lab is that, well, you don’t know whether it will work on your colleagues’ machine until you try rebuilding it on their machine. If you have a build system, it might take a few minutes to update the repository, rebuild the code, and run the tests. If you don’t have a build system, it could take all afternoon just to see if your new changes are compatible.
Let The Computers Do The Work
Scientists are good at creative insights, conceptual understanding, critical analysis, and consuming espresso. Computers are good at following instructions. Science would be more fun if the scientists could just give the computers the instructions and go grab an espresso.
Continuous integration servers allow just that. Based on your instructions, a continuous integration server can:
- check out new code from a repository
- spin up instances of supported operating systems (i.e. various versions of OSX, Linux, Windows, etc.).
- spin up those instances with different software versions (i.e. python 2.7 and python 3.0)
- run the build and test scripts
- check for errors
- and report the results.
Since the first step the server conducts is to check out the code from a repository, we’ll need to put our code online to make use of this kind of server (unless we are able/willing to set up our own CI server).
Set Up a Mean Git Repository on GitHub
test_mean.pyfiles can be the contents of a repository on GitHub.
- Go to GitHub and create a repository called mean.
- Clone that repository (git clone https://github.com:yourusername/mean)
- Copy the
test_mean.pyfiles into the repository directory.
- Use git to
pushthe two files to GitHub.
Your work on the mean function has both code and tests. Let’s copy that code into its own repository and add continuous integration to that repository.
What is required
It doesn’t need a build system, because Python does not need to be compiled.
- What does it need?
- Write the names of the software dependencies in a file called requirements.txt and save the file.
- In fact, why don’t you go ahead and version control it?
Travis is a continous integration server hosting platform. It’s commonly used in Ruby development circles as well as in the scientific python community.
To use Travis, all you need is an account. It’s free so someone in your group should sign up for a Travis account. Then follow the instructions on the Travis website to connect your Travis account with GitHub.
A file called
.travis.yml in your repository will signal to Travis that you want to
build and test this repository on Travis-CI. Such a file, for our purposes, is very simple:
language: python python: - "2.6" - "2.7" - "3.2" - "3.3" - "3.4" - "nightly" # command to install dependencies install: - "pip install -r requirements.txt" # command to run tests script: pytest
yml file syntax
The exact syntax of the
.travis.ymlfile is very important. Make sure to use spaces (not tabs). https://lint.travis-ci.org/ can be used to check for typographic errors.
You can see how the python package manager, pip, will use your requirements.txt file from the previous exercise. That requirements.txt file is a conventional way to list all of the python packages that we need. If we needed pytest, numpy, and pymol, the requirements.txt file would look like this:
numpy pymol pytest
- Add .travis.yml to your repository
- Commit and push it.
- Check the situation at your server
Some guidance on debugging problems if the tests fail to run can be found in the lesson discussion document.
We gave the example of Travis because it’s very very simple to spin up. While it is able to run many flavors of Linux, it currently doesn’t support other platforms as well. Depending on your needs, you may consider other services such as:
Servers exist for automatically running your tests
Running the tests can be triggered by a GitHub pull request
CI allows cross-platform build testing
.travis.ymlfile configures a build on the travis-ci servers
Many free CI servers are available