Best Practices in Scientific Computing

Kathryn (Katy) Huff

September 8, 2014

Split, Croatia

Science

builds and organizes knowledge
tests explanations about the universe
systematically,
objectively,
transparently,
and reproducibly.

Otherwise it's not science.

Computers

should...

improve efficiency,
reduce human error,
automate the mundane,
simplify the complex,
and accelerate research.

But scientists don't use them effectively.

“ Computational science is a special case of scientific research: the work is easily shared via the Internet since the paper, code, and data are digital and those three aspects are all that is required to reproduce the results, given sufficient computation tools. ” - Stodden, 2010.

Getting Started

“ Organized Skepticism. Scientists are critical: All ideas must be tested and are subject to rigorous structured community scrutiny.” - R.K. Merton, 1942

Data Storage

Good: pencil and paper
Better: spreadsheet
Best: database

Backing Up Files

Good: hope
Better: nightly emails
Best: remote version control

Managing Changes

Good: naming convention
Better: clever naming convention
Best: local version control

Getting It Done

“ It takes just as much time to write a good paper as it takes to write a bad one. ” - Polterovich, 2014

Analysis

Good: pencil and calculator
Better: spreadsheets, matlab, mathematica
Best: scripting, open source libraries, modern programming language

Multiple File Cleanup

Good: manually edit every file
Better: search and replace in each file
Best: scripted batch editing

Excecuting Workflows

Good: retype a series of commands
Better: bash script
Best: build system (make, configure, autoconf, automake, cmake, etc.)

Data Structures

Good: 100 string variables holding doubles
Better: lists of lists of doubles
Best: appropriate powerful data structures (dictionaries, hashed tables, etc.)

API Design

Good: single block of procedural code
Better: separate functions
Best: small, testable functions, grouped into classes, DRY

Variable Naming

Good: d1, d2, d3
Better: x, y, z
Best: p.x, p.y, p.z, p=Point(x,y,z)

File I/O

Good: none, hardcoded variables
Better: plain text input file, line-by-line homemade string parsing
Best: file parsing library (python argparse, xml rng, etc.)

Getting It Right

“ The scientific method’s central motivation is the ubiquity of error—the awareness that mistakes and self-delusion can creep in absolutely anywhere and that the scientist’s effort is primarily expended in recognizing and rooting out error. ” - Donoho, 2009.

Error Detection

Good: show results to experts
Better: integration testing
Best: unit test suite, continuous integration

Error Diagnostics

Good: re-re-read the code
Better: print statements
Best: use a debugger and a profiler

Error Correction

Good: fix code
Better: fix, add an exception
Best: fix, add an exception, add a test

Getting It Together

“ Two of the biggest challenges scientists and other programmers face when working with code and data are keeping track of changes (and being able to revert them if things go wrong), and collaborating on a program or dataset. ” - Wilson, et al. 2014.

Merging Collaborative Work

Good: single master copy, waiting
Better: emails and patches
Best: remote version control

Peer Review For Code

Good: separation of concerns
Better: shared repository
Best: peer-reviewed pull requests

“ just-in-time review of small code changes is more likely to succeed than large-scale end-of-work reviews. ” - Petre, Wilson 2014

Teamwork

Good: weekly research meetings, year-long tasks
Better: daily conversations, month-long goals
Best: pair programming, issue tracking

Software Handovers

Good: zip file, theory paper
Better: comments in code, example input file
Best: automated documentation (sphinx, doxygen), test suite

Getting It Out There