Running tests

Since we're responsible developers, we have written a full suite of unit tests for our application. For now, tests are run manually inside our Docker container. The Docker image used has py.test installed, as well as some coverage tools.

The only dependency to running tests is PostgreSQL. Docker again makes it very simple to run a PostgreSQL container and hook it up to our application container. Multiple strategies exist for this, from running Docker Compose to merely starting up a container with docker run and linking the containers manually. For simplicity, I use the latter option. See the targets in the repository Makefile for details.

To run tests, inside the container, we execute make tests. I have trimmed much of the output for brevity and clarity:

root@d8dd5cc4bb86:/code# make tests
py.test --cov=serverless/ --cov-report=html tests/
Connected to: postgresql://postgres:@cupping-rltest-postgres:5432/test_cupping_log
........
==== test session starts ====
platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /code, inifile:
plugins: mock-1.6.2, cov-2.5.1
collected 105 items

tests/test_cupping_model.py .........
tests/test_cupping_persistence.py ......................
tests/test_handler.py ...
tests/test_helpers.py .
tests/test_session_handler.py ...........................
tests/test_session_models.py ......
tests/test_session_persistence.py .....................................

--- coverage: platform linux, python 3.6.2-final-0 ---
Coverage HTML written to dir htmlcov

==== 105 passed in 2.04 seconds ===

The result is an htmlcov/index.html file that visually shows test coverage throughout the application and highlights lines that were not executed during the test run:

The preceding image is a test coverage report from the pytest and coverage libraries

Coverage may also be displayed on the console if we ask for it specifically:

----------- coverage: platform linux, python 3.6.2-final-0 -----------
Name Stmts Miss Branch BrPart Cover
------------------------------------------------------------------------------
serverless/cupping/__init__.py 0 0 0 0 100%
serverless/cupping/constants.py 11 0 0 0 100%
serverless/cupping/db/__init__.py 52 2 8 3 92%
serverless/cupping/db/mixins.py 33 14 6 1 51%
serverless/cupping/exceptions.py 5 0 0 0 100%
serverless/cupping/handlers/__init__.py 0 0 0 0 100%
serverless/cupping/handlers/decorators.py 9 0 0 0 100%
serverless/cupping/handlers/helpers.py 34 0 16 1 98%
serverless/cupping/handlers/session.py 43 4 4 0 91%
serverless/cupping/helpers.py 3 0 0 0 100%
serverless/cupping/models/__init__.py 2 0 0 0 100%
serverless/cupping/models/cupping.py 20 0 4 0 100%
serverless/cupping/models/session.py 14 0 4 0 100%
serverless/cupping/persistence/__init__.py 2 0 0 0 100%
serverless/cupping/persistence/base.py 7 0 0 0 100%
serverless/cupping/persistence/cupping.py 48 0 14 0 100%
serverless/cupping/persistence/queries.py 6 0 0 0 100%
serverless/cupping/persistence/session.py 41 0 10 0 100%
serverless/handler.py 32 3 2 1 88%
------------------------------------------------------------------------------
TOTAL 362 23 68 6 92%