pytest - Simple, rapid and fun testing with Python

Presenter Notes

About me

Florian Bruhin / The Compiler

  • python since 2011
  • testing since 2013
  • pytest since 2015
  • no idea what'll follow in 2017!
  • Writing a browser in Python since (December) 2013 - qutebrowser.org

pytest

  • Started in April 2015
  • Got part of core team
  • Contributed to various plugins
  • Giving pytest trainings

Presenter Notes

  • Talk about adopt pytest month

Why pytest?

Some easy code

1 def add_two(val):
2     return val + 2

Presenter Notes

Why pytest?

Some easy code

1 def add_two(val):
2     return val + 2

Test with unittest.py

1 import unittest
2 
3 class AddTwoTests(unittest.TestCase):
4 
5     def testAddTwo(self):
6         self.assertEqual(add_two(2), 4)
7 
8 if __name__ == '__main__':
9     unittest.main()

Presenter Notes

  • Reminds me of coffee, a certain island (Java), JUnit

Why pytest?

Some easy code

1 def add_two(val):
2     return val + 2

Test with pytest

1 def test_add_two():
2     assert add_two(2) == 4

Presenter Notes

  • Dave Halter: No API is the worst API
  • pytest: The best API is no API

Assertion rewriting

Running with python

1 Traceback (most recent call last):
2   File "tmp.py", line 9, in <module>
3     test_add_two()
4   File "tmp.py", line 6, in test_add_two
5     assert add_two(2) == 5
6 AssertionError

Presenter Notes

  • might be obvious here, but not in general
  • job of a test framework: useful information!

Assertion rewriting

Running with python

1 Traceback (most recent call last):
2   File "tmp.py", line 9, in <module>
3     test_add_two()
4   File "tmp.py", line 6, in test_add_two
5     assert add_two(2) == 5
6 AssertionError

Running with pytest

====================== FAILURES ======================
____________________ test_add_two ____________________

    def test_add_two():
>       assert add_two(2) == 5
E       assert 4 == 5
E        +  where 4 = add_two(2)

tmp.py:6: AssertionError
============== 1 failed in 0.01 seconds ==============

Presenter Notes

  • magic to make output more useful

More assertion rewriting

Nice output for lists...

    def test_eq_list():
>       assert [0, 1, 2] == [0, 1, 3]
E       assert [0, 1, 2] == [0, 1, 3]
E         At index 2 diff: 2 != 3
E         Use -v to get the full diff

...strings...

    def test_not_in_text_single():
        text = 'single foo line'
>       assert 'foo' not in text
E       assert 'foo' not in 'single foo line'
E         'foo' is contained here:
E           single foo line
E         ?        +++

and a lot of other stuff!

Presenter Notes

Parametrizing and fixtures

Presenter Notes

Parametrizing

What if we want to run the same test with various test data?

With unittest.py:

 1 import unittest
 2 
 3 class AddTests(unittest.TestCase):
 4 
 5     def testMinusOne(self):
 6         self.assertEqual(add_two(-1), 1)
 7 
 8     def testZero(self):
 9         self.assertEqual(add_two(0), 2)
10 
11     def testTwo(self):
12         self.assertEqual(add_two(2), 4)

Presenter Notes

  • a method for each testcase

  • 3.4: self.subTest

Parametrizing

With pytest:

1 import pytest
2 
3 @pytest.mark.parametrize('num, expected',
4                          [(-1, 1), (0, 2), (2, 4)])
5 def test_add(num, expected):
6     assert add_two(num) == expected

When running with -v:

tmp.py::test_add[-1-1] PASSED
tmp.py::test_add[0-2] PASSED
tmp.py::test_add[2-4] PASSED

Presenter Notes

Fixtures

A way to share objects or setup/teardown needed by tests.

1 @pytest.fixture
2 def somevalue():
3     return 42
4 
5 def test_func(somevalue):
6     assert somevalue == 42

Doing teardown as part of a fixture:

1 @pytest.yield_fixture
2 def db():
3     conn = some_sql_lib.connect(...)
4     cursor = conn.cursor()
5     yield cursor
6     conn.rollback()
7     conn.close()

Presenter Notes

Built-in fixtures

tmpdir

Provides an empty temporary directory for a test:

1 def test_one(tmpdir):
2     (tmpdir / 'foo').write("Hello World")
3 
4 def test_two(tmpdir):
5     assert not tmpdir.listdir()

monkeypatch

Allows you to patch attributes, environment variables, etc.:

1 def test_one(monkeypatch):
2     monkeypatch.setenv('FOO', 'test')
3     assert 'FOO' in os.environ
4 
5 def test_two():
6     assert 'FOO' not in os.environ

Presenter Notes

Plugins

Presenter Notes

pytest-cov

Coverage reports via coverage.py:

pytest-cov

Presenter Notes

Plugins

pytest-bdd

 1 Feature: Blog
 2     A site where you can publish your articles.
 3 
 4 Scenario: Publishing the article
 5     Given I'm an author user
 6     And I have an article
 7     When I go to the article page
 8     And I press the publish button
 9     Then I should not see the error message
10     And the article should be published

Presenter Notes

Plugins

pytest-sugar

pytest-sugar output

Presenter Notes

Plugins

Framework / Language integration

  • pytest-qt
  • pytest-django
  • pytest-twisted
  • pytest-cpp
  • oejskit

Others

  • pytest-xdist: Distribute tests over CPUs/hosts
  • pytest-catchlog: logging integration
  • pytest-timeout: Time out tests after a given time
  • ...and >180 more!

Presenter Notes

Plugins are easy!

 1 # conftest.py
 2 
 3 def pytest_addoption(parser):
 4     parser.addoption("--backend", choices=("webkit", "webengine"),
 5                      default="webkit")
 6 
 7 # test_foo.py
 8 
 9 def test_foo(request):
10     if request.config.getoption("--backend") == "webkit":
11         # ...
12     else:
13         # ...

Presenter Notes

Easy switching

  • There's unittest2pytest to rewrite your tests
  • Runs unittest.py tests
  • Runs nose tests too!

Presenter Notes

2016 sprint - http://s.cmpl.cc/sprint

pytest sprint

Presenter Notes

Contact / Resources

pytest

  • IRC: #pylib on Freenode
  • www.pytest.org

me / slides

  • me@the-compiler.org
  • http://t.cmpl.cc/sps16.html

training / contracting / funding

  • www.merlinux.eu

Presenter Notes