Pytest

Pytest looks for folder and files which start with test_.

Arrange Act Assert

Most functional tests follow the Arrange-Act-Assert model:

Arrange: set up the conditions for the test.
Act: by calling some function or method
Assert: that some end condition is true

Writing Tests

In Pytest does a test case look like this.

def test_always_passes():
        assert True, "Will always pass"

def test_always_fails():
        assert False, "Will always fail"

Or if you want to organise your tests in a class.

class TestSampleClass:
        def test_always_pass_in_class(self):
                assert True

        def test_always_fail_in_class(self):
                assert False

Assert

Pytest provides a wide range of assertion methods, such as assertequal, assertgreater and assertless, to compare values and test different conditions in Python functions and classes.

def test_addition():
    assert 2+2 == 4

Running Tests

Run pytest with pytest.

Options to use with pytest:

`-q`: make pytest less verbose (pytest -q)
`-m`: run tests with a specific mark (pytest -m regression)

Running Tests Continuously

To continuously run pytest on any file change, you can use a tool called pytest-watch.

You can also pass additional options to pytest-watch to control how it runs pytest, such as the number of workers or which files to include or exclude.

Marks

You can use pytest marks to categorize test cases. This can be used to run only a subset of the complete list of test cases.

You could for example create a test case with the mark `@pytest.mark.database_access`, to indicate this test requires access to the data base to run.

Pytest provides a few marks out of the box:

skip: skips a test unconditionally.
skipif: skips a test if the expression passed to it evaluates to True.
xfail: indicates that a test is expected to fail, so if the test does fail, the overall suite can still result in a passing status.
parametrize: creates multiple variants of a test with different values as arguments.

You can have more than one mark per test.

To see a list of all available markers do `pytest –markers=`.

Registering custom marks

It is possible to create custom marks. Those marks will generate warnings unless they are registered. The registering of marks removes the possibility of accidentally running no tests.

Do register custom markers in a pytest.ini file at the root of the repository. Then add your custom markers.

[pytest]
markers =
  marker1: This is a comment to mark "marker1".

If you want to mark all tests in a module with the same mark do.

pytestmark = pytest.mark.mark_name

Parameterize

For test cases which take a lot of input of the same type, is it possible to parameterize the input with a parametrize mark.

This function uses [subprocess](subprocess.md) and takes a brew package name and returns their currently installed version number.

"""dev.py module is used for development purposes.

Class Terms:

attributes are variables that belong to a class. Unlike class attributes, you can’t access instance attributes through the class. You need to access them through their containing instance.
methods are functions that belong to a class.
class.__init__: The constructor of a class. Self is used to refer to the instance of the class.
class.attribute: A variable that is shared by all instances of a class.
In general, you should use class attributes for sharing data between instances of a class. Any changes on a class attribute will be visible to all the instances of that class.
leading with an underscore: A convention to indicate that the attribute or method is intended to be private.
leading with two underscores: mangles the name and prevents use from outside the class
__str__: A special method that is called when an instance of a class is printed. Use to give a friendly name.
@classmethod: A function that is defined inside a class and is used to modify the state of an instance. A class method receives the class as an implicit first argument, just like an instance method receives the instance.
@staticmethod: A method that is bound to a class rather than its object. A static method does not receive an implicit first argument. A static method is also a method that is bound to the class and not the object of the class. This method can’t access or modify the class state. It is present in a class because it makes sense for the method to be present in class.
Use `isinstance()` to check if an object is an instance of a class.
Use `issubclass()` to check if a class is a subclass of another class.
vars() returns the dict attribute of the given object.
Property and Descriptor Based Attributes:
- https://realpython.com/python-classes/#property-and-descriptor-based-attributes

Data Classes:

Use `field` with `default_factory` to create a mutable default value.
Data class decorator takes the following parameters:
- init: Add .__init_₍₎ method? (Default is True.)
- repr: Add .__repr_₍₎ method? (Default is True.)
- eq: Add .__eq_₍₎ method? (Default is True.)
- order: Add ordering methods? (Default is False.)
- unsafe_hash: Force the addition of a .__hash_₍₎ method? (Default is False.)
- frozen: If True, assigning to fields raise an exception. (Default is False.)
class.__post_init_₍₎ is called after the init method.
Field support the types: int, float, str, bytes, bool, tuple, list, dict, set, frozenset, array, datetime, uuid
Field supports the following parameters:
- default: Default value of the field
- default_factory: Function that returns the initial value of the field
- init: Use field in .__init_₍₎ method? (Default is True.)
- repr: Use field in repr of the object? (Default is True.)
- compare: Include the field in comparisons? (Default is True.)
- hash: Include the field when calculating hash()? (Default is to use the same as for compare.)
- metadata: A mapping with information about the field

Tuple: - is a collection which is ordered and unchangeable. Allows duplicate members.

Create tuple >>> t = (1, 2, 3)
Concatenate two tuples >>> t + (4, 5, 6)
Multiply a tuple >>> t * 2
Unpacking tuple >>> a, b, *c = t
- If the asterisk is added to another variable name than the last, Python will assign values to the variable until
the number of values left matches the number of variables left.
delete tuple >>> del t
Tuple methods:
- count: Returns the number of times a specified value occurs in a tuple.
- index: Searches the tuple for a specified value and returns the position of where it was found.

Named Tuples:

Gives more readable code. Use dot notation to access.

"""

import json import subprocess as sup from shutil import which

class BrewPackage: num_instances = 0

def __init__{(self, package_name: str)}: self.package_name = package_name type(self).num_instances += 1 # Use type(self) instead of the class name to allow inheritance.

def package_version(self) -> str: """Display the version number of a brew package.

shutil.which() works like `which` on the CLI and returns true if a binary is found on PATH.

Parameters: self.package_name: Name of the brew package.

Returns: Version number of the brew package. """ if which(self.package_name) is not None: try: raw = sup.run(f"brew info {self.package_name} –json", capture_output=True, check=True, shell=True) string = raw.stdout.strip().decode("utf-8") jsons = json.loads(string) pkg_installed_version = jsons[0]["installed"][0]["version"] return pkg_installed_version except sup.CalledProcessError as e: return e.stderr.decode("utf-8") else: return f"{self.package_name} is not installed"

To test this function with a few different inputs can the following test be parametrized.

"""Tests for dev.py module""" import pytest from hax.dev import BrewPackage

@pytest.mark.parametrize("string, expected", [ ("az", "2.50.0₁"), ("brew_package", "brew_package is not installed")]) def test_brew_package_version(string, expected): """Test of brew_pkg_version function.

TODO: Change test to not test shutil or brew but just "my code". """ assert BrewPackage(package_name=string).package_version() == expected

This test is having the issue of not having mocked input data and will fail once the version of `azure-cli` is updated.

#### Fixtures [Fixtures](https://docs.pytest.org/en/latest/how-to/fixtures.html) are functions that can create data, test doubles, or initialize system state for the test suite. Any test that wants to use a fixture must explicitly use this fixture function as an argument to the test function. This makes it so that dependencies are always stated up front. Use of fixtures also reduce code repetition and making tests run faster.

<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides"> <caption class="t-above"><span class="table-number">Table 1:</span> Pytest Fixtures</caption>

<col class="org-left" /> </colgroup> <thead> <tr> <th scope="col" class="org-left">CLI Command</th> <th scope="col" class="org-left">Description</th> </tr> </thead>

<tbody> <tr> <td class="org-left"><code>pytest –fixtures</code></td> <td class="org-left">Show list of available fixtures</td> </tr> </tbody> </table>

@pytest.fixture def example_fixture(): return 1

def test_with_fixture(example_fixture): assert example_fixture == 1

A disadvantage of fixtures is that it might become difficult to see where a fixture is coming from if the code base is large or if fixtures are stored in different files.

Pytest looks for a file called conftest.py, which should be at the root of the repository. Any global configuration for pytest should be done here.

There are built-in fixtures:

cache: Cache object that persists between testing sessions.
capsys: Captures text written to STDOUT (standard out) and STDERR (standard error) and the output can be retrieved through the fixture.
tmp<sub>path</sub>: Creates a temporary directory unique to each test function, pass in as pathlib.Path object.
Testing Files Contents

If you want to create a file with pytest can you use the pytest fixture [tmp<sub>path</sub>](https://docs.pytest.org/en/7.1.x/reference/reference.html#std-fixture-tmp_path) to provide a space to temporarily store that file. The tmp<sub>path</sub> is always created as a sub-directory of the current path.

@pytest.fixture def sample_file(tmp_path): file = tmp_path / "sample.txt" file.write_text("Hello, world!") return file

def test_sample_file(sample_file): assert sample_file.read_text() == "Hello, world!"

In this example, the `sample_file` fixture takes the `tmp_path` fixture as a parameter. `tmp_path` provides a temporary directory that pytest creates and manages for us.

The `sample_file` fixture creates a file named `sample.txt` inside the temporary directory, writes the text "Hello, world!" to it, and returns the `Path` object representing the file.

The `test_sample_file` function takes the `sample_file` fixture as a parameter and tests that the file contains the expected text.

monkey-patches

monkey-patches are code that over-writes existing library calls, stubs out things that shouldn't be done during a test, tricks around with databases, networks, date/time etc.

monkey-patches are just like normal fixtures. If you, for example, want to add a monkepath which hinders the test from doing a network call, then do:

import pytest import requests

@pytest.fixture(autouse=True) def disable_requests_get(monkeypatch):

def patched_get(*args, **kwargs): raise RuntimeError("Bad! No network for you!")

monkeypatch.setattr(requests, "get", patched_get)

#### Filtering Tests Pytest can run all or only some of the tests. This is acomplished by filtering which tests you want to run. There are different ways of filtering:

Name-based filtering: Use -k parameter to limit pytest to run only those tests whose fully qualified names match a particular expression.
Directory scoping: By default, pytest will run only those tests that are in or under the current directory.
Test categorization: Use -m parameter to run include or exclude tests from particular categories that you define.

<span class="underline">Options of running tests</span>

-q to run the tests in quite mode. You still get failure info when running in quite mode.

#### Test duration Pytest can automatically show you which test are the slowest by using the `–durations=N` option, like `pytest –duration=3`.

## Reports We use [Allure](https://allurereport.org/) for test reports.

### Allure metadata Add metadata to tests with decorators.

```python @allure.title("Test Authentication") @allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @allure.tag("NewUI", "Essentials", "Authentication") @allure.severity(allure.severity_{level.CRITICAL}) @allure.label("owner", "John Doe") @allure.link("https://dev.example.com/", name="Website") @allure.issue("AUTH-123") @allure.testcase("TMS-456") def test_{authentication}(): … ```

Add metadata to tests with methodcalls.

```python def test_{authentication}(): allure.dynamic.title("Test Authentication") allure.dynamic.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") allure.dynamic.tag("NewUI", "Essentials", "Authentication") allure.dynamic.severity(allure.severity_{level.CRITICAL}) allure.dynamic.label("owner", "John Doe") allure.dynamic.link("https://dev.example.com/", name="Website") allure.dynamic.issue("AUTH-123") allure.dynamic.testcase("TMS-456") … ```

Hax

pytest.ini

[pytest]
addopts = --testmon
markers =
    web: Web tests

tests/test_conf.py

"""Tests for conf.py module"""
import pytest
from hax import conf


@pytest.fixture
def fixture_conf_file(tmp_path):
    """Create a TOML file in a temporary directory with dummy data."""
    file = tmp_path / "conf_file.toml"
    file.write_text('[HEADING]\nvariable_one = "number_one"')
    return file


@pytest.mark.limit_memory("2 MB")
def test_get_config(fixture_conf_file):
    """Test of get_config function."""
    config = conf.get_config(file=fixture_conf_file)
    assert config["HEADING"]["variable_one"] == "number_one"

tests/test_dev.py

"""Tests for dev.py module"""
import pytest
from hax.dev import BrewPackage


@pytest.mark.parametrize("string, expected", [
    ("az", "2.55.0"),
    ("brew_package", "brew_package is not installed")])
def test_brew_package_version(string, expected):
    """Test of brew_pkg_version function.

    TODO: Change test to not test shutil or brew but just "my code".
    """
    assert BrewPackage(package_name=string).version() == expected