Have you ever written a Python script that you were afraid to run? After investing hours of hard work into your code, you know the feeling: that moment of truth arrives, and you're about to test it. But then doubts creep in—what if a hidden bug lurks somewhere? Will the code perform as expected, or will it throw unforeseen errors? Don't worry; you're not alone.

Many Python developers have experienced this fear. But there's good news: a powerful tool can help you write better and more reliable Python code: Pytest. Pytest is a testing framework that can help you find and fix bugs before they cause problems. It provides a flexible syntax and is easy to use, even for beginners.

In this article, we will learn what testing is all about and how to write and run tests using pytest.

Prerequisites

Basic knowledge of Python is required, as pytest is built on Python
Python version 3.7+ installed

What is testing?

Testing in software development is the process of verifying that software performs as expected. It is an essential part of software development, as it helps to ensure that issues are identified and fixed early, making software reliable and bug-free.

There are many different types of tests, but they all have the same goal: to ensure that the software meets the requirements. Some common types of tests include:

Unit tests: Unit tests are the most basic type of test. They test individual units of code, such as functions or methods. Developers typically write unit tests for the code they write to ensure that it is working correctly.
Integration tests: Integration tests test how different units of code interact with each other. Usually, the task of writing integration tests falls on the developer, who takes on the responsibility of combining various units of code.
System tests: System tests test the entire system as a whole. Usually, it is the responsibility of a quality assurance (QA) engineer or tester to write system tests.

💡

A quality assurance (QA) engineer is a software professional who ensures the quality of software products.

People often think that testing is a waste of time. They think that it takes too long and that it doesn't help to find bugs. However, this is not the case. Testing can save you time and resources in the long run. It can also help you improve the quality of your code and increase your productivity.

Here are some of the benefits of testing:

Increased confidence in your code: Testing can help you increase your confidence in your code by assuring you that it is working correctly. This can help you feel more comfortable releasing your code to production and can also help reduce the number of bugs that are found in your code after it is released.
Early detection of bugs: Testing can help you detect bugs early in the development process, which can save you time and money in the long run. This is because bugs that are found early in the development process are typically easier to fix than bugs that are found later in the development process.
Improved code quality: Testing can help you improve the quality of your code by identifying and fixing bugs. This is because your code tends to be more reliable and easier to maintain.
Increased productivity: Testing can help you increase your productivity by making it easier to find and fix bugs. This can free up your time so that you can focus on other tasks, such as developing new features.

In Python, developers have multiple testing frameworks at their disposal, with two of the most popular ones being unittest and pytest. While both serve the same purpose, pytest has gained widespread popularity for its simplicity, readability, and reduced boilerplate code compared to unittest. This is why our focus will be on pytest.

What is Pytest?

Pytest is a Python testing framework that makes it easy to write and run tests. It provides several features that can help to simplify the process of testing, such as a flexible syntax, a rich set of assertions, and a powerful test runner.

💡

An assertion is a statement that is used to verify the behavior of your code.

To demonstrate the simplicity of pytest over other tools like unittest, we will start with a basic test example using unittest, followed by an equivalent test using pytest. By comparing these examples side by side, you'll witness firsthand how pytest streamlines the testing process, empowering developers to write efficient and expressive tests with minimal overhead.

Basic test example with unittest

# test_add.py
import unittest

class TestAddNumbers(unittest.TestCase):
    def test_add_numbers(self):
        self.assertEqual(add_numbers(1, 2), 3)

def add_numbers(a, b):
    return a + b

This test uses the unittest module to test the add_numbers function. The test asserts that the add_numbers function returns the correct value when passed the numbers 1 and 2 but look at how many steps it took to test a function as basic as this.

Same test example with pytest

# test_add.py
def test_add_numbers():
    assert add_numbers(1, 2) == 3

def add_numbers(a, b):
    return a + b

As you can see, the pytest example is more concise. There was no need to:

Import a module.
Create a class or inherit from it.
Create class methods.

The assert statement replaces the longer self.assertEqual() from unittest, resulting in cleaner and more readable test code. All that is required is a function prefixed with test_.

Installing Pytest

pytest can be easily installed with the pip command. To install it, open your terminal or command prompt and run this command:

>>> pip install pytest

This will install the latest version of pytest.

If you are using poetry, you can install pytest by running the following command:

>>> poetry add pytest

Once Pytest is installed, you can verify the installation by running the following command:

pytest --version

This will print the version of Pytest that is installed.

Writing your first test with Pytest

To make it simple, let's start by using the example we demonstrated earlier.

Create a simple Python function

Open your text editor and create a Python file. This file will contain the function we want to test, you can name it addition.py. Define a function to add two numbers in the addition.py file:

# addition.py

def add_two_numbers(num1, num2):
    return num1 + num2

Create a test file

When creating test files for testing in pytest, there are a couple of conventions to follow. Here are some of the conventions:

The file name should start with test_.
The test function should be prefixed with test_.
The file should contain only tests.
The tests should be written in a Python module (i.e., in a .py file).
The tests should use the pytest assertion library.

💡

When you run Pytest, it will automatically look for files prefixed with test_ and run the tests in it.

Create a test file following this convention; you can name it test_addition.py. This file will only contain test code.

💡

By convention, multiple test files should be grouped into a folder named test.

Create the test function

Define a function in the test file you just created. This function will be used to test the add_two_numbers function. You can name the test function test_add_two_numbers.

# test_addition.py

def test_add_two_numbers():

This test function doesn't do anything yet, we need to write assertion statements within the function to check that the code is working as expected.

Assertions in Pytest

When you create test cases, the goal is to compare actual results with expected outcomes. This is where assertions come into play. In software testing, assertions are statements that check the expected outcome of a piece of code. They are used to verify that the code is working as expected and to detect bugs.

An assertion is a boolean expression that evaluates to either true or false. If the expression evaluates to true, the assertion passes. If the expression evaluates to false, the assertion fails.

For example:

assert x == 10

In this case, if the value of x is not equal to 10, the expression evaluates to false and the assertion will fail. It means the expected outcome is not equal to the actual result, so the test fails.

Unlike some other testing frameworks, Pytest employs a simple and intuitive syntax for assertions. This approach reduces boilerplate code, making your test cases concise and easier to read.

Now that we have a basic understanding of assertions, we can use it in the test function we declared earlier.

In order to test the add_two_numbers function, we first need to import it into our test file. This can be done by adding the following import statement to the top of our test file:

# test_addition.py

from addition import add_two_numbers

Once we have imported the add_two_numbers function, we can use it in our test function.

# test_addition.py

from addition import add_two_numbers

def test_add_two_numbers():
    assert add_two_numbers(2, 3) == 5

This function uses the assert assertion method to check that the add_two_numbers function returns the correct value. The assert method takes two arguments: the expected value and the actual value. If the actual value is not equal to the expected value, the assert method will fail.

In this case, the expected value is 5, and the actual value is the result of calling the add_two_numbers function with the arguments 2 and 3. If the add_two_numbers function is working correctly, the assert method will not fail, and the test will pass. However, if the add_two_numbers function is not working correctly, the assert method will fail, and the test will fail.

Let's explore some essential Pytest assertion statements:

assert: The most basic assertion method in Pytest, verifies whether an expression is True. For example:
```
 def test_addition():
     assert 2 + 2 == 4
```
Equal to: Compares two values for equality. This assertion is useful for checking if two variables have the same value.
```
 def test_multiply():
     result = 3 * 5
     assert result == 15
```

Not Equal to: Verifies that two values are not equal to each other.

 def test_division():
     result = 10 / 2
     assert result != 5

In: verifies if a value is in a list of values.

   numbers=[1,2,3,4]

   assert 3 in numbers

Not in: verifies if a value is in a list of values.

 numbers=[1,2,3,4]

 assert 5 not in numbers

There are many other ways to use assertions. To learn more, check here.

Understanding Test Outcomes

When you run your test using Pytest, each individual test case undergoes a series of evaluations. These evaluations determine whether the test passes or fails. Understanding the various test outcomes will help you identify potential issues in your codebase.

Passed Tests: A test case is considered successful or "passed" when the expected outcomes match the actual results. This means your code performs as expected, and the test scenario is verified.
Failed Tests: Conversely, if an assertion within a test case fails, the test is marked as "failed." This indicates that the actual result did not align with the expected outcome, signaling the presence of an issue in your code.
Skipped Tests: In some scenarios, you may wish to skip certain tests temporarily. These tests are labeled as "skipped" and are not executed during the test run. Skipping tests can be useful when you're testing specific features that are still under development or not applicable in certain conditions.
Expected Failures: Sometimes, you might have tests that are expected to fail due to known issues or ongoing development. Such tests are tagged as "expected failures," and their failure does not indicate a new problem.

Let us run our test to see the outcome. To run the test, open your terminal or command prompt and run this command:

pytest <test_file_name.py>

Replace <test_file_name.py> with the name of your test file. In this case, test_addition.py. The command will look this way:

pytest test_addition.py

Outcome:

The screenshot shows the outcome of the test test_addition.py. The test passed, as indicated by the 1 passed in 0.04s message.

💡

We will discuss the [100%] later.

The test output is minimal and can make it difficult to read and debug tests, especially if they fail, which is why we need to make use of flags.

In testing, Flags are used to control the behavior of the test runner. They can be used to change the output of the test, the way the tests are run, and the way the test failures are handled.

We will explore some of the essential flags that pytest provides but before that, let us add another test function to our test file making two functions. This way, we will be able to see the benefit of using pytest flags.

# test_addition.py

from addition import add_two_numbers

def test_add_two_numbers():
    assert add_two_numbers(2, 3) == 5

# new function
def test_add_numbers_with_negative_numbers():
    assert add_two_numbers(-1, -2) == -3

We added a new function; test_add_numbers_with_negative_numbers. This function asserts that the add_two_numbers() function returns -3 when it is called with the arguments -1 and -2.

Now, let us explore some of the essential flags pytest provides.

The `-V` flag

The -v flag, also known as --verbose, is used during test runs to add extra detail to the test output. This enhances the understanding of test cases and their outcomes.

Try running your test file using the -v flag:

pytest -v test_addition.py

Outcome:

Unlike the previous test outcome, the -v flag displays the individual test outcomes.

test_addition.py::test_add_two_numbers PASSED                        [ 50%] 
test_addition.py::test_add_numbers_with_negative_numbers PASSED      [100%]

During the test execution, you might notice percentage values in the code output, such as 50% or 100%. These percentages represent the test coverage achieved during the test runs.

Test coverage is a metric that measures the proportion of your codebase that is covered by your test cases. A higher percentage indicates that a larger portion of your code is being tested.

For instance, when the output displays test_add_numbers_with_negative_numbers PASSED [100%]: This indicates that the test test_add_numbers_with_negative_numbers achieved 100% code coverage. All lines of code within the add_two_numbers() function were executed during this test, leaving no part of the function untested.

On the other hand, when the test output displays test_add_two_numbers PASSED [50%]: This means that the test test_add_two_numbers achieved 50% code coverage. In other words, the test case covered approximately half of the lines of code within the add_two_numbers() function. The other half of the code was not executed during this specific test run. This does not necessarily imply that the untested code is faulty, but it does highlight areas that require additional test cases to ensure complete coverage.

It's important to note that achieving 100% test coverage is not always the ultimate goal. Achieving 100% test coverage may require writing a large number of tests, which can be time-consuming.

it is more important to focus on writing tests that validate critical components of your code than it is to achieve 100% test coverage. If you can write tests that validate the critical components of your code, you will definitely catch the bugs in your code and improve the quality of your software.

Now, let us deliberately fail the test to see how pytest ensures readability. Change the expected value of the first test function from 5 to 10. For example:

def test_add_two_numbers():
    assert add_two_numbers(2, 3) == 10

This assertion will evaluate as false, making the test fail. Run the test again using the -v flag.

Outcome:

See how detailed the test result is. The test result gives information on:

The test that failed and the file it is located in.

  test_addition.py::test_add_two_numbers FAILED

The exact line where the error occurred.

  E       assert 5 == 10
  E        +  where 5 = add_two_numbers(2, 3)

  test_addition.py:6: AssertionError

The expression that caused the failure

  FAILED test_addition.py::test_add_two_numbers - assert 5 == 10

The number of tests that failed and passed.
```
  ============= 1 failed, 1 passed in 0.34s ============
```
Change the expected of the first test function back to 5.

The `K` flag

The K flag allows you to selectively run tests whose names match a given expression. For example:

pytest -k "negative"

This will execute all tests that contain "negative" in their names. In our code, only one test has "negative" in its name so only test_add_numbers_with_negative_numbers will be executed while the other will be deselected.

Here are some other flags you can try out:

-x, --exitfirst: When this flag is used, pytest stops the test run after the first test failure or error, making it easier to focus on resolving issues.
--maxfail=num: Limits the number of allowed test failures before terminating the test run. For example, pytest --maxfail=3 will stop the test run after three test failures.
--lf, --last-failed: This flag allows you to run only the tests that failed during the previous test run. It's useful for re-running and debugging failed tests.
--ff, --failed-first: Similar to --last-failed, but the failed tests are executed first in the subsequent test run.

Conclusion:

In this first part of our exploration into Pytest, we've covered fundamental testing concepts and learned how to leverage Pytest to write reliable Python code. From expressive assertions to insightful reporting with the -v flag, you now possess a good foundation in testing.

But there's more to come! In Part 2 of this article, we will dive into advanced testing techniques that will elevate your skills as a Python developer. We'll unravel the power of fixtures, which enable reusable setups and teardowns for tests, streamlining your test suite and enhancing maintainability.

Additionally, we'll explore the art of selectively skipping and marking tests, allowing you to optimize test runs based on specific criteria and test scenarios effectively.

We'll also explore mocking and patching with tests. These techniques empower you to simulate complex scenarios and dependencies, isolating your tests and ensuring their reliability.

Stay tuned for the upcoming continuation, where we will delve into these advanced concepts, enriching your testing skills and further enhancing your ability to write robust Python code.

Remember, comprehensive testing is an essential aspect of software development, contributing to the creation of stable and dependable applications. As you continue on your testing journey, keep honing your skills, and I look forward to sharing more valuable insights with you in Part 2.

Happy testing!

Seun Fashina

Seun Fashina

Getting Started with Pytest: The Python Testing Framework

A definitive guide to testing in Python using Pytest

Table of contents

Prerequisites

What is testing?

What is Pytest?

Installing Pytest

Writing your first test with Pytest

Create a simple Python function

Create a test file

Create the test function

Assertions in Pytest

Understanding Test Outcomes

The `-V` flag

The `K` flag

References

Getting Started with Pytest: The Python Testing Framework

A definitive guide to testing in Python using Pytest

Table of contents

Prerequisites

What is testing?

What is Pytest?

Installing Pytest

Writing your first test with Pytest

Create a simple Python function

Create a test file

Create the test function

Assertions in Pytest

Understanding Test Outcomes

The -V flag

The K flag

References

The `-V` flag

The `K` flag