Testing and Coverage Reports

To ensure that newly added code behaves as expected and that modifications or new additions do not change previously written code, gnss_lib_py uses unit testing, implemented using pytest.

The module pytest-cov is also used to generate code coverage reports. Code coverage reports detail which lines are run and which are missed during the execution of the tests. Higher coverage indicates that the code was run during unit tests and we aim to keep the code coverage above 90%.

Testing

The following is a description of how tests are structured and placed in the repository. We use pytest to write and run tests. A brief description of the naming convention and commonly used features is also included below. For more details, refer to the pytest documentation.

Running tests

  • To run tests, go to the parent directory and run

    poetry run pytest

  • To run tests from a specific file run

    poetry run tests/folder_name/test_file_name.py

  • To run a particular test, contained in a specific file, run

    poetry run pytest tests/folder_name/test_file_name.py::test_function

Naming convention and location of tests

  • Tests are placed outside the source code, in the tests/ diretory

  • The structure of the tests directory is expected to mirror the source directory. Eg. tests for functions in gnss_lib_py/utils/file_name.py must be placed in tests/utils/test_file_name.py

  • Following the pytest naming convention, test functions are named test_function

  • Fixtures should be named as fixture_name where name is the name of the fixture, defined using the @pytest.fixture(name="name") decorator before the fixture definition. More details on expected uses of fixtures can be found in the fixtures section.

Conventions for writing tests

  • Most tests in gnss_lib_py are unit tests. Unit tests use a known input-output pair to verify that the function being tested is working as expected. Unit tests should be used to verify nominal behaviour as well as any relevant edge cases

  • To verify that the function output matches the expected output, we recommend using the in-built assert statement, numpy’s testing functions, that are of the form np.testing.assert* and any additional module level testing functions, like pd.testing.*assert* for pandas.

  • Use fixtures @pytest.fixture to define certain fixed objects (tuples, strings etc.) that might be required by test functions. Fixtures can be named and these names can be used to access fixture outputs like regular variables

  • Fixtures can also be composed to create a fixture of a fixture.

  • gnss_lib_py already uses numerous fixtures for testing. Fixtures that are relevant only to tests in a single file are defined in that file. Fixtures that are used across multiple files are defined in conftest.py in the testing directory gnss_lib_py/tests/.

  • Before defining a new fixture, check to see if the fixture exists in conftest.py. If it does, you can refer to the fixture by name in your test function and use without having to define any additional methods.

  • Existing fixtures from conftest.py also define locations of test files or common test data that can be used for your own fixtures. Please check these existing fixtures to enable quick development of your own.

  • If you want to test the same function beheaviour for multiple input-output pairs, you can use the pytest.mark.parameterize function decorator before the test

  • When creating plots in a test, if you want to visually verify the plots,ensure that all plots are saved for checking later on. Plots that are created must be closed using plt.close() before the tests stop running

For additional details on pytest functionality mentioned above, check the pytest documentation.

Coverage Reports

In general, you should not submit new functionality without also providing corresponding tests for the code. Visual testing coverage reports can be found at the top of the GitHub repository. Similar reports can be generated locally, in terminal, with the following commands:

poetry run pytest --cov=gnss_lib_py/algorithms --cov=gnss_lib_py/parsers --cov=gnss_lib_py/utils --cov-report=html
poetry run coverage report

The total percentage of code covered (bottom right percentage) is the main number of priority.

To generate the coverage report and view as a webpage, use the following command:

poetry run pytest --cov=gnss_lib_py/algorithms --cov=gnss_lib_py/parsers --cov=gnss_lib_py/utils --cov-report=html

The generated coverage report can be accessed from the directory htmlcov/