Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error combining coverage when running with pytest-xdist and a custom coverage plugin #642

Open
slanzmich opened this issue Apr 19, 2024 · 4 comments

Comments

@slanzmich
Copy link

Summary

I am working on a custom coverage plugin that returns custom file tracers for Python files. When I use this coverage plugin together with pytest-xdist, it fails when combining the coverage data from the xdist workers (see traceback below).

To debug this, I added keep=True to the calls to Coverage.combine in pytest_cov/engine.py.

In an example with three xdist workers:

  • Four .coverage.* files are created: one from each worker proceses, and one from the master process.
  • All four coverage files contain the same set of files in the "files" table.
  • The coverage file generated from the master process has an empty "arc" table, and no entry in the "tracer" table.

When combining the coverage files, coverage checks if the "tracer" entries of all coverage files match. If a file is listed in the "files" table, but does not have an entry in the "tracer" table, it uses "" as the default tracer. This conflicts with the "tracer" info from the other data files, which causes coverage to error out.

To resolve this issue, pytest-cov could not create that extra, empty coverage data file from the master process.

Traceback

Traceback (most recent call last):
  File "<venv_path>/lib/python3.10/site-packages/_pytest/main.py", line 271, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "<venv_path>/lib/python3.10/site-packages/_pytest/main.py", line 325, in _main
    config.hook.pytest_runtestloop(session=session)
  File "<venv_path>/lib/python3.10/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "<venv_path>/lib/python3.10/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "<venv_path>/lib/python3.10/site-packages/pluggy/_callers.py", line 155, in _multicall
    teardown[0].send(outcome)
  File "<venv_path>/lib/python3.10/site-packages/pytest_cov/plugin.py", line 339, in pytest_runtestloop
    self.cov_controller.finish()
  File "<venv_path>/lib/python3.10/site-packages/pytest_cov/engine.py", line 46, in ensure_topdir_wrapper
    return meth(self, *args, **kwargs)
  File "<venv_path>/lib/python3.10/site-packages/pytest_cov/engine.py", line 355, in finish
    self.cov.combine()
  File "<venv_path>/lib/python3.10/site-packages/coverage/control.py", line 836, in combine
    combine_parallel_data(
  File "<venv_path>/lib/python3.10/site-packages/coverage/data.py", line 179, in combine_parallel_data
    data.update(new_data, aliases=aliases)
  File "<venv_path>/lib/python3.10/site-packages/coverage/sqldata.py", line 760, in update
    raise DataError(
coverage.exceptions.DataError: Conflicting file tracer name for '<project_path>/tests/test_example.py': '' vs 'example_plugin.ExampleCoveragePlugin'

Versions

Python 3.10.11 on Windows

coverage                         7.4.4
pytest                           7.4.3
pytest-cov                       5.0.0
pytest-xdist                     3.5.0

The same also fails on Linux with the same package versions.

Config

pytest is run via python -m pytest --cov --cov-config=pyproject.toml -v --numprocesses=3 --maxschedchunk=1.

coverage is configured as follows:

[tool.coverage.run]
branch = true
plugins = ["example_plugin"]
concurrency = ["thread", "multiprocessing"]
@nedbat
Copy link
Collaborator

nedbat commented Apr 19, 2024

I don't have a solution to the problem, but I'm curious what your custom plugin does for Python files?

@slanzmich
Copy link
Author

It tries to address what is also reported as nedbat/coveragepy#856: Find calls to functions that were compiled by TensorFlow's autograph functionality, and report the original function as covered.

@nedbat
Copy link
Collaborator

nedbat commented Apr 19, 2024

It tries to address what is also reported as nedbat/coveragepy#856: Find calls to functions that were compiled by TensorFlow's autograph functionality, and report the original function as covered.

Interesting! Keep me posted on the progress.

@ionelmc
Copy link
Member

ionelmc commented Sep 17, 2024

I have created an test for this but can't reproduce the problem: b9c99e7

Perhaps it's not enough to have an empty plugin and needs a custom tracer or something? I need a plugin that can reproduce the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants