3.7.0: pytest is failing in `tests/test_pytest_mock.py::test_standalone_mock` unit

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.7.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.7.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -p no:relaxed
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-mock-3.7.0, configfile: tox.ini
plugins: mock-3.7.0, cov-3.0.0, asyncio-0.17.2
asyncio: mode=auto
collected 72 items

tests/test_pytest_mock.py .........................................................F..............                                                                   [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________ test_standalone_mock ___________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-tkloczko/pytest-2/test_standalone_mock0')>

    def test_standalone_mock(testdir: Any) -> None:
        """Check that the "mock_use_standalone" is being used."""
        testdir.makepyfile(
            """
            def test_foo(mocker):
                pass
        """
        )
        testdir.makeini(
            """
            [pytest]
            mock_use_standalone_module = true
        """
        )
        result = testdir.runpytest_subprocess()
>       assert result.ret == 3
E       assert <ExitCode.OK: 0> == 3
E        +  where <ExitCode.OK: 0> = <RunResult ret=ExitCode.OK len(stdout.lines)=20 len(stderr.lines)=0 duration=0.46s>.ret

/home/tkloczko/rpmbuild/BUILD/pytest-mock-3.7.0/tests/test_pytest_mock.py:723: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
running: /usr/bin/python3 -mpytest --basetemp=/tmp/pytest-of-tkloczko/pytest-2/test_standalone_mock0/runpytest-0
     in: /tmp/pytest-of-tkloczko/pytest-2/test_standalone_mock0
============================= test session starts ==============================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-of-tkloczko/pytest-2/test_standalone_mock0, configfile: tox.ini
plugins: mock-3.7.0, cov-3.0.0, asyncio-0.17.2
asyncio: mode=legacy
collected 1 item

test_standalone_mock.py .                                                [100%]

=============================== warnings summary ===============================
../../../../usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
    config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)

test_standalone_mock.py::test_foo
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:317: DeprecationWarning: '@pytest.fixture' is applied to <fixture _mocker, file=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.7.0-2.fc35.x86_64/usr/lib/python3.8/site-packages/pytest_mock/plugin.py, line=388> in 'legacy' mode, please replace it with '@pytest_asyncio.fixture' as a preparation for switching to 'strict' mode (or use 'auto' mode to seamlessly handle all these fixtures as asyncio-driven).
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================== 1 passed, 2 warnings in 0.03s =========================
========================================================================= short test summary info ==========================================================================
FAILED tests/test_pytest_mock.py::test_standalone_mock - assert <ExitCode.OK: 0> == 3
======================================================================= 1 failed, 71 passed in 6.16s =======================================================================
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-6f5bceaa-27c1-4828-a3c4-0d1d4c62f501
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-6f5bceaa-27c1-4828-a3c4-0d1d4c62f501'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-f0b2eb80-1df2-41e3-b9c6-7ae4bdb46198
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-f0b2eb80-1df2-41e3-b9c6-7ae4bdb46198'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-aa20aba2-7e0e-4ebb-8719-a5e007e0d1c4
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-aa20aba2-7e0e-4ebb-8719-a5e007e0d1c4'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-d429fee2-5f3d-428e-92a2-6e0fc3e29972
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-d429fee2-5f3d-428e-92a2-6e0fc3e29972'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-0e7d96b3-02dc-4fec-88d8-b31f111ddb5d
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-0e7d96b3-02dc-4fec-88d8-b31f111ddb5d'
  warnings.warn(
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:80: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-9f197e27-3fb6-4540-9abc-37cc28021b4e
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-9f197e27-3fb6-4540-9abc-37cc28021b4e'
  warnings.warn(

[nicoddemus]

Hi @kloczek,

Can you confirm that the package mock is installed in that environment?

[nicoddemus]

I can reproduce the same error if I install mock in my test environment, so I'm pretty sure that's the reason.

I've changed that test's approach in #277, which I'm sure will work for you then. Do you need a new release with that fix?

[kloczek]

IMO it would be really better to switch to unittest.mock as in most of the cases now mock is treated as deprecated..

[The-Compiler]

This plugin has always used unittest.mock. The test is for an (optional) mock_use_standalone_module setting which makes it use mock instead.

[nicoddemus]

Exactly, thanks @The-Compiler.

Closing this then. 👍

[kloczek]

Even with installed 'mock` pytest is failing.
Additionally please have look on pytest warning.

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.7.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.7.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -p no:relaxed --deselect tests/test_pytest_mock.py::test_detailed_introspection --deselect tests/test_pytest_mock.py::test_detailed_introspection_async
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-mock-3.7.0, configfile: tox.ini
plugins: mock-3.7.0, asyncio-0.18.2
asyncio: mode=auto
collected 72 items / 2 deselected / 70 selected

tests/test_pytest_mock.py .........................................................F............                                                                     [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________ test_standalone_mock ___________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-tkloczko/pytest-155/test_standalone_mock0')>

    def test_standalone_mock(testdir: Any) -> None:
        """Check that the "mock_use_standalone" is being used."""
        testdir.makepyfile(
            """
            def test_foo(mocker):
                pass
        """
        )
        testdir.makeini(
            """
            [pytest]
            mock_use_standalone_module = true
        """
        )
        result = testdir.runpytest_subprocess()
>       assert result.ret == 3
E       assert <ExitCode.OK: 0> == 3
E        +  where <ExitCode.OK: 0> = <RunResult ret=ExitCode.OK len(stdout.lines)=16 len(stderr.lines)=0 duration=0.37s>.ret

/home/tkloczko/rpmbuild/BUILD/pytest-mock-3.7.0/tests/test_pytest_mock.py:723: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
running: /usr/bin/python3 -mpytest --basetemp=/tmp/pytest-of-tkloczko/pytest-155/test_standalone_mock0/runpytest-0
     in: /tmp/pytest-of-tkloczko/pytest-155/test_standalone_mock0
============================= test session starts ==============================
platform linux -- Python 3.8.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /tmp/pytest-of-tkloczko/pytest-155/test_standalone_mock0, configfile: tox.ini
plugins: mock-3.7.0, asyncio-0.18.2
asyncio: mode=legacy
collected 1 item

test_standalone_mock.py .                                                [100%]

=============================== warnings summary ===============================
../../../../usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
    config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= 1 passed, 1 warning in 0.02s =========================
============================================================================= warnings summary =============================================================================
tests/test_pytest_mock.py::test_patch_context_manager_with_context_manager
  /usr/lib/python3.8/site-packages/_pytest/python.py:192: PytestRemovedIn8Warning: Passing None has been deprecated.
  See https://docs.pytest.org/en/latest/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests for alternatives in common use cases.
    result = testfunction(**testargs)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
FAILED tests/test_pytest_mock.py::test_standalone_mock - assert <ExitCode.OK: 0> == 3
========================================================== 1 failed, 69 passed, 2 deselected, 1 warning in 4.36s ===========================================================

[nicoddemus]

Oh I forgot to merge #277. Should work now on the current main.

[kloczek]

What about push all those changes as new version? 😋

[nicoddemus]

@kloczek I usually avoid releasing versions without user-facing changes, that's why this has been sitting for a while.

But we have #296 open, which certainly warrants a new release once merged. 👍

[nicoddemus]

3.8.1 is out. 👍

[kloczek]

Looks clean now. Thank you 👍

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.8.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-mock-3.8.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-mock-3.8.1, configfile: tox.ini
plugins: mock-3.8.1, asyncio-0.18.2
asyncio: mode=auto
collected 73 items

tests/test_pytest_mock.py .........................................................................                                                                  [100%]

============================================================================ 73 passed in 3.92s ============================================================================