- 
                Notifications
    
You must be signed in to change notification settings  - Fork 232
 
Open
Description
This isn't truly an issue with pytest-bdd, but I thought this was worth mentioning given the popularity of the pytest-timeout plugin.
I noticed that if the pytest-timeout plugin is used, it can correctly fail a scenario, but all steps show as passed in the cucumberjson output. A minimal example:
test_timeout.py:
from pytest_bdd import scenario, when
import time
@scenario('timeout.feature', 'Long running operation')
def test_long_running():
    pass
@when('I wait for a really long time')
def wait_long_time():
    time.sleep(3)
timeout.feature:
Feature: Timeout demonstration
    Demonstration of a long-running operation
    Scenario: Long running operation
        When I wait for a really long time
then run pytest:
pytest -s tests/test_timeout.py --cucumberjson=output.json --timeout 1
to get the following logs:
=================================================================================================================================================== test session starts ====================================================================================================================================================
platform linux -- Python 3.12.7, pytest-8.3.5, pluggy-1.5.0
rootdir: /media/jason/e/pytest-bdd
configfile: pytest.ini
plugins: bdd-8.1.0, timeout-2.3.1
timeout: 1.0s
timeout method: signal
timeout func_only: False
collected 1 item                                                                                                                                                                                                                                                                                                           
tests/test_timeout.py OUTCOME: failed
F
========================================================================================================================================================= FAILURES =========================================================================================================================================================
____________________________________________________________________________________________________________________________________________________ test_long_running _____________________________________________________________________________________________________________________________________________________
fixturefunc = <function wait_long_time at 0x7130aa194360>, request = <FixtureRequest for <Function test_long_running>>, kwargs = {}
    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if is_generator(fixturefunc):
            fixturefunc = cast(
                Callable[..., Generator[FixtureValue, None, None]], fixturefunc
            )
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
venv/lib/python3.12/site-packages/_pytest/fixtures.py:898: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    @when('I wait for a really long time')
    def wait_long_time():
>       time.sleep(3)
E       Failed: Timeout >1.0s
tests/test_timeout.py:10: Failed
------------------------------------------------------------------------------------------------------------------------- generated json file: /media/jason/e/pytest-bdd/output.json -------------------------------------------------------------------------------------------------------------------------
================================================================================================================================================= short test summary info ==================================================================================================================================================
FAILED tests/test_timeout.py::test_long_running - Failed: Timeout >1.0s
but the following output.json shows the step passed:
[
  {
    "keyword": "Feature",
    "uri": "tests/timeout.feature",
    "name": "Timeout demonstration",
    "id": "tests/timeout.feature",
    "line": 1,
    "description": "Demonstration of a long-running operation",
    "language": "en",
    "tags": [],
    "elements": [
      {
        "keyword": "Scenario",
        "id": "test_long_running",
        "name": "Long running operation",
        "line": 4,
        "description": "",
        "tags": [],
        "type": "scenario",
        "steps": [
          {
            "keyword": "When",
            "name": "I wait for a really long time",
            "line": 5,
            "match": {
              "location": ""
            },
            "result": {
              "status": "passed",
              "duration": 0
            }
          }
        ]
      }
    ]
  }
]
pytest==8.3.5
pytest-bdd==8.1.0
pytest-timeout==2.3.1
Metadata
Metadata
Assignees
Labels
No labels