Severity: Medium
Where: tests/unit/test_ping.py (4 tests: test_ping_http_mode_confirmed_delivery_exits_zero, test_ping_direct_mode_reports_local_db, test_ping_json_output_is_machine_readable, test_ping_honors_custom_agent_id); the pollution source is elsewhere in tests/.
Observed: pytest tests/unit/test_ping.py alone → 10 passed. pytest tests/ (full suite) → the same 4 tests FAIL. Confirmed pre-existing by stashing all working-tree changes and re-running the full suite at HEAD: identical 4 failures, 3801 passed. The failure mode is that tj ping reports delivery to a real daemon instead of the expected mode: assert 'my-real-agent' in output fails against '⚠ Could not confirm local interception (the SDK failed to initialise a tracer).\n✓ Delivered to tj serve at http://127.0.0.1:7391 (confirmed received).', plus WARNING tokenjam.otel:provider.py:56 Span export failed: Connection Error: Connection already closed!.
So an earlier test in the full-suite ordering leaves global OTel/tracer or endpoint state behind (a configured exporter, a patched provider, or a live TJ_* endpoint env var) that test_ping then picks up. The tests are not wrong; they are the victim.
Why it matters beyond the 4 tests: the suite is currently red on every full run, which trains readers to ignore a red suite. It also means the isolation guarantee is already broken — whatever leaks here can leak into any other test that touches OTel, and the next such test to be written will fail mysteriously.
Fix direction: Bisect the ordering with pytest -p no:randomly tests/ -x plus --deselect halving, or run pytest tests/ --lf after a full run to find the first polluter. Likely candidates: a test that calls tokenjam.otel.provider setup without tearing the global tracer provider down, or one that sets a TJ_*/OTLP endpoint env var without monkeypatch. Fix the POLLUTER (autouse fixture resetting the global provider, or monkeypatch instead of raw os.environ), not the victim — do not add skips or xfails to test_ping.py.
Done when: pytest tests/ is green, pytest tests/unit/test_ping.py stays green, and the polluting test resets whatever global state it sets (ideally via an autouse fixture so the class of bug cannot recur).
Ref: Found while verifying an unrelated relearn change on #597 — the 4 failures were initially suspected to be new and were ruled out by a stash-and-rerun at HEAD.
Severity: Medium
Where:
tests/unit/test_ping.py(4 tests:test_ping_http_mode_confirmed_delivery_exits_zero,test_ping_direct_mode_reports_local_db,test_ping_json_output_is_machine_readable,test_ping_honors_custom_agent_id); the pollution source is elsewhere intests/.Observed:
pytest tests/unit/test_ping.pyalone → 10 passed.pytest tests/(full suite) → the same 4 tests FAIL. Confirmed pre-existing by stashing all working-tree changes and re-running the full suite at HEAD: identical 4 failures, 3801 passed. The failure mode is thattj pingreports delivery to a real daemon instead of the expected mode:assert 'my-real-agent' in outputfails against'⚠ Could not confirm local interception (the SDK failed to initialise a tracer).\n✓ Delivered to tj serve at http://127.0.0.1:7391 (confirmed received).', plusWARNING tokenjam.otel:provider.py:56 Span export failed: Connection Error: Connection already closed!.So an earlier test in the full-suite ordering leaves global OTel/tracer or endpoint state behind (a configured exporter, a patched provider, or a live
TJ_*endpoint env var) thattest_pingthen picks up. The tests are not wrong; they are the victim.Why it matters beyond the 4 tests: the suite is currently red on every full run, which trains readers to ignore a red suite. It also means the isolation guarantee is already broken — whatever leaks here can leak into any other test that touches OTel, and the next such test to be written will fail mysteriously.
Fix direction: Bisect the ordering with
pytest -p no:randomly tests/ -xplus--deselecthalving, or runpytest tests/ --lfafter a full run to find the first polluter. Likely candidates: a test that callstokenjam.otel.providersetup without tearing the global tracer provider down, or one that sets aTJ_*/OTLP endpoint env var withoutmonkeypatch. Fix the POLLUTER (autouse fixture resetting the global provider, or monkeypatch instead of rawos.environ), not the victim — do not add skips or xfails totest_ping.py.Done when:
pytest tests/is green,pytest tests/unit/test_ping.pystays green, and the polluting test resets whatever global state it sets (ideally via an autouse fixture so the class of bug cannot recur).Ref: Found while verifying an unrelated relearn change on #597 — the 4 failures were initially suspected to be new and were ruled out by a stash-and-rerun at HEAD.