Skip to content

Implementing no_cov feature for #131 #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ionelmc merged 3 commits into from
Oct 9, 2016
+51 −0
Merged

Implementing no_cov feature for #131 #135

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e391314
Implementing no_cov feature for #131
Oct 4, 2016
e78d309
tests don't want to work
Oct 4, 2016
c6478ce
rw flag does not work either
Oct 5, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,19 @@ subprocess. The python used by the subprocess must have pytest-cov installed.
do normal site initialisation so that the environment variables can be detected and coverage
started.

Coverage and debuggers
----------------------

When it comes to TDD one obviously would like to debug tests. Debuggers in Python use mostly the sys.settrace function
to gain access to context. Coverage uses the same technique to get access to the lines executed. Coverage does not play
well with other tracers simultaneously running. This manifests itself in behaviour that PyCharm might not hit a
breakpoint no matter what the user does. Since it is common practice to have coverage configuration in the pytest.ini
file and pytest does not support removeopts or similar the `--no-cov` flag can disable coverage completely.

At the reporting part a warning message will show on screen

WARNING: Coverage disabled by user!

Acknowledgements
================

Expand Down
25 changes: 25 additions & 0 deletions src/pytest_cov/plugin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ def pytest_addoption(parser):
group.addoption('--no-cov-on-fail', action='store_true', default=False,
help='do not report coverage if test run fails, '
'default: False')
group.addoption('--no-cov', action='store_true', default=False,
help='Disable coverage report completely (useful for debuggers) '
'default: False')
group.addoption('--cov-fail-under', action='store', metavar='MIN', type=int,
help='Fail if the total coverage is less than MIN.')
group.addoption('--cov-append', action='store_true', default=False,
Expand Down Expand Up @@ -131,11 +134,16 @@ def __init__(self, options, pluginmanager, start=True):
self.cov_total = None
self.failed = False
self._started = False
self._disabled = False
self.options = options

is_dist = (getattr(options, 'numprocesses', False) or
getattr(options, 'distload', False) or
getattr(options, 'dist', 'no') != 'no')
if options.no_cov:
self._disabled = True
return

if is_dist and start:
self.start(engine.DistMaster)
elif start:
Expand All @@ -144,6 +152,7 @@ def __init__(self, options, pluginmanager, start=True):
# slave is started in pytest hook

def start(self, controller_cls, config=None, nodeid=None):

if config is None:
# fake config option for engine
class Config(object):
Expand All @@ -170,6 +179,12 @@ def _is_slave(self, session):

def pytest_sessionstart(self, session):
"""At session start determine our implementation and delegate to it."""

if self.options.no_cov:
# Coverage can be disabled because it does not cooperate with debuggers well.py
self._disabled = True
return

self.pid = os.getpid()
if self._is_slave(session):
nodeid = session.config.slaveinput.get('slaveid',
Expand Down Expand Up @@ -207,6 +222,9 @@ def _failed_cov_total(self):
def pytest_runtestloop(self, session):
yield

if self._disabled:
return

compat_session = compat.SessionWrapper(session)

self.failed = bool(compat_session.testsfailed)
Expand All @@ -226,6 +244,13 @@ def pytest_runtestloop(self, session):
compat_session.testsfailed += 1

def pytest_terminal_summary(self, terminalreporter):
if self._disabled:
markup = {'red': True, 'bold': True}
msg = (
'WARNING: Coverage disabled by user!'
Copy link
Copy Markdown
Member

@ionelmc ionelmc Oct 4, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can only assume that you make this extra cruft here to make a longer message and use the pytest's warn system (in addition to writing to terminalreporter, of course)?

If so, you can use something like terminalreporter.config.warn(code='COV-U1', message=msg).

Also, I would mention "--no-cov" somehow in the warning message, just to be clear about what's going on.

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 4, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copied from elsewhere in the file. It uses the same technique. I was consistent with its style

Copy link
Copy Markdown
Member

@ionelmc ionelmc Oct 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, you're right. Still, I think it's good if we'd use the config.warn thing here ...

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the reason for not mentioning the no-cov flag in there is precisely to avoid spaghetti coding. You would constantly have to maintain this line whenever you decide to rename this variable. Unfortunate enough already that I have to copy it in the README.rst I know that Ctrl+R exists but some ppl. prefer typing. Not to mention --no-cov vs. no_cov. By all means I will make the changes but wanted you to know the reason just for the record.

Copy link
Copy Markdown
Member

@ionelmc ionelmc Oct 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we can grep/sed for --no-cov if we ever decide to rename 😀

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for the test

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I test this? Now the log line is simply not there in the stderr/stdout?

Copy link
Copy Markdown
Member

@ionelmc ionelmc Oct 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would send that message both to the warner and terminalwriter.

To test the warning you need to configure pytest to show warnings (add -rw to commandline).

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you duplicate the message? Also it writes on the console for me if I run py.test without any flags but the test framework does not pick those lines up. Just

     assert 'WCOV-U1 None Coverage disabled via --no-cov switch!' in  result.stdout.str()
E   assert 'WCOV-U1 None Coverage disabled via --no-cov switch!' in '============================= test session starts ==============================\nplatform darwin -- Python 2.6.8 -- p...D\ntest_no_cov.py::test_foo[9] PASSED\n\n==================== 10 passed, 1 warnings in 0.02 seconds ====================='
E    +  where '============================= test session starts ==============================\nplatform darwin -- Python 2.6.8 -- p...D\ntest_no_cov.py::test_foo[9] PASSED\n\n==================== 10 passed, 1 warnings in 0.02 seconds =====================' = <bound method LineMatcher.str of <_pytest.pytester.LineMatcher instance at 0x1050df248>>()

Copy link
Copy Markdown
Contributor Author

@kozmaz87 kozmaz87 Oct 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the -rw flag to the test does not help

)
terminalreporter.write(msg, **markup)
return
if self.cov_controller is None:
return

Expand Down
13 changes: 13 additions & 0 deletions tests/test_pytest_cov.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,19 @@ def test_no_cov_on_fail(testdir):
result.stdout.fnmatch_lines(['*1 failed*'])


def test_no_cov(testdir):
script = testdir.makepyfile(SCRIPT)

result = testdir.runpytest('-v',
'--cov=%s' % script.dirpath(),
'--cov-report=term-missing',
'--no-cov',
script)

assert 'WARNING: Coverage disabled by user!' in result.stdout.str()
assert result.ret == 0


def test_cov_and_failure_report_on_fail(testdir):
script = testdir.makepyfile(SCRIPT + SCRIPT_FAIL)

Expand Down