18

Is there a way to disable Pylint's duplicate-code message just for test files? All of the tests in our project are DAMP so the duplicated code is by design. I understand we can add # pylint: disable=duplicate-code throughout our tests, but would rather add some sort of rule that says all files under a test/ folder will have this rule disabled. Is there a way to do this?

To be more specific, I'm looking for something different from a 'run it twice' solution (which is what I've already fallen back on).

georgexsh
  • 15,984
  • 2
  • 37
  • 62
TinyTheBrontosaurus
  • 4,010
  • 6
  • 21
  • 34
  • 1
    not related to this question, but I do want to know is "DAMP" really a thing? just found a few random posts. – georgexsh Nov 29 '17 at 04:30
  • 1
    i think so? I only heard of it a few months ago, but it all seems to be related to testing. DRY = Don't repeat yourself. and DAMP = Descriptive and Meaningful Phrases. So basically, the tests we write are very repetitive, and that's on purpose. – TinyTheBrontosaurus Nov 29 '17 at 04:32
  • This is the best post i've seen about it: https://stackoverflow.com/questions/6453235/what-does-damp-not-dry-mean-when-talking-about-unit-tests – TinyTheBrontosaurus Nov 29 '17 at 04:32
  • 1
    There is an [issue #618](https://github.com/PyCQA/pylint/issues/618) which is [an ongoing effort of one of contributors](https://github.com/PyCQA/pylint/tree/per_dir_config). Contributions and help are more than welcome. – Łukasz Rogalski Nov 29 '17 at 15:28
  • See also Pylint [issue #214](https://github.com/PyCQA/pylint/issues/214) ("`duplicate-code` can't be disabled"). – akaihola Jul 09 '21 at 14:48

2 Answers2

12

It can be achieved with pylint plugin and some hack.

Assume we have following directory structure:

 pylint_plugin.py
 app
 ├── __init__.py
 └── mod.py
 test
 ├── __init__.py
 └── mod.py

content of mod.py:

def f():
    1/0

content of pylint_plugin.py:

from astroid import MANAGER
from astroid import scoped_nodes


def register(linter):
    pass


def transform(mod):
    if 'test.' not in mod.name:
        return
    c = mod.stream().read()
    # change to the message-id you need
    c = b'# pylint: disable=pointless-statement\n' + c
    # pylint will read from `.file_bytes` attribute later when tokenization
    mod.file_bytes = c


MANAGER.register_transform(scoped_nodes.Module, transform)

without plugin, pylint will report:

************* Module tmp.exp_pylint.app.mod
W:  2, 4: Statement seems to have no effect (pointless-statement)
************* Module tmp.exp_pylint.test.mod
W:  2, 4: Statement seems to have no effect (pointless-statement)

with plugin loaded:

PYTHONPATH=. pylint -dC,R --load-plugins pylint_plugin app test

yields:

************* Module tmp.exp_pylint.app.mod
W:  2, 4: Statement seems to have no effect (pointless-statement)

pylint read comments by tokenizing source file, this plugin change file content on the fly, to cheat pylint when tokenization.

Note that to simplify demonstration, here I constructed a "pointless-statement" warning, disable other types of message is trivial.

georgexsh
  • 15,984
  • 2
  • 37
  • 62
  • 1
    Turns out duplicate-code cannot be disabled per github.com/PyCQA/pylint/issues/214 ;-( – TinyTheBrontosaurus Dec 02 '17 at 21:14
  • 1
    and a very minor issue is that this causes an off-by-one-error on other reported issues since a line was added to the beginning of the `test.` files – TinyTheBrontosaurus Dec 02 '17 at 21:15
  • @TinyTheBrontosaurus here we insert a comment at the first line, to act as a module level command, the caveat is if the real first line of your source file is a statement, it will "consumed" the inserted comment command, [see also](https://github.com/PyCQA/pylint/blob/pylint-1.7.4/pylint/utils.py#L527-L591). – georgexsh Dec 05 '17 at 07:31
  • @TinyTheBrontosaurus line-no in reports should be correct, checker works on AST, which is not changed, at least they are correct in my test. – georgexsh Dec 05 '17 at 07:34
  • @TinyTheBrontosaurus if your first line is a comment or Docstring or blank, it will work as expected. – georgexsh Dec 05 '17 at 07:37
  • @TinyTheBrontosaurus as the Github issue you post pointed out, checkers/similar.py has a different code path, cannot be disabled with comments, so this answer is not helping unless pylint support this feature. – georgexsh Dec 05 '17 at 08:37
  • @TinyTheBrontosaurus I see, as checkers/similar.py re-read the whole file, rather than analyze on AST, line-no would be all have 1 extra offset. – georgexsh Dec 05 '17 at 08:54
  • Neat idea, but I cannot get your example to work with pylint 2.11.1 and python 3.9.5. If I manually copy the `# pylint: disable=...` comment into `test/mod.py`, then it works, but the plugin is not accomplishing this automatically – Addison Klinke Oct 11 '21 at 20:14
  • Really want this to work, but it doesn't for me in pylint 2.13.4. I've gone so far as to verify that the AsteroidManager.stream() bytes do have the pylint modeline at the top of the file, but for whatever reason, pylint ignores them. – dsummersl Apr 05 '22 at 12:35
3

There is the --disable or -d message control flag that can be used to selectively disable messages when called. You could therefore disable this message for all files under the test folder by running pylint on those files from within the project folder:

pylint -d duplicate-code test/

I was able to verify that I can cut out specific messages from all files in the directory, although I was not getting duplicate code errors and so could not check for that message.

You could also put this into a script that you run from your project's main directory. Something like:

#!/bin/bash
pylint src/
pylint -d duplicate-code test/

Alternatively, you could add # pylint: disable=duplicate-code to the top of each of the files for which you want to exclude these messages. It looks like that is about as far as file-wise selective exclusion flags goes for pylint.

Engineero
  • 12,340
  • 5
  • 53
  • 75
  • 4
    this looks like a 'run it twice' solution. – TinyTheBrontosaurus Nov 29 '17 at 15:27
  • @TinyTheBrontosaurus you might be stuck there. I can't find anything in the docs about applying global rules to specific folders, so you may just have to write a script to run your tests in one go on all of your files with different rules defined for the test folder. This is, as far as I can tell, the only way to blanket ignore specific message types for an entire folder. – Engineero Nov 29 '17 at 15:39
  • @TinyTheBrontosaurus I take that back. I might be on to something. Will update in a few. – Engineero Nov 29 '17 at 15:45
  • @TinyTheBrontosaurus it looks like [per-file exclusion is unpopular with pylint devs](https://github.com/PyCQA/pycodestyle/issues/381), although there is [some support](https://stackoverflow.com/questions/26656086/in-pylint-is-there-a-way-to-locally-disable-a-warning-and-then-undo-the-previous), but nothing, from what I can find, for excluding entire directories. It looks like you have to add `# pylint: disable=duplicate-code` to each of your files if you do not want to do it the way suggested here. – Engineero Nov 29 '17 at 16:09
  • @Engineero, adding `# pylint: disable=duplicate-code` to the top of `.py` files has no effect. See Pylint [issue #214](https://github.com/PyCQA/pylint/issues/214). – akaihola Jul 09 '21 at 14:51