How to set a pylint score threshold?

Question

I'd like to extract my pylint rating and set a threshold. Let me explain, for example, if the score is under 5, I want exit 1; And if my code is rated higher than 5, I want exit 0 and continue my Jenkins procedure.

Please, check @Gooseman answer. I think it is the best answer now. — Alexandre Paes, Aug 30 '20 at 23:37

score 39 · Answer 1 · edited Jun 06 '23 at 22:24

39

Since pylint 2.5.0 there is a new argument called --fail-under that resolves this question without needing external tools or scripts.

In this example, pylint will exit with error when score is under 8:

pylint --fail-under=8 python_code.py

edited Jun 06 '23 at 22:24

Cristian Ciupitu

20,270
7
50
76

answered Aug 27 '20 at 22:21

Gooseman

2,102
17
16

2

Thank you, for those who are here for Gitlab CI here's the command I'm using `pylint --fail-under=9 $(git ls-files '*.py')` – Bilal Jul 23 '21 at 16:08
1

pylint v2.11.1 does not obviously fail with `--fail-under=10` ? – El Dude Nov 04 '21 at 19:40
@ElDude maxium score of 10.0 by default. https://pylint.pycqa.org/en/latest/user_guide/output.html#score-section – Gooseman Nov 05 '21 at 13:13
I figured it, but it does not show a fail if < 10, it just shows the score. Need to parse exit value for proper fail as I understand from the docs... – El Dude Nov 05 '21 at 18:44

score 8 · Accepted Answer · answered Jun 22 '18 at 01:37

Here's a way to access the pylint API it in Python. The following code should be saved to a file and executed with first argument to the script to be module/file to lint:

import sys
from pylint import lint

THRESHOLD = 5

if len(sys.argv) < 2:
    raise ArgumentError("Module to evaluate needs to be the first argument")

run = lint.Run([sys.argv[1]], do_exit=False)
score = run.linter.stats['global_note']

if score < THRESHOLD:
    sys.exit(1)

score 4 · Answer 3 · edited Nov 04 '20 at 12:39

4

Install

> pip install pylint-fail-under

And you can check the threshold value as below

pylint-fail-under --fail_under=6.0 test_pylint_code.py (or path)

If the score is below 6.0 it returns a message

ERROR: score 5.3999999999999995 is less than fail-under value 6.0

Else it returns exit code 0.

Link to official documentation is https://pypi.org/project/pylint-fail-under/

edited Nov 04 '20 at 12:39

Community

1
1

answered Apr 19 '19 at 14:21

NMAK

209
2
9

score 0 · Answer 4 · edited Jun 06 '23 at 23:57

The fail-under option also didn't work for me. So, I explored the pylint module code and came across the logic which checks the score against the fail-under value, and then performs sys.exit with different digits:

0 if the score is above the threshold, and
nonzero if below threshold.

In both cases, it exits without raising an error or exception; hence, we no error is output to the console.

Modify the pylint/lint/run.py file as required to force code failure if the score is below some threshold:

if score_value >= linter.config.fail_under:
    sys.exit(0)
else:
    # We need to make sure we return a failing exit code in this case.
    # So we use self.linter.msg_status if that is non-zero, otherwise we just return 1.
    raise RuntimeError(f"ERROR: score {score_value} is less than fail-under value {linter.config.fail_under}")
    #sys.exit(8)

How to set a pylint score threshold?

4 Answers4

Linked