Static Code Analysis Tools -

Question

How Many of you use static code analysis tools during development and what is the amount of false positives you experience with the tool you use? This is for a research.

In order to succeed in research, you will have to ramp up your efforts eventually, I'm afraid. — barfuin, Nov 30 '17 at 09:27
Characteristics of PVS-Studio Analyzer by the Example of EFL Core Libraries, 10-15% of False Positives - https://www.viva64.com/en/b/0523/ — AndreyKarpov, Nov 30 '17 at 20:36
I will ramp it up, as soon as this semester gets to an end. Thanks for the suggestion though @ThomasJensen — Adithya, Dec 01 '17 at 15:14

score 2 · Answer 1 · answered Dec 04 '17 at 11:35

2

You can try CppDepend, it uses CQLinq a code query language to define the rules. You can customize easilly the rules to adapt it to your needs and remove the false positives detections.

answered Dec 04 '17 at 11:35

James from CppDepend Team

1,365
8
9

score 1 · Answer 2 · answered Feb 09 '18 at 19:36

Serious research will compare not just the rate of false positives to true positives (precision), but also the rate of false negatives, issues that are not reported by a tool (recall). What I expect you'll find is that a tool with less false positives will have more false negatives.

Finding the false negatives is not something trivial. There are test suites (like https://www.nist.gov/publications/juliet-11-cc-and-java-test-suite), but that is synthetic code and not production level code with multiple levels of indirection and complex flow of logic.

Good luck!

score 1 · Answer 3 · answered Feb 13 '18 at 13:54

As a very rough rule of thumb, I estimate a good commercial-grade static analysis tool will have 20% genuine false positives, and will find about 20% of genuine bugs. What’s more relevant in practice is that a much higher percentage of reported defects will be incorrectly classified as false positives, often because the developer, rightly or wrongly, thinks the bug is unlikely to happen. (FindBugs has a “Mostly Harmless” classification for this.) Discussion and references in my Dr Dobb’s article, http://pobox.com/~flash/Deploying_Static_Analysis.pdf.

Mixhab · Answer 4 · 2017-11-29T22:28:14.353

0

I'm using Coverity for 2 years. So far didn't get any false positive.

Previously I used Code Sonar. I got false positives once in a while.

edited Nov 29 '17 at 22:28

answered Nov 29 '17 at 22:18

Mixhab

421
3
14

1

If you never saw a false positive, you should be more aggressive and turn on more checkers. – Flash Sheridan Feb 13 '18 at 13:55

score 0 · Answer 5 · answered Feb 12 '18 at 16:32

I added rules progressively (via project specific config) and thus have very few false positives (or known ones that I know how to deal with). If you just take that standard FingBugs rules, you will have too many false positives, this is why people generally stop using it. In addition to this I added FindBugs contrib rules.

For JavaScript I use jsHint.

False positives depend on the active rules... I fear your research will be quite difficult due to this fact.

Static Code Analysis Tools -

5 Answers5