Tips to show similarities in files

Question

In a project, I found some css files that "smell" like there are copy-pasted rules in them.

I wonder what are your strategies for detecting copy-paste stuff in files.

Just of curiosity i'd like to hear your tips and tricks for showing file similarities!

Related question - http://stackoverflow.com/questions/2490884/why-is-copy-and-paste-of-code-dangerous — Oded, Jul 25 '10 at 17:46
possible duplicate of [How to find a similar code fragment?](http://stackoverflow.com/questions/709744/how-to-find-a-similar-code-fragment) — Ciro Santilli OurBigBook.com, Aug 31 '14 at 13:16

score 5 · Accepted Answer · edited Jul 17 '12 at 11:13

5

It is used for copy-paste-detection in source code (Java, C#, C, C++, COBOL, Ruby, JSP, ASP, HTML, XML, Visual Basic, Groovy), but you can run this on plain text files too.

edited Jul 17 '12 at 11:13

Sorcy

2,587
5
26
34

answered Oct 08 '09 at 09:32

The Chairman

7,087
2
36
44

is there a way to make simian show the actual similarities? On my test-run, it only shows a summary that "lines a to b in file X are similar to lines c to d in file Y"??? Showing the actual lines will help clarify a lot – Jesper Rønn-Jensen Oct 08 '09 at 12:02
Or better yet: is there a way to make it show diffs for the lines that are similar? – Jesper Rønn-Jensen Oct 08 '09 at 12:02
I'm afraid you'll have to check the results manually or write a tool that evaluates Simian results and attaches them to a file viewer. Maybe you could utilize a free diff tool (e.g. WinMerge, http://winmerge.org/) for that purpose. If you decide for the latter: I would be interested in such a solution too... – The Chairman Oct 11 '09 at 10:08
See my answer on CloneDR for a tool that explicitly shows similarities and differences. – Ira Baxter Feb 01 '10 at 18:45
@Jesper: Simian has a `-reportDuplicateText` option that instructs it to include the duplicated text in reports. – John Gietzen Mar 11 '11 at 18:44

score 4 · Answer 2 · edited May 23 '17 at 11:53

4

There is a Copy-Paste Detection (CPD) project on sourceforge; http://pmd.sourceforge.net/cpd.html

But even in large projects I find my own knowledge of the code to be a reliable (although not foolproof) detection mechanism.

Also see this question for other suggestions.

edited May 23 '17 at 11:53

Community

1
1

answered Oct 08 '09 at 09:25

Ed Guiness

34,602
16
110
145

1

The CPD tool seems very limited in languages. And does not have javascript or CSS – Jesper Rønn-Jensen Oct 08 '09 at 09:34
Thanks for pointing to "this question". Very similar to this, btw :) – Jesper Rønn-Jensen Oct 08 '09 at 10:19

Ira Baxter · Answer 3 · 2016-07-15T14:30:19.200

Our Semantic Designs CloneDR is a tool that detects copy-paste-edit blocks of code, for many languages: C, C++, Java, C++, COBOL, ECMAScript, PHP, VB6, VB.net, ...

It does using language-accurate parsers to build abstract syntax trees, corresponding to exact program structures, which are then compared for similarity. This means it is not confused in any way by whitespace, formmatting, comments, or even different "spelling" of literals (e.g., 3.14159 is the same as .00314150E3).

It generates a report that shows exactly how the blocks of code are similar, and precisely how they vary. You can see sample reports at the link.

Your link is not working? Could you please update your response? — Jesper Rønn-Jensen, Feb 01 '10 at 19:37

Tips to show similarities in files

3 Answers3

Linked