Best way to detect duplicate files in a certain folder android

Question

What is the best way to detect duplicate files in android, currently I am using files MD5# to detect duplicates, it works well but it is very slow and I don't know is it safer to user MD5# for detecting duplicates. Also it is not useful in the case of images when its resolution or format is different

Is there any better way to do this? I mean by using any library or algorithms

My case is that I was trying to build a duplicate detection app like the many in playstore as a part of my project

Actually this is not Android specific – Willi Mentzel Feb 06 '18 at 10:48 — Willi Mentzel, Feb 06 '18 at 10:48

score 1 · Answer 1 · edited Feb 06 '18 at 11:18

1

What are you trying to do?

If two image files have different formats and resolutions, then they are NOT duplicates, are they ?

You could speed up the MD5 process by first checking the file-size. If the file size is different, then there is no need to checksum, they are guaranteed to be different.
Perhaps build a binary tree using bits at specific offsets? For example, bit 1,2,4,8,16,32,64,128,256,512,128,1024,2048. Then only checksum files attached to the same node, that are the same file-size... Files attached to different nodes are guaranteed to be different.

We could help more if we knew more about the problem you are you trying to solve.

edited Feb 06 '18 at 11:18

Tejas Pandya

3,987
1
26
51

answered Feb 06 '18 at 10:26

cds84

343
1
14

@cds84 added my case in the question itself – AbuMaaiz Feb 06 '18 at 11:10
In that case, take a look at https://stackoverflow.com/questions/8644960/java-library-to-compare-image-similarity. – cds84 Feb 06 '18 at 11:19

Best way to detect duplicate files in a certain folder android

1 Answers1