How to search same fiile and compare in pandas

Question

I have some directory and files

Directory1

file1
file2
file3

Directory2

file1
file2
file3

I would like to compare each file in each directory.

I tried

'glob.glob'

and

'read csv'

I guess I should have applied

'diff' method.

But I couldnt figure out next step. How can I compare same name files in each directory?

Answer 1

if you could read csv files and convert to data frames then this link might be useful

Answer 2

Can you elaborate on what you mean by compare?

In order to get the list of files in your directory, you can use os.listdir('directory_path'), then you can iterate through the list and compare it with your reference file. Also, this link tells you how to compare the content of two dataframes in pandas.

Answer 3

You can generate a md5 checksum of the file content in a dict and search for equals checksums.

import glob
import hashlib

example = dict(('%s' % _, hashlib.md5(open('%s' % _, 'rb').read()).hexdigest()) for _ in glob.glob('*'))

{'file1': 'b026324c6904b2a9cb4b88d6d61c81d1', 'file2': '26ab0db90d72e28ad0ba1e22ee510510', 'file3': '26ab0db90d72e28ad0ba1e22ee510510', 'file4': '48a24b70a0b376535542b996af517398'}