Python encryption unicode error when converting from Python 2 to python 3

Question

I found some code which I want to incorporate into my Python encryption program. It should encrypt the files in the code's same directory, and I want it to target a directory. But, it's written in Python 2 and when I change around some code to fit Python 3, I get the following error:

Traceback (most recent call last):
  File "/home/pi/Desktop/Projects/FyleCript/Dev Files/encryption.py", line 77, in <module>
    encrypt(SHA256.new(password).digest(), str(Tfiles))
  File "/usr/lib/python3/dist-packages/Crypto/Hash/SHA256.py", line 88, in new
    return SHA256Hash().new(data)
  File "/usr/lib/python3/dist-packages/Crypto/Hash/SHA256.py", line 75, in new
    return SHA256Hash(data)
  File "/usr/lib/python3/dist-packages/Crypto/Hash/SHA256.py", line 72, in __init__
    HashAlgo.__init__(self, hashFactory, data)
  File "/usr/lib/python3/dist-packages/Crypto/Hash/hashalgo.py", line 51, in __init__
    self.update(data)
  File "/usr/lib/python3/dist-packages/Crypto/Hash/hashalgo.py", line 69, in update
    return self._hash.update(data)
TypeError: Unicode-objects must be encoded before hashing

But the code works perfectly in Python 2. I have tried looking for similar questions on SO and Googling, but no help.

Code:

def encrypt(key, filename):
        chunksize = 64 * 1024
        outFile = os.path.join(os.path.dirname(filename), "(encrypted)"+os.path.basename(filename))
        filesize = str(os.path.getsize(filename)).zfill(16)
        IV = ''
 
        for i in range(16):
                IV += chr(random.randint(0, 0xFF))
       
        encryptor = AES.new(key, AES.MODE_CBC, IV)
 
        with open(filename, "rb") as infile:
                with open(outFile, "wb") as outfile:
                        outfile.write(filesize)
                        outfile.write(IV)
                        while True:
                                chunk = infile.read(chunksize)
                               
                                if len(chunk) == 0:
                                        break
 
                                elif len(chunk) % 16 !=0:
                                        chunk += ' ' *  (16 - (len(chunk) % 16))
 
                                outfile.write(encryptor.encrypt(chunk))
 
 
def decrypt(key, filename):
        outFile = os.path.join(os.path.dirname(filename), os.path.basename(filename[11:]))
        chunksize = 64 * 1024
        with open(filename, "rb") as infile:
                filesize = infile.read(16)
                IV = infile.read(16)
 
                decryptor = AES.new(key, AES.MODE_CBC, IV)
               
                with open(outFile, "wb") as outfile:
                        while True:
                                chunk = infile.read(chunksize)
                                if len(chunk) == 0:
                                        break
 
                                outfile.write(decryptor.decrypt(chunk))
 
                        outfile.truncate(int(filesize))
       
def allfiles():
        allFiles = []
        for root, subfiles, files in os.walk(os.getcwd()):
                for names in files:
                        allFiles.append(os.path.join(root, names))
 
        return allFiles
 
       
choice = input("Do you want to (E)ncrypt or (D)ecrypt? ")
password = input("Enter the password: ") 

encFiles = allfiles()
 
if choice == "E" or 'e':
        for Tfiles in encFiles:
                if os.path.basename(Tfiles).startswith("(encrypted)"):
                        print("%s is already encrypted" %str(Tfiles))
                        pass
 
                elif Tfiles == os.path.join(os.getcwd(), sys.argv[0]):
                        pass
                else:
                        encrypt(SHA256.new(password).digest(), str(Tfiles))
                        print("Done encrypting %s" %str(Tfiles))
                        os.remove(Tfiles)
 
 
elif choice == "D" or 'd':
        filename = input("Enter the filename to decrypt: ")
        if not os.path.exists(filename):
                print("The file does not exist")
                sys.exit()
        elif not filename.startswith("(encrypted)"):
                print("%s is already not encrypted" %filename)
                sys.exit()
        else:
                decrypt(SHA256.new(password).digest(), filename)
                print("Done decrypting %s" %filename)
                os.remove(filename)
 
else:
        print("Please choose a valid command.")
        sys.exit()

Can anyone help me with this problem? I have used a Python 2 to 3 tool, but it still didn't work.

Also, could you please fix the directory problem? It isn't necessary, but I would like it.

EDIT: I have replaced str with bytes and bytearray but it returns the same error.

Did you try using `bytes` instead of `str`? Python 2 strings are just arrays of bytes; Python 3 strings are Unicode, and as the error message says, they need to be encoded into bytes before doing binary things like encryption. — Tom Zych, Jul 15 '18 at 17:22
Your example does not seem to fulfill minimal verifiable criteria. — guidot, Jul 15 '18 at 17:24
The modified code with `bytes` instead of `str` still returns the same error — Trooper Z, Jul 15 '18 at 17:25
You have implicit strings too, e.g. `IV`. I don’t have this package on my computer, can’t experiment easily. I recommend you look at the docs and make sure you’re passing a valid type to `encrypt` and anything else that gives an error, then work backwards until all the types are correct. Note that `bytes` is immutable; use `bytearray` when you need something mutable. — Tom Zych, Jul 15 '18 at 17:32
What I would like is for someone to give me some straightforward code or edited parts of my code because the substitution of `str` with `bytes` or `bytearray` doesn't work. — Trooper Z, Jul 15 '18 at 17:53
`chr()` is problematic, too. I would collect the values into a list without `chr()` or any other conversion and then pass that list to `bytes()` when you're done looping. — tripleee, Jul 15 '18 at 18:29
Not in a place where I can test code but `ivlist=[]; for i in range(16): ivlist.append(random.randint(0, 0xFF)); IV=bytes(ivlist)` ... and obviously more elegant with a list comprehension so you can avoid the stupid temporary variable. — tripleee, Jul 15 '18 at 18:36
@tripleee If you could please edit my whole code and put it in the answer section so I can make this answer answered — Trooper Z, Jul 15 '18 at 18:40
Feel free to answer this yourself. You can mark it "community wiki" if you'd rather abstain from gaining reputation from it and mark the answer as a community effort. Obviously either way don't hesitate to lift text from my comments if you like. — tripleee, Jul 15 '18 at 18:50
@tripleee But I am having trouble putting this code you put into my code. — Trooper Z, Jul 15 '18 at 18:51
So do I, on a mobile device far away from anything with a decent keyboard or screen. — tripleee, Jul 15 '18 at 18:52
try looking at the .decode('utf8') and .encode('utf8') string methods (or maybe 'ascii' instead of 'utf8', depending on your files. — Louic, Jul 15 '18 at 20:45

Louic · Accepted Answer · 2018-07-15T22:59:12.633

3

Your 'password' variable is a string, but SHA256.new expects bytes (to allow for unicode, for example). That you need bytes is listed in the Crypto.Hash.SHA256 documentation.

The solution is to encode the password to bytes before hashing. This is almost literally what the error message says (if you know that all strings in python 3 are unicode objects):

TypeError: Unicode-objects must be encoded before hashing

The solution is for example to encode with utf8 :

SHA256.new(password.encode('utf8')).digest()

edited Jul 15 '18 at 22:59

answered Jul 15 '18 at 22:45

Louic

2,403
3
19
34

This works but there's a new error: `ValueError: IV must be 16 bytes long` – Trooper Z Jul 16 '18 at 00:42
1

Please start a new question if you want to ask about this, and make sure you provide details and show what you have tried already. – Louic Jul 16 '18 at 06:23

Python encryption unicode error when converting from Python 2 to python 3

1 Answers1

Linked