Issues in Reading MemoryObject of file

Question

• I have a function who reads MemoryObject of file. It is reading most of the files but for one file it is raising UnicodeDecodeError.

here is my code

def read(file):
    """

    :param file: File Memory Object (submitted from POST)
    :return: File Iterable object
    """

    file = StringIO(file.read().decode())
    return csv.DictReader(file, delimiter=',')

• File which raise no issue.
• File which is rasing issue.

Complete error is as follows : UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

• In other question where user have asked questions of similar errors. they are using open() to streamline the file but I already have streamline MemoryObject so can't use open()

Answer 1

Your file is already opened in binary mode: decode is a method of bytes, not str.

For your problem, the encoding and errors parameter of bytes.decode works the same as for open. You can apply the appropriate encoding, or ignore errors:

def read(file, encoding: str = 'utf-8', errors: str = 'strict'):
    """
    :param file: File Memory Object (submitted from POST)
    :return: File Iterable object
    """

    file = StringIO(file.read().decode(encoding=encoding, errors=errors))
    return csv.DictReader(file, delimiter=',')

Note that you either must know the encoding, or suppress errors by ignoring them. You can try different encodings to find one that works, but in the end you must know what your data means.