2

I am trying to read some txt files, tab-delimited data and using the code below, it shows me the following error:

for row in reader:

Error: line contains NULL byte

The code is basically trying to to get some information from specific columns. An abridged version of it below:

with open('07table.txt', encoding = "utf-8", errors='ignore') as fo:
    reader = csv.reader(fo, delimiter='\t', quoting=csv.QUOTE_MINIMAL)

For the past week, I tried to investigate if the files contain NULL bytes. I've used the code below to check whether this is true, and to my surprise I didn't find any NULL bytes in the files:

if '\x00' in open('07table.txt').read():
    print("you have null bytes in your input file")
else:
    print("you don't")

Just to be on the safe side, I also ran a code that replaces NULL bytes in my txt files, but I still have the same error.

Another thing that I have tried so far is to change to an older version of Spyder and downgrade the Python version to 3.5.14 because I thought it might be an issue with the environment I have installed. This has had some effects in the sense that my code successfully reads two files that were previously supposed to contain NULL bytes. However, when it gets past these 2 files, the code renders the same error as posted above.

At this point, I am very sure that the files do not contain NULL bytes. Any thoughts on why I could get this error?

nik7
  • 806
  • 3
  • 12
  • 20
TheWalküre
  • 45
  • 4
  • Have you tried this link? https://stackoverflow.com/questions/7894856/line-contains-null-byte-in-csv-reader-python. Which mentions trying to open the file with utf-16 encoding. – apoorva kamath Jun 02 '20 at 19:02
  • Thank you for your reply! I forgot to mention, but I actually tried that suggestion. For every file I try to decode using UTF-8, UTF-16, or latin-1. Most of the files are encoded in UTF-8. – TheWalküre Jun 02 '20 at 19:25

1 Answers1

0

I found what the issue was. I was reading the files from an external hard drive formatted in NFTS, while the code was running on a macOS formatted in HFS.

After formatting the external drive to match the formatting on my laptop, the problem of null bytes disappeared.

TheWalküre
  • 45
  • 4