linux lines counting not working with python code

Question

I am using WC -l to count number of lines in a text document. However I got a problem here.

I got a python code that wrote different combination of numbers into different files. each file contain each number of the combination on a separate line.

When I use wc -l for it, it didnt count the last line!

below is the python code:

import os
import itertools
lst = [6,7,8,9,12,19,20,21,23,24,26,27,28,29,43,44]
combs = []
for i in xrange(1, len(lst)+1):
els = [list(x) for x in itertools.combinations(lst, i)]
combs.extend(els)
for combination in els:
  combination_as_strings = map(str, combination)
  filename = "_".join(combination_as_strings) + ".txt"
  filename = os.path.join("Features", filename)
  with open(filename, 'w') as output_file:
     output_file.write("\n".join(combination_as_strings))

Thanks in advance,

Ahmad

Answer 1

The join you are using is placing newlines between lines, but not at the end of the last line. Hence, wc is not counting the last line (it counts the number of newlines).

Add output_file.write("\n") at the end of your script, in the with clause:

  with open(filename, 'w') as output_file:
     output_file.write("\n".join(combination_as_strings))
     output_file.write("\n")

Answer 2

I think you are seeing a variant of this:

$ printf '1\n2\n3' | wc -l

at the Bash prompt type this -- prints 2 because there is no final \n

Compare to:

$ printf '1\n2\n3\n' | wc -l

which prints 3 because of the final \n.

Python file methods do not append a \n to their output. To fix you code, either use writelines like so:

with open(filename, 'w') as output_file:
    output_file.writelines(["\n".join(combination_as_strings),'\n'])

or print to the file:

with open(filename, 'w') as output_file:
     print >>output_file, "\n".join(combination_as_strings)

or use a format template:

with open(filename, 'w') as output_file:
     output_file.write("%s\n" % '\n'.join(combination_as_strings))

Answer 3

Why not use writelines?

output_file.writelines(line+"\n" for line in combination_as_strings)

Answer 4

The wc command counts the number of new line characters in the file (\n).

So if you have 10 lines on a file, it'll return 9, because you will have 9 new line characters.

You could make it work as you want by adding an empty new line at the end of each file.