split bytes variable on newline

Question

This is an unusual request, and I would appreciate some guidance! :)

I have a python variable, for simplicity we can call it 'output'

When I print output I get the following:

b"word1\nword2\nword3\n"

I would love it if I could print

word1
word2
word3
word4

I have tried to split the variable, and feed it to a for loop with zero success. I am happy to write the output to a file in the OS and use bash to resolve the issue as well.

Thanks!

Does this answer your question? [split byte string into lines](https://stackoverflow.com/questions/13857856/split-byte-string-into-lines) — Josh Correia, Oct 08 '20 at 22:25

score 8 · Answer 1 · answered Apr 07 '16 at 17:12

It sounds like you are using Python 3 and invoking the equivalent of

>>> print(b"foo\nbar\nbaz")
b'foo\nbar\nbaz'

This is str(bytes) in action: the (Unicode) string representation of a byte string. By decoding it first you get something that Python 3 will print more elegantly.

>>> print(b"foo\nbar\nbaz".decode('utf-8'))
foo
bar
baz

birdoftheday · Accepted Answer · 2016-04-07T16:43:21.767

6

You'll need to do this (see the string.split function for more details)...

for word in output.decode('utf-8').split('\n'):
    print word

And you don't need to print word - you can do anything you want with it. This loop will iterate over every line in output.

edited Apr 07 '16 at 16:43

answered Apr 07 '16 at 15:51

birdoftheday

816
7
13

Calling split with no args will split on more than just `\n` (which may or may not matter here) – Two-Bit Alchemist Apr 07 '16 at 15:53
1

If `\n` is being printed, that means that the actual string includes `\\n`. – zondo Apr 07 '16 at 15:55
@jeff_h is there any reason you're using bytes instead of a string? – birdoftheday Apr 07 '16 at 15:56
I have tried splitting before I get the error: for i in output.split('\n'): TypeError: a bytes-like object is required, not 'str' – jeff_h Apr 07 '16 at 15:57
I am using the variable name output, this is how it is created: output = ssh_stderr.read() + ssh_stdout.read() – jeff_h Apr 07 '16 at 15:57
The data that forms the variable is coming through paramiko – jeff_h Apr 07 '16 at 15:57
@jeff_h Check this out: http://stackoverflow.com/questions/606191/convert-bytes-to-a-python-string If you first decode the data into a string before doing what I (or any other answer) said, it should work. – birdoftheday Apr 07 '16 at 16:01
@stanpines testing that idea now with print(output.decode("utf-8")) – jeff_h Apr 07 '16 at 16:03
@stanpines you were correct, if you add it as an answer I can mark it as resolved :) – jeff_h Apr 07 '16 at 16:11
@jeff_h how's that? :) – birdoftheday Apr 07 '16 at 16:43
@stanpines You don't need the split :) – jeff_h Apr 11 '16 at 10:45

score 2 · Answer 3 · answered Apr 07 '16 at 15:53

2

To me it sounds like your string has escaped newlines. str.split won't help you here, nor str.splitlines. You need to decode the escapes:

>>> print s
word1\nword2\nwored3\n
>>> print s.decode('string-escape')
word1
word2
wored3

answered Apr 07 '16 at 15:53

wim

338,267
99
616
750

Interesting, so I should try: print(output.decode('\\n')) ? :) – jeff_h Apr 07 '16 at 16:00

score 2 · Answer 4 · answered Jul 08 '20 at 01:35

Assuming Python 3. You have a bytes string (bytes), different from a unicode string (str).

bstring = b"word1\nword2\nword3\nword4"

The newline '\n' is the same as u'\n' (unicode string) and different from b'\n'. If you are interested simply in the splitting you can use b'\n' as separator:

for w in bstring.split(b'\n'):
    print(w)

This will print bytes strings b'word1', ... If you want regular strings you have to decode before or after the splitting like shown in the other solutions:

for w in bstring.decode().split('\n'):
    print(w)

This uses your default encoding (normally utf-8), if you want a different one you can pass it as argument to decode(). You can find more in the Python manual

score -1 · Answer 5 · answered Apr 07 '16 at 15:51

-1

output = b"word1\nword2\nword3\nword4\n"
for w in output.split():
    print(w)

answered Apr 07 '16 at 15:51

dsh

12,037
3
33
51

split bytes variable on newline

5 Answers5