Python 3 bytes formatting

Question

In Python 3, one can format a string like:

"{0}, {1}, {2}".format(1, 2, 3)

But how to format bytes?

b"{0}, {1}, {2}".format(1, 2, 3)

raises AttributeError: 'bytes' object has no attribute 'format'.

If there is no format method for bytes, how to do the formatting or "rewriting" of bytes?

score 52 · Accepted Answer · edited Oct 10 '21 at 04:32

As of Python 3.5, % formatting will work for bytes, too!

This was part of PEP 461, authored by Ethan Furman:

PEP: 461
Title: Adding % formatting to bytes and bytearray
Version: $Revision$
Last-Modified: $Date$
Author: Ethan Furman <ethan at stoneleaf.us>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-01-13
Python-Version: 3.5
Post-History: 2014-01-14, 2014-01-15, 2014-01-17, 2014-02-22, 2014-03-25,
               2014-03-27
Resolution:


Abstract
========

This PEP proposes adding % formatting operations similar to Python 2's ``str``
type to ``bytes`` and ``bytearray`` [1]_ [2]_.


Rationale
=========

While interpolation is usually thought of as a string operation, there are
cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the
work needed to make up for this missing functionality detracts from the overall
readability of the code.


Motivation
==========

With Python 3 and the split between ``str`` and ``bytes``, one small but
important area of programming became slightly more difficult, and much more
painful -- wire format protocols [3]_.

This area of programming is characterized by a mixture of binary data and
ASCII compatible segments of text (aka ASCII-encoded text).  Bringing back a
restricted %-interpolation for ``bytes`` and ``bytearray`` will aid both in
writing new wire format code, and in porting Python 2 wire format code.

Common use-cases include ``dbf`` and ``pdf`` file formats, ``email``
formats, and ``FTP`` and ``HTTP`` communications, among many others.

PEP 461 was accepted by Guido van Rossum on March 27, 2014:

Accepted. Congrats with marshalling yet another quite contentious discussion, and putting up with my last-minute block-headedness!

From this, we can obviously conclude that % is no longer scheduled for deprecation (as was announced with Python 3.1).

score 30 · Answer 2 · answered Jan 09 '14 at 13:20

30

Another way would be:

"{0}, {1}, {2}".format(1, 2, 3).encode()

Tested on IPython 1.1.0 & Python 3.2.3

answered Jan 09 '14 at 13:20

Schcriher

913
10
16

17

This doesn't work if your format arguments are bytes. – user3467349 Mar 01 '15 at 17:49
6

This does not work when you need to insert hard byte codes in the stream that are not supported by Unicode. – EvertW Jun 15 '20 at 10:30

mechanical_meat · Answer 3 · 2013-03-29T20:17:12.743

18

Interestingly .format() doesn't appear to be supported for byte-sequences; as you have demonstrated.

You could use .join() as suggested here: http://bugs.python.org/issue3982

b", ".join([b'1', b'2', b'3'])

There is a speed advantage associated with .join() over using .format() shown by the BDFL himself: http://bugs.python.org/msg180449

edited Mar 29 '13 at 20:17

answered Mar 29 '13 at 20:05

mechanical_meat

163,903
24
228
223

Thanks! The message by BDFL is great find, too. However, this does not solve formatting of, say, floats. It would be interesting to know if there is any other way besides `str(some_bytes).encode()` – Ecir Hana Mar 29 '13 at 20:25
Understood. A `.format()` method for bytes does appear to be on track for inclusion in 3.4+ – mechanical_meat Mar 29 '13 at 20:39
1

@bernie: But the `__format__` method of an object and built-in `format` return `str`. How will that work with `bytes` and encodings? – Eryk Sun Mar 29 '13 at 23:54
4

@bernie: [`bytes.format()` pep is withdrawn](https://www.python.org/dev/peps/pep-0460/). [printf-style formatting (%-interpolation) is introduced in Python 3.5 instead](https://www.python.org/dev/peps/pep-0461/) – jfs Aug 01 '15 at 12:26

score 18 · Answer 4 · edited Oct 16 '19 at 19:40

18

I found the %b working best in Python 3.6.2, it should work both for b"" and "":

print(b"Some stuff %b. Some other stuff" % my_byte_or_unicode_string)

edited Oct 16 '19 at 19:40

pts

80,836
20
110
183

answered Nov 09 '17 at 16:39

Ivan Bilan

2,379
5
38
58

2

it should be `%b` not `%a` for byte string, otherwise 'b' prefix is included – panda-34 Sep 12 '18 at 12:50
It doesn't work in Python 2.7.16 though, so it's not backward-compatible. – pts Oct 16 '19 at 19:40
2

It doesn't work with unicode string. Only b'' is supported. – Maxim Suslov Aug 26 '20 at 01:22

Nurbol Alpysbayev · Answer 5 · 2020-11-01T18:14:14.757

13

For Python 3.6+ you can use this nice and clean syntax:

f'foo {bar}'.encode() # a byte string

edited Nov 01 '20 at 18:14

answered Aug 20 '20 at 09:46

Nurbol Alpysbayev

19,522
3
54
89

4

This does formatting on `str`, not `bytes`. For example, it won't give the intended result if `bar` is a `bytes` object. Or if you want to put non-ASCII byte values in the format string. – interjay Mar 18 '21 at 12:30
@interjay of course there are all sorts of cases, where my solution wouldn't be helpful, including the one you mentioned. But this doesn't negate the fact there are remaining cases, including the one I had, when the solution works. – Nurbol Alpysbayev Mar 19 '21 at 14:02
1

This should be the accepted answer because describes the simplest and best solution. – SLDem Jul 28 '21 at 06:42
4

This is effectively a duplicate of [Schcriher's answer](/a/21021433/4518341) and has the exact same problems. – wjandrea Aug 26 '21 at 01:31

score -4 · Answer 6 · answered May 22 '20 at 03:58

-4

I've found this to work.

a = "{0}, {1}, {2}".format(1, 2, 3)

b = bytes(a, encoding="ascii")

>>> b
b'1, 2, 3'

answered May 22 '20 at 03:58

binny

649
1
8
22

3

This is a duplicate of [Schcriher's answer](/a/21021433/4518341), just using a different encoding process – wjandrea Aug 26 '21 at 01:35

Python 3 bytes formatting

6 Answers6

Linked

Related