7

I think I understand python bytes objects, but supporting bitwise operations on byte strings seems like such an obvious feature. I don't understand why it is not supported.

>>>'abcdefg'.encode('ascii')
b'abcdefg'

Okay. I went from a string to something like the byte representation of my string in ascii.

So when I try:

>>> a = 'abcdefg'.encode('ascii')
>>> a ^ a
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for ^: 'bytes' and 'bytes'

Why? Why doesn't python support this? Is there something I don't understand about bytes objects that makes this unfeasible or ambiguous?

Elias Zamaria
  • 96,623
  • 33
  • 114
  • 148
cowlicks
  • 1,037
  • 1
  • 12
  • 20
  • 2
    There are work arounds for this, Check out this stackoverflow thread: http://stackoverflow.com/questions/2612720/how-to-do-bitwise-exclusive-or-of-two-strings-in-python – Brian Cain Sep 18 '15 at 18:14
  • 1
    @BrianCain I realize there are workarounds. I was asking why it is not supported. – cowlicks Sep 18 '15 at 18:15
  • @BrianCain there is a comment in that question basically saying that python should support this. So maybe the answer is "No, python should do this"? – cowlicks Sep 18 '15 at 18:16
  • Probably no one has proposed a PEP for this yet – Chad S. Sep 18 '15 at 18:25
  • 1
    [this](https://mail.python.org/pipermail/python-dev/2006-March/061980.html) mailing list post seems relevant. The conclusion appears to be "put it in a PEP and we'll think about it" – Kevin Sep 18 '15 at 18:36
  • 1
    defining xor for bytes is not so straightforward if you consider data of different length - see [this](https://mail.python.org/pipermail/python-list/2004-October/279803.html) python mailinglist answer for reference – mata Sep 18 '15 at 18:37
  • What should `b'a' ^ b'ab'` evaluate to? I can thing of four options off the top of my head, depending on whether you pad `'a'` to `'\x00a'` or `'a\x00'`, or you return a one-byte answer with `'a'` xor'd with either the first or second byte of `'ab'`. – chepner Sep 18 '15 at 18:56
  • 1
    @chepner -- So you cannot conceive of the obvious `ValueError -- array lengths must match` ??? – Patrick Maupin Sep 20 '15 at 04:47
  • That's not much more interesting than the current `TypeError`, so I ignored it. – chepner Sep 20 '15 at 12:36

2 Answers2

5

This feature has been proposed on the python bug tracker. And there is proposed patch (which I wrote after being annoyed by this). But the current feedback is negative, and more is needed if it is to be included.

cowlicks
  • 1,037
  • 1
  • 12
  • 20
  • Wow. Just followed your link, and...what a rabbit-hole that discussion is. As an embedded coder, it just seems logical to me that if you're in a domain where you need bytes and bytearrays, you're not far off from needing bitwise operations. Sad. – Matt Jul 17 '20 at 18:36
0

It is not supported primarily because it is only rarely needed in "normal" Python code, and by the time you do need it you are probably already using a third-party package, like NumPy, that has such things (and more!) built-in and very performant... at least, much more performant than standard Python would be.

Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
  • 4
    Hello Ethan, NumPy is a heavyweight dependency (it needs a c compiler). And it is not portable to jython, or pypy. So responding to "why doesn't standard python support this"? With "because a heavyweight non-portable third party library already does that" does not seem like a sufficient answer to me. Also the python cryptographic authority's [cryptography](https://github.com/pyca/cryptography) does not seem to depend on NumPy. So I'm not sure this is even correct. – cowlicks Jan 09 '16 at 11:52
  • @cowlicks: The actual response was "it is only rarely needed in 'normal' Python code". – Ethan Furman Apr 05 '23 at 16:42