4

I am used to the python byte literal syntax representing bytes as hex values such as b'\x7a' is the hex value 0x7a. I however have run into an issue that I don't know how it is working. I am using the ssl library for the first time and creating random values with ssl.RAND_bytes(...) It is returning strings with characters that alter the value (say when doing an int.from_bytes(...)) Example strings I have received:

b'\x12\x1f)\x8b\xe0\xd7LD'
b'\x808\x8a(\x02\xb3S\xc9\xabW_\n\xf3\xbb\x80o'
b'\xde\xef\x81%FMB=Ps'
b'9\x81\x90\xdc\xfa\x98\xbd\x1f^;'

What I don't understand are those non hex value characters, such as the 9 preceding the \x81 or the MB=Ps. I would like to understand what these extra values represent and how I can interpret them when debugging my code. I know python has no issue with these, it is me who is at a loss.

Note: I am on python3.3 on windows 7 64bit

Bear
  • 486
  • 2
  • 8
  • 16

3 Answers3

7

Byte array notation converts certain values to their textual representation:

>>> b'\x12\x41\x42'
b'\x12AB'

In this case the hex number 41 is the ASCII value for capital letter A.

hdante
  • 7,685
  • 3
  • 31
  • 36
  • 3
    Ok I wasn't aware these were byte arrays, when I called `type` on them it just said bytes which confused me further! Thank you. So if the hex value is also an ascii val it is automatically replaced during print. Is there a way for me to display the pure hex values? The only way I can think is something like `hex(int.from_bytes(val, 'little'))` but that seems so clunky. I have never worked with bytes in python like this before so there is probably something simple I am missing. I just want to be able to read the hex values for myself to make sure what I am doing is working correclty. – Bear May 31 '13 at 17:23
  • @Bear See [this question](http://stackoverflow.com/questions/2340319/python-3-1-1-string-to-hex). – Aya May 31 '13 at 17:26
  • Those values are random, so they're textual garbage. Trying to show them in a formatted way is not useful. If you really want to format them, just iterate over them and print the way you want: for b in val: print('%x' % b). – hdante May 31 '13 at 17:34
  • Aya Checking out the comment. @hdante I just wanted to see the actual byte values as that is important for what I am doing since I am sending info to a server that is not written in python I just wanted to visually compare the two in a common format. the for is probably enough for that. Thanks. – Bear May 31 '13 at 17:53
  • 1
    The following works in at least Python 3.5 to display a hex/ascii "mixed display" bytes object in just hex: `byte_array.hex()` where `byte_array` is type `bytes` – Adam P Nov 26 '16 at 15:39
7

If the byte can be represented with a printable ASCII character that character is used instead of a hex escape, here is some code for generating a table of the representation for all byte values (columns are decimal, hex, and byte representation):

>>> all_bytes = bytes(range(256))
>>> for i in range(256):
...     print('{:<8}{:<#8x}{}'.format(i, i, all_bytes[i:i+1]))
... 
0       0x0     b'\x00'
1       0x1     b'\x01'
2       0x2     b'\x02'
3       0x3     b'\x03'
4       0x4     b'\x04'
5       0x5     b'\x05'
6       0x6     b'\x06'
7       0x7     b'\x07'
8       0x8     b'\x08'
9       0x9     b'\t'
10      0xa     b'\n'
11      0xb     b'\x0b'
12      0xc     b'\x0c'
13      0xd     b'\r'
14      0xe     b'\x0e'
15      0xf     b'\x0f'
16      0x10    b'\x10'
17      0x11    b'\x11'
18      0x12    b'\x12'
19      0x13    b'\x13'
20      0x14    b'\x14'
21      0x15    b'\x15'
22      0x16    b'\x16'
23      0x17    b'\x17'
24      0x18    b'\x18'
25      0x19    b'\x19'
26      0x1a    b'\x1a'
27      0x1b    b'\x1b'
28      0x1c    b'\x1c'
29      0x1d    b'\x1d'
30      0x1e    b'\x1e'
31      0x1f    b'\x1f'
32      0x20    b' '
33      0x21    b'!'
34      0x22    b'"'
35      0x23    b'#'
36      0x24    b'$'
37      0x25    b'%'
38      0x26    b'&'
39      0x27    b"'"
40      0x28    b'('
41      0x29    b')'
42      0x2a    b'*'
43      0x2b    b'+'
44      0x2c    b','
45      0x2d    b'-'
46      0x2e    b'.'
47      0x2f    b'/'
48      0x30    b'0'
49      0x31    b'1'
50      0x32    b'2'
51      0x33    b'3'
52      0x34    b'4'
53      0x35    b'5'
54      0x36    b'6'
55      0x37    b'7'
56      0x38    b'8'
57      0x39    b'9'
58      0x3a    b':'
59      0x3b    b';'
60      0x3c    b'<'
61      0x3d    b'='
62      0x3e    b'>'
63      0x3f    b'?'
64      0x40    b'@'
65      0x41    b'A'
66      0x42    b'B'
67      0x43    b'C'
68      0x44    b'D'
69      0x45    b'E'
70      0x46    b'F'
71      0x47    b'G'
72      0x48    b'H'
73      0x49    b'I'
74      0x4a    b'J'
75      0x4b    b'K'
76      0x4c    b'L'
77      0x4d    b'M'
78      0x4e    b'N'
79      0x4f    b'O'
80      0x50    b'P'
81      0x51    b'Q'
82      0x52    b'R'
83      0x53    b'S'
84      0x54    b'T'
85      0x55    b'U'
86      0x56    b'V'
87      0x57    b'W'
88      0x58    b'X'
89      0x59    b'Y'
90      0x5a    b'Z'
91      0x5b    b'['
92      0x5c    b'\\'
93      0x5d    b']'
94      0x5e    b'^'
95      0x5f    b'_'
96      0x60    b'`'
97      0x61    b'a'
98      0x62    b'b'
99      0x63    b'c'
100     0x64    b'd'
101     0x65    b'e'
102     0x66    b'f'
103     0x67    b'g'
104     0x68    b'h'
105     0x69    b'i'
106     0x6a    b'j'
107     0x6b    b'k'
108     0x6c    b'l'
109     0x6d    b'm'
110     0x6e    b'n'
111     0x6f    b'o'
112     0x70    b'p'
113     0x71    b'q'
114     0x72    b'r'
115     0x73    b's'
116     0x74    b't'
117     0x75    b'u'
118     0x76    b'v'
119     0x77    b'w'
120     0x78    b'x'
121     0x79    b'y'
122     0x7a    b'z'
123     0x7b    b'{'
124     0x7c    b'|'
125     0x7d    b'}'
126     0x7e    b'~'
127     0x7f    b'\x7f'
128     0x80    b'\x80'
129     0x81    b'\x81'
130     0x82    b'\x82'
131     0x83    b'\x83'
132     0x84    b'\x84'
133     0x85    b'\x85'
134     0x86    b'\x86'
135     0x87    b'\x87'
136     0x88    b'\x88'
137     0x89    b'\x89'
138     0x8a    b'\x8a'
139     0x8b    b'\x8b'
140     0x8c    b'\x8c'
141     0x8d    b'\x8d'
142     0x8e    b'\x8e'
143     0x8f    b'\x8f'
144     0x90    b'\x90'
145     0x91    b'\x91'
146     0x92    b'\x92'
147     0x93    b'\x93'
148     0x94    b'\x94'
149     0x95    b'\x95'
150     0x96    b'\x96'
151     0x97    b'\x97'
152     0x98    b'\x98'
153     0x99    b'\x99'
154     0x9a    b'\x9a'
155     0x9b    b'\x9b'
156     0x9c    b'\x9c'
157     0x9d    b'\x9d'
158     0x9e    b'\x9e'
159     0x9f    b'\x9f'
160     0xa0    b'\xa0'
161     0xa1    b'\xa1'
162     0xa2    b'\xa2'
163     0xa3    b'\xa3'
164     0xa4    b'\xa4'
165     0xa5    b'\xa5'
166     0xa6    b'\xa6'
167     0xa7    b'\xa7'
168     0xa8    b'\xa8'
169     0xa9    b'\xa9'
170     0xaa    b'\xaa'
171     0xab    b'\xab'
172     0xac    b'\xac'
173     0xad    b'\xad'
174     0xae    b'\xae'
175     0xaf    b'\xaf'
176     0xb0    b'\xb0'
177     0xb1    b'\xb1'
178     0xb2    b'\xb2'
179     0xb3    b'\xb3'
180     0xb4    b'\xb4'
181     0xb5    b'\xb5'
182     0xb6    b'\xb6'
183     0xb7    b'\xb7'
184     0xb8    b'\xb8'
185     0xb9    b'\xb9'
186     0xba    b'\xba'
187     0xbb    b'\xbb'
188     0xbc    b'\xbc'
189     0xbd    b'\xbd'
190     0xbe    b'\xbe'
191     0xbf    b'\xbf'
192     0xc0    b'\xc0'
193     0xc1    b'\xc1'
194     0xc2    b'\xc2'
195     0xc3    b'\xc3'
196     0xc4    b'\xc4'
197     0xc5    b'\xc5'
198     0xc6    b'\xc6'
199     0xc7    b'\xc7'
200     0xc8    b'\xc8'
201     0xc9    b'\xc9'
202     0xca    b'\xca'
203     0xcb    b'\xcb'
204     0xcc    b'\xcc'
205     0xcd    b'\xcd'
206     0xce    b'\xce'
207     0xcf    b'\xcf'
208     0xd0    b'\xd0'
209     0xd1    b'\xd1'
210     0xd2    b'\xd2'
211     0xd3    b'\xd3'
212     0xd4    b'\xd4'
213     0xd5    b'\xd5'
214     0xd6    b'\xd6'
215     0xd7    b'\xd7'
216     0xd8    b'\xd8'
217     0xd9    b'\xd9'
218     0xda    b'\xda'
219     0xdb    b'\xdb'
220     0xdc    b'\xdc'
221     0xdd    b'\xdd'
222     0xde    b'\xde'
223     0xdf    b'\xdf'
224     0xe0    b'\xe0'
225     0xe1    b'\xe1'
226     0xe2    b'\xe2'
227     0xe3    b'\xe3'
228     0xe4    b'\xe4'
229     0xe5    b'\xe5'
230     0xe6    b'\xe6'
231     0xe7    b'\xe7'
232     0xe8    b'\xe8'
233     0xe9    b'\xe9'
234     0xea    b'\xea'
235     0xeb    b'\xeb'
236     0xec    b'\xec'
237     0xed    b'\xed'
238     0xee    b'\xee'
239     0xef    b'\xef'
240     0xf0    b'\xf0'
241     0xf1    b'\xf1'
242     0xf2    b'\xf2'
243     0xf3    b'\xf3'
244     0xf4    b'\xf4'
245     0xf5    b'\xf5'
246     0xf6    b'\xf6'
247     0xf7    b'\xf7'
248     0xf8    b'\xf8'
249     0xf9    b'\xf9'
250     0xfa    b'\xfa'
251     0xfb    b'\xfb'
252     0xfc    b'\xfc'
253     0xfd    b'\xfd'
254     0xfe    b'\xfe'
255     0xff    b'\xff'
Andrew Clark
  • 202,379
  • 35
  • 273
  • 306
3

When calling repr() on a bytestring it tries to show bytes which represent usual ASCII characters as such. Only the "unusual" ones are shown as \x..

Michael Butscher
  • 10,028
  • 4
  • 24
  • 25