54

I have a string. In that string are double backslashes. I want to replace the double backslashes with single backslashes, so that unicode char codes can be parsed correctly.

(Pdb) p fetched_page
'<p style="text-align:center;" align="center"><strong><span style="font-family:\'Times New Roman\', serif;font-size:115%;">Chapter 0<\\/span><\\/strong><\\/p>\n<p><span style="font-family:\'Times New Roman\', serif;font-size:115%;">Chapter 0 in \\u201cDreaming in Code\\u201d give a brief description of programming in its early years and how and why programmers are still struggling today...'

Inside of this string, you can see escaped unicode character codes, such as:

\\u201c

I want to turn this into:

\u201c

Attempt 1:

fetched_page.replace('\\\\', '\\')

but this doesn't work -- it searches for quadruple backslashes.

Attempt 2:

fetched_page.replace('\\', '\')

But this results in an end of line error.

Attempt 3:

fetched_page.decode('string_escape')

But this had no effect on the text. All the double backslashes remained as double backslashes.

zzz
  • 2,515
  • 4
  • 28
  • 38
  • 5
    Those aren't double backslashes in the string, which is why you can't get rid of them - it's just an artifact of displaying the string. Your problem is bigger than you think. – Mark Ransom Jul 19 '11 at 18:56
  • You're right -- the problem was bigger than I had originally thought. My solution involved reworking the way I was extracting data so as to prevent the extra backslashes from getting into my strings in the first place (instead of trying to strip them out after the fact). – zzz Jul 21 '11 at 12:04
  • @zzz did you solve this problem? – Lead Developer Jun 14 '19 at 15:30

6 Answers6

28

Python3:

>>> b'\\u201c'.decode('unicode_escape')
'“'

or

>>> '\\u201c'.encode().decode('unicode_escape')
'“'
max5555
  • 495
  • 5
  • 9
27

You can try codecs.escape_decode, this should decode the escape sequences.

schlamar
  • 9,238
  • 3
  • 38
  • 76
16

I'm not getting the behaviour you describe:

>>> x = "\\\\\\\\"
>>> print x
\\\\
>>> y = x.replace('\\\\', '\\')
>>> print y
\\

When you see '\\\\' in your output, you're seeing twice as many slashes as there are in the string because each on is escaped. The code you wrote should work fine. Trying printing out the actual values, instead of only looking at how the REPL displays them.

Jeremy
  • 1
  • 85
  • 340
  • 366
  • 11
    but the question was "How to replace a double backslash with a single backslash" and not how replace a quadruple with a double backslash! –  Apr 09 '19 at 17:08
6

To extend on Jeremy's answer, your problem is that '\' is an illegal string because \' escapes the quote mark, so your string never terminates.

murgatroid99
  • 19,007
  • 10
  • 60
  • 95
2

Interesting question, but in reality, you have only one slash symbol. It's just a way how it represents in python. If you make a list of symbols which string contains? like:

[s for s in string_object]

it shows every symbol and represents "" as "\", but you don't have to be confused about it. It is the single symbol actually. So, in the case of my example, it's just not a double backslash.

real example:

>>> [s for s in 'usnDu\\NgAnA{I']
['u', 's', 'n', 'D', 'u', '\\', 'N', 'g', 'A', 'n', 'A', '{', 'I']
Alex Kay
  • 23
  • 1
  • 3
2

It may be slightly overkill, but...

>>> import re
>>> a = '\\u201c\\u3012'
>>> re.sub(r'\\u[0-9a-fA-F]{4}', lambda x:eval('"' + x.group() + '"'), a)
'“〒'

So yeah, the simplest solution would ms4py's answer, calling codecs.escape_decode on the string and taking the result (or the first element of the result if escape_decode returns a tuple as it seems to in Python 3). In Python 3 you'd want to use codecs.unicode_escape_decode when working with strings (as opposed to bytes objects), though.

JAB
  • 20,783
  • 6
  • 71
  • 80
  • Hmm, this seems like a good solution. However, when I run it verbatim on the python 2.7.1 interactive terminal, I get '\\u201c\\u3012' as the result instead of '“〒' – zzz Jul 19 '11 at 20:52