0

I know similar questions exist for this topic but I've gone through them and still couldn't get it.

My python program retrieves a subsection of html from a page using a regular expression. I just realised that I hadn't accounted for html special characters getting in the way.

say I have:

regex_title = ['I went to the store', 'Itlt's a nice day today', 'I went home for a rest']

I obviously want to change lt' to a single quote '.

I've tried variations of:

for each in regex_title:
    if 'lt'' in regex_title:
        str.replace("lt'", "'")

but had no success. What am I missing.

NOTE: The purpose is to do this without importing any more modules.

falsetru
  • 357,413
  • 63
  • 732
  • 636
doxyl
  • 49
  • 5

6 Answers6

3

str.replace does not replace in-place. It returns the replaced string. You need to assigned back the return value.

>>> regex_title = ['I went to the store', 'Itlt's a nice day today',
...                'I went home for a rest']
>>> regex_title = [s.replace("lt'", "'") for s in regex_title]
>>> regex_title
['I went to the store', "It's a nice day today", 'I went home for a rest']
falsetru
  • 357,413
  • 63
  • 732
  • 636
2

If your task is to unescape HTML, then better use unescape function:

>>> ll = ['I went to the store', 'Itlt's a nice day today', 'I went home for a rest']
>>> import HTMLParser
>>> h = HTMLParser.HTMLParser()
>>> print map(h.unescape, ll)
['I went to the store', u"Itlt's a nice day today", 'I went home for a rest']
stalk
  • 11,934
  • 4
  • 36
  • 58
  • +1 for recommending `map()`. Though I'd probably suggest list comprehensions to a Python newbie as the OP appears to be, since their syntax looks a bit more like the `for` loop that his instinct will be pushing him to write. – rmunn Oct 03 '14 at 06:28
1

You need to change your code to this :

for each in regex_title:
    if 'lt'' in each:
        each.replace("lt'", "'")

But it doesn't change your list so you need to pass the replaced index to list:

>>> for each in regex_title:
...         if 'lt'' in each:
...             regex_title[regex_title.index(each)]=each.replace("lt'", "'")
... 
>>> regex_title
['I went to the store', "It's a nice day today", 'I went home for a rest']
>>>
Mazdak
  • 105,000
  • 18
  • 159
  • 188
  • There's a MUCH better way to do this, using list comprehensions: `new_list = [s.replace("lt'", "'") for s in old_list]`. But as I said in my answer, better to use HTMLParser. – rmunn Oct 03 '14 at 06:25
  • thanks for reminding i know that there is a lot ways for this aim , but at first i attempt to refine the OP's idea ! then i will add another ways ! – Mazdak Oct 03 '14 at 06:27
  • If you need indexes, use `enumerate`. Using `list.index` inside a loop is not efficient. – falsetru Oct 03 '14 at 06:30
  • i think using `enumerate` in this case decrease the performance because it need to store the index for every index in every cycle ! while we just need one index ! – Mazdak Oct 03 '14 at 06:36
  • i see that and dont find anything ! about our topic ! – Mazdak Oct 03 '14 at 06:42
  • `list.index` has O(n) time complexity (same for `x in list`). – falsetru Oct 03 '14 at 06:52
  • yes ,exactly !but there is not any thing about comparison between `enumerate` and `list.index` i think its desire for a good benchmarking .... any way thanks for proposition – Mazdak Oct 03 '14 at 07:01
1

You don't explain why you want to avoid importing standard library modules. There are very few good reasons to deny yourself the use of Python's included batteries; unless you have such a reason (and if you do, you should state it), you should use the functionality provided to you.

In this case, it's the unescape() function from the html module: 1

from html import unescape

titles = [
    'I went to the store',
    'It's a nice day today',
    'I went home for a rest'
]

fixed = [unescape(s) for s in titles]

>>> fixed
['I went to the store', "It's a nice day today", 'I went home for a rest']

Reimplementing html.unescape() yourself is

  1. Pointless.
  2. Error-prone.
  3. Going to mean constantly going back and adding new cases when new HTML entities crop up in your data.

1 Since Python 3.4, anyway. For previous versions, use HTMLParser.HTMLParser.unescape() as per @stalk's answer.

Community
  • 1
  • 1
Zero Piraeus
  • 56,143
  • 27
  • 150
  • 160
  • It was the requirement of a project to not import other modules. I'm sure this will be useful for someone in the future though, so there you go. Cheers. – doxyl Oct 03 '14 at 06:59
  • Worth to point, that this code is for python3, not for python2 (`html` module is in python3 only) – stalk Oct 03 '14 at 07:00
0

Instead of doing this yourself, you'd be better off using the HTMLParser library, as described in https://stackoverflow.com/a/2087433/2314532. Read that question and answer for all the details, but the summary is:

import HTMLParser
parser = HTMLParser.HTMLParser()
print parser.unescape(''')
# Will print a single ' character

So in your case, you'd want to do something like:

import HTMLParser
parser = HTMLParser.HTMLParser()
new_titles = [parser.unescape(s) for s in regex_title]

That will unescape any HTML escape, not just the ' escape that you asked about, and process the entire list all at once.

Community
  • 1
  • 1
rmunn
  • 34,942
  • 10
  • 74
  • 105
0

Try like this:-

 regex_title = ['I went to the store', 'Itlt's a nice day today', 'I went home for a rest']
 str=','.join(regex_title)
 str1=str.replace("lt'","'");    
 print str1.split()
Hussain Shabbir
  • 14,801
  • 5
  • 40
  • 56