Python not recognizing quotes

Question

What I want to do is very straight forward.

Retrieve text from a file
If the text contains any quotes, get the text inside the quotes.

To do that I am using this regex, borrowed from another post.

re.findall('"([^"]*)"', text)

The problem I am running into though, is that the particular quotes that are contained in my text files aren't being recognized as quotes.

For example:

text = #get text from a file

print(text) 
#Outputs: 'this is a "test"'

print(re.findall('"([^"]*)"', text))
#Outputs: []

But if I type the string directly in as a variable it functions correctly.

text = 'this is a "test"'

#The same regex outputs ['test']

So I believe that my problem here is something to do with the encoding. That being said type(text) returns str.

Edit: Solution I found thanks to @rmharrison Here is what is now working

import re
from unidecode import unidecode

text = # Text From File

cleaned_text = unidecode(text)

print(re.findall('"([^"]*)"', cleaned_text))

#This successfully outputs text inside quotes.

Are you using Python2 or Python3? And, what does `print(repr(text))` reveal? — Robᵩ, Jul 13 '17 at 02:22
Please provide a short reproducible test case. See [mcve] for why and how. — Robᵩ, Jul 13 '17 at 02:25
I am using python 3 and print(repr(text)) reveals 'this is a "test"' — Tyler Bell, Jul 13 '17 at 02:28
Especially if you're on windows, I suspect you're using 'non-standard' double-quotes, e.g. the ones with explicit begin- and end-quote. There are *many* supported unicode [`Quotation_Mark`][1]. — rmharrison, Jul 13 '17 at 02:33
@rmharrison yes you are right. Do you know how I can either #1 get the regex to recongize them or #2 just replace them with the standard? — Tyler Bell, Jul 13 '17 at 02:35
I am going to try this from unidecode import unidecode line = unidecode(line) — Tyler Bell, Jul 13 '17 at 02:37
Per #1, there doesn't seem to be a magic bullet other than enumerating all the ones you want to match, see: https://stackoverflow.com/a/3128913/3684641 — rmharrison, Jul 13 '17 at 02:39
Thought the unicodedata native-module might have a lookup to return all unicode characters of the same "type". No cigar: https://docs.python.org/2/library/unicodedata.html — rmharrison, Jul 13 '17 at 02:43
You may want to actually submit that as an answer and accept it so this question is marked as answered. — kirkpatt, Jul 13 '17 at 20:04

score 0 · Answer 1 · answered Jul 13 '17 at 20:19

Solution I found thanks to @rmharrison Here is what is now working

import re
from unidecode import unidecode

text = # Text From File

cleaned_text = unidecode(text)

print(re.findall('"([^"]*)"', cleaned_text))

#This successfully outputs text inside quotes.

Python not recognizing quotes

1 Answers1