48

Given an RFC822 message in Python 2.6, how can I get the right text/plain content part? Basically, the algorithm I want is this:

message = email.message_from_string(raw_message)
if has_mime_part(message, "text/plain"):
    mime_part = get_mime_part(message, "text/plain")
    text_content = decode_mime_part(mime_part)
elif has_mime_part(message, "text/html"):
    mime_part = get_mime_part(message, "text/html")
    html = decode_mime_part(mime_part)
    text_content = render_html_to_plaintext(html)
else:
    # fallback
    text_content = str(message)
return text_content

Of these things, I have get_mime_part and has_mime_part down pat, but I'm not quite sure how to get the decoded text from the MIME part. I can get the encoded text using get_payload(), but if I try to use the decode parameter of the get_payload() method (see the doc) I get an error when I call it on the text/plain part:

File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/
email/message.py", line 189, in get_payload
    raise TypeError('Expected list, got %s' % type(self._payload))
TypeError: Expected list, got <type 'str'>

In addition, I don't know how to take HTML and render it to text as closely as possible.

MarianD
  • 13,096
  • 12
  • 42
  • 54
Chris R
  • 17,546
  • 23
  • 105
  • 172
  • 1
    I found a useful solution to something similar in http://ginstrom.com/scribbles/2007/11/19/parsing-multilingual-email-with-python/ – beldaz Jul 11 '13 at 03:21

5 Answers5

108

In a multipart e-mail, email.message.Message.get_payload() returns a list with one item for each part. The easiest way is to walk the message and get the payload on each part:

import email
msg = email.message_from_string(raw_message)
for part in msg.walk():
    # each part is a either non-multipart, or another multipart message
    # that contains further parts... Message is organized like a tree
    if part.get_content_type() == 'text/plain':
        print part.get_payload() # prints the raw text

For a non-multipart message, no need to do all the walking. You can go straight to get_payload(), regardless of content_type.

msg = email.message_from_string(raw_message)
msg.get_payload()

If the content is encoded, you need to pass None as the first parameter to get_payload(), followed by True (the decode flag is the second parameter). For example, suppose that my e-mail contains an MS Word document attachment:

msg = email.message_from_string(raw_message)
for part in msg.walk():
    if part.get_content_type() == 'application/msword':
        name = part.get_param('name') or 'MyDoc.doc'
        f = open(name, 'wb')
        f.write(part.get_payload(None, True)) # You need None as the first param
                                              # because part.is_multipart() 
                                              # is False
        f.close()

As for getting a reasonable plain-text approximation of an HTML part, I've found that html2text works pretty darn well.

Jarret Hardie
  • 95,172
  • 10
  • 132
  • 126
  • That is an excellent explanation...that covers exactly what I've already got; I can, as noted, locate and extract the bare payload of the part. However, I can _not_ decode the part if it's decoded, nor can I render the text/html part to text if no text/plain part is available. – Chris R Sep 23 '09 at 13:45
  • (on re-read -- sorry, coffee is lacking!) Well, okay, so you've solved my HTML to text problem :) – Chris R Sep 23 '09 at 13:46
  • My bad... clearly not enough coffee last night when I answered. I've amended the answer, hopefully with what you need. – Jarret Hardie Sep 23 '09 at 14:50
  • 2
    Cool.. How can I check if the part is encoded? Where do I see the part's Content-Transfer-Encoding attribute? – Chris R Sep 23 '09 at 17:54
  • Use part.get_param('content-transfer-encoding') to see the attribute – Jarret Hardie Sep 23 '09 at 18:51
  • 5
    Actually, use part.get("content-transfer-encoding"), since it's just a header. Not part of the content-type header. Also, instead of `part.get_payload(None, True)` you can use `part.get_payload(decode=True)`, which I think is a little clearer. – Wodin Apr 14 '14 at 15:47
1

Flat is better than nested ;)

from email.mime.multipart import MIMEMultipart
assert isinstance(msg, MIMEMultipart)

for _ in [k.get_payload() for k in msg.walk() if k.get_content_type() == 'text/plain']:
    print _
guneysus
  • 6,203
  • 2
  • 45
  • 47
  • This blindly extracts all `text/plain´ parts with no attention to which one is "right". – tripleee Dec 02 '15 at 10:11
  • @tripleee Generally we use one plain, one html part, and several image parts. Even if there is more than plain parts, how do you know which one right? – guneysus Dec 02 '15 at 22:28
  • 1
    In the typical case, with a toplevel `multipart/alternative` where only one part is `text/plain`, that one. In the more general case, I don't think there is a single right answer, because it depends on the purpose of your application and the preferences of the recipient. – tripleee Dec 03 '15 at 05:24
  • 4
    In all fairness, the accepted answer has the same problem. – tripleee Dec 03 '15 at 05:26
1

Try my lib for IMAP: https://github.com/ikvk/imap_tools

from imap_tools import MailBox, AND

# Get date, subject and body len of all emails from INBOX folder
with MailBox('imap.mail.com').login('test@mail.com', 'pwd') as mailbox:
    for msg in mailbox.fetch():
        print(msg.date, msg.subject, len(msg.text or msg.html))

See html2text: https://pypi.org/project/html2text/.

And may be msg.text is enough

Vladimir
  • 6,162
  • 2
  • 32
  • 36
0

To add on @Jarret Hardie's excellent answer:

I personally like to transform that kind of data structures to a dictionary that I can reuse later, so something like this where the content_type is the key and the payload is the value:

import email

[...]

email_message = {
    part.get_content_type(): part.get_payload()
    for part in email.message_from_bytes(raw_email).walk()
}

print(email_message["text/plain"])
Seraf
  • 850
  • 1
  • 17
  • 34
0

#This is what I have for a gmail account using the app password method.

from imap_tools import MailBox
import email
my_email = "your email"
my_pass = "app password"
mailbox =  MailBox('imap.gmail.com').login(my_email, my_pass)

for msg in mailbox.fetch('Subject "   "', charset='utf8'):
    print("Message id:",msg.uid)
    print("Message Subject:",msg.subject)
    print("Message Date:", msg.date)
    print("Message Text:", msg.text)
Woody
  • 1