Python: how to save a webpage (as html file) from the link in the Email

Question

I get every day one specific email with a link in it. Then I klick the link which opens a browser, and then I save the webpage as a html file. That is what I have to do every day. Until now I do it manually, but I guess that it is a way to do it with python. I only know for now how to save an attachment from an email using python. But I don't know what to do with a link. Have somebody some experience with it?

Thanks in advance

Common StackOverFlow etiquette is to post what you have tried, what works, and what doesn't. Have you done any research of your own on this topic? What avenues have you pursued? Do you have a copy of the code you have tried to implement this functionality? — Ryan Bananahands, Feb 11 '19 at 18:39

Andrew F · Accepted Answer · 2019-02-11T18:51:28.047

2

You can do this with the requests package. Assuming you've already extracted the link as a string

link = 'https://...'

from requests import get
resp = get(link)
with open('todays-file.html', 'wb') as fOut:
    fOut.write(resp.content)

You will probably want to add handling if the link is bad (i.e. does not return a 20x status code), but that's the general idea.

edited Feb 11 '19 at 18:51

answered Feb 11 '19 at 18:47

Andrew F

2,690
1
14
25

1

I think you meant `resp.content` or `resp.text`, not `resp.data`. – John Gordon Feb 11 '19 at 18:51
Fixed. Good catch! – Andrew F Feb 11 '19 at 18:51
Thank You. But how can i extract the link? The link has always the same name in the email (namely "Version for mobile devices"), but the actual link is every day different. – Denis Feb 12 '19 at 09:24
1

@Denis Extracting the link depends on whatever format you're processing the emails in. If it always has the same text, you probably want to look for any anchors (``), then filter out only the one with the matching text, then get the link from the `href` attribute. If you post a sample (with personal info removed) and whatever script you have so far, I can make a more specific suggestion. – Andrew F Feb 12 '19 at 09:27
Thanks a lot. I was able to do it. Unfortunately, i still cannot get the content of the webpage: the created html file contains only "Access Denied (authentication_failed)". I suppose it is some problem with the certificate. – Denis Feb 12 '19 at 11:56
It's possible that the link takes you to a page that checks for authentication via a cookie (e.g. when you logged in and clicked "Remember Me"). Requests [does support cookies](http://docs.python-requests.org/en/master/user/quickstart/#cookies) but the process of extracting it from your browser and re-purposing it would almost definitely be tricky. You may also want to look at [Postman](https://getpostman.com) which allows you to [intercept browser requests](https://learning.getpostman.com/docs/postman/sending_api_requests/capturing_http_requests). – Andrew F Feb 12 '19 at 13:31

Python: how to save a webpage (as html file) from the link in the Email

1 Answers1