I'm stuck in loading the data from xml file

Question

import requests
try:
    import xml.etree.cElementTree as et
except ImportError:
    import xml.etree.ElementTree as et

user_key = authorized_key
doc_name = "F-C0032-001"

api_link = "http://opendata.cwb.gov.tw/opendataapi?dataid=%s&authorizationkey=%s" % (doc_name,user_key)
report = requests.get(url=api_link).text #The problem is here

Error:

Traceback (most recent call last):
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\connection.py", line 96, in create_connection
    raise err
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1010, in _send_output
    self.send(msg)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 950, in send
    self.connect()
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 200, in connect
    conn = self._new_conn()
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001F0DC29F940>: Failed to establish a new connection: [WinError 10061]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\util\retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='opendata.cwb.gov.tw', port=80): Max retries exceeded with url: /opendataapi?dataid=F-C0032-001&authorizationkey= (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001F0DC29F940>: Failed to establish a new connection: [WinError 10061]'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/Python Books/Python/Ch17 Flask Web API/weatherdata.py", line 14, in <module>
    report = requests.get(url=api_link,headers=headers).text
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\domin\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='opendata.cwb.gov.tw', port=80): Max retries exceeded with url: /opendataapi?dataid=F-C0032-001&authorizationkey= (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001F0DC29F940>: Failed to establish a new connection: [WinError 10061]'))

Actually my aim is to read the data and select the related location from the api_link but however the file is too bulky so is there any good ways or methods to properly read the related data from the xml file without any connection error messages? Since I do not understand what the error messages telling me how to deal with so if possible could anyone give some related solutions?

As for now the issue is not related to xml - I remove the tag. — balderman, Oct 01 '21 at 08:58
Please include the _full_ error and remove all the code that is not even called to get a [mcve]. — Ulrich Eckhardt, Oct 01 '21 at 09:09
The code added in revision 3 does not help at all. The error has nothing to do with XML processing. — mzjn, Oct 01 '21 at 09:12
Indeed. It's rather making a query against some API where the remote side (accidentally or intentionally) doesn't behave as expected. You'll have to either fix your request or the remote side API. — Ulrich Eckhardt, Oct 01 '21 at 09:19
You are making a request to `opendata.cwb.gov.tw` which is down right now. That's why you are not getting a response. Edit: looks like the endpoint is not returning a valid response (both with http and https). Wait for some time and try again. — Pranava Mohan, Oct 01 '21 at 09:23

Quentin Demange · Answer 1 · 2021-10-01T08:52:52.747

0

it looks like the error is not reading the xlm but when you send the request to the server, this error:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='opendata.cwb.gov.tw', port=80): Max retries exceeded with url: /opendataapi?dataid=F-C0032-001&authorizationkey=

Is send when you're sending too many requests from same ip address in short period of time. Have you tried this ? https://stackoverflow.com/a/47475019/17044313 It should look like this:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

user_key = Authorization Key
doc_name = 'F-C0032-001'

api_link = 'http://opendata.cwb.gov.tw/opendataapi?dataid=%s&authorizationkey=%s'%(doc_name,user_key)

session = requests.Session()
retry = Retry(connect=3, backoff_factor=0.5)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

session.get(api_link)

edited Oct 01 '21 at 08:52

answered Oct 01 '21 at 08:39

Quentin Demange

15
3

so how should I edit my code with the features of requests? – Dominic Ching Hei Oct 01 '21 at 08:44
I edited my response to respond to your comment. – Quentin Demange Oct 01 '21 at 08:53
I have used the above code but still facing the same issue – Dominic Ching Hei Oct 01 '21 at 08:55
so do I have to give a variable name in order to get the content from the ```api_link```? – Dominic Ching Hei Oct 01 '21 at 09:00
`session.get(api_link).text`should return the content of the request – Quentin Demange Oct 01 '21 at 09:14
I tried to print ```session.get(api_link).text``` but still getting the same error message so any ways to solve the connection problem? – Dominic Ching Hei Oct 01 '21 at 13:30

I'm stuck in loading the data from xml file

1 Answers1