0

Problem

I can open the following link on my browser just fine: 
https://www.hapag-lloyd.com/en/online-business/track/track-by-container-solution.html?container=HLBU9517848

However , If I use Python requests I will get a <Response [403]>

Upon further investigation of this response, I have noticed that in the response text of this non-authorized request, something like this appeared :

<p class="display-later" style="display:none;">Please complete 
following CAPTCHA to get access to the Hapag-Lloyd's website.</p>
<p><form id="challenge-form" class="challenge-form managed-form" action="/en/online-business/track/track-by-container-solution.html?container=HLBU9517848&amp;__cf_chl_f_tk=ubmYYNVSxf5CVa3wq85K19Cb6kHn.6V14MgS52EOPpU-1658760627-0-gaNycGzNB70" method="POST" enctype="application/x-www-form-urlencoded">
    <div id='cf-please-wait'>
      <div id='spinner'>
        <div id="cf-bubbles">
            <div class="bubbles"></div>
            <div class="bubbles"></div>
            <div class="bubbles"></div>
        </div>
      </div>
      <p data-translate="please_wait" id="cf-spinner-please-wait">Please stand by, while we are checking your browser...</p>  
      <p data-translate="redirecting" id="cf-spinner-redirecting" style="display:none">Redirecting...</p>
      </div>

This website is using a security service to protect itself from online attacks. The action I just performed triggered the security solution. Completing the CAPTCHA proves I'm a human and gives me access to the Hapag-Lloyd's website.

Minimal Reproducible Example

url = "https://www.hapag-lloyd.com/en/online-business/track/track-by-container-solution.html?container=HLBU9517848"

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36'}

container_data = requests.get(url, proxies={"http":"http://95.66.151.101:8080"}, headers=headers)

print(container_data)

Solution

  • What are possible fixes ?

  • This answer only mentions changing User-Agent and http headers to deal 403 responses, which doesn't work on this case.

  • I believe I will only be on the right path if I follow something like this.

  • Any advice or link regarding this issue are welcome.

Simao
  • 371
  • 2
  • 13

0 Answers0