0

I successfully ran the PhantomJS+Selenium in Python 3.5 in my laptop, but when I used it on AWS EC2 CentOS 6 environment, I did not get error but got the content showing:

<html><head><meta content="no-cache" http-equiv="Pragma">
<meta content="-1" http-equiv="Expires">
<meta content="no-cache" http-equiv="CacheControl">
<meta content="IE=edge" http-equiv="X-UA-Compatible">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<link href="data:;base64,iVBORw0KGgo=" rel="shortcut icon">
<script>

(function(){
    var securemsg;
    var dosl7_common;

window["bobcmn"] = "11111011111010200000002200000005200000000224b7a934200000096300000000300000000300000006/TSPD/300000008TSPD_101300000004http200000000200000000";

Can someone help me to figure out what was the reason and possible solutions? Thank you so much!

Lobster
  • 1
  • 1
  • 1
    Could be that AWS IP-adress range is blocked by that site due to frequent attacks/scraping/etc. – Vaviloff Mar 29 '17 at 16:22
  • Thanks, it has just been tested for a few times. – Lobster Mar 29 '17 at 20:23
  • Not quite sure I got you. What has been tested a few times? To what results? – Vaviloff Mar 30 '17 at 07:42
  • I only tested my codes to fetch the web page for a couple of times, so I don't think the IP address is blocked due to frequent scraping. – Lobster Mar 30 '17 at 11:47
  • Didn't say that :) It could be blocked due to activity by a previous user of that IP. Whole ranges of Amazon Ips could be blocked by some parties due to that. That actually was a case once discussed here. – Vaviloff Mar 30 '17 at 16:58
  • The bottom point is - try to deploy this code at some other provider. For example, here's a [ref link](https://m.do.co/c/2ff6cab4a36b) for free $10 at Digital Ocean. – Vaviloff Mar 30 '17 at 17:03
  • Thanks for the advice, I will try. – Lobster Mar 31 '17 at 02:35

1 Answers1

0
http://stackoverflow.com/questions/34707532/python-post-request-not-returning-html-requesting-javascript-be-enabled

Please take a look at Backgammon's answer, very good point, basically, you encountered bot-defeat system. headless browser like PhantomJS can be detected and against. I guess if it's debugging, you can use selenium & Chrome.

Larry Song
  • 1,086
  • 9
  • 13