PhantomJS failing to open HTTPS site

Question

I'm using the following code based on loadspeed.js example to open up a https:// site which requires http server authentication as well.

var page = require('webpage').create(), system = require('system'), t, address;

page.settings.userName = 'myusername';
page.settings.password = 'mypassword';

if (system.args.length === 1) {
    console.log('Usage: scrape.js <some URL>');
    phantom.exit();
} else {
    t = Date.now();
    address = system.args[1];
    page.open(address, function (status) {
        if (status !== 'success') {
            console.log('FAIL to load the address');
        } else {
            t = Date.now() - t;
            console.log('Page title is ' + page.evaluate(function () {
                return document.title;
            }));
            console.log('Loading time ' + t + ' msec');
        }
        phantom.exit();
    });
}

Its failing to load the page all the time. What could be wrong here? Are secured sites to be handled any differently? The site can be accessed successfully from browser though.

I'm just starting with Phantom right now and find it too good to stop playing around even though i'm not moving forward with this issue.

score 154 · Accepted Answer · edited May 07 '18 at 09:13

154

I tried Fred's and Cameron Tinker's answers, but only --ssl-protocol=any option seem to help me:

phantomjs --ssl-protocol=any test.js

Also I think it should be way safer to use --ssl-protocol=any as you still are using encryption, but --ignore-ssl-errors=true will ignore (duh) all ssl errors, including malicious ones.

edited May 07 '18 at 09:13

Hesham Yassin

4,341
2
21
23

answered Jul 10 '14 at 14:24

JLarky

9,833
5
36
37

1

i had to use these 3 arguments and it resolved: "--web-security=false", "--ssl-protocol=any", "--ignore-ssl-errors=true". webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true', "--web-security=false", "--ssl-protocol=any"]) – Abdul Khalid Nov 16 '17 at 19:32

score 123 · Answer 2 · edited Dec 24 '22 at 01:51

123

The problem is most likely due to SSL certificate errors. If you start phantomjs with the --ignore-ssl-errors=yes option, it should proceed to load the page as it would if there were no SSL errors:

phantomjs --ignore-ssl-errors=yes [phantomOptions] script.js [scriptOptions]

I've seen a few websites having problems with incorrectly implementing their SSL certificates or they've expired, etc. A complete list of command line options for phantomjs is available here: http://phantomjs.org/api/command-line.html.

edited Dec 24 '22 at 01:51

starball

20,030
7
43
238

answered Aug 31 '12 at 17:53

Cameron Tinker

9,634
10
46
85

2

Thank you, this resolved my problem as well. My case, I didn't get Cert error on browser so it was very confusing but I did curl verbose and noticed that one that didn't work was using Wildcard Cert (ie: CN=*.example.com). It would be nice if phantomjs can return verbose reason as to why this is failed. – naoko Apr 15 '13 at 14:50
7

Holy monkeys. I wish the SSL error were explained, only stack overflow and the pink unicorn helped me track down the root cause. Thanks guys – SimplGy Apr 16 '13 at 15:21
Glad this is helping people. Perhaps PhantomJS will need an update to provide verbose SSL errors with another command line argument. I know that SSL error information is available in Qt, but most times people just suppress the errors without handling them explicitly. – Cameron Tinker Apr 16 '13 at 15:50
I know I'm late to the game, but I found that adding the option *after* the script name didn't work - you need to call it in order: `phantomjs --ignore-ssl-errors=yes script.js` – Simon Jan 16 '14 at 20:15
Yes, the `--ignore-ssl-errors=yes` option should come before the script name. Thank you for pointing this out. – Cameron Tinker Jan 16 '14 at 21:20
Does it give you any error message? I haven't tried this method on phantomjs 1.9.7, but it should work since the `--ignore-ssl-errors=yes` option is still available. You might also change the SSL protocol as suggested by Fred's answer below. – Cameron Tinker Mar 26 '14 at 13:12
I tried this and it didn't work but then I realized I was putting the --ignore-ssl-errors=yes option *after* the script argument (script.js). The order matters! phantomjs --ignore-ssl-errors=yes script.js. It works like a charm when you get the order right. – Brady Holt Oct 26 '14 at 19:29
My example command shows that phantom options must be first. Then you can specify the script and the script options. I'm glad you got it working though! – Cameron Tinker Oct 26 '14 at 20:27
I'd love it if one the more specific flags suggested (like `--ssl-protocol=tlsv1`) would be the **RIGHT** solution in all cases, but it seems like this one is (currently) the only fix for hosts with the SNI extension. [This issue](https://github.com/ariya/phantomjs/issues/12440) suggests that PhantomJS currently doesn't support those, so `--ignore-ssl-errors=yes` is the only workable fix suggested here. – John Larson Mar 27 '15 at 07:19

score 71 · Answer 3 · answered Oct 17 '14 at 04:00

71

Note that as of 2014-10-16, PhantomJS defaults to using SSLv3 to open HTTPS connections. With the POODLE vulnerability recently announced, many servers are disabling SSLv3 support.

To get around that, you should be able to run PhantomJS with:

phantomjs --ssl-protocol=tlsv1

Hopefully, PhantomJS will be updated soon to make TLSv1 the default instead of SSLv3.

answered Oct 17 '14 at 04:00

Micah

17,584
8
40
46

4

Version 1.9.8 switches the default to TLSv1: https://github.com/ariya/phantomjs/issues/12670 – Andy Triggs Oct 24 '14 at 10:56
Fixed it for me, SSLv3 was disabled on the server – Chris Herring Nov 10 '14 at 07:17
2

Note that updating to PhantomJS 1.9.8 leads to a [new bug](https://github.com/ariya/phantomjs/issues/12697). – Artjom B. Nov 20 '14 at 17:59
This should be the accepted answer. Disabling SSL or allowing any protocol are not good solutions in my opinion. Thanks for sharing. – Dominic P Dec 23 '14 at 20:14
Didn't work by itself for me. I needed to also add --ignore-ssl-errors. But I can't see anything wrong with the certificate -- signed by a valid CA, not expired and matches hostname. (Chrome displays green lock icon without complaint.) – wrschneider Jan 21 '15 at 15:58
3

How can I get more verbose debugging out of SSL handshake problems? – wrschneider Jan 21 '15 at 15:59
This wasn't enough to get my tests working, I had to add `--ignore-ssl-errors=true` to it too. – etagwerker Jan 27 '15 at 14:41
This worked for me, but I did not have to set `--ignore-ssl-errors=true`. I was hitting https://*.herokuapp.com – Sam Mar 07 '15 at 07:43

score 24 · Answer 4 · answered Jun 14 '13 at 05:20

24

experienced same issue...
--ignore-ssl-errors=yes was not enough to fix it for me, had to do two more things:
1) change user-agent
2) tried all ssl-protocols, the only one that worked was tlsv1 for the page in question
Hope this helps...

answered Jun 14 '13 at 05:20

Fred

635
9
18

2

Changing ssl-protocols to tlsv1 worked for me as well, thanks! – Dave Stibrany Oct 17 '13 at 21:20
1

+1 Fixed my problem, lucky you are not here with me as I would kiss you right now lol, cheers – Martin Dec 20 '13 at 17:51
This is exactly what I needed. Thank you very much! Instead of tlsv1 I used "any" and this works as well. (seems a bit safer should I visit other ssl websites. – user1841243 Jul 30 '14 at 07:02
1

change user-agent to what? – Lucas Steffen Sep 19 '19 at 16:23

score 17 · Answer 5 · edited Jul 22 '16 at 13:55

17

I experienced the same problem (casperjs 1.1.0-beta3/phantomjs 1.9.7). Using --ignore-ssl-errors=yes and --ssl-protocol=tlsv1 solved it. Using only one of the options did not solve it for me.

edited Jul 22 '16 at 13:55

evandrix

6,041
4
27
38

answered Dec 08 '14 at 10:35

Jojje

763
1
8
10

This trick fixed my problem on codeship CI, running a phantomjs + behat suite (after ~10 hours of debugging and trying out things) – ZeNC Jun 05 '16 at 23:45
yeah. after some silly hours your comment made my day! – Yarimadam Sep 25 '16 at 18:22
FIXED my issues!!! – wizpig64 Oct 11 '21 at 23:53

score 1 · Answer 6 · edited Oct 22 '18 at 13:20

1

I was receiving

Error creating SSL context" from phantomJS (running on CentOS 6.6)

Building from source fixed it for me. Don't forget to use the phantomjs that you built. (instead of the /usr/local/bin/phantomjs if you have it)

sudo yum -y install gcc gcc-c++ make flex bison gperf ruby openssl-devel freetype-devel fontconfig-devel libicu-devel sqlite-devel libpng-devel libjpeg-devel
git clone git://github.com/ariya/phantomjs.git
cd phantomjs
git checkout 2.0
./build.sh
cd bin/
./phantomjs <your JS file>

edited Oct 22 '18 at 13:20

gotqn

42,737
46
157
243

answered Apr 07 '15 at 17:02

Mert Değirmenci

31
3

1

I don't know why people rate your answer negatively. After trying all solutions mentioned above, and days of struggle this is the only solution that worked for me. phantomJS is quite annoying to compile because of qt, but it's worth it. – FlorianB Nov 20 '16 at 16:58

score 0 · Answer 7 · edited Jan 25 '14 at 15:08

If someone is using Phantomjs with Sahi the --ignore-ssl-errors option needs to go in your browser_types.xml file. It worked for me.

<browserType>
    <name>phantomjs</name>
    <displayName>PhantomJS</displayName>
    <icon>safari.png</icon>
    <path>/usr/local/Cellar/phantomjs/1.9.2/bin/phantomjs</path>
    <options>--ignore-ssl-errors=yes --debug=yes --proxy=localhost:9999 /usr/local/Cellar/phantomjs/phantom-sahi.js</options>
    <processName>"PhantomJS"</processName>
    <capacity>100</capacity>
    <force>true</force>
</browserType>

score 0 · Answer 8 · edited Jun 20 '20 at 09:12

What about `shebang`?

If you're using shebang to execute phantomjs scripts, use the following shebang line

#!/usr/bin/phantomjs --ignore-ssl-errors=yes
    
var system = require('system');
var webpage = require('webpage');

// ... rest of your script

Use any of the above answers. i personally like --ignore-ssl-errors=yes since it's irrelevant to validate my loopback web servers' self-signed certificate.

score 0 · Answer 9 · answered Dec 09 '17 at 20:12

None of the other answers here helped me; it may be that the specific site(s) I was working with were too picky with their HTTP headers. This is what worked:

var page = webpage.create();
page.customHeaders = {
    "Connection": "keep-alive"
};

I found out that PhantomJS was using "Keep-Alive" (capitalized), and the connection was not being kept alive. :)

score 0 · Answer 10 · answered Jan 03 '18 at 20:57

I was getting SSL Handshake Failed yesterday. I tried many combinations of phantomJS options (--ignore-ssl-errors=yes etc.), but none of them worked.

Upgrading to phantomJS 2.1.1 fixed it.

I used the phantomJS installation instructions at https://gist.github.com/julionc/7476620, changing the phantomJS version to 2.1.1.

score 0 · Answer 11 · answered Dec 11 '18 at 00:49

On the machine you are trying to run phantomjs on to connect to a remote server, run "openssl ciphers." Copy and paste the ciphers listed into the --ssl-ciphers="" command line option. This tells the connecting web server which ciphers are available to use to communicate with your client. If you don't set the ones available on your own machine, it can use any cipher your machine does not understand that the default modern browsers do that are used for the default setting.

score 0 · Answer 12 · answered Apr 11 '22 at 17:57

0

phantomjs --web-security=false --ignore-ssl-errors=true scripts.js

answered Apr 11 '22 at 17:57

Pushplata

552
4
10

score -2 · Answer 13 · answered Oct 12 '18 at 10:17

-2

The only thing that worked for me was upping phantomjs from 1.9x to 2.x ;)

answered Oct 12 '18 at 10:17

giorgio79

3,787
9
53
85

PhantomJS failing to open HTTPS site

13 Answers13

What about `shebang`?

Linked

Related

PhantomJS failing to open HTTPS site

13 Answers13

What about shebang?

Linked

Related

What about `shebang`?