32

So I wrote a script in PHP that requires JavaScript to run with it. I've had a hard time finding the solution on how to run the JavaScript with PHP at the same time using some Linux tool to make it automatic (since running it in my browser isn't a choice, I need it to be a crontab). Using the php command isn't a choice either since it does not run JavaScript. So I found a solution; it was Lynx.

So here is my problem: for some reason, the JavaScript runs in my browser just fine, but not in Lynx. Using Lynx the page just loads without any JavaScript involvement. Is there a way to debug JavaScript in Lynx, similar to what Google Chrome has where you can view what's going on from the console. What's causing the script not to run in Lynx?

Edit:

So apparently Lynx does not support JavaScript. In that case, my question is, what does? I need to run this script as a browser would but from the Linux command prompt. And if OS matters in this case, I am running the latest version Ubuntu.

unrealapex
  • 578
  • 9
  • 23
TheNotSoWise
  • 869
  • 2
  • 10
  • 15
  • 2
    You should better use a browser-automation solution like Selenium or my own library, [dryscrape](https://github.com/niklasb/dryscrape) inside a virtual framebuffer (Xvfb) for that. – Niklas B. Feb 19 '12 at 21:53
  • What exactly are you trying to accomplish? Chances one of these is not the case: 1) It's best to run from a Linux cron job, 2) It's best run through JavaScript loaded in a web page – sh-beta Feb 19 '12 at 22:01
  • @sh-beta, I am trying to login to a site that uses a password encryption in javascript to encrypt the password before it gets sent to the server. Since I didn't want to rewrite the whole library they have to php, I directly use their javascript functions, encrypt the password, and send it off to the server using POST. – TheNotSoWise Feb 19 '12 at 22:09
  • @user1215232 adjusted my answer to reflect this – sh-beta Feb 19 '12 at 22:29
  • (for inbound Google users) Nowadays, you should be fine using Node.js. Place the shared logic in a different layer, and create a Node.js app to consume that. The browser app will also consume that logic. Then, you can easily automate your Node.js app with crontab. – Jorjon Apr 22 '20 at 07:08

4 Answers4

22

Lynx doesn't support Javascript.

Update 1

Based on your summary of what you're trying to do (login to a site that requires JavaScript for the password encryption) I'd strongly recommend you look at using Selenium or another browser automation package. Even if you get the password submission working properly without such a layer, the site's probably going to have other issues if you use a minimalist browser like Lynx.

sh-beta
  • 3,809
  • 7
  • 27
  • 32
  • Tried to use selenium. Error I have: "Message: 'The browser appears to have exited before we could connect. The output was: Error: no display specified\n'" That also tends to happen when I use the plain "firefox --newtab *link*" command – TheNotSoWise Feb 19 '12 at 22:42
  • It looks like to get firefox to work, I need to have a display, which is not possible in my situation since this is a VPS. Is there a work around for this? If I can get firefox to work, then I don't think I need selenium. – TheNotSoWise Feb 19 '12 at 22:48
  • 1
    I think this is a new question. – sh-beta Feb 20 '12 at 17:54
  • 1
    You are wrong. JS supported by lynx, try: lynx "www.google.ru/search?client=lynx&q=lynx&e=UTF-8&oe=UTF-8" – Николай Лубышев Jul 07 '16 at 07:02
7

What you need is called a Headless Browser. For example PhantomJS is one of them. I think it is the most popular one.

Alexander Skiller
  • 303
  • 2
  • 7
1

If you only want to execute Javascript from the command line I would advise you something like nodejs or otto. The have a much smaller performance footprint as a whole headless browser.

If you need more than just plain Javascript, e.g. the DOM or so, you should try a headless browser (Chrome, PhantomJS) as suggested by others.

If you not only need a browser, but want to do automated testing with that browser, Selenium is still the way to go.

It really depends on your use case how much you need.

Update: Since a few months it is possible to use Chrome as a headless Browser now (Firefox is on its way too).

JepZ
  • 1,159
  • 14
  • 27
0

Selenium is the way to go, but as you notice, it needs a display. Which then needs to be launched, and yada yada yada, now you have quite a complex setup going on!!

Thankfully, there's yet another way to automate all that, and it's through an automation framework. I recommend Jenkins. It already has an addon to launch an x server.

So the approximate pipeline is as follows:

  • Forget about crontab
  • Setup a jenkins ubuntu server (with x installed)
  • Install the jenkins xvncserver(?) addon -- I think that's what it's called. Anyway this will launch an XHOST for you automatically.
  • Write selenium webdriver scripts that go to the location in question (easiest * is to use the Selenium IDE)
  • Have the selenium IDE output the webdriver scripts to any format, then have jenkins run it. I personally use the Java format, then use ant to build them.

As you can imagine, this is quite a setup, but it technically will do what you are asking. I'm sorry it's not as simple as piping wget to lynx.

Another way to go is just rewrite your webpage to DO STUFF without assuming a browser, for example, take a GET parameter that puts it in "auto mode" and then it will assume it's run from a cron and do it's happiness through a shorter circuit.

Jonathan
  • 6,741
  • 7
  • 52
  • 69