0

hi everyone I would like to create a small bot to help me on binary option. i am not an expert on python but actualy I can read a web page and retrieve a precise value in a tag, but the information what I need is on a web application and not in the source code of the web page. I am not an expert of eb application and I want to know if I retrieve a value displayed on the application with python. here is a link to the picture of the application: "http://comparatif-options-binaires.fr/wp-content/uploads/2014/05/optionweb-analyse-technique-ow-school.jpg"

user3710113
  • 13
  • 2
  • Be aware that their [T&C document](http://www.optionweb.com/optionweb-com-terms-conditions/) specifically prohibits this: `Non authorized Automatic or semi automatic trading mechanism integrated by the client on the browser or client side, that does not involve human execution, will be considered back door API or system abuse that may result to an account suspension and/or position cancellation.` – Jamie Bull Jun 05 '14 at 17:01
  • I don't want to use it to trade but to warn me if a level of overbought or oversold has been reached. for example if rsi value returned by the bot is> 70 or <30 then there's oversold or surachat.on can not stay 24/24 to the website. I do not think this is a violation of the terms of the contract. programe just alert me with a sound if an event occurs – user3710113 Jun 05 '14 at 17:20
  • That sounds reasonable. I'd look into the options suggested by @Ewan below then. – Jamie Bull Jun 05 '14 at 17:39

2 Answers2

1

I think the problem you face here is the value you need is being loaded via Javascript of some sort (though without access to the web application and no visible effort from your code I can't be sure).

Expanding on @sabhirams answer (and agreeing that requests and BeautifulSoup are excellent libraries for static text) I would recommend having a look at the following:

  1. Selenium - automates web browser usage in python (so will run the full javascript).
  2. Webkit - Again another headless browser for python that has some excellent SO questions on the matter.
  3. Ghost.py - attempts to make the Webkit experience a little smoother.
  4. pyv8 - something a bit more barebones, pyv8 is a python wrapper for the Google V8 Javascript engine and can be used to run the javascript on the page and, hopefully, extract the element you need.

And if you're really not settled with python why not look at using a Javascript headless browser to run the javascript like PhantomJS.

As mentioned before; Respect others when scraping and be aware there may be consequences if you are caught.

Community
  • 1
  • 1
Ewan
  • 14,592
  • 6
  • 48
  • 62
  • thank you a lot for your answer.I'll do some research on this list and I'll let you know if I solved the problem. – user3710113 Jun 05 '14 at 18:17
-1

I think you mean you want to build a script which can scrape a given webpage, and extract a certain value out of a given target DOM element.

I dont currently have the time to write this code for you, but it should be rather simple to put together. Here are some modules which might help you:

  1. Request - Use this to fetch a given webpage into your py script
  2. BeautifulSoup - Feed the above "DOM text" to beautiful soup, and you will be able to more easily manipulate the HTML page (fetch your var of interest etc...).

EDIT: As pointed out in the comments above, please consider the Terms and Conditions of the web-service you are trying to scrape info from.

sabhiram
  • 897
  • 7
  • 10
  • 1
    thank you bro.for your answer. I know a little beautifulsoup but I think he only read the source code of a webpage. I'll dig more in this if I get the below to find info on web application – user3710113 Jun 05 '14 at 17:25