1

I have been trying to scrape share market usinf "rvest" package from this url: http://finans.mynet.com/borsa/canliborsa/#A which requires to sign-up. I created dummy account for you to try. Username and password below are real and work okay. What I have come up with is the following:

library("rvest")
library("data.table")

url<- "http://uyeler.mynet.com/login/login.asp?rurl=http%3A%2F%2Ffinans.mynet.com%2Fborsa%2Fcanliborsa%2F&formname=finans#A"
session<-html_session(url)
form<- html_form(session)[[1]]

login<- set_values(form, "username" ="muharrem_babaogul_1991","password"="q1w2e3")
submit_form(session,login)

jumped<-jump_to(session,url = 'http://finans.mynet.com/borsa/canliborsa/#A')

page<- read_html(jumped)
page<-html_nodes(page,xpath='//*[@id="canliLeftColumn"]/div[3]/table')
page<-  html_table(page)
head(page[[1]])

And the result:

[1] Hisse         Hisse         Hisse         Son           Alış         
[6] Satış         %Fark         En Düşük      En Yüksek     AOF          
[11] Hacim (Lot)   Hacim (TL)    Son İşlem     Ekle / Kaldır
<0 rows> (or 0-length row.names)

As you can see, I can reach the table with xpath, I get the column names but without any data inside. Table is completely empty. Is there anyone who can help ? Thanks in advance.

Barsssk
  • 93
  • 7
  • The tables are generated via javascript. It looks like the data you are interested is store is a file `data.fdata`, but I am not sure how to request it from a secure page. Take a look at this answer: http://stackoverflow.com/questions/40638511/using-rvest-to-grab-data-returns-no-matches/40642507#40642507, this might help. – Dave2e Nov 21 '16 at 20:08
  • Yes @Dave2e, data comes from fdata so I cant get it, didnt know about it till now. But still I cant manage to scrape it anyways. Because fdata url ( [here](http://finans.mynet.com/borsa/canliborsadata/data.fdata) ) also needs a login. And I cant use the way above (sumbit_form and jump_to) because it gives error on read_html(jumped) function that file is not a html file. So I tried the solution on this [link](https://stackoverflow.com/questions/24723606/scrape-password-protected-website-in-r) for my problem and came up with code on following comment below and still does not work. – Barsssk Nov 22 '16 at 08:14

1 Answers1

2

Problem solved. Thanks for help @Dave2e . I found out that jump_to function can scrape the data and contains it in $response. Code below gives the data as txt only spliting it remained.

url<- "http://uyeler.mynet.com/login/login.asp?rurl=http%3A%2F%2Ffinans.mynet.com%2Fborsa%2Fcanliborsa%2F&formname=finans#A"
session<-html_session(url)
form<- html_form(session)[[1]]

login<- set_values(form, "username" ="muharrem_babaogul_1991","password"="q1w2e3")
submit_form(session,login)


jumped<-jump_to(session,url = 'http://finans.mynet.com/borsa/canliborsadata/data.fdata')

file<-content(jumped$response,as="text")
Barsssk
  • 93
  • 7