2

I am trying to get response of youtube.com using java with JSoup.

I am able to get the response of youtube using JSoup as follows, it returns the desktop website's response

         String str = "https://www.youtube.com/";
         doc = Jsoup.connect(str)
                    .userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36")
                    .get();

Same way, I am trying to get the response for mobile version to this same site as follows,

            doc = Jsoup.connect("https://"+url2.getHost()+"/search?q="+q)
                        .userAgent("Mozilla/5.0(Linux; U; Android 2.2; en-gb; LG-P500 Build/FRF91) AppleWebKit/533.0 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1")
                        .get();

But this gives only desktop/laptop version response and not the mobile response.

How to get the mobile response from jsoup.

Thanks in advance.

Am_I_Helpful
  • 18,735
  • 7
  • 49
  • 73

2 Answers2

1

I think that maybe your User-Agent isn't quite correct.

I've just tried it with the following and appear to have hit the mobile YouTube site:

String mob = "https://m.youtube.com/";
         mobile = Jsoup.connect(mob)
                    .userAgent("Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1")
                    .get();

Update I've had a look in more detail and I believe that the page content is being modified by some Javascript once the DOM has loaded.

Looking at the HTML returned by the JSoup code above I get this (note the content div is empty):

 <body id="body" class="atom fusion-tn">

  <div id="player"></div>
  <div id="guide-layout-container">
    <div id="guide-container"></div>
    <div id="content-container">
      <div id="content"></div>
    </div>
    <div id="guide-overlay"></div>
    <div id="lightbox"></div>
    <div id="toast"></div>
    <div id="content-overlay"></div>
  </div>
  <div id="_yt_orientation_detect"></div>

  </body>

Comparing to the HTML viewed in Chrome's dev tools I see this:

enter image description here

JSoup is just an HTML parser, not a web browser. In order to do achieve what you require, I think you might need to look at something like this Is there a way to embed a browser in Java?

Community
  • 1
  • 1
Adam Rice
  • 880
  • 8
  • 20
  • It delivers the response. but the response says this page is error. Have you read and noticed that? –  Mar 31 '16 at 07:16
  • I don't get an error when I run that. It loads the mobile site, albeit without any content. Looking in Chrome's dev tools, I think the content is loaded over a separate request. What error are you getting? – Adam Rice Mar 31 '16 at 08:44
  • reponse html content says "Please enable JavaScript on your browser. " –  Apr 04 '16 at 11:08
  • @Ratwanska have a look at the update to my answer. JSoup is not a web browser - it's just an HTML parser. – Adam Rice Apr 04 '16 at 12:02
  • Adam, Thanks for your explanation. In the same way I can get facebook and other sites with mobile response... youtube only the problem.. –  Apr 07 '16 at 05:48
0

instead of:

.userAgent("Mozilla/5.0(Linux; U; Android 2.2; en-gb; LG-P500 Build/FRF91) AppleWebKit/533.0 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1")

try:

.userAgent("Opera/12.02 (Android 4.1; Linux; Opera Mobi/ADR-1111101157; U; en-US) Presto/2.9.201 Version/12.02")

other options find on:

http://www.useragentstring.com/pages/useragentstring.php

and search for :

Mobile Browsers

Community
  • 1
  • 1
calincan mircea
  • 5
  • 3