Is it possible to read another url's dom structure?

Question

Obviously modifying it would be out of the question.

But you would think just reading it should not be a problem?

If i have my .js running on someone's system and I want to analyze the DOM of another URL , client side, is there a way to do this?

Something simple like pull the title tag or pull the url...maybe load the site into an iframe to accomplish this?

If you're asking whether you can inspect the DOM of a document in another origin, then no. It would allow stealing user-data in forms, or just served in text. — Mike Samuel, May 21 '12 at 19:36
The same origin policy prevents you to do that. You can have a look at http://stackoverflow.com/questions/3076414 — Mic, May 21 '12 at 19:37
Why do you want to do that from a web client instead of a serverside program like a crawler/bot? — Tharabas, May 21 '12 at 19:40
You want to parse a page by javascript ( I mean see the html of it ?) — Jashwant, May 21 '12 at 19:41
document.getElmentbyId().contentWindow.document for the said iframe...I'll figure out the syntax later. — CS_2013, May 21 '12 at 19:42
@CS_2013: `document.getElementById('myiframe').contentWindow.document`. It was deleted because that won't work if the iFrame is from a different domain than your page. — gen_Eric, May 21 '12 at 19:43
@Mic - do those methods require control of the other domain? — CS_2013, May 21 '12 at 21:20

score 13 · Answer 1 · answered May 31 '16 at 19:37

You can do this using xmlhttp

function getSourceAsDOM(url)
{
    xmlhttp=new XMLHttpRequest();
    xmlhttp.open("GET",url,false);
    xmlhttp.send();
    parser=new DOMParser();
    return parser.parseFromString(xmlhttp.responseText,"text/html");      
}

score 4 · Accepted Answer · edited May 23 '17 at 12:25

4

If I am getting your question right,

A cross domain example by using yql,

var url = 'xyz.com'; // website you want to scrape
var yql = 'http://query.yahooapis.com/v1/public/yql?q=' + encodeURIComponent('select * from html where url="' + url + '"') + '&format=json&callback=?';  
$.getJSON(yql,function(data){
    if (data.results[0]){  
        console.log(data = data.results[0].replace(/<script[^>]*>[\s\S]*?<\/script>/gi, ''));  // The scraped data (the whole webpage)
    }
});

Reference: How can i get Equivalent method of HttpwebRequest in javascript

edited May 23 '17 at 12:25

Community

1
1

answered May 21 '12 at 19:45

Jashwant

28,410
16
70
105

yahoo query language...I'm googled out..can you just give me a brief synopsis? – CS_2013 May 21 '12 at 20:25
some sort of yahoo api that does the parsing for you? – CS_2013 May 21 '12 at 20:26
It does a lot of things. You will get everything [here](http://developer.yahoo.com/yql/console/). What it does, how it does, how to do it and more ? – Jashwant May 21 '12 at 20:28
It acts as a proxy. Since a server can parse any page, it can... and then it sends result back to you as `jsonp`. Since, `jsonp` is cross domain, you can use it from any domain :) – Jashwant May 21 '12 at 20:30
I don't want to hit another server, though that looks cool there are about 3 different ways to do it mentioned in the comment by Mic. – CS_2013 May 21 '12 at 21:08
SO is saying to avoid discussion. Mark as answer if my answer helped you :) – Jashwant May 21 '12 at 21:17
Actually...the ways above require access to second origin...your way requires...3rd party. – CS_2013 May 21 '12 at 22:56

score 1 · Answer 3 · answered May 21 '12 at 19:38

1

If the domains do not match you will not be able to do this due to a security exception. If however you control the other domain, you should research adding a cross domain file to allow access via javascript.

answered May 21 '12 at 19:38

Steve Binder

2,270
1
14
6

1

@SteveBinder: It was deleted. – gen_Eric May 21 '12 at 19:40

score 1 · Answer 4 · answered May 21 '12 at 19:40

1

You could get the html source with a AJAX GET request. An then you can search in the html code or assign it to an iframe/...

answered May 21 '12 at 19:40

Marduk

101
3

4

Not if the URL is from another domain. – gen_Eric May 21 '12 at 19:42

Is it possible to read another url's dom structure?

4 Answers4