14

I know this question has been asked before, but none of the answers have been working for me! I am doing a school project and I would like to get the HTML (to parse it for my project) returned by the dynamic schedule files on my schools server.

The page I would like the HTML of is: https://telaris.wlu.ca/ssb_prod/bwckschd.p_disp_dyn_sched

I think that CORS is not enabled for the school server files and I do not know if it supports JSONP...

How do I set up the cross-domain request to get the HTML from this page?

I have tried:

$.ajax({
    type:'POST',
    url: 'https://telaris.wlu.ca/ssb_prod/bwckschd.p_disp_dyn_sched',
    headers: {
      'Access-Control-Allow-Origin': '*'
   },
   contentType: 'text/html',
   crossDomain:true
}).done(function( data ) {

});

and I get the error:

XMLHttpRequest cannot load https://telaris.wlu.ca/ssb_prod/bwckschd.p_disp_dyn_sched. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access. The response had HTTP status code 501.

When I add:

dataType:'jsonp'

I get the error:

GET https://telaris.wlu.ca/ssb_prod/bwckschd.p_disp_dyn_sched?callback=jQuery21108736664191819727_1416964243449&_=1416964243450 400 (Bad Request) jquery.min.js:4send jquery.min.js:4n.extend.ajax jquery.min.js:4(anonymous function)

Any help is greatly appreciated!

Liam
  • 27,717
  • 28
  • 128
  • 190
Jack C
  • 1,044
  • 2
  • 12
  • 22
  • 6
    Uhhh, you can't do cross domain requests from a browser if the source domain doesn't support it. It's designed that way on purpose for browser security reasons. Non-browser sources (like other servers or proxies) can get the data because the cross domain access is implemented in the browser. So, if you want that data, you will have to get it from somewhere other than via a browser. You can command line tools (like wget) or your own scripting tools like Python or NodeJS to fetch the data. You just can't get it from a browser. – jfriend00 Nov 26 '14 at 01:28
  • If it's a one-time thing to get the HTML, you can even just load the web page into a browser and do File/Save As to save the web page to your hard drive. That will be the page source and will not include any DOM objects created via javascript which is the same as any tool will retrieve. – jfriend00 Nov 26 '14 at 01:30
  • @jfriend00 So there is no way to do it on a browser? For the record this will be implemented on a hybrid mobile application using jquery mobile and cordova for android.. but I would like to be able to test it first to see that my request works. – Jack C Nov 26 '14 at 01:32
  • Guys, more can use apache/nginx/etc proxy feature, and receive actually response from self host. – Deep Nov 26 '14 at 01:32
  • @jfriend00 It will not be a one time thing, it is for a hybrid mobile application and I will have to call it each time the application opens to parse the data. – Jack C Nov 26 '14 at 01:33
  • 3
    Repeat after me. There is no way to do it in a browser (for security reasons). There is no way. You will have to involve a 3rd agent (either a proxy or your own server) that you can request the data from and it can fetch it from the web site for you and then return it to you. – jfriend00 Nov 26 '14 at 01:33
  • FYI, if you just want to display the data, perhaps you could use an iframe? But if you want the actual data in your app, the iframe won't allow you to access the data. – jfriend00 Nov 26 '14 at 01:35
  • @jfriend00 I do not want to display the data. I would like to parse the HTML and use certain parts of it for later in the application. – Jack C Nov 26 '14 at 01:37
  • Possible duplicate of [No 'Access-Control-Allow-Origin' header is present on the requested resource—when trying to get data from a REST API](https://stackoverflow.com/questions/43871637/no-access-control-allow-origin-header-is-present-on-the-requested-resource-whe) – Liam May 02 '18 at 13:04

2 Answers2

18

Browsers enforce "same-origin" access control unless the site explicitly allows cross origin requests (either via CORS or JSONP). So, if the site you are trying to access does not allow cross origin requests, then you cannot get the data directly from the site using only a browser. The browser is enforcing the same origin restrictions requested by the target site.

This is NOT security for a server at all as there are many ways around it. It only applies to one specific type of access from a browser (though that one specific type of access protection is useful).

This means to get the data into a browser you will need to use some sort of third party agent (other than the browser) that can get the data for you. The two most common ways of doing that are:

  1. Your own server. You make a request of your own server to get some content from some other server. Your server then fetches the data from the other server and returns it to you in the browser.

  2. A proxy server. There are some preconfigured proxy servers that are built just for doing what is described in option #1. You can either use a proxy service or install your own proxy server to do this for you or configure your own web server to have this capability.

So, you can't bypass cross origin restrictions from a cooperating browser. But, you can bypass them from a server. This is because CORs restrictions are implemented only in the browser. They aren't a server-enforced restriction. The browser asks the target server what CORs policies are in play and enforces them in the browser only. Some other server making a request to that server does not need to pay any attention to CORs policies.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • 1
    What is the alternate way? means `Angular` or `React` can resolve it? – 151291 Feb 01 '19 at 06:35
  • Why is a third party agent allowed to access a site not allowing cross origin requests? How secure is that? –  Aug 18 '19 at 22:22
  • 1
    @PauliSudarshanTerho - You would benefit from doing a little reading about how the web works. A browser makes a request to your site. That browser could be any IP address on the internet in the world. It's just some random client's computer somewhere that wants to access your site. Since you want users to be able to access your site, your web server responds and sends them the data they are asking for. The same is true for regular page requests as for web APIs. Any web API is open to the world unless you explicitly require some form of authentication in your web server. – jfriend00 Aug 19 '19 at 00:01
  • 1
    @PauliSudarshanTerho - CORs is a browser-specific security feature. It ONLY affects requests coming from a cooperating browser. It does not affect at all requests coming from any other type of source. That's how the web works. CORS is not really security. ALL it protects you from is someone else embedding Javascript in their web page that would contact your site when viewed from a cooperating browser. Web API requests from a web page don't actually come from your site. They come from some random client's computer/IP address. If you were the downvote today, you might rethink that. – jfriend00 Aug 19 '19 at 00:03
  • 1
    @PauliSudarshanTerho - The two usual forms of protection that web sites build in are: 1) Rate limiting to protect a rougue client from consuming excessive resources on your site and 2) Some form of user login from which "proper" use of the site APIs can be tracked and when abused that login can be banned. There is no such thing as preventing use from outside of your web page. It really can't be done. Web APIs are open to any computer on the web. That's how the web works unless you require user/account authentication for every single request. – jfriend00 Aug 19 '19 at 00:07
  • 1
    @PauliSudarshanTerho - So, as you can see CORs protection is easy to bypass from your server. You just ask your server to get some data from the other web site. Since CORS is only enforced between a cooperating browser and a server, there will not be any CORS enforcement from your server to this other server. That is the way of the web. If you really want to prevent that, you have to implement credentials and then figure out how normal users can still use your site. – jfriend00 Aug 19 '19 at 00:10
  • 1
    Thank you for the lessons. Don't expect I have a server and know lesser than you. My understanding is that origin is defined by http, domain and port, that means origin = server. Cross origin then means communication between **two** servers. I have no server so I need your solution (2) and PO need solution (1). If cross origin is not allowed none of the two answers is possible. PO have a server but it does not help if that server go through another server requesting the server with CORS not set or it is set to not allowing access. Can you correct what I said? –  Aug 19 '19 at 01:31
  • 1
    @PauliSudarshanTerho - Well, to get access to cross origin data in a browser from a site that restricts what origins can access its data, you need an external server of some type in order to bypass browser-implemented cross origin restrictions. I don't know what else to say. If ted.com implements cross origin restrictions so that only web pages from ted.com can access it's API, then Javascript in web pages from alice.com can't get directly access ted.com APIs. Javascript in alice.com web pages would need to ask some server to get the data from ted.com on their behalf and forward it to them. – jfriend00 Aug 19 '19 at 02:20
  • *Cross origin restrictions*? Nah.. Cross origin is about allowing.. From beginning there was the SoP only alowing same origin. It was not enough so more allowed by CORS. History of Cross origin in a nutshell. –  Aug 19 '19 at 02:21
  • 1
    @PauliSudarshanTerho - Why are you picking on this? Now you're picking on details of terminology. Is there something wrong with the concept of this answer? If not, I'll move on to better places to put my effort. CORS stands for cross origin resource sharing. When cross origin sharing is not specifically allowed by the target, then one runs into cross origin restrictions. – jfriend00 Aug 19 '19 at 02:32
  • 1
    @PauliSudarshanTerho - And, for your info, people did JSONP to work around cross origin issues and that was an abomination of a way to do things. At least CORS is a nicer way to allow cross origin access. But, it ONLY applies to requests made from a cooperating browser because it's a client-side implementation, therefore server to server does not have any of these restrictions. – jfriend00 Aug 19 '19 at 02:35
  • It is not wrong in what you say (see the link in this comment) and it is because of it is possible to go around CORS that makes it meaningless or actually very dangerous if everyone believes it is secure. My first questions is still open to the world to think about: *How secure is that?* https://medium.com/netscape/hacking-it-out-when-cors-wont-let-you-be-great-35f6206cc646 –  Aug 19 '19 at 02:40
  • @PauliSudarshanTerho - Same origin restrictions is NOT security in any definition of the word. All it prevents is websiteB accessing websiteA APIs from Javascript in the webSiteB page. That's all it stops. There are many ways around it (but they require help from a server or code running somewhere other than a browser). And, same origin protection ONLY works from a cooperating browser. I'm going to move on to other questions now. – jfriend00 Aug 19 '19 at 02:42
  • @PauliSudarshanTerho - If you have further questions about same-origin protections, you should probably post your own question. We are not supposed to be using comments for a long running discussion about something that isn't directly related to the original question asked here. – jfriend00 Aug 19 '19 at 02:48
  • Yes... JSONP is json padded by HTML ` –  Aug 19 '19 at 02:55
0

I know how frustrating it will be when technical limitations makes life more harder.

For the situation like this, you can try the Javascript library: Xdomain
You can check the reference article for the same here: http://hayageek.com/cross-domain-ajax-jquery-without-cors/
The YCombinator discussion may help, if you have issues: https://news.ycombinator.com/item?id=8023844

Other way is, use your server to get the HTML page, and push to your application as and when required. Something like this. Assume you have added one iframe as 'my_external_content.php'. The code should be

<?php

    $externalURL = "http://otherdomain.com";

    // requires fopen wrappers
    $externalData = file_get_contents($externalURL);

    echo $externalData;
?>

Please refer the other question on SO for some reference: Ways to Circumvent Same Origin Policy

Hope this helps your or others who are in same situation.

Herman J. Radtke III
  • 1,804
  • 1
  • 24
  • 29
Vinod Tigadi
  • 859
  • 5
  • 12
  • This helped me after realising that I would not be able to access the content of a webpage that does not allow cross origin requests. You have a typo in your code where you forgot the $ symbol from the variable being echo'd. – dading84 Jul 09 '17 at 09:37