How to use ajax (or simple, in case ajax is not possible) result from a website to my site?

Question

I am developing a site which need a huge data dynamic in nature. I can't afford such a huge data on my server but there are lot of sites using it. Let me clarify with an example.

Suppose a site hotel.com have list of all hotels. It allows user to search hotels with ajax query. As soon as user type a letter it shows possible hotel names with that letter and user selects out of them.

I am designing a site which need hotel names but i don't have them all and lot of names keep on adding.

So how can I design a page so that when I enter a letter in my site, I get all relevant results from hotel.com in a drop-down menu?

EDIT 1

---

Based on replies, here are more specification. Based on microspino relpy, I think i can manage space if i scrap the data periodically. I wont be needing all the data once. So you can forget the space limitation. All I need is data which is not available to me directly.

EDIT 2

---

Based on solutions, i thought of a possible workaround that i can let user enter the hotel names manually and i can cross check their validity at backend based on the data on the hotel.com site. What shall i use to it at backend i.e fetch results from the site and compare them?

Answer 1

Unless hotel.com make their data available via JSON-P, you can’t, because AJAX requests can only be made to the same server from which the JavaScript file containing them was loaded, for security reasons.

Aside from that, I don’t think hotel.com would be particularly happy with you scraping all their data without permission. They’re paying for a server that can hold all that data, yet apparently you want to use that for free.

Answer 2

Ajax request generally don't work cross domain. However there are some exceptions, IE will allow you to POST data but not look at the response, and JSONP can be set up as mentioned if both sides are willing participants.

However saying that "this can't be done" isn't exactly 100% correct. You can't do this with javascript alone but you CAN set up a proxy locally to do it for you. Any server side programming language can handle grabbing information from another web site. In the past I've set up a local JSP proxy to pull the data and then just pointed my ajax call at the local JSP instead of the remote site.

See my answer here: stackoverflow answer for cross domain ajax calls which has another link to an example I posted. Java/JSP Proxy example

Setting up a JSP, ASP, PHP, etc...Proxy is your best bet and will give you what you want.

Answer 3

You can't.

You can't add a widget on your site that makes the user construct an AJAX request to hotel.com and steals their data. AJAX does not work cross-domain.

Unless

Unless hotel.com was friendly enough to explicitly allow this for you by hosting the hotel-api.js file on their server, which you can include on your page. It will provide a JSON object which you can use in your further scripts.

But there are other solutions: read more about cross domain AJAX here.

Answer 4

You may use JQuery and its crossdomain Ajax capabilities as described in http://usejquery.com/posts/the-jquery-cross-domain-ajax-guide

However I am not sure whether it would be legible to use other site's data for commercial use.

Answer 5

Given that you have not provided your tech setup I will go very generic. I prefer to give you a sort of solution, based on Ruby, but your goal could be accomplished by using any general purpose programming language.

THE GENERAL STRATEGY

I would schedule a cron job on your server that fetch the names every week, by scraping the hotels.com website. Scrapi is a nice rubygem to do this. If the hotels.com needs some sort of interaction to retrive the info you need, you can also use Mechanize. You could store the names on your SQL db and if you want a no-SQL solution that is also extremely fast I'd like to mention Redis (see below after 'adding complexity').

I will then make a URL endpoint on your own backend that render a JSONified version of the names list. This can be easily accomplished with the json rubygem but if you need more power/flexibilty jBuilder will help you a lot.

I will then fetch/query that local endpoint for names completition, to avoid cross-domain-request issues. There is a jQuery plugin called autocomplete that would do the rest.

It's worth to mention that Ruby on Rails 3.1 already embed the jQuery-UI and everything you need to create the website with that answer-JSON endpoint. It would be a nice learning experience but I'm a Rails partisan... :)

ADDING COMPLEXITY

Depending on your requirements/time maybe you can add Redis to the mix. It's a fast C coded key-value store with bindings for a lot of programming languages. It's a nice fit when you need to deal with fast queries a large key-value dataset. This question on SO (...SO it's really like wikipedia for programmers!!!) cover a something similar to what you asked. Salvatore Sanfilippo has also posted an entire tutorial on his website about using Redis for an auto-complete scenario. Your scheduled script would populate redis instead of writing the list somewhere on the filesystem or on a db. Of cours this needs some effort ;)