10

This is a multi-part question. I just watched a very interesting presentation on YQL by the lead developer (a graduate of my MS program). While it was very compelling, and I am looking forward to trying it out, I am wondering if anyone knows of alternative frameworks for querying multiple web service APIs to make them appear seamless, the apparent purpose of YQL?

Yahoo's strategy has been to create XML schema definitions that bind a given web service's parameters into their YQL Open Table query parameters, which I think is very clever. Is there any tool that attempts (perhaps I am naive here) to automate the discovery of parameters in say a REST API? I am aware that with SOAP APIs, because there is a published WSDL, it makes automation easier, but is there yet no way to do this with REST? Is anyone trying?

Sam McAfee
  • 10,057
  • 15
  • 60
  • 64
  • I'm very skeptical that an auto-discovery tool exist for REST API's since the same Entity may have many different representations. And may define ad-hoc what parameters it accepts. WADL attempts to make things better, but I think it's dead in the water since it goes against the minimalistic mind set of REST developers. Good Question. +1 – Allain Lalonde Sep 24 '09 at 13:45

3 Answers3

6

Yes people are trying to produce description languages for REST. The most popular effort is WADL. There are lots of questions about WADL here on SO. Is it a good idea? In my opinion no.

REST does not need a discovery model beyond what it already has with hypermedia, because is trying to solve a problem at a different architectural layer than web services. Web services deliver data to an application's business logic/domain model. REST is about delivering content and behaviour to a presentation layer.

How about an analogy? Think of the different between an object and struct in C++. A struct is just simple data that some client process is going to manipulate. That's what a web service does, it returns a chunk of data, a struct. Sure maybe it did a bunch of server side processing to produce the result, but the end result is a lump of data. A REST interface delivers an object. i.e. It contains both data and the methods that can be used to manipulate that object. By definition, if you understand the uniform interface and you understand the returned media type, you already know what you can do with the response. Discovery mechanisms are redundant.

If you find this hard to believe, the think about the web. How does a web browser discover web pages? The web has no formalized discovery mechanism, and yet there is a world of information out there that we can discover with a web browser.

Darrel Miller
  • 139,164
  • 32
  • 194
  • 243
  • I don't agree with this answer, I don't think that REST limits only to delivering content (and behaviour) to a "presentation layer". And I consider a bad practice binding behaviour to REST. – ElLocoCocoLoco Feb 20 '16 at 23:59
  • @ElLocoCocoLoco If you would help me understand which of the REST constraints are being violated by "binding behaviour to REST" and the negative system effects of those violations then maybe I will be able to understand why you consider it a "bad practice". – Darrel Miller Feb 21 '16 at 19:22
  • Do you understand French? https://fr.wikipedia.org/wiki/Representational_State_Transfer In delivering "behaviour" I assume you are talking about the 6th (optional constraint) Code-On-Demand of REST services. If that is the case then that is usually considered a bad practice because "a state becomes dependent on the client and not the server which contradicts Rule 2." If you are talking about point 4.3 "responses explain their nature" even on this case we need some times some services to explain nature of results before executing the request itself (Adaptative/Auto discovery Systems) – ElLocoCocoLoco Feb 22 '16 at 14:39
  • @ElLocoCocoLoco Yes I understand French. No I'm not talking about COD. The presence of link relation types in responses describe the available behaviour and the interaction mechanism. The presence of a "next" link in a response identifies to a client whether it can transition to the "next" resource or not. This is a description of potential behaviour. – Darrel Miller Feb 22 '16 at 19:49
  • I think you are conferring to much logic to what is a REST service, and your ideas are in an upper layer but not in REST itself. http://rest.elkstein.org/ has a good definition about what is REST. "REST is a lightweight alternative to mechanisms like RPC (Remote Procedure Calls) and Web Services (SOAP, WSDL, et al.)" I dont agree with your analogy about classes and structures. WADL (or another) just a mean to define "contracts". As specified here: http://stackoverflow.com/questions/1312087/what-is-the-reason-for-using-wadl, nothing more. Regards. – ElLocoCocoLoco Feb 22 '16 at 22:40
  • @ElLocoCocoLoco Have you read this? http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven Can I respectfully ask that you spend some more time researching what REST really is before you start educating others? – Darrel Miller Feb 23 '16 at 00:56
  • I d't care about educating others, I just give my opinion based on experience in real life. I let Bookworms educate others, they have time to do it, and make them feel intelligent. I think the first dissertation about REST has completely slipped from the author hands, mainly because the terrible lack of practical details in many aspects (you can see it in the link you generously posted, no a single line of practical implementation) "If evolution would have followed only rules without trail and error we would be holding from a tree branch some where in Africa right now or even better".Closed ;) – ElLocoCocoLoco Feb 23 '16 at 14:32
  • @ElLocoCocoLoco Don't let the fact that you can't see the practical implications of Fielding's words let you believe they don't exist. I've been building actual production REST based systems for 8 years, applying the REST constraints as outlined in the dissertation. If you have no interest in educating others about what you have learned, then I suggest you keep your opinions to yourself. I've spent many years here on SO trying to help people not make the same mistakes I did and I don't appreciate the inference that I do it to feel intelligent. – Darrel Miller Feb 23 '16 at 16:02
1

There are two ways to find information. Either you use a 100% unambiguous language or you use a natural language. Anything in between like YQL is doomed to fail because it delivers neither and works well only with the examples its authors tout.

I blogged about this at http://zscraper.wordpress.com/2012/05/30/enough-with-crawling-2. My personal stance is that you'll always get the most accurate results if you do your homework first, i.e. study the target domain and figure out how to query it unambiguously.

To answer your question and give you an alternative -- try Bobik. This is a cloud-backed scraping service that you control via REST API. Compose your "queries" in traditional syntax (Bobik supports Javascript, JQuery, XPATH and CSS) and call Bobik to run them from any client-side environment (webpages, mobile apps, or your server).

Hope this helps.

Yevgeniy
  • 1,313
  • 2
  • 13
  • 26
  • 3
    The website, http://usebobik.com, no longer exists. I also believe the service is no longer available. – j.raymond Nov 08 '13 at 14:35
1

There is this little website http://zachgrav.es/yql/tablesaw/ which indeed auto-discovers parameters in a REST api and turns it into a YQL compatible table.

Sean McRaghty
  • 295
  • 2
  • 6
  • 17