Alternatives to Yahoo Pipes

Question

I've used Yahoo Pipes for some prototyping, but I'm ~~unwilling~~ reluctant to use it in full production as we'll have no control over it uptime etc.

Is there an (ideally open source) alternative that I can run on my own server? Or any application that can convert a pipe into PHP or Ruby?

Between Yahoo's servers and yours, I would bet than Yahoo's uptime is better than yours ^^ — Pascal MARTIN, Jan 07 '10 at 11:38
@Pascal MARTIN Ha! Agreed. But they throttle and I'm not convinced it will even exists as a company in a years time. Or they could just pull the plug or start to charge. Just too risky. — Richard Stelling, Jan 07 '10 at 11:56
The question is now closed so i can't post it as answer, bot for the sake of documentation, I came a cross Huginn(https://github.com/cantino/huginn) which is a open source & self hosted alternative, which gained a lot of attraction after pipe sunset. — Rabin, Apr 27 '16 at 12:19

score 9 · Answer 1 · answered Mar 24 '12 at 12:27

What about this? http://pipes.deri.org/

"DERI Pipes: Open Source, Extendable, Embeddable Web Data Mashups

Inspired by Yahoo's Pipes, DERI Pipes is an engine and graphical environment for general Web Data transformations and Mashup. Supports RDF, XML, Microformats, JSON and binary streams. Use it as a "Web Pipe" or embedded in your applications Works as a mashup command Line tool Supports SPARQL, XQUERY, Several scripting languages. Extend it as needed DERI Pipes, in general, produce as an output streams of data (e.g. XML, RDF,JSON) that can be used by applications. However, when invoked by a normal browser, they provide a end user GUI for the user to enter parameter values and browse the results (see the examples below) ."

From what I can tell, DERI Pipes has long been abandoned as an active open source project. The last commit was about 3 years ago, and there's been no sign of activity since then. — Jack, Jan 10 '13 at 18:24

score 8 · Accepted Answer · answered Oct 04 '10 at 08:44

You can generate a Python code equivalent of a simple Yahoo pipe with pipe2py: http://github.com/ggaughan/pipe2py

If a generated code equivalent of Yahoo Pipes block you require isn't part of the distribution, pipe2py is easily extended/ (Please consider contributing any extensions back...)

score 4 · Answer 3 · answered Jan 07 '10 at 11:58

In our company, we've been using Yahoo Pipes for quite a long time and we're still using it for some specific and more complex feeds.

Otherwise, we are developing a simple Sinatra-based application to work like Yahoo Pipes and return the result as a standardized XML/JSON response. It's quite simple to create such application, however I agree with Pascal. Between Yahoo's servers and yours, relying on Yahoo! should be more effective than relying on a custom server.

The only Pipes downside we've been experiencing is the API throttle. We've partially solved the issue caching the responses on our side and implementing a custom throttling mechanism to dynamically slow down requests when Pipes returns an HTTP error.

Good to hear a success story. But it would be great to have an open source alternative that you could write your own modules for. — Richard Stelling, Jan 07 '10 at 12:25

score 2 · Answer 4 · answered Nov 20 '12 at 05:46

2

It seems to have gone dormant, but for the sake documenting, there's WebHookIt, based on Node and MongoDB.

answered Nov 20 '12 at 05:46

Su'

2,128
20
22

score 2 · Answer 5 · answered Nov 28 '12 at 06:04

2

Please take a look at apigee , they have visual interface as well as xml base scripting engine. It allows you to process any api, xml, json or text. they have

answered Nov 28 '12 at 06:04

shahid610169

370
2
11

mmm is their engine opnesource or is it a SAS based model? – user1172468 Apr 09 '13 at 23:54

score 2 · Answer 6 · edited Jun 18 '12 at 04:47

2

Maybe you should check Apatar - not sure what your requirements are, but maybe it helps?

edited Jun 18 '12 at 04:47

Chad Levy

10,032
7
41
69

answered Feb 09 '11 at 15:49

Hale

21
1

score 1 · Answer 7 · answered Jan 07 '10 at 12:47

Regarding uptime, you could cache the output of Yahoo Pipes on your server in case the feed goes AWOL for a short time (though as PascalMartin said, their uptime is very likely better than yours).

It's unlikely that Yahoo would suddenly close Yahoo Pipes but if they did I'm sure clones would start springing up. Worst case scenario it wouldn't be too difficult to write a custom program/script to fetch a bunch of feeds and filter them on your criteria.

score 0 · Answer 8 · answered Apr 03 '14 at 19:32

There is a very similar tool called Quadrigram. It has over 500 operation modules that you chain together to create data processes. It also has 40+ combinable visualizations. It is web-based, and allows you to publish interactive prototypes online.

www.quadrigram.com

Alternatives to Yahoo Pipes

8 Answers8

Linked