1

I know Airflow is called workflow manager, nifi dataflow manager, but what this means exactly? The best explanation so far was that nifi cares about data while airflow cares about tasks, but I don't quite get this definition, and I couldn't find any other good explanation/article/video that explains how to integrate this systems, if it is a good idea or is better to use each one in their own.

Also I was thinking if it is better StreamSets or NiFi, I think streamsets looks better in UI and monitor the data, but I heard that depends on the case, that nifi is better if I only ingest data, but again I can't find much information about this questions.

set92
  • 322
  • 4
  • 13
  • This question is not for stackoverflow. It's too wide and depends on personal opinion. Just note that streamsets is not opensourse: https://streamsets.com/subscriptions And airflow and nifi for different kind of job. – daggett May 13 '19 at 06:02
  • StreamSets is Apache 2.0 licensed open source: https://streamsets.com/opensource and https://github.com/streamsets/datacollector/blob/master/LICENSE.txt – metadaddy May 14 '19 at 01:07
  • Your second question has been answered at https://stackoverflow.com/questions/36899612/difference-between-apache-nifi-and-streamsets – metadaddy May 14 '19 at 01:09

1 Answers1

1

As you said, Airflow is a workflow manager. It means that it only tells other programs to run. It doesn't process data, but tells other to run.

NiFi and StreamSets on the other hand, process data, transform it, receive it and send it. That's why they are dataflow managers.

metadaddy
  • 4,234
  • 1
  • 22
  • 46
Ben Yaakobi
  • 1,620
  • 8
  • 22