3

Is there any way to add arbitrary data in HDFS to apache atlas? Having installed HDP 3.1 for evaluation, this appears to not be possible (eg. only data that is sqooped in, placed in a hive table, or some other narrow set of atlas-visible operations). We have ETL processes that bring in various forms of flatfile data (eg. parquet, tsv, etc.) and for none of which have I figured out how to get to show up on atlas. Is there any way to do this? Basically, would like to have this data be taggable by Ranger and discoverable via Atlas. Any ideas on how this can be done?

lampShadesDrifter
  • 3,925
  • 8
  • 40
  • 102
  • A workaround for adding arbitrary HDFS items manually (though this specific workaround can't be converted to a programmatic one) is to use the Atlas entity creation UI, see https://community.hortonworks.com/articles/182877/manually-creating-entities-using-atlas-ui.html. Am also looking into using the Atlas REST API and will post with more info later. – lampShadesDrifter Aug 06 '19 at 21:20

0 Answers0