0

I am currently having Hadoop-2, PIG, HIVE and HBASE. I have an inputdata. I have loaded that data in HDFS. I want to create staging data in this environment.

My query is -

In which BigData component, I should create Staging Table(Pig/HIVE/HBASE) ; this will have data coming in based on a condition? Later, we might want to run MapReduce Jobs with complex logic on it.

Please assist

user3343543
  • 143
  • 2
  • 14
  • Anywhere you want. Pig is not an option as it does not have a metastore. Hive if you want SQL Like queries. HBase based on your access patterns. – Venkat Jul 15 '15 at 10:37
  • Hello Venkat. Thanks for the reply.If I create it in HIVE and then i want to run mapReduce programs on top of it. How will I achieve that? And one more thing, what do you exactly means by access patterns? Any example? – user3343543 Jul 15 '15 at 10:51

2 Answers2

0

Anywhere you want. Pig is not an option as it does not have a metastore. Hive if you want SQL Like queries. HBase based on your access patterns.

When you run a Hive query on top of data it is converted into MR.

When you create it in Hive use Hive Queries & not MR. If you are using MR then use Pig. You will not benefit creating a Hive table on top of data.

Venkat
  • 1,810
  • 1
  • 11
  • 14
0

Hive: If you have OLAP kind of workload and dont need realtime read/write.

HBase: If you have OLTP kind of workload. You need to do realtime/streaming read/write. Some batch or OLAP processing can be done by using MapReduce. SQL-like querying is possible by using Apache Phoenix.

You can run MapReduce job on HIVE and HBase both.

Anil Gupta
  • 1,116
  • 2
  • 8
  • 13