1

I came across something which showed that you can write analytic functions in Hive.

For example: for a wordcount example, we can write in hive as well. the difference would be that in Aster data it is already in-built whereas in hive we have to write it.

What will be the difference? why go for which?

topchef
  • 19,091
  • 9
  • 63
  • 102
NJ_315
  • 1,863
  • 7
  • 22
  • 30
  • Are you using SQL-H in Aster to work with Hadoop nodes? Or you compare Aster and hive independently? – topchef Jul 06 '13 at 13:22
  • i know that there is SQL-H in Aster. its just that I came across something that Hive can do the same word count problems. So can Hive do all the at least basic analytic functions if I can write them? – NJ_315 Jul 08 '13 at 05:21
  • i want to know the difference between Aster and Hive, if Hive can do analytic functions then just like Aster? – NJ_315 Jul 08 '13 at 05:22

2 Answers2

1

There is a good research study based on a lab study which compares the time to develop and execute analytic functions in both the Teradata Aster discovery platform and Hadoop/Hive. The systems were run side by side to show what workloads are appropriate for each system. There is a good working example of a "day in the life" of an analyst and the time/effort required.(disclosure: i work for Teradata which acquired Aster Data 2 years ago) http://www.asterdata.com/resources/assets/ESG-Lab-Validation-Teradata-Aster-MapReduce-Platform.pdf

user2571199
  • 11
  • 1
0

Theoretically, yes, Hive should be able to do all the same as both feature Java code and map-reduce frameworks. I am not a user of Hadoop/Hive but my understanding is that Hive is a layer on top of Hadoop and everything Hive does (including analytical extensions written in Java) will get translated into Hadoop jobs. You may want to ask Hive-directed question on how/what it takes to do it.

On contrary, Aster SQL/MR is native to Aster database. By native I mean that Java runs within each Aster node as part of Aster SQL/MR framework, which in turn is integral part of Aster database engine. All data manipulations will be consistent with the data model, data distribution keys, etc. In Aster, while using its SQL/MR functions (including Java-based) user never leaves premises of SQL and the data model. At the same time SQL/MR is polymorphic to table definitions adapting for arbitrary models (all within Aster SQL). Maybe you want to investigate how this would work in Hive.

Another point to make is that Aster offers rich set of high-level analytical functions out of the box so that writing custom Java SQL/MR may not be necessary. Thus, word counting example could be executed using nGram function and aggregate SQL.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
topchef
  • 19,091
  • 9
  • 63
  • 102
  • So, such analytic functions can be done in Hive theoretically, but wont be easy as using Aster? – NJ_315 Jul 08 '13 at 06:15
  • Easy is relative and I don't want to compare to Hive/Hadoop I have very limited knowledge of. The best path would be getting Hive/Hadoop and Aster side-by-side and prototyping simple functions in both. Sorry for no definite answer. – topchef Jul 08 '13 at 06:29