Pivoting a large table in Hive

Asked Jul 07 '17 at 21:03

Active Jul 08 '17 at 05:33

Viewed 297 times

I have a very large table and I need to pivot it. Something similar to this: (How to transpose/pivot data in hive?) I need to go from here

 | ID   |   Code   |  cat     |   count | 
 | 1    |    A     |   p      |   e     | 
 | 2    |    B     |   q      |   f     |
 | 3    |    B     |   p      |   f     |
 | 3    |    B     |   q      |   h     |
 | 3    |    B     |   r      |   j     |
 | 3    |    C     |   t      |   k     |

to here:

 | ID   |   Code   |  p   |   q |  r  |   t |
 | 1    |    A     |   e  |     |     |     |
 | 2    |    B     |      |   f |     |     |
 | 3    |    B     |   f  |   h |  j  |     |
 | 3    |    C     |      |     |     |  k  |

But, the categories I have are several around (2000) and I don't want to write a separate select for each category. How could I do this?

edited Jul 08 '17 at 05:33

David דודו Markovitz

42,900
6
64
88

asked Jul 07 '17 at 21:03

Shehzaad

"separate select"? – David דודו Markovitz Jul 08 '17 at 05:35
Did you consider writing code? – Has QUIT--Anony-Mousse Jul 08 '17 at 16:48
You should share what you have tried already – Shubham Srivastava Jul 09 '17 at 06:50
@Anony-Mousse, I did. But I didnt write distributed code as I don't know how to. Code on a single machine which tried to pivot the csv took forever to run. – Shehzaad Jul 10 '17 at 17:28

Pivoting a large table in Hive

0 Answers0