0

I dump a Hive table as follows

hive -e 'select * from sometable' | sed 's/[\t]/,/g' > /tmp/us.csv

I drop a destination table and create it again based on a model table.

drop table blargh; create table blargh like modeltable;

modeltable is partitioned on a field called mkt_cd so now blargh is too.

I run a script against /tmp/us.csv and modify the timestamp field, adjusting it to be now() in the format YYYY-mm-DD HH:MM:SS and I write out a file new.csv. The old CSV /tmp/us.csv had a timestamp in this format which was old and we needed to refresh it.

Finally I try to load with the new CSV file:

hive> load data inpath '/path/to/new/new.csv' into table blargh partition(mkt_cd); FAILED: NullPointerException null

This error occurs even if I take the "head" (a few rows) of new.csv as my input. In the editor, it looks good and there are no empty lines. In addition, this error also occurs if I simply use the original CSV /tmp/us.csv file before I changed it. What could be a reason here?

Mark Ginsburg
  • 2,139
  • 4
  • 17
  • 31
  • Does this answer your question? [What is a NullPointerException, and how do I fix it?](https://stackoverflow.com/questions/218384/what-is-a-nullpointerexception-and-how-do-i-fix-it) – Ken White Feb 02 '21 at 01:06
  • No, that has to do with Java programming and in this case I am running some HiveQL in a JVM. The low-level Java error was certainly unexpected for me so at a higher level I am wondering what in a "load table from CSV" can cause it. – Mark Ginsburg Feb 02 '21 at 02:10
  • couple of things i can think of 1. since none row is getting loaded, is modeltable and sometable are of same structure, data type? 2. The timestamp should bein this format `yyyy-MM-dd HH:mm:ss`. pls not month is uppercase MM. 3. do you have permission to load into the DB? 4. you havent shared load into script so i am not sure if that has an y issue 5. you havent shared create table so not sure if that has an issue. – Koushik Roy Feb 02 '21 at 05:12

0 Answers0