3

I am trying to hive-import a mysql table using sqoop which has a column that stores json data containing new line chars. For handling hive default delimiters, I tried to change the line delimiter using sqoop's --lines-terminated-by '\u002' option. Sqoop gave error saying --lines-terminated-by takes only '\n'. --hive-drop-import-delim option allows to import data by dropping new line characters. Is there a way to import data intact into hive as text file?

learner
  • 31
  • 4
  • Can you try to import data using Cntrl+A as delimiter [ --fields-terminated-by '\001' ] and skipping the --lines-terminated-by and load into Hive table – Abhi Mar 22 '16 at 16:00
  • Abhi, Yes. I tried with --fields-terminated-by option. Problem is rows gets terminated with new line character. Since the column data is having new lines, I am getting more records in hive as compared to mysql. I want to import data intact without loosing new line chars in the column data.. – learner Mar 22 '16 at 16:12
  • Why not to use parquet or avro instead of text file? – fi11er Mar 22 '16 at 18:10
  • Currently we have setup sqoop-1.4.5 version and also we have some requirement to store it as text file. – learner Mar 22 '16 at 18:33
  • Did anyone get a solution for this? – Sachin Aug 21 '20 at 06:11

0 Answers0