0

I am trying to insert a lot of rows into a table (my target is to insert a billion dummy rows). This is what I did:

  1. I wrote the billion SQL lines corresponding to the inserts into a file.
  2. ./hbase shell < insert.sql

First few inserts go through fine, but very soon I keep on running into this error

put 'test', 'row19115', 'cf:a', '19115'

ERROR: java.lang.OutOfMemoryError: unable to create new native thread

Here is some help for this command:
Put a cell 'value' at specified table/row/column and optionally
timestamp coordinates.  To put a cell value into table 't1' at
row 'r1' under column 'c1' marked with the time 'ts1', do:

  hbase> put 't1', 'r1', 'c1', 'value', ts1
  • How to fix this error / Is there a better way to do it?
  • Also, is there some other way to do the inserts rather than writing them to a file everytime?
Moeb
  • 10,527
  • 31
  • 84
  • 110
  • Another option is to prepare your data in a `HFile` format and then efficiently bulk insert into HBase. See: http://hbase.apache.org/book.html#arch.bulk.load . Also see section `12.9.2.4` at http://hbase.apache.org/book/trouble.rs.html whether it helps. – Lorand Bendig Nov 11 '12 at 20:54
  • You may want to have a look at this: http://stackoverflow.com/questions/8750764/what-is-the-fastest-way-to-bulk-load-data-into-hbase-programmatically – Cihan Keser Nov 12 '12 at 05:43

0 Answers0