0

I have a multiple HBase tables. Each table is having Millions of records. What is the best or fastest way to count the number of records. Through below program i will get the count but i want some some fastest way to count the records.

def getTotalRecords(connection: Connection, tableName: String): Long = {
    val startTime = System.currentTimeMillis();
    val table = connection.getTable(TableName.valueOf(Bytes.toBytes(tableName)))
    var resultScanner: ResultScanner = table.getScanner(new Scan());
    var count: Long = 0;
    while (resultScanner.next() != null) {
      count = count + 1;
    }
    val endTime = System.currentTimeMillis();
    val timeDiff = endTime - startTime
    println(s"$tableName - Time taken $timeDiff")
    count
  }
Ranga Reddy
  • 2,936
  • 4
  • 29
  • 41
  • Have you looked at `RowCounter`? You can put the table names in a Parallel collection and return a row count on each into a `List` which you can then `.sum` up. See post about counting records in Hbase: https://stackoverflow.com/questions/11375098/hbase-quickly-count-number-of-rows – airudah Jul 27 '17 at 11:44
  • Possible duplicate of [Hbase quickly count number of rows](https://stackoverflow.com/questions/11375098/hbase-quickly-count-number-of-rows) – Alexander Kuznetsov Jul 27 '17 at 12:32
  • I don't think it's a complete duplicate @Alexander Kuznetsov. He's asking for the fastest way to do this in Java/Scala and Hbase. – airudah Jul 27 '17 at 13:20
  • Hi @AlexanderKuznetsov. It is not a duplicate a question because i want to write a scala/java program to get the count of table in hbase. I have written a program above mentioned code. But it is taking more time to count million or billion records. I asked is there any better way to get the count of table. – Ranga Reddy Jul 27 '17 at 13:58
  • you can run RowCounter from your java program several times for each table and sum the results. the fastest way to count is to use mapreduce and this is exactly what RowCounter perform under the hood. – AdamSkywalker Jul 28 '17 at 11:39

0 Answers0