HappyBase is a developer-friendly Python library to interact with Apache HBase. HappyBase is designed for use in standard HBase setups, and offers application developers a Pythonic API to interact with HBase. Below the surface, HappyBase uses the Python Thrift library to connect to HBase using its Thrift gateway, which is included in the standard HBase 0.9x releases.
Questions tagged [happybase]
95 questions
10
votes
1 answer
PySpark dataframe.foreach() with HappyBase connection pool returns 'TypeError: can't pickle thread.lock objects'
I have a PySpark job that updates some objects in HBase (Spark v1.6.0; happybase v0.9).
It sort-of works if I open/close an HBase connection for each row:
def process_row(row):
conn = happybase.Connection(host=[hbase_master])
# update HBase…
Alex Woolford
- 4,433
- 11
- 47
- 80
7
votes
2 answers
Spark can't pickle method_descriptor
I get this weird error message
15/01/26 13:05:12 INFO spark.SparkContext: Created broadcast 0 from wholeTextFiles at NativeMethodAccessorImpl.java:-2
Traceback (most recent call last):
File "/home/user/inverted-index.py", line 78, in
…
Theofilos Mouratidis
- 1,146
- 12
- 32
6
votes
1 answer
How to handle BigTable Scan InvalidChunk exceptions?
I am trying to scan BigTable data where some rows are 'dirty' - but this fails depending on the scan, causing (serialization?) InvalidChunk exceptions.
the code is as follows:
from google.cloud import bigtable
from google.cloud import…
Josh Reuben
- 289
- 2
- 8
4
votes
1 answer
unable to upload pdf files of size more than 10MB in Hbase via python happybase - HDP 3
We are using HDP 3. We are trying to insert PDF files in one of the columns of a particular column family in Hbase table. Developing environment is python 3.6 and the hbase connector is happybase 1.1.0.
We are unable to upload any PDF file greater…
areddy
- 373
- 3
- 7
- 18
4
votes
1 answer
Updating HBase data with HappyBase
I am trying to write a function to update Data from a table saved in HBase. I have a function that will get called to Update it and I have a pretty good start but I am a little bit lost on the end of finishing it. I can update single rows based one…
Travis
- 657
- 6
- 24
4
votes
2 answers
Connection to Hbase using python is failing
I am trying to connect to Hbase using python
sample code used
import happybase
connection = happybase.Connection(myhost,port, autoconnect=True)
# before first use:
connection.open()
print(connection.tables())
which is giving error as…
Anoop R
- 545
- 3
- 8
- 19
4
votes
3 answers
AWS Hbase: Invalid method name: 'getTableNames'
I am using Hbase 0.94.18 on AWS EMR. I am planning to use HappyBase as it looks very promising. Unfortunately I faced this issue on the very first attempt:
conn = happybase.Connection(port=9200,compat='0.94')
conn.tables()
Traceback (most recent…
Ayan Guha
- 750
- 3
- 10
4
votes
2 answers
Error while Happybase connection with hbase
>>>import happybase
>>>cnx=happybase.Connection('localhost')
Traceback (most recent call last):
File "", line 1, in
File "/home/seyf/envname/local/lib/python2.7/site-packages/happybase/api.py", line 121, in __init__
self.open()
File…
Seyf Tunisien
- 41
- 1
- 2
3
votes
1 answer
Parallel scan requests to HBase in Java and Python have different performance
Statement
We have 10 machines HBase cluster and billions of rows inside. Every row consists of one column family and ~20 columns. We need perform frequent scan requests which contains start row prefix and end row prefix. Usually every scan returns…
maxteneff
- 1,523
- 12
- 28
3
votes
1 answer
Decoding HappyBase data from HBase
While trying to decode the values from HBase, i am seeing an error but it is apparent that Python thinks it is not in UTF-8 format but the Java application that put the data into HBase encoded it in UTF-8 only
a =…
Bala Kumaran
- 31
- 2
3
votes
1 answer
ImportError: No module named happybase
I have a mapreduce program that connects to Hbase using happybase. I am getting the following error
File
/hadoop/yarn/local/usercache/aprakash/appcache/application_1407169690715_0025/container_1407169690715_0025_01_000002/./testhappybase.py, line…
user3919905
- 31
- 2
2
votes
1 answer
How can I scan over sets of rows in HappyBase with a single API call?
I want to scan a big-table for a list of IDs (or prefixes of IDs) (using Python HappyBase).
Is there any way to do it on server side? That is, I'd like to send a list of start/stop rows to be scanned in one API call rather than performing a long…
RELW
- 189
- 1
- 14
2
votes
0 answers
thriftpy.transport.TTransportException: TTransportException(type=1, message="Could not connect to ('localhost', 9090)")
I'm facing Thrift error while connecting to HBase through Python. Please find the error trace below-
File "/usr/local/lib/python3.6/dist-packages/happybase/pool.py", line 147, in connection
connection.open()
File "/usr/local/lib/python3.6/dist-…
Dinesh
- 1,135
- 2
- 15
- 22
2
votes
1 answer
ProtocolError: No protocol version header
I have hortonworks cluster with versions as below:
hbase = 1.1.2
happybase=1.1.0
hortonworks hadoop cluster = HDP-2.6.2.0
When I try to run simple example:
import happybase
server = "server-address"
connection = happybase.Connection(server)
print…
Piotr Sobolewski
- 2,024
- 4
- 28
- 42
2
votes
1 answer
Filter doesn't work when using happybase to scan an HBase table with Chinese character
I have a table in HBase where Chinese characters are stored in a certain column, say 'FLT:CREW_DEPT'. Now I need to filter out all the rows that 'FLT:CREW_DEPT' equals a certain value. When doing this in hbase shell, it works fine, shown as…
Roger Zhong
- 123
- 9