2

I want to connect Python to hive using pyhive. I'm using the below python script to be executed on my local.

#!/usr/bin/env python
# coding: utf-8
from pyhive import hive
from TCLIService.ttypes import TOperationState
def mysql_connect(host, port, username):
 conn = hive.Connection(host=host, port=port, username=username)
 return conn.cursor()

cursor = mysql_connect("localhost", 50070, "hduser")
cursor.execute("show databases")
print_log(cursor)

I have pyhive present to the location /usr/local/lib/python2.7/dist-packages but I end up in the following output

vaibhav@vaibhav-Lenovo-G570:~/Desktop/Python/Automation$ ./pyhive_test.py
Traceback (most recent call last):
  File "./pyhive_test.py", line 9, in <module>
    cursor = mysql_connect("localhost", 50070, "hduser")
  File "./pyhive_test.py", line 6, in mysql_connect
    conn = hive.Connection(host=host, port=port, username=username)
  File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 131, in __init__
    self._transport.open()
  File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/__init__.py", line 80, in open
    status, payload = self._recv_sasl_message()
  File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/__init__.py", line 101, in _recv_sasl_message
    payload = read_all_compat(self._trans, length)
  File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/six.py", line 31, in <lambda>
    read_all_compat = lambda trans, sz: trans.readAll(sz)
  File "/home/vaibhav/.local/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 60, in readAll
    chunk = self.read(sz - have)
  File "/home/vaibhav/.local/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 132, in read
    message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

Edits 1. Name of the files was changed from Pyhive to pyhive_test

  1. pyhive.py was deleted from the directory

Possible solution tried: 1.There were installed two versions of python2.7 and python 3.4. I uninstalled Python3.4 but still the folder seems to be present in /usr/local/lib/. I ran some of the commands below to check where my python is installed and packages available in PYTHONPATH

vaibhav@vaibhav-Lenovo-G570:~$ which -a python
/usr/bin/python
vaibhav@vaibhav-Lenovo-G570:~$ python -c "import sys, pprint; pprint.pprint(sys.path)"
['',
 '/home/vaibhav',
 '/usr/lib/python2.7/dist-packages',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-x86_64-linux-gnu',
 '/usr/lib/python2.7/lib-tk',
 '/usr/lib/python2.7/lib-old',
 '/usr/lib/python2.7/lib-dynload',
 '/home/vaibhav/.local/lib/python2.7/site-packages',
 '/usr/local/lib/python2.7/dist-packages',
 '/usr/lib/python2.7/dist-packages/PILcompat',
 '/usr/lib/python2.7/dist-packages/gtk-2.0',
 '/usr/lib/pymodules/python2.7',
 '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

2. Got the reference from the link mentioned here they mentioned to use it in virtual enviroment or use clean anaconda. Not used any of them how and don't know how it will affect already existing configuration .

3.I installed Pyhive using sudo, so i changed permission following this link but still got the same problem.

user7422128
  • 902
  • 4
  • 17
  • 41

1 Answers1

0

The file you're starting is named pyhive.py.

When you do

from pyhive import hive

inside your pyhive.py, then it will try to import hive from your module, not from the pyhive library.

Please name the file you're starting something else and avoid it to use names of existing modules/libraries.

From the docs:

When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

  • the directory containing the input script (or the current directory).
  • PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
  • the installation-dependent default.
Community
  • 1
  • 1
Mike Scotty
  • 10,530
  • 5
  • 38
  • 50
  • I changed it to pyhive_test, still getting the same error – user7422128 Jun 25 '17 at 09:33
  • Could you please update your question and add the new error message? Did you make sure there's no longer a file ``pyhive.py`` in the same directory as ``pyhive_test.py``? – Mike Scotty Jun 25 '17 at 09:38
  • I see you have updated the question, but you clearly still have a file ``pyhive.py`` in the same directory. Please remove it. – Mike Scotty Jun 25 '17 at 09:52
  • @codaholic Seems your original question "Python Pyhive module cannot import name hive" is now answered, while you now got a different problem with ``hive.Connection``. I suggest you create a new question if you need assistance with the new problem and accept my answer if it was helpful in solving the original problem. – Mike Scotty Jun 25 '17 at 10:18
  • Accepted. I will post a new question if you know about the new issue please post your answer there as well – user7422128 Jun 25 '17 at 10:43