3

I am trying to connect to Apache Hive from a Perl script but I'm getting the following error:

Thrift::TException=HASH(0x122b9e0)

I am running with Hadoop version 2.7.0, Hive version 1.1.0, and Thrift::API::HiveClient version 0.003. Here is the script I am using:

#!/usr/bin/perl

use English;
use Thrift::API::HiveClient;

connecttoHive();

sub connecttoHive {
    my $client = Thrift::API::HiveClient->new( host => 'localhost', port => 10000 );

    $client->connect() or die "Failed to connect";

    $client -> execute('select count(1) from Koushik.emp2');
    my $result = $client -> fetchAll();
}

Could this be caused by a version issue or is it something else?

I also tried running the following script, which comes with the Thrift-API-HiveClient-0.003 distribution:

#!/usr/bin/env perl
use lib 'lib';
use Data::Dumper;
use Try::Tiny;
use Thrift::API::HiveClient;
#use Moose;

my ($host, $port) = (localhost => 10000);

try sub {
  my $client = Thrift::API::HiveClient->new( host => $host, port => $port );
  $client->connect;
  print "Connected\n";
  $client->execute(
    q{ create table if not exists t_foo (foo STRING, bar STRING) }
  );
  $client->execute('show tables');
  print Dumper $client->fetchAll;
  print Dumper $client->getClusterStatus;
  print Dumper $client->get_fields( 'default', 't_foo');
},
catch sub {
  print "ZOMG\n";
  print Dumper($_);
  exit 1;
};

I get the following output:

hduser@ubuntu:~/perl_script$ perl test-thrift.pl
Connected
ZOMG
$VAR1 = bless( {
                 'message' => 'Missing version identifier',
                 'code' => 0
               }, 'Thrift::TException' );

After enabling NOSASL authentication on my HiveServer2 by modifying the hive-site.xml, I am now getting a different error:

hduser@ubuntu:~/perl_script$ perl test-thrift.pl
Connected
ZOMG
$VAR1 = bless( {
                 'message' => 'Invalid method name: \'execute\'',
                 'code' => 1
               }, 'TApplicationException' );

It worked using Thrift::API::HiveClient2

hduser@ubuntu:~/perl_script$ cat test-thrift-client2.pl
#!/usr/bin/env perl
use lib 'lib';
use Data::Dumper;
use Try::Tiny;
use Thrift::API::HiveClient2;
#use Moose;

my ($host, $port) = (localhost => 10000);

try sub {
  my $client = Thrift::API::HiveClient2->new( host => $host, port => $port );
  $client->connect;
  print "Connected\n";
  $client->execute(
    q{ create table if not exists t_foo (foo STRING, bar STRING) }
  );
  $client->execute('show tables');
  print Dumper $client->fetch;
# print Dumper $client->getClusterStatus;
# print Dumper $client->get_fields( 'default', 't_foo');
},
catch sub {
  print "ZOMG\n";
  print Dumper($_);
  exit 1;
};

hduser@ubuntu:~/perl_script$ perl test-thrift-client2.pl
Connected
$VAR1 = [
          [
            'drv_cdr_mp'
          ],
          [
            'emp1'
          ],
          [
            'emp3'
          ],
          [
            'emp_1'
          ],
          [
            'emp_bucket'
          ],
          [
            'emp_incr_test'
          ],
          [
            'emp_rslt'
          ],
          [
            'log_detail'
          ],
          [
            't_foo'
          ],
          [
            'test1_emp1'
          ]
        ];
$VAR2 = '';
Koushik Chandra
  • 1,565
  • 12
  • 37
  • 73
  • Where does the error come from? From the `new`, `connect`, `execute`, or `fetchAll`? Also, can you dump the hash to see what's inside? – choroba Mar 04 '16 at 15:36
  • The error is coming from `execute`. Could you please let me know where from can I get hash error details. – Koushik Chandra Mar 04 '16 at 15:41

2 Answers2

0

Since you are using HiveServer2, try using Thrift::API::HiveClient2. If that doesn't work, you could put some debug prints into the client to see if there is anything going on vis-a-vis the network, e.g. timeouts.

JenVander
  • 325
  • 2
  • 11
  • Do you mean should I try with `Thrift::API::HiveClient2`? – Koushik Chandra Mar 25 '16 at 05:31
  • Yes give that a try -- I'll update the answer, I realize I didn't word it very well :) – JenVander Mar 25 '16 at 14:08
  • It worked using `Thrift::API::HiveClient2`(I have updated the original post with the changed code and result) but why it is not working using `Thrift::API::HiveClient`? – Koushik Chandra Mar 28 '16 at 04:23
  • I thought it might have to do with SASL support, however I have recently run into someone that claims to have turned off SASL in their server and had a similar problem executing (auth worked but not exec). My next guess is that the API changed, creating a rev-lock for client/server versions. – JenVander Mar 28 '16 at 14:12
0

Thrift::API::HiveServer is for HiveServer, which has been deprecated for a long time. All reasonably recent Hadoop distros now feature HiveServer2, for which you need Thrift::API::HiveServer2. Also, check the get_tables and get_columns methods, that'll give you better information than what you tried in your example.

David Morel
  • 1
  • 3
  • Suggestions & pointers will likely suitable as comment but not full answer. Add few more description about the methods you have mentioned to make it a meaningful answer. - [From Review](http://stackoverflow.com/review/first-posts/12355414) – Raju May 15 '16 at 00:12