7

after configuring hadoop I could run hdfs

then install hive and edit the conf file to make it run on tez by default, but running into some special issue when using hive directly:

hive

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tez/dag/api/SessionNotRunning
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:353)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Version Info: hadoop: 2.5 hive 0.13 tez 0.41

anyone met this before?

seems not like a PATH related error.

shengsgs
  • 91
  • 1
  • 1
  • 4

3 Answers3

8

My peoblem is：

Could not open connection to jdbc:hive2://localhost:10000: java.net.ConnectException: Connection refused (state=08S01,code=0)

Here is my solution and the progress to figure it out:

when run the below command to start hiveserver2

hive --service hiveserver2

the log tells the Excetion：

    Error starting HiveServer2 on attempt 1, will retry in 60000ms
java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration

but continue to look up we find that 

2018-11-16T18:45:14,836  INFO [main] server.HiveServer2: HS2 interactive HA not enabled. Starting tez sessions..
2018-11-16T18:45:14,836  INFO [main] server.HiveServer2: Starting/Reconnecting tez sessions..

so the reason is thad the default setting disable the HS2 interactive HA configuration. just chage to true to fix this in the hive-site.xml 

<property>
    <name>hive.server2.active.passive.ha.enable</name>
    <value>false</value> # change false to true
</property>

Problem solved !!!

part of the logs(releated to the Tez):

2018-11-16T18:45:14,835  INFO [main] server.HiveServer2: Web UI has started on port 10002

2018-11-16T18:45:14,836  INFO [main] server.HiveServer2: HS2 interactive HA not enabled. Starting tez sessions..
2018-11-16T18:45:14,836  INFO [main] server.HiveServer2: Starting/Reconnecting tez sessions..

2018-11-16T18:45:14,836  INFO [main] server.HiveServer2: Initializing tez session pool manager
2018-11-16T18:45:14,847  INFO [main] server.HiveServer2: Shutting down HiveServer2
2018-11-16T18:45:14,847  INFO [main] service.AbstractService: Service:ThriftBinaryCLIService is stopped.
2018-11-16T18:45:14,847  INFO [main] service.AbstractService: Service:OperationManager is stopped.
2018-11-16T18:45:14,848  INFO [main] service.AbstractService: Service:SessionManager is stopped.
2018-11-16T18:45:14,850  INFO [main] service.AbstractService: Service:CLIService is stopped.
2018-11-16T18:45:14,850  INFO [main] service.AbstractService: Service:HiveServer2 is stopped.
2018-11-16T18:45:14,847  INFO [main] thrift.ThriftCLIService: Thrift server has stopped
2018-11-16T18:45:14,866  INFO [main] server.HiveServer2: Stopping/Disconnecting tez sessions.
2018-11-16T18:45:14,866  INFO [main] server.HiveServer2: Stopped tez session pool manager.
2018-11-16T18:45:14,869  WARN [main] server.HiveServer2: Error starting HiveServer2 on attempt 1, will retry in 60000ms
java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession$AbstractTriggerValidator.startTriggerValidator(TezSessionPoolSession.java:74) ~[hive-exec-3.1.1.jar:3.1.1]
parsons jim
  • 141
  • 2
  • 3
2

I temporarily solved this by add hiveconf to force hive use mr engine not tez;

like this:

hive -hiveconf hive.execution.engine=mr -e "my sql"

but as I want to use tez, anyone could help?

shengsgs
  • 91
  • 1
  • 1
  • 4
0

For me I had to add the below dir to the classpath. This is mapr specific but other distro will have similar path

/opt/mapr/tez/tez-0.9/lib/*:/opt/mapr/tez/tez-0.9/*

MikeKulls
  • 873
  • 1
  • 10
  • 22