I am trying to setup hive in my local machine, using apache_hive_2.3.7 and apache_hadoop_3.3.6. When I try to run hive getting below error:
[sgopi@localhost v2.3.7]$ hive
/usr/bin/which: no hbase in (/home/sgopi/.local/bin:/home/sgopi/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin:/etc/hive/v2.3.7/bin:/etc/hadoop/v3.3.6/bin)
Logging initialized using configuration in jar:file:/etc/hive/v2.3.7/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.hadoop.hive.ql.plan.TableDesc.setProperties(TableDesc.java:133)
at org.apache.hadoop.hive.ql.plan.TableDesc.<init>(TableDesc.java:71)
at org.apache.hadoop.hive.ql.plan.PlanUtils.getDefaultTableDesc(PlanUtils.java:458)
at org.apache.hadoop.hive.ql.exec.Utilities.<clinit>(Utilities.java:681)
at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:709)
at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:654)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:553)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private volatile java.lang.String java.net.URI.string accessible: module java.base does not "opens java.net" to unnamed module @73ee04c8
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
at org.apache.hadoop.hive.common.StringInternUtils.<clinit>(StringInternUtils.java:57)
... 16 more
Tried using hive --service hiveserver2, but this comman keeps running indefinitely without any error/warnings.
[sgopi@localhost bin]$ hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console
/usr/bin/which: no hbase in (/home/sgopi/.local/bin:/home/sgopi/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin:/etc/hive/v2.3.7/bin:/etc/hadoop/v3.3.6/bin)
2023-08-30 11:19:31: Starting HiveServer2
When I check the running process, hiveserver2 is running with -Xmx256m argument. I have updated all the parameter values where the value was -Xmx256m to -Xmx1g. However the command is still using -Xmx256m. Is it taking time because of less memory? Is there a way to increase this to 1GB/2GB?
sgopi 15079 14307 3 11:19 pts/2 00:00:11 /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java -Dproc_jar -Djava.net.preferIPv4Stack=true -Dproc_hiveserver2 -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/etc/hive/v2.3.7/conf/parquet-logging.properties -Dyarn.log.dir=/etc/hadoop/v3.3.6/logs -Dyarn.log.file=hadoop.log -Dyarn.home.dir=/etc/hadoop/v3.3.6 -Dyarn.root.logger=INFO,console -Djava.library.path=/etc/hadoop/v3.3.6/lib/native **-Xmx256m** -Dhadoop.log.dir=/etc/hadoop/v3.3.6/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/etc/hadoop/v3.3.6 -Dhadoop.id.str=sgopi -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /etc/hive/v2.3.7/lib/hive-service-2.3.7.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console
I tried searching for -Xmx256m value in both hive, hadoop dirs but there is no parameter with this value.
[sgopi@localhost v3.3.6]$ pwd
/etc/hadoop/v3.3.6
[sgopi@localhost v3.3.6]$ grep -ir Xmx256 *
[sgopi@localhost v3.3.6]$ cd /etc/hive/v2.3.7/
[sgopi@localhost v2.3.7]$ grep -ir Xmx256 *
[sgopi@localhost v2.3.7]$
Hive metastore is already running, using mysql for hive metastore.
[sgopi@localhost v2.3.7]$ ps -ef|grep hive
sgopi 12083 1 0 10:10 ? 00:00:37 /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java -Dproc_jar -Dproc_metastore -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/etc/hive/v2.3.7/conf/parquet-logging.properties -Dyarn.log.dir=/etc/hadoop/v3.3.6/logs -Dyarn.log.file=hadoop.log -Dyarn.home.dir=/etc/hadoop/v3.3.6 -Dyarn.root.logger=INFO,console -Djava.library.path=/etc/hadoop/v3.3.6/lib/native -Xmx256m -Dhadoop.log.dir=/etc/hadoop/v3.3.6/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/etc/hadoop/v3.3.6 -Dhadoop.id.str=sgopi -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /etc/hive/v2.3.7/lib/hive-metastore-2.3.7.jar org.apache.hadoop.hive.metastore.HiveMetaStore
sgopi 15869 14307 0 11:40 pts/2 00:00:00 grep --color=auto hive
[sgopi@localhost v2.3.7]$ netstat -lntp|grep 12083
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp6 0 0 :::9083 :::* LISTEN 12083/java
[sgopi@localhost v2.3.7]$
***Edit-1: Updated Heap size from 256MB to 2GB, as highlighted Here, Still same issue.
