6

I can run hive on Tez,But can not see any job in tez ui.

And it will drive me Crazy!

enter image description here

and the user and name are null in timelineserver enter image description here

the config is blow: tez-site.xml

<property>
      <name>tez.history.logging.service.class</name>
      <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value>
</property>
<property>
       <description>URL for where the Tez UI is hosted</description>
       <name>tez.tez-ui.history-url.base</name>
       <value>http://10.0.0.51:8080/tez-ui</value>
</property>

and yarn-site.xml

<property>
  <name>yarn.timeline-service.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.timeline-service.hostname</name>
  <value>0.0.0.0</value>
</property>
<property>
  <name>yarn.timeline-service.http-cross-origin.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.system-metrics-publisher.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.timeline-service.generic-application-history.enabled</name>
  <value>true</value>
</property>
<property>
   <name>yarn.timeline-service.webapp.address</name>
   <value>${yarn.timeline-service.hostname}:8188</value>
</property>
<property>
  <name>yarn.timeline-service.webapp.https.address</name>
  <value>${yarn.timeline-service.hostname}:2191</value>
</property>

And the url:

http://10.0.0.51:8188/ws/v1/timeline/TEZ_DAG_ID  
http://10.0.0.51:8188/ws/v1/timeline/TEZ_APPLICATION_ATTEMPT  
http://10.0.0.51:8188/ws/v1/timeline/TEZ_APPLICATION

All of them,I just get the same reponse below:

{
entities: [ ]
}
leocook
  • 191
  • 1
  • 13

2 Answers2

3

In my case found that it was YARN ACL problem. So the following helped me:

yarn.acl.enable = false
or
yarn.admin.acl = activity_analyzer,yarn,dr.who,admin

As a bonus, gathered full configuration for HDP 3.1 + TEZ 0.9.2:

YARN configuration:

yarn.timeline-service.enabled = true
yarn.acl.enable = false
yarn.admin.acl = activity_analyzer,yarn,dr.who,admin
yarn.timeline-service.webapp.address = <host>:8188
yarn.timeline-service.version = 2,0f
yarn.timeline-service.hostname = <host>
yarn.timeline-service.http-cross-origin.enabled = true
yarn.timeline-service.http-cross-origin.allowed-origins = *
yarn.resourcemanager.system-metrics-publisher.enabled = true
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes = org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl

TEZ configuration:

yarn.timeline-service.enabled = true
tez.tez-ui.history-url.base = http://<host>/tez-ui/
tez.am.tez-ui.history-url.template = __HISTORY_URL_BASE__?viewPath=/#/tez-app/__APPLICATION_ID__
tez.history.logging.service.class = org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService
tez.dag.history.logging.enabled = true
tez.am.history.logging.enabled = true
tez.allow.disabled.timeline-domains = true

Hive configuration:

hive_timeline_logging_enabled = true
hive.exec.pre.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook
hive.exec.post.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
hive.exec.failure.hooks = org.apache.hadoop.hive.ql.hooks.ATSHook

HDFS configuration:

hadoop.http.filter.initializers = org.apache.hadoop.security.HttpCrossOriginFilterInitializer
Pavel
  • 424
  • 3
  • 12
  • Hi @Pavel . I want to keep `yarn.acl.enable = true` but tez-ui failed to display. Any suggestion? Thanks – kholis Aug 10 '20 at 10:50
  • Thanks for including all the necessary config! This worked with ``yarn.acl.enable = true`` for me also. – runr Dec 07 '20 at 08:04
0

As far as I see in Timeline Server UI you do not have information about jobs. In Timeline and Tez UI, you can't see information about jobs that you ran before Timeline Server was enabled since Timeline Server uses LevelDB storage and should publish information during a job running.

Oleksandr Shevchenko
  • 1