2

I am trying to start tomcat service in linux with elastic APM agent (java). But every time I start the service, getting these errors

SEVERE: ContainerBase.addChild: start:
    org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/mis-all-1.0.0]]
    at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:440)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1007)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:983)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
    at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1296)
    at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2037)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.VerifyError: (class: javax/servlet/GenericServlet, method: init signature: (Ljavax/servlet/ServletConfig;)V) invokedynamic bytecode is not supported in this class file version
    at java.lang.Class.getDeclaredFields0(Native Method)
    at java.lang.Class.privateGetDeclaredFields(Class.java:2509)
    at java.lang.Class.getDeclaredFields(Class.java:1819)
    at org.apache.catalina.util.Introspection.getDeclaredFields(Introspection.java:106)
    at org.apache.catalina.startup.WebAnnotationSet.loadFieldsAnnotation(WebAnnotationSet.java:270)
    at org.apache.catalina.startup.WebAnnotationSet.loadApplicationServletAnnotations(WebAnnotationSet.java:138)
    at org.apache.catalina.startup.WebAnnotationSet.loadApplicationAnnotations(WebAnnotationSet.java:70)
    at org.apache.catalina.startup.ContextConfig.applicationAnnotationsConfig(ContextConfig.java:417)
    at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:891)
    at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:388)
    at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5536)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
    ... 10 more
    
May 20, 2021 5:48:15 AM org.apache.catalina.startup.HostConfig deployDirectory
SEVERE: Error deploying web application directory []
java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/mis-all-1.0.0]]
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1011)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:983)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
    at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1296)
    at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2037)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
    at java.lang.Thread.run(Thread.java:745)

I am using below commands to set Environmental variables for APM agent

export CATALINA_OPTS="$CATALINA_OPTS -javaagent:/app/elastic-apm-agent-1.23.0.jar"
export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.service_name=Stage-pricing"
export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.application_packages=org,com,com.qr.jadu"
export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.server_url=http://ip:8200"
export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.global_labels='env=Stage'"

and logback-classic version 1.0.6

Please suggest a solution or any steps I am missing, thanks in advance.

Update by kriegaex, copied from the OP's comment:

  • Server version: Apache Tomcat/7.0.106 Server
    • built: Sep 16 2020 08:33:41 UTC
    • Server number: 7.0.106.0 OS
  • OS: Linux OS Version: 4.14.35-2047.502.4.1.el7uek.x86_64
  • Architecture: amd64
  • JVM Version: 1.7.0_301-b09
  • JVM Vendor: Oracle Corporation APM
  • Server version: 7.12.1
  • APM Agent language and version: java, 1.23. Also apm agent supports 7u60+, 8u40+, 9, 10, 11.
aksh_18
  • 21
  • 2
  • 1
    Either the `elastic-apm-agent` modifies the bytecode version of the `javax.servlet.GenericServlet` class (which Tomcat compiles with Java 8) or your application contains a prehistoric version of `servlet-api.jar` (it shouldn't contain any version). – Piotr P. Karwasz May 21 '21 at 11:46
  • Thanks @PiotrP.Karwasz for the reply, i have mentioned details below – aksh_18 May 24 '21 at 07:53

1 Answers1

2
Caused by: java.lang.VerifyError: (
  class: javax/servlet/GenericServlet,
  method: init
  signature: (Ljavax/servlet/ServletConfig;)V)
invokedynamic bytecode is not supported in this class file version

Piotr is right. Your Elastic APM Agent - whatever that thing does - transforms the byte code of void GenericServlet.init(ServletConfig) in such a way as to utilise invokedynamic byte code instructions, not checking or expecting that the code it weaves into is from an ancient Java version. Invokedynamic was introduced in Java 7, therefore your JAR must contain older versions. Either exclude it from transformation, if the agent can be configured that way, or upgrade the JAR, either using a more recent version or recompiling the API sources to a more recent (Java 7+) byte code version. Most agents probably rather expect Java 8+, but I am not sure about yours.

kriegaex
  • 63,017
  • 15
  • 111
  • 202
  • The strange thing is: 1. Tomcat's application classloader always looks for `javax.servlet.GenericServlet` in the parent classloader (cf. [source code](https://github.com/apache/tomcat/blob/f0627aa10346539995e4546275b58d02bd7323ca/java/org/apache/catalina/loader/WebappClassLoaderBase.java#L2582)), so an ancient version of `servlet-api.jar` bundled with the application is ignored, 2. The `servlet-api.jar` in Tomcat 9 (since the question is tagged `tomcat9`) uses bytecode version 52 (Java 8). – Piotr P. Karwasz May 22 '21 at 07:32
  • 1
    Chances are, the OP is using a much older Tomcat version. The logback-classic 1.0.6 library he uses is from 2012, for example. At that time, JDK 7 was the current one, but probably not everyone had updated at the time yet and there were lots of Java 6 libraries around. I gues, we have some kind of legacy application situation here, but the OP did not explain anything about it, so this is pure speculation. The question is clearly lacking detail. – kriegaex May 22 '21 at 09:33
  • Thanks @kriegaex for reply, below are the details about the version of tomcat etc. Server version: Apache Tomcat/7.0.106 Server built: Sep 16 2020 08:33:41 UTC Server number: 7.0.106.0 OS Name: Linux OS Version: 4.14.35-2047.502.4.1.el7uek.x86_64 Architecture: amd64 JVM Version: 1.7.0_301-b09 JVM Vendor: Oracle Corporation APM Server version: 7.12.1 APM Agent language and version: java, 1.23 Also apm agent supports 7u60+, 8u40+, 9, 10, 11 these versions of java – aksh_18 May 24 '21 at 08:01
  • 1
    That confirms kriegaex: Tomcat 7.0 was compiled with Java 6. You need to upgrade to Tomcat 8.5/9.0 or compile Tomcat 7.0 yourself with a more recent version of Java. – Piotr P. Karwasz May 24 '21 at 08:21
  • 1
    Such details belong in the question. For the purpose, there exists an "Edit" button. I edited your question this time for you, so you see how it is done. I also fixed your code blocks and other formatting, then added the details from your comment above. Now they are readable and not one long spaghetti string. – kriegaex May 24 '21 at 12:48
  • Thanks @kriegaex for the feedback, I will take care – aksh_18 May 25 '21 at 09:17
  • Thnaks @PiotrP.Karwasz : ) – aksh_18 May 25 '21 at 09:18
  • The way to thank people on StackOverflow is to accept and upvote their answers. – kriegaex May 26 '21 at 01:59