0

I am trying to monitor GC information using VisualVM from local laptop and connecting to Neo4j (enterprise) application running on Oracle JDK 8 on remote EC2 instance running ubuntu 16. I am able to connect using JMX connection but the issue is I can't view anything on VisualGC (plugin). The message is "Not supported for this JVM".

Based on this link and link, I ran the jstatd command on the remote but that didn't solve the problem. Next, looked at this link, I ran jstatd as neo4j user as this is the user under which Neo4j service runs. If I run jstatd as ubuntu user, the below command works but doesn't show any data on VisualGC.

sudo -u neo4j jstatd -p 8888 -J-Djava.security.policy=/home/ubuntu/tools.policy -J-Djava.rmi.server.hostname=neo4j_ip_address

Running above command as neo4j user gave this error:

Could not create remote object access denied ("java.util.PropertyPermission" "java.rmi.server.ignoreSubClasses" "write") java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.rmi.server.ignoreSubClasses" "write")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
        at java.security.AccessController.checkPermission(AccessController.java:884)
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
        at java.lang.System.setProperty(System.java:792)
        at sun.tools.jstatd.Jstatd.main(Jstatd.java:139)

Are these the correct permissions for tools.policy file?

-rwxr-xr-x 1 neo4j  adm    112 Mar 21 15:41 tools.policy

Contents of tools.policy:

grant codebase "file:/usr/lib/jvm/java-8-oracle/lib/tools.jar" {
   permission java.security.AllPermission;
};

How do I get the VisualGC to work in this scenario? Are there any logs for VisualVM to know what's going on? Any other tools that I can use to visually see GC statistics? Any help is much appreciated. Thanks.

user0000
  • 369
  • 2
  • 7

1 Answers1

0

What happen if you try to run jstatd with full path?

sudo -u neo4j /usr/lib/jvm/java-8-oracle/bin/jstatd -p 8888 -J-Djava.security.policy=/home/ubuntu/tools.policy -J-Djava.rmi.server.hostname=neo4j_ip_address

Tomas Hurka
  • 6,723
  • 29
  • 38
  • No error this time. It doesn't not show anything on the terminal, seems like the service is running. I verified that by running sudo lsof -i -P -n and can see jstatd listening on port 8888. But I am still not able to view anything on VisualGC. Are there any additional steps? – user0000 Mar 22 '19 at 15:24
  • You did not mentioned your remote host configuration in VisualVM. Do you still use JMX connection? Did you added your remote host directly? Did you specify non-default `jstatd` port (8888) in remote host advanced configuration? – Tomas Hurka Mar 22 '19 at 18:30
  • On VisualVM, I have specified remote host and port as 8888. I also tried running jstatd on default port of 1099 and used port redirection on the remote (due to firewall restrictions) sudo iptables -t nat -A PREROUTING -p tcp --dport 8888 -j REDIRECT --to-port 1099 - not successful. apparently, the issue seems to be with the n/w connectivity between my local and ec2 instance, able to now view VisualGC. what's not clear to me is telnet on that port was successful but why would VisualGC show "jvm not supported" message ? Thanks! – user0000 Mar 22 '19 at 20:37
  • I guess that `jstatd` opens additional connections similarly to JMX. You can test `jvmstat` from command-line (without running VisualVM) via `jps remote_EC2_IP:8888`. If configured correctly, this command should return `jstatd` and `neo4j` processes running on remote EC2. See man page for `jps` for more details. – Tomas Hurka Mar 25 '19 at 14:25
  • I am accepting your reply as answer to show my appreciation for your time to provide suggestions. Thanks again! – user0000 Mar 25 '19 at 15:11