1

In my kubernetes application, I am running a java application from a readOnly root directory (ReadOnly root is a must). I do have a mount (folder) that is writable, however when I run:

jmap -dump:live,format=b,file=/dumps/temp/dump.hprof 1

I get the error:

Exception in thread "main" java.io.IOException: Read-only file system
    at java.base/java.io.UnixFileSystem.createFileExclusively(Native Method)
    at java.base/java.io.File.createNewFile(File.java:1035)
    at jdk.attach/sun.tools.attach.VirtualMachineImpl.createAttachFile(VirtualMachineImpl.java:298)
    at jdk.attach/sun.tools.attach.VirtualMachineImpl.<init>(VirtualMachineImpl.java:76)
    at jdk.attach/sun.tools.attach.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:58)
    at jdk.attach/com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:207)
    at jdk.jcmd/sun.tools.jmap.JMap.executeCommandForPid(JMap.java:128)
    at jdk.jcmd/sun.tools.jmap.JMap.dump(JMap.java:208)
    at jdk.jcmd/sun.tools.jmap.JMap.main(JMap.java:114)

I have run the java application with the following options: java -XX:MaxRAMPercentage=90.0 -XX:InitialRAMPercentage=15.0 Dlogback.configurationFile="${CUSTOM_LOGBACK_FILE:-/opt/custom-logback.xml}" -DconfigFile="${SETTINGS_FILE}" -jar "/${ARTIFACT}.jar

Dhruva
  • 11
  • 2
  • 1
    [This bug report](https://bugs.openjdk.org/browse/JDK-8230085) is about MacOS but seems to be the same issue, writable filesystems (or parts) within a read-only filesystem. So you could check whether the fix to this issue also solves your issue (by trying one of the fixed versions listed there). – Holger Jul 20 '22 at 10:09
  • cannot upgrade to java 14 at the moment. If I run the same java application with the same parameters from a directory that does have write access, the command does work. I am foxed as to why this would happen – Dhruva Jul 20 '22 at 11:32
  • 1
    There is a list of versions with backported fixes in the linked report, like 7u251 or 8u231. – Holger Jul 20 '22 at 11:34
  • I am using 11.0.15 which should have had the fix right? – Dhruva Jul 20 '22 at 11:55
  • 1
    I think so. I just noticed that the problem isn’t the heap dump file, as according to the stack trace, it didn’t even get so far. It’s failing at the attempt to attach already, most probably trying to create a file for the inter process communication. See [this answer](https://stackoverflow.com/a/45836660/2711488) for a description of the procedure. – Holger Jul 20 '22 at 14:44
  • In the same environment, if I run the application through a writable folder, I do not get this issue. But we want to run it through root, so as to avoid having to create a separate writable mount in other similar pods – Dhruva Jul 21 '22 at 06:26
  • 1
    By “run … through a writable folder”, do you mean the program location or the current directory or something else? – Holger Jul 21 '22 at 06:27
  • the directory from where we run the command for the jar. The jar may be located anywhere. Btw, I saw procedure description, trying to find out a way to create the /tmp/ folder it requires in a writable location – Dhruva Jul 21 '22 at 08:18
  • 1
    So does the actual current directory matter to the program logic? If not, you could simply use the knowingly writable directory `/dumps/temp/` instead of creating a new writable directory for it. – Holger Jul 21 '22 at 08:29
  • right, that is possible, but we will have to change the dockers for each pod. My thinking is that it should be possible for jcmd (or jmap) to use a writeable folder through some option. rather that relying on the location the jvm was started in. – Dhruva Jul 21 '22 at 09:30
  • 1
    Do I get this correctly, that no writable `/tmp` directory exists? So perhaps, jcmd and the JVM use the process’s current directory as alternative for the `.attach_pid…` file. That would explain why it works if the current directory is writable. It’s not possible to provide an option for jcmd to use an arbitrary location, because this protocol requires the JVM to check the particular location. So both sides must agree on the location. – Holger Jul 21 '22 at 16:07
  • right we must run it through some writable dir – Dhruva Jul 22 '22 at 17:28

0 Answers0