12

I am looking towards some approach where by using Java agent or instrumenting classes (preferably something at lower level than user classes) to intercept all object creation in JVM (new or any alternative ways to create Object), There is a similar question which doesn't focus on Java agent or something lower than instrumenting user classes

jmj
  • 237,923
  • 42
  • 401
  • 438
  • You should be able to [do this with AspectJ](http://stackoverflow.com/questions/12184391/how-to-define-an-aspectj-pointcut-that-picks-out-all-constructors-of-a-class-tha). – Mick Mnemonic Mar 10 '16 at 19:08
  • won't it [just](https://stackoverflow.com/questions/95419/what-are-all-the-different-ways-to-create-an-object-in-java) cover `new` cases ? – jmj Mar 10 '16 at 19:11
  • If you'd add pointcuts to `clone()` and `ObjectInputStream.readObject()`, then shouldn't you have everything covered? [Here's](http://stackoverflow.com/questions/17338788/aspectj-pointcut-on-constructor-object) one more related question. – Mick Mnemonic Mar 10 '16 at 19:32
  • What do you want to do which a memory profiler doesn't do? – Peter Lawrey Mar 11 '16 at 07:54
  • For reference: I created POC here https://github.com/JigarJoshi/jvm-objects-inspector – jmj May 25 '16 at 06:58

2 Answers2

18

Java Objects can be created in several different ways.

  1. From Java code, when a Java method, either interpreted or compiled, executes one of the following bytecode instructions: new, newarray, anewarray, multianewarray.
  2. From native code, when native methods, including those in standard class library, call one of JNI functions: NewObject, NewObjectArray, NewStringUTF, NewDirectByteBuffer, etc.
  3. Directly from VM runtime, when a new object is created internally by JVM, for example, in response to Object.clone(), Throwable.getStackTrace(), Class.getInterfaces(), etc.

Unfortunately, there is no single point where you can collect objects from all these sources. However, there are means for intercepting all of them.

  1. Objects instantiated from Java can be caught by an Instrumentation agent. The agent needs to define a ClassFileTransformer that will scan the bytecode of all loaded classes for object-creating instructions and modify it.

    Note: there is no need to intercept all new instructions, you can instrument Object() constructor instead. But you still need to intercept array allocation instructions.

  2. JNI functions can be intercepted by JVMTI agent. You need to define your own native hooks for NewObjectArray, NewStringUTF etc. and then replace JNI function table. See JVMTI Reference for the details.

  3. Objects created by the VM can be caught by JVMTI Event Callback mechanism. The desired event is VMObjectAlloc.

    Note: JVM will not post VMObjectAlloc event for objects allocated from Java or by JNI functions.

All other ways of object instantiation (cloning, reflection, deserialization) fall into one of the above categories.

Get JDK 8 Demos and Samples from Oracle Java SE Downloads website.
There is a sample JVMTI agent for exactly this question.

Look under

  • jvmti/heapTracker
  • jvmti/hprof
Community
  • 1
  • 1
apangin
  • 92,924
  • 10
  • 193
  • 247
  • Thanks Apangin, for option# 1 It [doesn't intercept](https://gist.github.com/JigarJoshi/ba76fbd31050aa655404) `java.lang.Object` for custom class's construction – jmj Mar 11 '16 at 07:22
  • @JigarJoshi I'm not sure I understand the issue. Have you specified `canRetransform=true` for your transformer? – apangin Mar 11 '16 at 08:50
  • By posting that output I meant that when I ran with that agent plugged in, it intercepts all class loading, not the `java.lang.Object` so may be it isn't plugged into class loader which is responsible to load it, and Yes, `inst.addTransformer(new InstrumentClass(), true);` and it has `Can-Retransform-Classes: true` set in MANIFEST.MF as well – jmj Mar 12 '16 at 06:00
  • 1
    @JigarJoshi Core system classes like `Object`, `String`, `Class` etc. are obviously loaded before any Java agent starts. In order to appy transformations to these classes, you need to call [`retransformClasses`](http://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses-java.lang.Class...-). – apangin Mar 12 '16 at 12:31
  • 1
    @JigarJoshi Alternatively you may use JVMTI-level instrumentation. If you install [`ClassFileLoadHook`](http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#ClassFileLoadHook) in [`Agent_OnLoad`](http://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#onload), it will be called for all classes, starting from `java.lang.Object`. – apangin Mar 12 '16 at 12:44
  • Thanks @apangin, this really helped – jmj Mar 15 '16 at 06:54
  • Could you share how could I trace array initialization ? – jmj Mar 17 '16 at 22:06
  • also `hprof` is able to track user class's initialization, I will review the source but if you know is it purely native agent based implementation afaik, how is it done (only if you know offhand) – jmj Mar 17 '16 at 22:07
  • 1
    @JigarJoshi It uses JVMTI `ClassFileLoadHook` event to instrument *all* loaded classes. Each `newarray`, `anewarray` and `multianewarray` bytecode is supplemented with `invokestatic` to the instrumentation method. – apangin Mar 17 '16 at 22:51
  • 1
    For reference I created POC here https://github.com/JigarJoshi/jvm-objects-inspector – jmj May 25 '16 at 06:58
2

You can take a look at this opensource java agent created by devexperts team https://github.com/Devexperts/aprof It provides nice reports to detect where memory is allocated. But, as i know, it doesn't intercept new objects created via JNI or sun.misc.Unsafe.allocateInstance in current version

It is pure java agent which manipulates bytecode with ASM. Before each object allocation aprof inserts method call which traks allocation size and location stack (where this allocation occurs)

Alex K
  • 56
  • 3
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/low-quality-posts/12481942) – Martin Frank May 26 '16 at 12:03
  • I wanted use comments for this link but a can't because of reputation... Now i described shortly how it works, is that enough? – Alex K May 26 '16 at 13:26
  • Thanks! - do you have some numbers around performance implication due to agent ? – jmj May 26 '16 at 21:10
  • I don't have my own benchmarks, but, of course we cannot measure a system without affecting it performance. What they offer: 1) "It has no performance impact on dense garbage-free computation code, various getters, etc" 2) "Long-running methods retrieve LocationStack from ThreadLocal only once" 3) "HotSpot optimizes this ugly code quite well" 4) "Does not allocate memory during stable operation" Presentation: http://www.slideshare.net/elizarov/aprof-jocker-2014 – Alex K May 27 '16 at 09:28