How can I modify a java.lang class on the fly?

Question

I'm looking for a way to add fields to an Thread on the fly by rewriting the byte code and reloading the class, not sure if it is at all possible. Any pointers welcome. I found some info on modifying and loading a class, and I know JRebel can seamlessly hot swap your code but not sure if the same approach/tools apply here.

The motivation here is exploring a theoretically better alternative to thread local objects. Should the method work I should be able to replace thread local with an annotation and the result should outperform current JDK implementation.

PS: Please save me the "root of all evil speech"

Clarifying use case:

Imagine I have a class with a ThreadLocal:


class A {
   ThreadLocal&ltCounter&gt counter;
   ...
   counter.get().inc()
}

I'd like to replace that with an annotation:


class A {
   @ThreadLocal
   Counter counter;
   ...
   counter.inc()
}

But instead of the above code getting generated I'd like to mutate Thread such that Thread will now have an Acounter field and the actual code will be:


class A {
   // Nothing here, field is now in Thread
   ...
   Thread.currentThread().Acounter.inc()
}

would you add what you currently have or worked on ..the code so we can better help you please ? — grepit, Jun 07 '13 at 14:15
Sorry, this is in the preliminary research phase, no code available at the moment. I'll edit the Q to clarify intention. — Nitsan Wakart, Jun 07 '13 at 14:41

score 11 · Accepted Answer · edited Nov 15 '16 at 09:09

At present it is impossible to redefine a class at runtime such that the redefinition will result in new methods or fields. This is due to the complexity involved in scanning the heap for all existing instances and transforming them + their references + potential Unsafe field offset base updaters (like AtomicFieldUpdater).

This limitation may be lifted as part of the JEP-159 but as discussed on the concurrency-interest mailing group this is a big impact change so may never happen at all.

Using Javaassist/similar will allow the transformation of a class to a new class with new methods/fields. This class can be loaded by a ClassLoader and used at runtime, but it's definition will not replace existing instances. So it will not be possible to use this method combined with an agent to redefine the class as an instrumentation redefinition is limited such that: "The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields ..." see here.

So for now, NO.

score 5 · Answer 2 · answered Jun 07 '13 at 15:02

5

If you would like to change the behaviour of "class" at runtime, you could try javassist. API is here

answered Jun 07 '13 at 15:02

Chris

5,584
9
40
58

I'm looking for a way to change classes that are already loaded and allocated. I can find enough documentation to support changing a class as it's loaded, but not how to replace a running Thread with my new version. – Nitsan Wakart Jun 07 '13 at 15:28

score 5 · Answer 3 · answered Jun 12 '13 at 12:20

I have seen custom class loading solution that dynamically reloaded JARs - you define one ClassLoader per JAR file and use it to load the classes from that JAR; to reload entire JAR you just "kill" its ClassLoader instance and create another one (after you replace the JAR file).

I don't think it's possible to tweak Java's internal Thread class this way because you don't have control over System ClassLoader. A possible solution is to have a CustomThreadWeaver class that would generate a new class extending Thread with the variables you need and use a custom DynamicWeavedThreadClassLoader to load them.

Good luck and show us your monster when you succeed ;-)

Possible · Answer 4 · 2015-10-14T15:59:40.823

Possible using instrumentation, and possibly libraries like javassist to modify code on fly. (however, adding and removing fields, methods or constructors are currently not possible)

//Modify code using javassist and call CtClass#toBytecode() or load bytecode from file
byte[] nevcode;
Class<?> clz = Class.forName("any.class.Example");
instrumentationInstace.redefineClasses(new ClassDefinition(clz, nevcode));

Do not forget to add Can-Redefine-Classes: true to your java agent's manifest.

Real example - optimizing java < 9 string.replace(CharSequence, CharSequence) using javassist:

String replace_src = 
    "{String str_obj = this;\n"
    + "char[] str = this.value;\n"
    + "String find_obj = $1.toString();\n"
    + "char[] find = find_obj.value;\n"
    + "String repl_obj = $2.toString();\n"
    + "char[] repl = repl_obj.value;\n"
    + "\n"
    + "if(str.length == 0 || find.length == 0 || find.length > str.length) {\n"
    + "    return str_obj;\n"
    + "}\n"
    + "int start = 0;\n"
    + "int end = str_obj.indexOf(find_obj, start);\n"
    + "if(end == -1) {\n"
    + "    return str_obj;\n"
    + "}\n"
    + "int inc = repl.length - find.length;\n"
    + "int inc2 = str.length / find.length / 512;\ninc2 = ((inc2 < 16) ? 16 : inc);\n"
    + "int sb_len = str.length + ((inc < 0) ? 0 : (inc * inc2));\n"
    + "StringBuilder sb = (sb_len < 0) ? new StringBuilder(str.length) : new StringBuilder(sb_len);\n"
    + "while(end != -1) {\n"
    + "    sb.append(str, start, end - start);\n"
    + "    sb.append(repl);\n"
    + "    start = end + find.length;\n"
    + "    end = str_obj.indexOf(find_obj, start);\n"
    + "}\n"
    + "if(start != str.length) {\n"
    + "    sb.append(str, start, str.length - start);\n"
    + "}\n"
    + "return sb.toString();\n"
    +"}";


ClassPool cp = new ClassPool(true);
CtClass clz = cp.get("java.lang.String");
CtClass charseq = cp.get("java.lang.CharSequence");

clz.getDeclaredMethod("replace", new CtClass[] {
        charseq, charseq
}).setBody(replace_src);

instrumentationInstance.redefineClasses(new ClassDefinition(Class.forName(clz.getName(), false, null), clz.toBytecode()));

score 2 · Answer 5 · edited May 23 '17 at 12:08

2

This seems to be a question of using the right tool for the job. A similar question has been asked here: Another Stack Overflow Question and the Javaassist byte code manipulation library was a possible solution.

But without further detail into the reasons why this is being attempted, it seems like the real answer is to use the right tool for the job. For example, with Groovy the ability to dynamically add methods to the language.

edited May 23 '17 at 12:08

Community

1
1

answered Jun 17 '13 at 18:48

droozen

176
4

Adding a new method is not like adding a new field, so that doesn't help. If you know how to add a new field on the fly using Javaassist please tell me. The motivation is an intellectual exercise, a thought/idea at this point. – Nitsan Wakart Jun 17 '13 at 20:45
Ah, I see. I haven't used Javaassist, myself, but I did find this tutorial that claims to be able to add a field: http://www.csg.is.titech.ac.jp/~chiba/javassist/tutorial/tutorial2.html#add – droozen Jun 17 '13 at 20:57

score 1 · Answer 6 · answered Jun 18 '13 at 05:17

1

You could try creating a JVM Agent that makes use of the java.lang.instrument API and more specifically make use of the retransform method that " facilitates the instrumentation of already loaded classes" and then make use of Javassist (or ASM) as mentioned to deal with the bytecode.

More info on the java.lang.instrument API

answered Jun 18 '13 at 05:17

Rafael Oltra

1,239
9
15

Sadly the docs claim that is not possible: "The redefinition may change method bodies, the constant pool and attributes. The redefinition __must not add, remove or rename fields__ ..." see [here](http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/Instrumentation.html#redefineClasses(java.lang.instrument.ClassDefinition...)) – Nitsan Wakart Jun 18 '13 at 06:45

score 0 · Answer 7 · answered Jun 18 '13 at 11:47

0

To do what you want, the simpler alternative would be to use a subclass of Thread, run it, and then inside that thread execute the code from your example (together with a cast of currentThread() to your subclass).

answered Jun 18 '13 at 11:47

kutschkem

7,826
3
21
56

note that this approach should also work with dynamically gerated subclasses created through, e.g. javassist. – kutschkem Jun 18 '13 at 11:48
That's not what I want though... I want to add fields on the fly. Not doing it on the fly is as you describe. – Nitsan Wakart Jun 19 '13 at 07:23
But for what you describe, you should use dynamic subclasses imho. What if you have multiple Threads, where one uses A and the second uses B. Do you really want to generate a field in the same Thread class for both? Also, the problem i see is that ThreadLocals are members of an instance. You should therefore generate one object per A instance, not one field for the A class (ok i assume the example was only there to demonstrate the problem) – kutschkem Jun 19 '13 at 07:54
well never mind, seeing that a ThreadLocal needs to be present in ALL threads, i guess you are right that you need to somehow smuggle your field in the Thread class. – kutschkem Jun 19 '13 at 07:56

score -2 · Answer 8 · answered Jun 13 '13 at 13:34

-2

What you are attempting to do is not possible.

Since you already know about ThreadLocal, you already know what the suggested solution is.

Alternatively, you can sub-class Thread and add your own fields; however, only those threads that you explicitly create of that class will have those fields, so you will still have to be able to "fall back" to using a thread local.

The real question is "why?", as in "why is a thread local insufficient for your requirements?"

answered Jun 13 '13 at 13:34

cpurdy

1,177
5
12

That's your real question, not mine :). Measure the difference in performance between having a field on Thread and using ThreadLocal, that's why. Why is it not possible? There are products out there, like JRebel which claim to be able to hot-swap your code at runtime, so why is it not possible to hot swap Thread? – Nitsan Wakart Jun 13 '13 at 20:21
But what was incorrect about the answer? BTW, JRebel works by changing the classes at load time, adding an additional field to any class that it needs to be able to modify. You could do that yourself by changing the Thread.class in the rt.jar, since you don't seem to like reasonable suggestions. – cpurdy Aug 09 '17 at 19:22

How can I modify a java.lang class on the fly?

8 Answers8