2

I am creating several ByteBuddy classes (using DynamicTypeBuilder) and loading them. The creation of these classes and the loading of them happens on a single thread (the main thread; I do not spawn any threads myself nor do I submit anything to an ExecutorService) in a relatively simple sequence.

I have noticed that running this in a unit test several times in a row yields different results. Sometimes the classes are created and loaded fine. Other times I get errors from the generated bytecode when it is subsequently used (often in the general area of where I am using withArgumentArrayElements, if it matters; ArrayIndexOutOfBoundsErrors and the like; again other times this all works fine (with the same inputs)).

This feels like a race condition, but as I said I'm not spawning any threads. Since I am not using threads, only ByteBuddy (or the JDK) could be. I am not sure where that would be. Is there a ByteBuddy synchronization mechanism I should be using when creating and loading classes with DynamicTypeBuilder.make() and getLoaded()? Maybe some kind of class resolution is happening (or not happening!) on a background thread or something at make() time, and I am accidentally somehow preventing it from completing? Maybe if I'm going to use these classes immediately (I am) I need to supply a different TypeResolutionStrategy? I am baffled, as should be clear, and cannot figure out why a single-threaded program with the same inputs should produce generated classes that behave differently from run to run.

My pattern for loading these classes is:

  1. Try to load the (normally non-existent) class using Class#forName(name, true, Thread.currentThread().getContextClassLoader()).
  2. If (when) that fails, create the ByteBuddy-generated class and load it using the usual ByteBuddy recipes.
  3. If that fails, it would be only because some other thread might have created the class already. In this unit test, there is no other thread. In any case, if a failure were to occur here, I repeat step 1 and then throw an exception if the load fails.

Are there any ByteBuddy-specific steps I should be taking in addition or instead of these?

Laird Nelson
  • 15,321
  • 19
  • 73
  • 127
  • Interesting question and good explanation, even though a reproducing [MCVE](https://stackoverflow.com/help/mcve) would be even better. As you were mentioning unit tests, just let me double-check if maybe you run several unit tests in parallel in the same JVM, maybe via Maven Surefire or so. Or if other test methods or classes are being run in the same JVM before your test runs. It might be sensitive to execution order then because it is about class-loading. – kriegaex Sep 09 '20 at 02:31
  • 1
    A good thought and I've certainly made that mistake many years ago. But not this time. I'm not often completely stumped but this issue certainly has me questioning things! – Laird Nelson Sep 09 '20 at 02:55
  • Looks like a case for Rafael. If you could give him something to work with, probably he could find the root cause quickly. – kriegaex Sep 09 '20 at 03:01
  • One thing I'm going to look into is the semantics of the https://javadoc.io/static/net.bytebuddy/byte-buddy/1.10.14/net/bytebuddy/implementation/MethodCall.html#withArgumentArrayElements-int-int-int- method (where I already found a bug in 1.10.14). Guessing something is up when the array size is 0, but how this would result in an NPE with one run and an `ArrayIndexOutOfBoundsException` with another is beyond me currently. – Laird Nelson Sep 09 '20 at 03:07

2 Answers2

0

Phew! I think I can chalk this up to a bug in my code (thank goodness). Briefly, what looked like concurrency issues was (most likely) an issue with accidentally shared classnames and HashMap iteration order: when one particular subclass was created-and-then-loaded, the other would simply be loaded (not created) and vice versa. The net effect was effects that looked like those of a race condition.

Laird Nelson
  • 15,321
  • 19
  • 73
  • 127
0

Byte Buddy is fully thread-safe. But it does attempt to create a class every time you invoke load what is a fairly expensive operation. To avoid this, Byte Buddy offers the TypeCache mechanism that allows you to implement an efficient cache.

Note that libraries like cglib offer automatic caching. Byte Buddy does not do this since the cache uses all inputs as keys and references them statically what can easily create memory leaks. Also, the keys are rather inefficient which is why Byte Buddy chose this approach.

Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192
  • Thanks for this information. Once a class is loaded, what are the use cases for caching it? As you well know, `ClassLoader#loadClass()` caches loads. What use cases did you have in mind for `TypeCache`? – Laird Nelson Sep 12 '20 at 14:04
  • If you want to create an on-demand proxy, for instance. You generate the class for each proxy base when requested for the first time and return the previous value on every subsequent attempt. – Rafael Winterhalter Sep 12 '20 at 14:20
  • But won't that on-demand class get loaded by a classloader, and hence never re-generated? I mean: if the caller first tries `loadClass(proxyClass)` and it fails b/c the class doesn't exist, then you generate it with ByteBuddy, then `loadClass(proxyClass)` again—why is another cache needed here? Subsequent `loadClass` calls will not regenerate the proxy. (I feel like I'm missing something.) – Laird Nelson Sep 12 '20 at 19:27
  • By default, Byte Buddy generates random names such that this behavior would not be implicit. And even with identical names, Byte Buddy generates a new class loader each time. Only if you use the injection strategy, allow preexisting classes and define the same name each time this would be the outcome. – Rafael Winterhalter Sep 13 '20 at 17:53
  • Aha! Right; it was the random name part I was forgetting (in my case I use deterministic names and the injection strategy). Thanks! – Laird Nelson Sep 13 '20 at 18:42