How to reliably set up post-compilation bytecode enhancement builder in Eclipse?

Question

I need to set up an Eclipse project with an additional builder that enhances the Java bytecode produced by an earlier builder (ideally Eclipse's own). I managed to get this builder to run and enhance the Eclipse Java builder output properly but seconds later Eclipse re-runs its Java builder and resets the bytecode back. It does not rerun my enhancement builder.

My setup

• Imported as a "Gradle project" into Eclipse 2019-12 (with Buildship).
• Added manually (and automated with Gradle) a custom Ant builder (that ends up calling Gradle) to enhance the code that Eclipse Java builder produces in bin/main, in place. This builder is set to run on Manual Build and Auto Build and not After a "Clean" or During a "Clean".
• By default the above ends up having three builders, top-to-bottom: 1. Gradle Project Builder, 2. Java Builder and 3. my bytecode enhancement builder (yes, it is listed last).

Alternatives I tried

1. Some combinations of setting my builder to run after/during a "Clean" as well without success. Not sure what exact events these relate to, really.
2. Had the builder refresh the project after ... and also not - did not help.

3. Try to remove the Java Builder using the following bit in Gradle script (didn't work - it comes back on its own):

   eclipse {
       project {
           file {
               whenMerged { projectFile ->
                   projectFile.buildCommands.removeAll { it.name == 'org.eclipse.jdt.core.javabuilder' }
               }
           }
       }
    }
   

4. Tried disabling the Java builder manually and have my bytecode enhancement builder also build the files itself (using Gradle). This stores the following file org.eclipse.jdt.core.javabuilder.launch file with the following content ... but upon restart the builder is re-enabled:

   <?xml version="1.0" encoding="UTF-8" standalone="no"?>
   <launchConfiguration type="org.eclipse.ant.AntBuilderLaunchConfigurationType">
   <booleanAttribute key="org.eclipse.ui.externaltools.ATTR_BUILDER_ENABLED" value="false"/>
   <stringAttribute key="org.eclipse.ui.externaltools.ATTR_DISABLED_BUILDER" value="org.eclipse.jdt.core.javabuilder"/>
   <mapAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS"/>
   <booleanAttribute key="org.eclipse.ui.externaltools.ATTR_TRIGGERS_CONFIGURED" value="true"/>
   </launchConfiguration>
   

5. I tried (and failed) to find if there is some workspace file (as opposed to project file) being changed (as well) to disable the Java builder.

Questions

1. What is the "proper" way to set up Eclipse for post-compilation bytecode enhancement?
2. What causes Eclipse to re-run previous builders without re-running mine?
3. Is there a way to fix (1)?
4. How to reliably disable the Java builder?

Can anyone help? Thanks!

UPDATE Additional details I added 12 builders and made them all append output to the same log file to research. The 12 extra builders are just informational - 4 before the Java Builder, 4 between the Java and the enhancement builder and 4 after the enhancement builder. Each of the 12 run in only one of the four conditions (hence 3x4). They are arranged as follows:

1. Gradle Project Builder
2. 1a-after-clean (runs only After a "Clean")
3. 1b-manual (runs only During manual builds)
4. 1c-auto (runs only During auto builds)
5. 1d-during-clean (runs only During a "Clean")
6. Java Builder
7. 2a-after-clean (runs only After a "Clean")
8. 2b-manual (runs only During manual builds)
9. 2c-auto (runs only During auto builds)
10. 2d-during-clean (runs only During a "Clean")
11. Bytecode Enhancement Builder
12. 3a-after-clean (runs only After a "Clean")
13. 3b-manual (runs only During manual builds)
14. 3c-auto (runs only During auto builds)
15. 3d-during-clean (runs only During a "Clean")

Each of the 12 informational builders writes time, its name and the size of a chosen test class. Unenhanced it is 46243 bytes long. When enhanced it becomes 53338 bytes long.

Here's the log after running "Clean" on this project alone ("Build automatically" is enabled):

20:19:19
1d-during-clean
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:10 Test.class

20:19:19
2d-during-clean
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:10 Test.class

20:19:20
1c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:10 Test.class

20:19:27
2c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class


Buildfile: /.../some-ant.xml

run-gradle:
        [echo] Running Gradle: --parallel :...:enhanceEclipseBytecode
        ...
        [java] > Task :...:enhanceBytecode
        [java] Enhanced class: ...Test in ...
        ...
        [java] Enhanced 205 classes.
        [java] > Task :...:enhanceEclipseBytecode
        [java] BUILD SUCCESSFUL in 15s
        [java] 2 actionable tasks: 2 executed
BUILD SUCCESSFUL
Total time: 15 seconds
20:19:44
1c-auto
-rw-r--r--  1 Learner  ...\...  53338  3 Mar 20:19 Test.class

20:19:46
1c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:46
2c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:46
3b-manual
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:46
3c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:46
3d-during-clean
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:57
1c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:57
2c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:57
3b-manual
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:57
3c-auto
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

20:19:57
3d-during-clean
-rw-r--r--  1 Learner  ...\...  46243  3 Mar 20:19 Test.class

UPDATE 2: Minimum example to reproduce

1. Create a folder - name it what you wish.

2. In that folder create build.grade file with the following content:

   buildscript {
       repositories {
           mavenCentral()
       }
   
       dependencies {
           classpath 'org.hibernate:hibernate-gradle-plugin:5.4.2.Final'
       }
   }
   
   plugins {
       id 'java'
       id 'eclipse'
   }
   
   apply plugin: 'org.hibernate.orm'
   
   repositories {
       mavenCentral()
   }
   
   dependencies {
       implementation 'org.hibernate.javax.persistence:hibernate-jpa-2.1-api:1.0.0.Final'
   }
   
   hibernate {
       sourceSets = [ project.sourceSets.main ]
       enhance {
           enableLazyInitialization = true;
           enableDirtyTracking = true;
           enableAssociationManagement = false;
           enableExtendedEnhancement = false;
       }
   }
   

3. Create a src/main/java/learner/TestEntity.java in there too as follows:

   package learner;
   
   import javax.persistence.*;
   
   @Entity
   public class TestEntity {
       @Id
       @Column(name = "id", nullable = false, updatable = false)
       private Long id = null;
   
       @Column(name = "name", columnDefinition = "TEXT")
       private String name = null;
   
       public Long getId() {
           return id;
       }
   
       public void setId(final Long id) {
           this.id = id;
       }
   
       public String getName() {
           return name;
       }
   
       public void setName(final String name) {
           this.name = name;
       }
   }
   

4. Execute gradle compileJava. Open the resulting build/classes/java/main/learner/TestEntity.class binary in an ASCII or hex viewer and observe stuff like $$_hibernate_write_name in there.

5. Import this project into Eclipse (say 2019-12) as a Gradle project and build it. Open the resulting bin/main/learner/TestEntity.class and observe none of that.

Answer 1

There were a few things I was unclear/wrong about and couldn't find relevant documentation to learn the details. Here's a summary of what needs to be known to get this right (some of which I got right from the get go, but not everything):

1. Gradle/Buildship integration in Eclipse attempts to make use of Eclipse's internal compiler. This is by Eclipse's design and has its own advantages during development ... as well as disadvantages during times like this - inability to leverage the external/production builders (Gradle model in this case) to do the building. For this reason any bytecode enhancement (plugins or not) operating inside Gradle will have no effect in Eclipse at all (unless they do something Eclipse-specific). (I got this right)
2. To perform bytecode enhancement in Eclipse (not apt or what some Eclipse plugin could do), one has to add custom builders and position them after the default Java builder (manually or automatically). (I got this right too)
3. Out of box Eclipse offers two kinds of builders - "Ant" and "Program". No Gradle there. "Program" kind is not cross-platform, only Ant is. Ant can be used to do things or launch Gradle in a cross-platform way (Java exec on Gradle's main() method). (I got this right too)
4. [I WAS WRONG HERE] Eclipse offers four "events" to which one can bind an Ant builder: After a "Clean", Manual Build, Auto Build and During a "Clean". I did not understand when do they run. For example, I thought that "During a Clean" runs while the clean is happening and that After a "Clean" is there to run after that, to allow the custom builder to do its own after-clean cleaning. I thought that this would be followed by Auto build if "Build automatically" is enabled or "Build immediately" is checked in the "Clean" dialog. This is NOT the case. After a "Clean" actually refers to a build step, not cleaning and will NOT be followed by the *Auto Build" step (that step only runs when one saves an edit and "Build Automatically" is enabled. In the builder's *.launch file this is much more apparent - the After a "Clean" is actually called full and Auto and Manual builds are called auto and incremental. This means that en enhancement builder has to be set to run on After a "Clean" in addition to Auto Build and Manual Build and should NOT be set to run on *During a "Clean". My mistake was to only set it to run for Auto and Manual builds.
5. [I WAS WRONG HERE TOO] I specified the working set of relevant resources in the Build Options tab for the enhancement builder. I set those resources to be the source code (to be enhanced) and build.gradle (containing the enhancer). Since these do not change on most builds, Eclipse chooses not to run the builder. The now obvious 20-20 vision truth is that relevant resources for this builder are the Java builder's output binaries (and build.gradle), not the Java source code. However, this isn' the entirely correct choice (in isolation) either as Eclipse, in our case, ends up in an infinite loop - it thinks that the enhancer changed the binaries and, as it is set to run when binaries changed, runs the build again. We cannot NOT set the relevant resources at all as that seems to mean "everything/anything". The enhancer must be made in such a way to NOT even touch the files that are already enhanced [UPDATE] and that isn't enough. Read on.

I still don't definitively know why the informational builders I used to research this have their output appended to the common log file in order that isn't chronological. I can only assume that this has to do with Eclipse's output buffering and periodic writing to these files somehow.

[UPDATE 1]

6. [I DIDN'T KNOW THIS] Eclipse has a workspace setting (checkbox, overridable per project) in Preferences -> Java -> Compiler -> Building -> Output Folder called "Rebuild class files modified by others" officially described as "Indicate whether class files which have been modified by others should be rebuilt to undo the modification.". By default it is unchecked/off, which seems right for this case. This, however, does not work as advertised. Whatever the setting is, Eclipse's Java Builder will react to its output changes (as if they were inputs, I call this a defect) and "undo the modification", causing infinite build loops. I found this symptom reported many times - search for this. With "correct" setup and no hacking the Java builder keeps undoing the modifications and Eclipse keeps re-running them, causing the infinite loop regardless of the setting.
7. [HACK THAT SEEMS TO WORK PRESENTLY] In addition to having everything above set up correctly I modified the enhancer to ensure two things (either only one or both may be required, not sure): (a) that existing *.class files are NOT deleted and recreated but rewritten and (b) that their last modified time is changed back to what it was before the enhancement. This seems to trick the Eclipse's modification detection enough to break out of the loop even though the file sizes are different. This is with Eclipse 2019-12 (4.14.0.v20191210-0610) and it may stop working with any update. I hope they fix the infinite build loop defect by then.