14

For my team, I'd like to configure maven/eclipse build to automatically generate Java code from *.proto files (in a project that uses gRPC). Currently one needs to run mvn generate-source or mvn protobuf:compile (as in plugin usage page). Or what is the same add Run configuration to invoke maven goal compile.

Whenever Eclipse Maven project is refreshed (Alt+F5) or IDE is restarted, project is rebuilt but without what should appear in target/generated, thus turning project into red. So one need to generate and refresh project (F5). UPDATE Eclipse has needed source folders configured in .clathpath file.

As I know that should be m2e connector, but I could only find one https://github.com/masterzen/m2e-protoc-connector for the oldest Googles plugin com.google.protobuf.tools:maven-protoc-plugin, that is even not mentioned currently at https://github.com/grpc/grpc-java

We use exactly referenced/recommended

  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>

that is:

<build>
  <extensions>
    <extension>
      <groupId>kr.motd.maven</groupId>
      <artifactId>os-maven-plugin</artifactId>
      <version>1.4.1.Final</version>
    </extension>
  </extensions>
  <plugins>
    <plugin>
      <groupId>org.xolstice.maven.plugins</groupId>
      <artifactId>protobuf-maven-plugin</artifactId>
      <version>0.5.0</version>
      <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.1.0:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.1:exe:${os.detected.classifier}</pluginArtifact>
      </configuration>
      <executions>
        <execution>
          <goals>
            <goal>compile</goal>
            <goal>compile-custom</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Related:

Community
  • 1
  • 1
Paul Verest
  • 60,022
  • 51
  • 208
  • 332
  • There should be Java sources in `target/generated-sources`, but it is empty on clean / restart. – Paul Verest Nov 04 '16 at 15:53
  • That is exactly what `protoc` Protocol Buffers compiler does: it generates java from .proto files – Paul Verest Nov 04 '16 at 15:57
  • @Gimby I would love to use only Java http://stackoverflow.com/questions/40425211/java-idl-for-protobuf-protocl-buffers-in-java – Paul Verest Nov 04 '16 at 16:05
  • @Gimby Generated sources are exactly supposed to end up inside `target`, the build directory. The reason is that they are temporary files used to build and compile the project, but they are not sources in the Maven term, because they are generated from another source. Typically, generated sources should not be versioned-control; what you want on CVS is the actual source, not a generated product from the source. (The same way as you wouldn't version-control the built JAR). – Tunaki Nov 04 '16 at 16:07
  • 1
    I find it strange though that your Eclipse removes the `target/generated` folder. It shouldn't do that, even on hard refresh. Once the folder is added to the build path (this has to be done manually -- if not done by the `buildhelper-maven-plugin`), Eclipse should always keep it. Which version of Eclipse are you using? Maybe it could use an upgrade, along with m2e. – Tunaki Nov 04 '16 at 16:11
  • 1
    Can you post your current POM? Also, can you answer [the previous comment](http://stackoverflow.com/questions/40426366/automatically-generate-java-from-proto-with-maven-m2e-in-eclipse-ide#comment68102924_40426366) I left? I see in the source code of the plugin that it [correctly adds the generated files](https://github.com/xolstice/protobuf-maven-plugin/blob/master/src/main/java/org/xolstice/maven/plugin/protobuf/AbstractProtocCompileMojo.java#L72) to the maven project, so in Eclipse, once you add it as source folder, and don't `clean` without re-compiling, it should still work. – Tunaki Nov 08 '16 at 16:02
  • pom.xml part is exactly as on https://github.com/grpc/grpc-java Well, `target/generated` is not removed, but its content is empty. – Paul Verest Nov 08 '16 at 16:21
  • Your title asks to automatically generate but the question says the problem is that the folder with the generated sources is disappearing. Once you refresh the project and the folder comes back do the generated `.java` files reflect changes in the `.proto` files without refreshing? I ask because I have a solution that doesn't remove the generated sources folder when you open Eclipse but the `.java` files don't automatically reflect changes in the `.proto` files, so it answers what I think the question asks but not what the title asks. – Captain Man Nov 08 '16 at 16:56
  • At this point I thought I said that "Eclipse removes `target/generated` folders, but it was not me, but @Tunaki in comments above. Eclipse still has source folders for `target/generated` but they are empty, after [re]start(i.e. workspace build) or after Maven project is refreshed (Alt+F5). Anyways it is not important if folders are there, but whether they have generated java sources. Please add any solution you have as answer. – Paul Verest Nov 09 '16 at 11:10

3 Answers3

11

Instead of using org.xolstice.maven.plugins:protobuf-maven-plugin my team has used com.github.os72:protoc-jar-maven-plugin to generate the message classes. I believe they are the same since under the hood they all seem to be using the tools from Google.

I am not using any m2e connectors for this plugin (Edit: protoc-jar-maven-plugin's m2e connector is bundled with it so no extra installation is needed, which is why it seemed like I wasn't using one, but technically I was, but this doesn't really matter). Unfortunately the changes in the .proto file are not "automatically" propagated to the generated .java files, you need to manually run Maven or trigger the project to be built in Eclipse (instructions below), but fortunately the target/generated-sources file is not vanishing or emptying or anything strange like what you describe.

If you want to rebuild the .java files from the .proto classes without using mvn clean compile from the command line you can clean the Eclipse project . Project → Clean... → select your project → Select build option (only shows if you have "Build Automatically" from the Project menu is unchecked).

I was able to do this in the latest Eclipse Neon (it will probably work in later ones too, but I don't know for certain).

Below is the POM I am using. I don't think it requires any special explanation, my solution is to simply use a different plugin than the one you are using. (If some explanation is needed I'll be happy to provide it though.)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>io.github.jacksonbailey</groupId>
    <artifactId>protobuf-m2e-sample</artifactId>
    <version>0.1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.1.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>com.github.os72</groupId>
                <artifactId>protoc-jar-maven-plugin</artifactId>
                <version>3.1.0.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <protocVersion>3.1.0</protocVersion>
                            <inputDirectories>
                                <include>src/main/resources</include>
                            </inputDirectories>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
Captain Man
  • 6,997
  • 6
  • 48
  • 74
  • Voted up. Thanks. However this does not answer "how to Automatically" – Paul Verest Nov 10 '16 at 08:30
  • If you do `mvn clean` and then start Eclipse, will it get generated Java file during initial workspace build? The goal is that new developers can get code into Eclipse, and be able to run it. – Paul Verest Nov 10 '16 at 08:32
  • @PaulVerest I told you it was only a partial answer but you told me to answer anyways :) -- As for the second comment, I thought it wouldn't but it actually does. I am updating the answer (it's still not as "automatic" as you probably would like) – Captain Man Nov 10 '16 at 16:46
  • 1
    @PaulVerest Updated. However if I run `mvn clean` then open Eclipse the `.java` files aren't made with the initial workspace build, but still if I clean the project through Eclipse it works. – Captain Man Nov 10 '16 at 17:01
  • I have found answer for `protobuf-maven-plugin`. It would be super great to have it for `protoc-jar-maven-plugin`. The sample is https://github.com/paulvi/com.example.grpc.protoc-jar-maven-plugin – Paul Verest Nov 11 '16 at 16:30
  • Different output comparing to `protobuf-maven-plugin` https://github.com/os72/protoc-jar-maven-plugin/issues/20 – Paul Verest Nov 11 '16 at 16:38
  • @PaulVerest, popped back because someone upvoted this. I know this is an ancient answer. I haven't used gRPC/Protobuf in past year or so but since this answer I feel pretty confident I've use a Protobuf plugin that was able to properly build the `target/generated-sources` as you edit the `proto` files as well as having `target/generated-sources` be a source folder in Eclipse. Paul, if you have figured this out I think it would be good to add since it doesn't seem like a large amount of people use this stuff and this may still be a top result in search engines. :) Either way is fine though – Captain Man Nov 03 '20 at 18:20
3

for protobuf-maven-plugin

Thanks to sergei-ivanov answer in https://github.com/xolstice/protobuf-maven-plugin/issues/16, that gave link https://github.com/trustin/os-maven-plugin#issues-with-eclipse-m2e-or-other-ides :

One need to download os-maven-plugin-x.x.x.Final.jar (the version as in your pomx.ml) and put it into the <ECLIPSE_HOME>/plugins directory.

After that Eclipse will generate source on project clean, including after Maven -update project... (Alt+F5), but not after Project -> Build (or with default Build Automatically). Also on IDE start it will not compile.

Yes, that is illogical:

Project - Clean will generate and compile Java source
but
Project - Build will not.

P.S. Raised Bug 507412

Paul Verest
  • 60,022
  • 51
  • 208
  • 332
  • Eclipse's Maven integration is just ...bizarre... it always was, because instead of treating the pom.xml as the authoritative source for builds, it basically scans it to create an authoritative eclipse build configuration, which always seems to be "just a bit off" in the corner cases. So, in the clean builds, Eclipse will deep scan all folders to "find out" the new stuff, but in a "build" it tends to trust its alternate universe of the Maven config, which often misses generated items. The m2e plugin is not much more than a "signal Eclipse to rescan" plugin. – Edwin Buck Nov 11 '16 at 16:17
  • My answer builds at the same time in the same way with the same limitations, am I missing something? – Captain Man Nov 11 '16 at 16:34
  • 1
    One issues added for the answer as comment. The bounty is yours anyways. – Paul Verest Nov 11 '16 at 17:31
1

Both eclipse and vscode can automatically compile proto when changed.

            <plugin>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.6.2</version>
                <executions>
                    <execution>
                        <phase>initialize</phase>
                        <goals>
                            <goal>detect</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:3.12.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.32.1:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                    <encoding>UTF-8</encoding>
                </configuration>
            </plugin>

See: https://github.com/trustin/os-maven-plugin#issues-with-eclipse-m2e-or-other-ides

Community
  • 1
  • 1
Roc King
  • 431
  • 6
  • 18