1

I encountered the common "org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "EPSG" found for object of type "EngineeringCRS" while trying to run my packaged jar on an Apache Flink cluster.

I already had gt-epsg-hsql added as a dependency when I got the above error. I managed to fix that by following the instructions on the official Geotools building FAQ and rearranging the dependency order in my pom.xml file and using maven-shade-plugin.

But it seems now I encounter a new exception which is this one.

Caused by: java.lang.IllegalArgumentException: Argument "ellipsoid" should not be null.
    at org.geotools.referencing.GeodeticCalculator.<init>(GeodeticCalculator.java:195) ~[?:?]
    at org.geotools.referencing.GeodeticCalculator.<init>(GeodeticCalculator.java:188) ~[?:?]

Spent the whole day pulling my hair trying to debug this but had no success. It all works when running it on a local cluster via my IDE but packaging and deploying the jar to my cluster causes these errors.

I tried emulating the suggested POM that the original asker had listed in this question and modified my pom.xml file to the current one attached below but it still gives the same error. Incase anyone needs to see my full pom.xml for details, it is hosted here on Textbin.

<dependencies>
    <!-- GeoTools Related -->
    <dependency>
        <groupId>org.geotools</groupId>
        <artifactId>gt-shapefile</artifactId>
        <version>${geotools.version}</version>
    </dependency>
    <dependency>
        <groupId>org.geotools</groupId>
        <artifactId>gt-epsg-hsql</artifactId>
        <version>${geotools.version}</version>
    </dependency>
    <dependency>
        <groupId>org.geotools</groupId>
        <artifactId>gt-swing</artifactId>
        <version>${geotools.version}</version>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13.2</version>
        <scope>test</scope>
    </dependency>

</dependencies>

    <plugins>
                <!-- Java Compiler -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.1</version>
                    <configuration>
                        <source>${java.version}</source>
                        <target>${java.version}</target>
                    </configuration>
                </plugin>
    
                <!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. -->
                <!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-shade-plugin</artifactId>
                    <version>3.0.0</version>
                    <executions>
                        <!-- Run shade goal on package phase -->
                        <execution>
                            <phase>package</phase>
                            <goals>
                                <goal>shade</goal>
                            </goals>
                            <configuration>
                                <artifactSet>
                                    <excludes>
                                        <exclude>org.apache.flink:force-shading</exclude>
                                        <exclude>com.google.code.findbugs:jsr305</exclude>
                                        <exclude>org.slf4j:*</exclude>
                                        <exclude>log4j:*</exclude>
                                    </excludes>
                                </artifactSet>
                                <filters>
                                    <filter>
                                        <!-- Do not copy the signatures in the META-INF folder.
                                        Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                        <artifact>*:*</artifact>
                                        <excludes>
                                            <exclude>META-INF/*.SF</exclude>
                                            <exclude>META-INF/*.DSA</exclude>
                                            <exclude>META-INF/*.RSA</exclude>
                                        </excludes>
                                    </filter>
                                </filters>
                                <transformers>
                                    <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                        <mainClass>DataGen.StreamingJob</mainClass>
                                    </transformer>
                                    <!-- This bit merges the various GeoTools META-INF/services files         -->
                                    <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
                                </transformers>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-jar-plugin</artifactId>
                </plugin>
            </plugins>

Any ideas on how to fix this error?

EDIT: Apache Flink version 1.13.0. Geotools 28-SNAPSHOT. Java version "1.8.0_333". Adding the contents of META-INF/services:

├── com.fasterxml.jackson.core.JsonFactory
├── com.fasterxml.jackson.core.ObjectCodec
├── it.geosolutions.imageio.compression.CompressorSpi
├── it.geosolutions.imageio.compression.DecompressorSpi
├── java.sql.Driver
├── javax.annotation.processing.Processor
├── javax.cache.spi.CachingProvider
├── javax.imageio.spi.ImageInputStreamSpi
├── javax.imageio.spi.ImageOutputStreamSpi
├── javax.imageio.spi.ImageReaderSpi
├── javax.imageio.spi.ImageWriterSpi
├── javax.measure.spi.FormatService
├── javax.measure.spi.ServiceProvider
├── javax.measure.spi.SystemOfUnitsService
├── javax.measure.spi.UnitFormatService
├── javax.media.jai.OperationRegistrySpi
├── javax.media.jai.OperationsRegistrySpi
├── javax.xml.bind.JAXBContext
├── org.apache.flink.client.deployment.ClusterClientFactory
├── org.apache.flink.core.execution.PipelineExecutorFactory
├── org.apache.flink.runtime.security.contexts.SecurityContextFactory
├── org.apache.flink.runtime.security.modules.SecurityModuleFactory
├── org.apache.flink.runtime.state.changelog.StateChangelogWriterFactory
├── org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonFactory
├── org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.ObjectCodec
├── org.apache.flink.table.factories.Factory
├── org.apache.flink.table.factories.TableFactory
├── org.apache.kafka.common.config.provider.ConfigProvider
├── org.apache.logging.log4j.core.util.ContextDataProvider
├── org.apache.logging.log4j.message.ThreadDumpMessage$ThreadInfoFactory
├── org.apache.logging.log4j.spi.Provider
├── org.apache.logging.log4j.util.PropertySource
├── org.geotools.coverage.grid.GridCoverageFactory
├── org.geotools.coverage.grid.io.footprint.FootprintLoaderSpi
├── org.geotools.data.DataStoreFactorySpi
├── org.geotools.data.FileDataStoreFactorySpi
├── org.geotools.feature.AttributeTypeFactory
├── org.geotools.filter.FunctionExpression
├── org.geotools.filter.FunctionFactory
├── org.geotools.filter.expression.PropertyAccessorFactory
├── org.geotools.http.HTTPClientFactory
├── org.geotools.referencing.factory.gridshift.GridShiftLocator
├── org.geotools.referencing.operation.MathTransformProvider
├── org.geotools.renderer.crs.ProjectionHandlerFactory
├── org.geotools.renderer.style.ExternalGraphicFactory
├── org.geotools.renderer.style.MarkFactory
├── org.geotools.styling.StyleFactory
├── org.geotools.swing.tool.InfoToolHelper
├── org.geotools.util.ConverterFactory
├── org.jaitools.numeric.Processor
├── org.jboss.marshalling.ProviderDescriptor
├── org.opengis.coverage.processing.Operation
├── org.opengis.feature.FeatureFactory
├── org.opengis.feature.type.FeatureTypeFactory
├── org.opengis.filter.FilterFactory
├── org.opengis.filter.expression.Function
├── org.opengis.referencing.crs.CRSAuthorityFactory
├── org.opengis.referencing.crs.CRSFactory
├── org.opengis.referencing.cs.CSAuthorityFactory
├── org.opengis.referencing.cs.CSFactory
├── org.opengis.referencing.datum.DatumAuthorityFactory
├── org.opengis.referencing.datum.DatumFactory
├── org.opengis.referencing.operation.CoordinateOperationAuthorityFactory
├── org.opengis.referencing.operation.CoordinateOperationFactory
├── org.opengis.referencing.operation.MathTransformFactory
├── reactor.blockhound.integration.BlockHoundIntegration
└── tech.units.indriya.spi.NumberSystem

1 directory, 67 files
Komal
  • 17
  • 5
  • can you check what is in the `META-INF/services` file in the output and add it to your question – Ian Turton Feb 04 '23 at 10:57
  • Technically java 8 is no longer supported by GeoTools at 28.x so may be try with 11 and see if that helps? – Ian Turton Feb 06 '23 at 08:44
  • @IanTurton So I tried upgrading to 11. Still get the same error unfortunately. Also tried using geotools version 24.5, but same results. – Komal Feb 07 '23 at 10:00

1 Answers1

0

Okay so the problem has nothing to do with Geotools.

The problem was in these specific lines of the my code:

crs = Params.coordinateReferenceSystem;
GeodeticCalculator gc = new GeodeticCalculator(crs);

and the crs was being set to null because Params.coordinateReferenceSystem is defined as a static variable. The scope of the static variables doesn't span (nor is it replicated) across all nodes in a distributed system.

The result of which it works perfectly on the local IDE but fails on the cluster.

Komal
  • 17
  • 5