Provider "gs" not installed

Question

i've found two similar posts here but one hasn't been answered and the other was about android. I have a spring boot project and I want to access GCP Storage files within my application.

Locally everything works fine I can access my bucket and read as well as store files in my storage. But when i upload it to gcp kubernetes I get the following exception:

"java.nio.file.FileSystemNotFoundException: Provider "gs" not installed at java.nio.file.Paths.get(Paths.java:147) ~[na:1.8.0_212] at xx.xx.StorageService.saveFile(StorageService.java:64) ~[classes!/:0.3.20-SNAPSHOT]

My line of code where it appears is like follows:

public void saveFile(MultipartFile multipartFile, String path) {
    String completePath = filesBasePath + path;
    
    Path filePath = Paths.get(URI.create(completePath)); // <- exception appears here
    Files.createDirectories(filePath);
    multipartFile.transferTo(filePath);
        
}

}

The completePath could result in something like "gs://my-storage/path/to/file/image.jpg"

I have the following dependencies:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-storage</artifactId>
    <version>1.2.6.RELEASE</version>
</dependency>
<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-nio</artifactId>
    <version>0.122.5</version>
</dependency>

Does anyone have a clue where to look at? The only real difference except for the infrastructure is that i don't explicitly don't use authentication on kubernetes as it is not required according to the documentation

When using Google Cloud libraries from a Google Cloud Platform environment such as Compute Engine, Kubernetes Engine, or App Engine, no additional authentication steps are necessary.

Answer 1

It looks like the conventional Spring boot packaging here isn't packaging the dependency in the needed way. Usually you'll see something like:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>2.4.5</version>
    <executions>
        <execution>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>

However, for the "gs" provider to be accessible it needs to be in the 'lib/' folder. You can package it manually by copying the dependencies and then creating the JAR (this is based on the springboot-helloworld sample:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <executions>
      <execution>
          <id>copy-dependencies</id>
          <phase>prepare-package</phase>
          <goals>
              <goal>copy-dependencies</goal>
          </goals>
          <configuration>
              <outputDirectory>
                  ${project.build.directory}/lib
              </outputDirectory>
          </configuration>
      </execution>
  </executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
    <archive>
        <manifest>
            <addClasspath>true</addClasspath>
            <classpathPrefix>lib/</classpathPrefix>
            <mainClass>
              com.example.appengine.springboot.SpringbootApplication
            </mainClass>
        </manifest>
    </archive>
</configuration>
</plugin>

Originally posted on GitHub.

Answer 2

Following Averi Kitsch's answer and using the same springboot-helloworld example, I was able to get it working locally after updating pom.xml. However, much like it did for you, it only worked when I ran it locally and would fail when I deployed it on Google Cloud. The issue was that the Dockerfile I was using was ignoring all of the jar files in the /target/lib directory, and I needed to copy that directory to the image. Note that I used Google Cloud Run, but I believe this should work for most deployments using Dockerfile.

Here's what I ended up with:

Dockerfile

FROM maven:3.8-jdk-11 as builder

WORKDIR /app
COPY pom.xml .
COPY src ./src

RUN mvn package -DskipTests

FROM adoptopenjdk/openjdk11:alpine-jre

COPY --from=builder /app/target/springboot-helloworld-*.jar /springboot-helloworld.jar

# IMPORTANT! - Copy the library jars to the production image!
COPY --from=builder /app/target/lib /lib

CMD ["java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/springboot-helloworld.jar"]

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example.appengine</groupId>
  <artifactId>springboot-helloworld</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.5.4</version>
    <relativePath/>
  </parent>

  <properties>
    <maven.compiler.target>11</maven.compiler.target>
    <maven.compiler.source>11</maven.compiler.source>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-nio</artifactId>
      <version>0.123.8</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>copy-dependencies</id>
            <phase>prepare-package</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
            <configuration>
              <outputDirectory>
                ${project.build.directory}/lib
              </outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
              <mainClass>
                com.example.appengine.springboot.SpringbootApplication
              </mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

Answer 3

The gs:// syntax is not universal; it's supported by certain Google tools and services (gsutil, for example) but it's not a known by many URL libraries.

You could change to use the HTTP syntax for one of the GCS APIs, e.g.,

https://my-storage.storage.googlapis.com/path/to/file/image.jpg

Note that the above HTTP URL will require authentication, so probably instead you would want to use a library like google-cloud-java that supports making authenticated GCS requests.

Answer 4

just mounh the storage bucket in the your server use local path see below: https://medium.com/@antrixsh/how-to-mount-cloud-storage-bucket-with-gcp-compute-engine-ba7c95ad5349