Best practices for including large resource folder into java project

Question

I'm working on a project where having access to a big resource folder (structure with thousand of little images) is required. The client want to offer the app through a native installation (which includes the JVM that the app require to run). He doesn't want to pack that resources as a folder in the app because it would create a folder structure as big as the original in the final user's hard drive (the folder doesn't take much space but it has many little files), plus the folder could be stealed easily by simply copying it. Giving this, I can't package all the app with a resource folder in a jar file, as far as i know jar files are not installables. Another requirement is that client needs certain flexibility to add some files in a installed app folders structure to add new features to the program. So an installation is the only way (i think) to obtain this.

I've tried to pack them in a jar file, include it in the build path and tried to access it but i failed even with all the research i've made through various sites. Tried getResources() in a million ways but it was impossible to get a simple directory inside the jar file meanwhile doing it from a folder outside the jar is really easy. I need to get access to a directory in order to get a list of files it cointains.

Arrived to this point. I've started to ask myself if i'm facing this problem on the best way so i wanted to ask you all: how would you package the resources you need in a native java app with this requirements?

I'm even thinking about create some kind of encryption proccess to create a single file with all the information and simply temporarily decrypt it when needed at runtime but i think there would be a simpler and cleaner way to face this.

Thank you in advance

EDIT: As you asked for, i'm adding the code of what i've tried:

this is the project structure

project
├───src
│   ├───main
│   │   └───java
│   │       ├───model <--- where my class is
│   │       ├───controllers
│   │       ├───utilities
│   │       └───views
│   ├───resources <--- this is where is (formerly) accessed the content i need
|   |   ├─── jarfile.jar <--- i've placed file here and included to build path
│   │   └───-various folders and files -
│   └───test
└───target

inside the jar file there are the packages src.resources.blalblaba and inside of this, the folder i need

Way1:

getResources replacing jar file "." with "/" tried with paths: "src/resources/blablabla/folderINeed","src/resources/src/resources/blablabla" (due to possible duplicity), "folderINeed", "blablabla/folderINeed" -> URI always get NullPointerException with message "null"

public void loadContent(String contentPath) throws Exception
{ 
    File resources= null;
    File[] listFiles = null;

    URI uri = getClass().getClassLoader().getResource(contentPath).toURI();
    resources= new File(uri);
    listFiles = resources.listFiles();

    //do some file proccessing and load them
}

Way 2: paths used "folderINeed","src/resources/blablabla/folderINeed","blablabla/folderINeed","../../../resources/blablabla/folderINeed" <--- URL return null but, at least, doesn't raise a NullPointerException.

public void loadContent(String contentPath) throws Exception
{ 
    // conseguimos todas las carpetas de animaciones
    File resources;
    File[] listFiles = null;

    URL url = MyClass.class.getResource(contentPath);
    if (url == null) {
         // error - missing folder
    } else {
        resources = new File(url.toURI());
        listFiles = resources.listFiles();
    }
}

Way 3: some complex code using class JarFile that didn't work for me and was oriented to get a simple file, not a folder. Obtained here

Answer 1

src/main/java/ is the build convention of maven, so I'll assume it is that.

Then one normally would pack the images as read-only resources in its own jar, a separate maven project jarfile.

jarfile/src/main/resources/ would be the root directory for the images. For instance: jarfile/src/main/resources/icons16/new.png would then be accessed by something like:

getClass().getResource("/icons16/new.png"); // Class' package relative path
getClass().getResourceAsStream("/icons16/new.png");
getClassLoader().getResource("icons16/new.png"); // Absolute class path path

In the original project one would add a dependency in the pom.xml to jarfile.

If you do no use maven, the jarfile.jar must be on the class path.

File cannot be used with resources, but getResourceAsStream most often is feasible.

---

To read a resource directory, the directory path should be rather unique, and it is simply a list of file names in the directory resource.

    InputStream in = App.class.getResourceAsStream("/icons16");
    try (BufferedReader rd = new BufferedReader(new InputStreamReader(in, "UTF-8"))) {
        String line;
        while ((line = rd.readLine()) != null) {
            System.out.println("line: " + line);
        }
    }

Answer 2

MY WRONGS:

• As @Joop Egen told in his answer, one of my problem was my folder structure in the project. I was not following the maven convention of putting the resource folder in the src/main/ folder it's because of this all the solutions i were trying didn't have the necessary scope to get the resource folder.

• I didn't know how jar files work with java. For java .jar files are a collection (not a Collection of java) of jar entries to every single file within them. In my single case, the .jar file was created using the Eclipse export wizard and didn't have any reference to folders, it just had references to files. So it's simply IMPOSSIBLE to get a folder with all its content.

• I used the java JarFile class to manage the content but this class doesn't offer methods to manage files like java File class does. So it's not as easy to do as it is with another kind of files.

WHAT I'VE DONE:
I've developed a code to read all the file entries in the .jar, discriminate the ones i was interested to. And then extract them to a directory within the app. By doing this i had standard access to them and, if i want to, i can simply remove the directory when the application closes. I was trying to use them directly from the jar but jar files are zip files so in some moment that inner files need to be extrated from the jar to somewhere as OS do with zip files. It's can a be a temp directory or not.

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import org.apache.commons.io.FileUtils;

public class App
{
  public static void main(String[] args)
  {
    try
    {

      //needed attributes
      String pathToJar = "./src/main/blablalba/fileName.jar";
      String folderYouWantToRetrieveFromTheJar = "/folderIWant";
      String pathOfFilesWithinTheJar="src/resources/blablabla/"+folderYouWantToRetrieveFromTheJar+"/";
      String tempDirectoryWhereExtract="./src/main/resources/temp";

      //creating the temp directory
      File tempDirectoryReference = new File(tempDirectoryWhereExtract);
      if (!tempDirectoryReference.exists())
      {
        Files.createDirectory(tempDirectoryReference.toPath());
      }

      //searching what entries i need
      JarFile jar = new JarFile(pathToJar);
      final Enumeration<JarEntry> entries = jar.entries(); 
      List<JarEntry> targetEntries = new ArrayList<>();
      while (entries.hasMoreElements())
      {
        JarEntry entry = entries.nextElement();
        //if the entry is what i need 
        if (entry.getName().startsWith(pathOfFilesWithinTheJar))
        { 
          targetEntries.add(entry);
        }
      }
      //extract every target entry from the .jar
      for (JarEntry entry : targetEntries)
      {
        //in order to copy the structure i will get only the point where folderIWant is present
        int index = entry.getName().indexOf(folderYouWantToRetrieveFromTheJar);
        String newTemporaryPath = tempDirectoryReference.getPath().toString()+"/"+entry.getName().substring(index);
        extractFileFromJar(jar, entry, new File(newTemporaryPath));

      }

      jar.close();
      //(optional) clean after use
      FileUtils.deleteDirectory(tempDirectoryReference);


    }
    catch (IOException e)
    {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
  }

  public static void extractFileFromJar (JarFile jarFile, JarEntry targetEntry, File destinyPath)
  {
    try
    {
      if (!destinyPath.getParentFile().exists())
      {
        createFolderStructure(destinyPath);
      }
      else
      {
        Files.createFile(destinyPath.toPath());
      }

      InputStream inputStream = jarFile.getInputStream(targetEntry); 
      FileOutputStream outputStream = new java.io.FileOutputStream(destinyPath);
      while (inputStream.available() > 0) {  
          outputStream.write(inputStream.read());
      }
      outputStream.flush();
      outputStream.close();
      inputStream.close();
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }
  }


  private static void createFolderStructure(File destinyPath)
  {
    File parentPath = destinyPath.getParentFile();
    try
    {
      if (parentPath.exists())
      {
          Files.createFile(destinyPath.toPath());
      }
      else
      {
        Files.createDirectories(destinyPath.getParentFile().toPath());
        Files.createFile(destinyPath.toPath());
      }
    }
    catch(IOException e)
    {
      System.err.println(e.getMessage());
    }
  }
}