The Java IOException "Too many open files" shown in the Jenkins build log is caused by your Maven build manager.
It is not an uncommon violation of a system limit¹ that can happen in build systems, especially when those are builds and workloads on remote machines that are seeing heavier builds for the first time. This could be the case here with your Jenkins setup.
However, it is a bit of speculation as you wrote that when you invoke the build manager on the command line, you do not hit that limit but have not shared that it is not on the same machine, so take the last sentence with a grain of salt.
The normal procedure is to file a change (request) about the limit of open files/file descriptors/file handles (for the build user(s) Maven as the build manager is executed under, a.k.a. "ulimit").
If it is a self-managed system, consult your operational manual that covers this system and check whether such settings are already under configuration (I list some more resources below). If so, you should then be able to file the change directly, according to the operation instructions and related manuals.
¹ limits.conf(5), sysconf(8), dockerd(8), systemd.exec(5)
Additional Resources
It's all about the ulimit but system configurations differ a lot today, and often administration service is gone. Some pointers and Q&A resources so that you can start building your own knowledge base in case you have to (this is all Linux):
Excerpt from $ bash -c 'help ulimit':
ulimit: ulimit [-SHabcdefiklmnpqrstuvxPT] [limit]
Modify shell resource limits.
Provides control over the resources available to the shell and
processes it creates, on systems that allow such control.
Options:
-S use the `soft' resource limit
-H use the `hard' resource limit
-a all current limits are reported
[...]
-n the maximum number of open file descriptors
(c.f. How do I change the number of open files limit in Linux?)
The ulimit command on shell you can use to test and try things, e.g. compare with your shell and the remote system, limit your shell to a setting of the remote and try to reproduce etc.
And from dockerd(8):
--default-ulimit=[]
Default ulimits for containers.
(c.f. how to set ulimit / file descriptor on docker container the image tag is phusion/baseimage-docker)
Not caring much about Docker here, this is merely a reminder that if you have Jenkins / Maven run dockerized, you may find additional places where and how to configure the ulimits (the last excerpt points to only one of those).
The same applies to systems with systemd, specifically systemd.exec(5) is of interest then, excerpt:
Table 1. Resource limit directives, their equivalent ulimit shell
commands and the unit used
┌─────────────────┬───────────────────┬─────────────────────┐
│Directive │ ulimit equivalent │ Unit │
├─────────────────┼───────────────────┼─────────────────────┤
[...]
├─────────────────┼───────────────────┼─────────────────────┤
│LimitNOFILE= │ ulimit -n │ Number of File │
│ │ │ Descriptors │
├─────────────────┼───────────────────┼─────────────────────┤
(c.f. Jenkins slave using pipeline ignores system defined limits)
And compare with Resolution section B. Ulimit Settings [just for Linux OS] in the Prepare Jenkins for Support article by Cloudbees for more input.
