When you build with maven on a multicore / multi-CPU machine it would often be possible to build different subprojects in parallel. Is there a way to do this with maven? Is there a plugin for this / whatever?
Asked
Active
Viewed 1.2e+01k times
4 Answers
307
Maven 3 (as of beta 1) now supports parallel builds as an experimental feature.
For example,
mvn -T 4 clean install # Builds with 4 threads
mvn -T 1C clean install # 1 thread per cpu core
mvn -T 1.5C clean install # 1.5 thread per cpu core
Full documentation can be found on the Maven wiki: Parallel builds in Maven 3 - Apache Maven - Apache Software Foundation.
Sergey Vyacheslavovich Brunov
- 17,291
- 7
- 48
- 81
hallidave
- 9,579
- 6
- 31
- 27
-
9What does _1.5_ threads mean? If I have a single core system, does that mean 1 thread, but a dual core system results in 3 threads? – Saad Malik Apr 28 '14 at 19:35
-
5@SaadMalik Yes, exactly, 3 threads on dual core, 6 threads on quad core systems, etc. – t0r0X Aug 06 '14 at 11:25
-
1Is there a argument we can use for the JVM, so we can use this in eclipse? – Jason Huntley Oct 13 '15 at 14:13
-
Eclipse support does not invoke maven externally. – Thorbjørn Ravn Andersen Oct 14 '15 at 14:03
-
Does `mvn -T 2C clean install` do the same trick? Can we modify this numbers? Can I use `-T 99C` and mvn will use max of my CPU cores? How to calculate optimum? – Andrew Feb 25 '21 at 09:41
-
2What the point in using more that 1 thread per CPU? – Enbugger May 26 '21 at 10:28
-
2@Enbugger some tests may cause threads to partially idle during tests, for example when waiting for endpoint or database responses. – Marv Nov 19 '21 at 09:04
35
The suggested solutions are great, but I wanted to add something to the answers here regarding the test stability during parallel builds.
So, when Maven parallel build is used:
mvn -T 4 clean install # Builds with 4 threads
mvn -T 1C clean install # 1 thread per cpu core
mvn -T 1.5C clean install # 1.5 thread per cpu core
Some issues with tests can appear. Note any behavior in tests which is different between serial and parallel test execution. Most of the times it happens do to improper test isolation resource-wise.
For example, test1 manipulate db entry with key 12345, which is hard-coded and test2 uses the same entry! It can't be good…
It's a situation that should be considered in the first place, but sometime it's forgotten and could lead to different problems once the switch to parallel maven build is made.
In case that happens, and you still want to use the parallel execution at least in some of the occasions, you can (of course, besides trying to fix the test and make them properly isolated) to disable Maven test runs using -DskipTests argument:
mvn clean install -T 4 -DskipTests
Sergey Vyacheslavovich Brunov
- 17,291
- 7
- 48
- 81
Johnny
- 14,397
- 15
- 77
- 118
-
2https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3 – Ahmed Nabil Apr 02 '20 at 20:26
-
11This is not the point of the question and it isn't good advice. Tests that break in parallel execution are bad tests, They may also be signaling bad code design. It is better to listen and act on it. Skipping tests to solve this is just sweeping something under the rug. – corlaez Oct 18 '20 at 03:37
-
How it's related? When someone introduces parallel execution in his tests, *it can happen*. It's just pragmatism. – Johnny Oct 19 '20 at 06:35
-
2When running parallel builds, their test phases are also parallel, therefore make sure the debugging is disabled, as the allocation of the debug port breaks silently the parallel builds. – Rusi Popov Dec 27 '20 at 11:33
13
Some of the CI build applications (e.g. hudson) can build multiple maven projects at the same time (and even on multiple machines).
Support for this in maven 'standalone' would also be nice, a quick look through the maven issue tracker gave me: http://jira.codehaus.org/browse/MNG-3004
jor
- 711
- 3
- 7
-
I think the Hudson functionality for multiple maven projects at the same time is broken as is indicated in the MNG-3004 JIRA comments. – Dougnukem Dec 14 '09 at 17:12
-
4
If you came to this question looking to sort out your build server and you're not using one that deals with maven natively the magic flag you are looking for is this:
-Dmaven.repo.local=someNoneGlobalDir
do that for each one of your builds and you can let them run all at the same time rather than having everything that uses maven in a queue!
JonnyRaa
- 7,559
- 6
- 45
- 49