12

The global build of our application (30 Maven modules) is taking too much time (15 minutes). This includes units and integration tests. The majority of the time is consumed by the integrations tests (60%).

Our tech stack comprises of Spring, Spring MVC, Spring Batch etc. and Maven. Our developers are not motivated to keep this practice (Build All before commit)

Since I want to improve the build time I am suggesting these scenarios:

  • Parallel build : mvn -T 1C is not going to work as this consumes all resources of developer machine which prevents the developer from doing other things.
  • Organize module by profile (front, batch, connector, commons) is not going to work either as our modules are inter-dependent and we must do the build all.

Do you have some suggestions to improve the build time of large projects?

Thanks in advance.

nabster
  • 1,561
  • 2
  • 20
  • 32
Nabil
  • 1,771
  • 4
  • 21
  • 33
  • I have run into this many times on projects I have worked on. I don't consider a 15 minute build particularly long. Could you try motivating the developers by sacking the ones that don't run the tests before committing? – mikea Jan 13 '14 at 12:10
  • 2
    We commit many times in a day , waiting 15 minutes for every commit is very frustrating . Specially when you are resolving bugs and regression . So i don't think it's because we have one or two developer not test minded but because it's a real pain . – Nabil Jan 13 '14 at 12:12
  • 1
    Integrationtests may not necessary in developer's build. – Grim Jan 13 '14 at 12:13
  • Maybe you could speedup your tests? – Behe Jan 13 '14 at 12:17
  • I have some tips here: http://stackoverflow.com/questions/161698/how-can-i-speed-up-my-maven2-build/18911835#18911835 – Sinto Jan 13 '14 at 12:17
  • y not skip some test steps? – Rugal Jan 13 '14 at 12:22
  • 1
    if `mvn -T 1C` is consuming too much resources, than you should find the bottleneck and upgrade your dev machines appropriately. You might also try `mvn -T n` where `n` is some constant to use a constant pool size which is smaller than `1C` – SpaceTrucker Jan 13 '14 at 14:54

5 Answers5

12

Don't run the integration tests in the normal profile, let the developers check in after running unit tests only.

Run integration tests on a separate server (a build server or continuous integration server, like Jenkins or similar). Have the build server email the developers that checked in bad code.

In our work office, we also have big screens showing green/yellow/red flags for each module, so everyone can see if a module is unstable (and who has checked in since the last stable build).

Harald K
  • 26,314
  • 7
  • 65
  • 111
8

  1. Adjust memory configurations to optimum for eg: add this line to mvn.bat set MAVEN_OPTS=-Xmx512m -XX:MaxPermSize=256m

  2. Clean phase of mvn normally deletes target folder. Instead if we are renaming target folder the cleaning phase will be much faster.

  3. -Dmaven.test.skip=true will skip the test execution.

  4. Add -Denforcer.skip=true to mvn command line argument (This is enforcing versions of maven, jdk etc ,we can skip it after initial runs)

  5. Disable non-critical operations during build phase: Analysis, javadoc generation, source packaging. This will save huge time.

  6. Spawnig new process also helps in time improvement -Dmaven.junit.fork=true (fork the junit tests into a new process) -Dmaven.compile.fork=true (forks the compilation)

Hope it helps.

Sinto
  • 920
  • 5
  • 10
2

The pricipal time is consumed by the integrations tests (60%).

Typically you don't want to run integration-tests during a build on developer machines. Just exclude them. But make sure you run them on your continuous integration server.

Puce
  • 37,247
  • 13
  • 80
  • 152
1

Have a build server that automatically does a full build and run the tests once someone checks in. If the build or tests fail immediately flag it in some suitably prominent way including emailing the person who did the check in.

Tim B
  • 40,716
  • 16
  • 83
  • 128
1

So our developers are not motivated to keep this practice ... ... we consume all ressources of developer machine so during the build the developer cannot do any things

If the main reason why developers do not want to build all before commit is that during the maven build their computer is useless (due to the maven using all available resources), let them specify how much resources maven should use.

Example (using Windows cmd), instead of:

mvn clean package

you can use:

start /affinity 3 mvn clean package

In this case maven will use 2 cpu cores (I have 8 cpu cores) so during the maven build lifecycle I can still work on my machine.

The pricipal time is consumed by the integrations tests (60%)

I would say that tests are important but you can save some time skipping tests:

mvn clean package -DskipTests

More info about processor affinity could be found here: http://www.techrepublic.com/blog/windows-and-office/change-the-processor-affinity-setting-in-windows-7-to-gain-a-performance-edge/

vitfo
  • 9,781
  • 6
  • 29
  • 30