-1

I am using Flyway to deploy to 300!!! Databases. What I've done is created a cmd file that has the flyway command - 1 line each for each of the 300 databases.

Like this:
flyway -configFile= -url= migrate.

There are 300 such lines in the cmd file and its works great when the execution is sequential - obviously it also takes more time.

If I want to run it in Parallel i'll have to add 'start' at the beginning of the command like so:

start flyway -configFile= <conf file path> -url=<DB URL> migrate

The problem am facing with this approach is - after migration on about 100 DB's the Microsoft-Windows-Resource-Exhaustion-Detector detects a Low Virtual Memory condition. Now I know how to set the JVM memory and have tried various options but I don't think it's making any difference at all. I think FlyWay is completely ignoring the JVM settings.

I've 16GB physical memory on the Windows server(6-bit) where Flyway is being executed. When the parallel run happens - I've checked in the Task Manager and the usage doesn't go beyond 3 GB. JRE version is 64 bit. I set _Java_Options. I also tried setting the JVM memory in Java Control panel but nothing seems to work.

Thank you

Mihir
  • 11
  • 5

1 Answers1

0

You will need to limit the number of migrations you run in parallel. From your description it appears you are trying to run 300 which is using too much RAM. Try 10 in parallel and work up from there.

See this question Best strategy to run multiple flyway migration in parallel for a linux based approach. Installing git for windows with git bash is a convenient way to get linux tools to try this.

Hamish Carpenter
  • 831
  • 4
  • 9
  • In the FlyWay installation there's a file : \jre\lib\amd64\jvm.cfg. In that file - i added the following : -Xms64m -Xmx64m -Xss8m -XX:-UseParallelGC. Now I've 30 threads running in paralled and there's 9 GB of free memory before the start of the execution.What am i doing wrong? 20 threads succeed. For the remaining 10 I get this error: Error occurred during initialization of VM Could not reserve enough space for object heap. – Mihir Oct 02 '17 at 15:44
  • Not sure from here sorry. Perhaps 20 is your parallel rate limit. – Hamish Carpenter Oct 02 '17 at 22:01