Is this normal behaviour or is there something wrong with the application?
Without more information, it's impossible to say.
This could be normal behaviour. Memory in the JVM is freed up by garbage collection. The garbage collector cleans up objects that are no longer being used. It has its own rules for deciding when and how to do that. In particular, it tries to free up memory when the application needs it. If the application is idle and doesn't need any memory to allocate new objects, the garbage collector might not run.
On the other hand, it might be that your application is still holding references to some data, preventing it from being garbage collected. If there's any variable or data structure in your application that still holds a reference to an object, that object won't be garbage collected. Over time, this would eventually mean that the garbage collector would not be able to free up any new memory and the application would crash.
If you want to be sure, you should try running your test several times. If your application is working well, you will see the memory graph drop right down when the heap starts to get full, and then gradually rise again. This results in a characteristic "sawtooth" memory usage graph.