Although there are other answers that cover this in some way, I want to have it for someone else that has similar problem and does notice the solution.
After setting up my own annotation processor and it properly working via maven, I got annoyed by being forced to rebuild source with maven on each change that needs the processor to do some magic.
Setting up eclipse to use my annotation processor required me to close annotation processor project so m2e-apt can put processor jars into .factorypath .
But then it stopped working for some reason, and I could not find the reason,