I am a student of artificial intelligence. I am trying to use the ART library to perform attacks on a binary classifier doing speaker recognition. However, due to some technical problems, I had to resort to training from scratch, using a Keras tutorial. Now, even trying a trivial attack like FGSM, I notice some ambiguous behaviour. First, the security evaluation curve decreases, then peaks and then decreases again, which makes my nose twitch. After that, the classifier's performance on the attacked samples does not fall below 50 per cent, which should not be correct.
I wanted to know if anyone else had experienced this problem and if so how they solved it. Thank you very much in advance.