While Ninjas answer provided a way to close many activities on stack, note that the way Android was built, no application may interact directly with another... this as a security feature blocks most un wanted software, as well as malwares/virus/etc.
The only actual way to forcefully close an application is to either block resources for it (by using the Android Settings tool to put the application in "not started" state), or to go to the below system (mostly a linux), and deny resources, this one is tricker, but possible.
If I open the same task killer app again within seconds, the same app will again be there which was claimed to be killed earlier. Now is this a fault of the app or the Android OS?
It is the OS "fault", but it behaves EXACTLY as expected. No program should be able to alter/edit your program. Just like ninjas answer, he basically asked the system to disregard any activities on that stack. But if muliple stacks are present... well... nope then, its the APP fault.
Not only here, even with apps having an Exit option, I have noticed that clicking the exit button does not terminate it. So where is the problem?
A programmer should NOT exit his application, because its not a full program, you are basically running commands inside another program, hence the rules must be obeyed. Once you start, the memory management is out of your hands, so you cannot/should not "exit" applications.
Is there a proper way yet to close an app when we want and make it behave normally?
Dont understand this one. In the application lifecycle, everything is either behaving normally (or you know, stopped: example
has stopped working), or in the ready state