If you want to maintain the previous app in the foreground and have your app appear over the top, you could use the SYSTEM_ALERT_WINDOW mechanism. This is what Facebook Messenger ("chat heads"), Evernote, Pocket etc use to draw a window over the top of apps in the foreground, normally briefly to show the user that an action has taken place in an app that's not currently in the foreground.
It's not something create an entire app in however, as this would break how a user expects an Android app to work, and could cause you some usability issues down the road. It's also known to be open to abuse, so some users are averse to app that make use of it, which could lose your users' trust if it's not clear why you want your app to work in that way.
But for cases where you want to provide user with some action option that can't be effectively done as a rich notification, it serves a purpose.
Here's an example of how to implement it.
In addition, a longer article explaining what it is and how it works.
Updated Info in response to comments (put here so I can use markdown)
In the example there is an Activity called Main that only exists to start the service, so the Service belongs to the app's package.
Permissions on Android are handled per-package, so you need to declare the permissions that you want a Service to have access to in the AndroidManifest.xml file of the project that the Service belongs to.
Once running, OverlayShowingService gets a handle on the WindowManager for the current display. The system's WindowManager is run as a system service - WINDOW_SERVICE, and is what the OS uses to display apps and Activities that are currently running.
Each app effectively runs within its own Window (which is incidentally why Android N's new multi-window feature can let more than one app run concurrently without needing to change the app lifecycle - they already exist in separate windows, just on top of each other rather than side by side).
So once running, OverlayShowingService programmatically creates the View to be shown over the top (in the case of the example, a Button), and adds that view to the system-wide WindowManager by calling addView(..):
wm.addView(overlayedButton, params);
This immediately adds the View to the screen at the coordinates specified by x and y.
If the Service did not have the SYSTEM_ALERT_WINDOW permission at this point, it would not be allowed to add the view to the system-level WindowManager, as otherwise any malicious or badly made app could add a view to the screen over the top of all other running apps, and capture screen inputs.
Re: How Notifications work
They are also created using a system service. Notifications are added in a similar way to the way that Views are added to the WindowManager:
If you want to make something equivalent to a custom Notification shade, you'll need to create a system service of your own, which is going down the route of compiling your own ROM and replacing parts of the OS for a specific purpose. If you get to that point, your users will need to be rooted and/or running your custom ROM.
Update - Worked Example
I had a spare few minutes this morning so put together a working demo of intercepting the back button from a SYSTEM_ALERT_WINDOW. You can clone it and run it directly to see how the interception works, and there are a few brief comments in the code to explain the steps.
Only the back button is intercepted, but you can use the same method to handle other key types. Be aware however that there are so many different types of Android device out there with hardware and software keys, running on different versions of Android, that it may not be possible to make a solution that works reliably on every device. Especially for things like the menu button, which most newer devices no longer have; when you see the menu dots in most apps, it's handled within the app itself rather than as a system-level interaction, so unless you modify those apps directly, that action cannot be intercepted.
Have a good one.
