In general, I would say that if an element of a parent class is needed in some but not all of the children then an intermediate parent should be introduced.
When defining an inheritance hierarchy, it's a logical assumption that the children of a parent should share all properties of that common ancestor. This is akin to the way a biological taxonomy would work, and it helps to keep the code clean.
So let's have a look at the objects in your system (we'll use the "is a" relationship to help us figure out the inheritance hierarchy):
Button, Knob, Lever, and Switch
Each of these might indeed be called "Devices", but when we say "devices" most people will probably think of digital devices like phones or tablets. A better word for describing these objects might be "controls" because these objects help you to control things.
Are all objects Controls? Indeed they are, so every object will have Control as a parent in its inheritance hierarchy.
Do we need to further classify? Well your requirements are to have an on/off status, but it does not make sense for every control to have on/off status, but only some of them. So let's further divide these into Control and ToggleControl.
Is every Control a ToggleControl? No, so these are separate classes of objects.
Is every ToggleControl a Control? Yes, so ToggleControl can inherit from Control.
Are all objects properly classified and are all parent attributes shared by all children? Yes, so we're done building our inheritance hierarchy.
Our inheritance hierarchy thus looks like this:
Control (Code shared by all controls)
/ \
/ \
Knob ToggleControl (Code shared by all controls that can also be toggled - Adds isOn)
\
\
Button, Lever, Switch
Now, to the other part of your question:
Lever and Switch need a method Throw() that toggles isOn; Button uses isOn but does not use Throw() to handle it. Does this affect your placement of isOn, and where would you define the Throw() method?
Firstly, "throw" is a reserved word (at least in Java), so using method names that are similar to reserved words might cause confusion. The method might be better named "toggle()".
Button should (in fact it must) use toggle() to toggle it's isOn since it is a togglable control.