First of all you need to understand the difference between screen density and screen size.
The screen size is the physical size, measured as the screen's diagonal. Android groups all actual screen sizes into four generalized sizes: small, normal, large, and extra large.
A phone might have a screen size of 4.9 inch which would be considered normal, both Nexus 7 (the old one from 2012 and the new one from 2013) have a screen size of 7" which is large, the Nexus 10 has a screen size of 10" and that's extra large.
The screen density on the other hand is the quantity of pixels within a physical area of the screen; usually referred to as dpi (dots per inch). For example, a "low" density screen has fewer pixels within a given physical area, compared to a "normal" or "high" density screen.
For simplicity, Android groups all actual screen densities into four generalized densities: low, medium, high, and extra high (plus the new xxhdpi).
An old Nexus 7 has the same screen size as a new Nexus 7 but the old one has a resolution of 1280x800 which is 216 dpi or hdpi while the new one has a resolution of 1920×1200 pixels which is 323 dpi or xhdpi (more pixels within the same physical area means higher pixel density in dpi).
An image in the drawable folder will have the same physical size on small, normal, large and x-large screens if the screens have the same screen density. Because the screens have different sizes the image will take up a different fraction of the screen. On small screens it will take up a larger part in percentage than on a large screen.
Nothing will change if the same image is in one of the screen size folders (drawable-small, drawable-normal, drawable-large, drawable-xlarge) but you can decide to put a larger version of the image in drawable-xlarge. In that case the image would be larger on a Nexus 10 than on a new Nexus 7 (both have xhdpi pixel density).
If the screens have a different pixel density that same image will look differently though. The image would be half the size on an xhdpi screen compared to an mdpi screen (because the xhdpi screen has approximately double the pixel density):
http://developer.android.com/images/screens_support/density-test-bad.png
In case of an icon you usually want it to have the same size on different screens. That's why e.g. menu icons for mdpi screens are 32x32 and those for xhdpi screens are 64x64 and both are in the appropriate drawable folder (drawable-mdpi and drawable-xhdpi):
http://developer.android.com/images/screens_support/density-test-good.png
Now when do you use the pixel density and when do you use the screen size drawable folders?
Pixel density folders are used if the image should have the same physical size on screens with different screen densities which is usually what you want. If you use the same image for an old and a new Nexus 7 it would have a different size even as the screens have the same physical size and that's not what you want. So using density dependent images is imperative.
Screen size folders are used if you want an image to have a different physical size on small, normal, large and x-large screens. If I have a grid navigation with 6 icons on the main screen and I don't want to make use of the extra screen real estate on larger screens (e.g. by adding more icons), then I would provide a small image for the small screen and a large image for the large screen.
You would still have to provide density dependent images on top of the screen size dependent images as explained before (example old Nexus 7 vs. new Nexus 7).
So in theory you would need 16 different resources for the same image (4 screen sizes in 4 screen densities or with the new xxhdpi density even 5 densities -> 20 resources).
Now of course no one wants to create that many resources especially if you have a lot of images.
One approach is to use the Dashboard as someone has suggested:
http://developer.android.com/about/dashboards/index.html#Screens
and pick the most commonly used combinations which are small/ldpi, normal/mdpi, normal/hdpi and normal/xhdpi (81% of all devices). That way you bring down the resources to just 4.
Another approach is to provide resources for either screen size or for screen density (again only 4 resources needed) and then do some scaling in code.
If you have screen density dependent resources then you would use e.g. this https://stackoverflow.com/a/5016350/534471 to scale the images down (never up).
If you have screen size dependent resources then you would use this http://developer.android.com/reference/android/util/DisplayMetrics.html
to scale the images down.
There's a nice example for all this here (including source code):
https://www.captechconsulting.com/blog/steven-byle/understanding-density-independence-android
Now for you specific problem you could use one generic image in the folder drawable. The size of that image should be so that it won't have to be up-scaled (because that would look ugly). You define the button in the layout like this:
<ImageButton
android:id="@+id/myButton"
android:src="@drawable/myDrawable"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:scaleType="centerCrop"
android:adjustViewBounds="true"
/>
Together with this piece of code the button scales to 10% of the screen:
Display display = getWindowManager().getDefaultDisplay();
Point screenSize = new Point();
display.getRealSize(screenSize);
int size = Math.min(screenSize.x, screenSize.y);
int buttonSize = Math.round(size * 0.1f);
ImageButton button = (ImageButton) findViewById(R.id.myButton);
button.setMaxWidth(buttonSize);
button.setMaxHeight(buttonSize);
How large should the original image be?
A Nexus 10 has probably the highest screen resolution of all Android devices at the moment. The 1600 pixels will translate to 3200 density independent pixels on its xhdpi display. 10% of 3200 is 320. If you use a 320x320 image then you will get a good result on all existing devices.
There's a catch to this approach though.
320x320 is pretty large (possibly 24/32 bit color depth) and thus you might run into memory issues. If you provide the same resource in the density dependent drawable folders you can lower the memory footprint for hdpi, mdpi and ldpi devices:
- drawable-xhdpi: 320x320
- drawable-hdpi: 240x240
- drawable-mdpi: 160x160
- drawable-ldpi: 120x120
The screen size drawable folders could be used to further improve this (smaller screens need smaller images) but then you'll have to provide the 16 or 20 resources as mentioned before. In the end it's a trade-off between memory footprint / speed on one side and maintainability / time to create the resources / apk size on the other side.