I just had a similar problem. I wanted to publish an app to be downloaded only on smartphones. I will introduce tablet support later. So I needed a filter that makes my app available only for smartphones. The app could not be install on Nexus 5 or Galaxy S5.
To fix this I did as suggested in the dev documentation (screen distribution) (see also answer on stackoverflow):
<manifest ... >
<compatible-screens>
<!-- all small size screens -->
<screen android:screenSize="small" android:screenDensity="ldpi" />
<screen android:screenSize="small" android:screenDensity="mdpi" />
<screen android:screenSize="small" android:screenDensity="hdpi" />
<screen android:screenSize="small" android:screenDensity="xhdpi" />
<!-- all normal size screens -->
<screen android:screenSize="normal" android:screenDensity="ldpi" />
<screen android:screenSize="normal" android:screenDensity="mdpi" />
<screen android:screenSize="normal" android:screenDensity="hdpi" />
<screen android:screenSize="normal" android:screenDensity="xhdpi" />
</compatible-screens>
...
<application ... >
...
<application>
</manifest>
But that’s not enough. Some xxhdpi devices will not be supported. You may be tempted to copy/paste the last line and add “xxhdpi”, but tt won’t work (I just tested it and the APK signing failed with this error: "error APT0000: String types not allowed (at 'screenDensity' with value 'xxhdpi')") ;)
To support xxhdpi check the dev documentation (compatible screens element).
Note: This attribute currently does not accept xxhdpi as a valid
value, but you can instead specify 480 as the value, which is the
approximate threshold for xhdpi screens.
Thus you need to add 480, not xxhdpi, as follows:
<screen android:screenSize="small" android:screenDensity="480" />
<screen android:screenSize="normal" android:screenDensity="480" />
Now, coming back to the Nexus 5 and Galaxy S5 not being supported. Let’s check some specifications:
LG Nexus 5: http://www.gsmarena.com/lg_nexus_5-5705.php
- Screen size is 4.95 inches.
- Density: 445 ppi
Samsung Galaxy S5: http://www.gsmarena.com/samsung_galaxy_s5-6033.php
- Screen size: 5.1 inches
- Density: 432 ppi
So according to the dev documentation (screen support):

Figure 1. Illustration of how Android roughly maps actual sizes and
densities to generalized sizes and densities (figures are not exact).
As you can see in the illustration above, between 4 and 5 inches is a bit “tricky” (please comment if I'm wrong) as the screen can be “normal” or “large”. So basically to be on the safe side I had to add support for large:
<screen android:screenSize="large" android:screenDensity="ldpi" />
<screen android:screenSize="large" android:screenDensity="mdpi" />
<screen android:screenSize="large" android:screenDensity="hdpi" />
<screen android:screenSize="large" android:screenDensity="xhdpi" />
<screen android:screenSize="large" android:screenDensity="480" />
I just tested on the Play Store and my colleague owning a Nexus 5 was able to download the app. Like this, anyway, some small tablets or phablets (screen size between 4 and 7 inches) will still be able to download my app, but at least the newest tablets with xlarge screen (larger than 7 inches) won’t. In my case this is acceptable… hope this helps and…
…please comment if I’m wrong about the zone between 4 and 5 inches which is not really clear to me. Which is normal or large…