Unfortunately there's not a really good way of removing tablets. I ended up just turning those devices off directly in the developer console.
For disabling screen sizes in the manifest you need to use this
<compatible-screens>
<!-- all small size screens -->
<screen android:screenSize="small" android:screenDensity="ldpi" />
<screen android:screenSize="small" android:screenDensity="mdpi" />
<screen android:screenSize="small" android:screenDensity="hdpi" />
<screen android:screenSize="small" android:screenDensity="xhdpi" />
<!-- all normal size screens -->
<screen android:screenSize="normal" android:screenDensity="ldpi" />
<screen android:screenSize="normal" android:screenDensity="mdpi" />
<screen android:screenSize="normal" android:screenDensity="hdpi" />
<screen android:screenSize="normal" android:screenDensity="xhdpi" />
</compatible-screens>
instead of what you have listed. The problem is this will also remove large phones, however in the case of google cardboard that might be ok since some of those phones won't fit in cardboard anyway.