If you control the whole network stack, you can perform DNS service discovery (DNS-SD) through Android's NsdManager. I recommend this guide from the documentation:
https://developer.android.com/training/connect-devices-wirelessly/nsd
In short, it boils down to:
mNsdManager.discoverServices(
SERVICE_TYPE, NsdManager.PROTOCOL_DNS_SD, mDiscoveryListener);
You can cross-reference the service you wish to resolve to by inspecting the response from dns-sd
on Mac:
$ dns-sd -Z . .
_http._tcp PTR myserver._http._tcp
myserver._http._tcp SRV 0 0 80 myserver.local. ; Replace with unicast FQDN of target host
myserver._http._tcp TXT ""
In the NsdManager.DiscoveryListener callback, you can obtain the corresponding IP address from NsdServiceInfo.getHost()
.