I am looking for such a thing as well. I just found http://www.3dar.us/ which seems like a good place to start for iOS. It isn't open source, which is a bummer.
Edit:
I have tried out Mixare, and I am really impressed by it. I have compiled it for Android. The only part that gave me a hard time was making sure the project was using the Google API v10, and that I had it installed.
Mixare does easily support Points of interest (POI) Check out their wikipage for more about it, basically you just feed it JSON. I am personally going use CouchDB with GeoCouch to serve up data, with free hosting provided by iriscouch
The JSON should look like the following:
{
"status": "OK",
"num_results": 1,
"results": [
{
"id": "2827",
"lat": "46.43893",
"lng": "11.21706",
"elevation": "1737",
"title": "Penegal",
"distance": "9.756",
"has_detail_page": "1",
"webpage": "http%3A%2F%2Fwww.suedtirolerland.it%2Fapi%2Fmap%2FgetMarkerTplM%2F%3Fmarker_id%3D2827%26project_id%3D15%26lang_id%3D9"
}
]
}
The most wonderful thing is that it is GPLed!