SwiftUI custom map annotations are not shown continuously during pan and zoom

Question

Setup:

My app has a SwiftUI Map that can show annotations and the user location. It is possible to pan or zoom the map. Here is the relevant part of my code:

struct MapViewSWUI: View {
    @Binding var show_map_modal: Bool
    @ObservedObject var mapViewModel: MapViewModel
    @State private var region: MKCoordinateRegion
    // …  

var body: some View {   
  // …  
        let placesForBuyLocationNames = mapViewModel.placesForBuyLocationNames()
        let annotations = placesForBuyLocationNames.map { Annotation(name: $0.name, coordinate: $0.coordinate) }
        Map(coordinateRegion: $region,
                interactionModes: .all,
                showsUserLocation: true,
                annotationItems: annotations) { annotation in
            MapAnnotation(coordinate: annotation.coordinate, anchorPoint: CGPoint(x: 0.5, y: 1)) {
                GeometryReader { geo in
                    AnnotationView(annotation: annotation,
                                                 region: region, 
                                                 mapViewModel: mapViewModel, 
                                                 annotationDragged: $annotationDragged)
                    .onAppear {
                        mapViewModel.annotationNamesAndOrigins[annotation.name] = geo.frame(in: .global).origin
                    }
                }
                .frame(width: 30, height: 30)
            }
        }
}

Problem:

When annotations are shown, and the map is panned or zoomed, the user location is shown continuously during the operation. However this is not true for the custom annotations:
Simulator: The custom annotations blink during the operation, as if they are repeatedly removed and added again to the map.
Device: The custom annotations disappear during the operation. When the operation is finished, the annotations re-appear again.

Question:

How do I have to modify the code so that custom annotations are shown continuously like the user location?