Flutter zoomable image with overlays

Question

I'm trying to make a custom widget that the user can zoom and pan an image and tap to overlay points on the image. I think I'm close but can't quite get the points to line up under the tap.

I copied the relevant zoom-pan from here: https://github.com/flutter/flutter/blob/master/examples/layers/widgets/gestures.dart

I also have my GestureDetector pulled out into its own widget as per this post to get the correct RenderBox for global to local offset conversion flutter : Get Local position of Gesture Detector

import 'package:flutter/material.dart';

class ImageWithOverlays extends StatefulWidget {
  final List<FractionalOffset> fractionalOffsets;
  ImageWithOverlays(this.fractionalOffsets);

  @override
  _ImageWithOverlaysState createState() => _ImageWithOverlaysState();    
}


class _ImageWithOverlaysState extends State<ImageWithOverlays> {
  Offset _startingFocalPoint;
  Offset _previousOffset;
  Offset _offset = Offset.zero;

  double _previousZoom;
  double _zoom = 1.0;

  @override
  Widget build(BuildContext context) {
    return GestureDetector(
        onScaleStart: _handleScaleStart,
        onScaleUpdate:_handleScaleUpdate,
        onTapDown: (details){          
          setState(() {                        
            widget.fractionalOffsets.add(_getFractionalOffset(context, details.globalPosition));             
          });
        },
        onDoubleTap: _handleScaleReset,
        child: Transform(
        transform: Matrix4.diagonal3Values(_zoom, _zoom, 1.0) + Matrix4.translationValues(_offset.dx, _offset.dy, 0.0),         
        child: Center(
          child: Container(
              child:Stack(
              children: <Widget>[
                Image.network(
                  'https://picsum.photos/250?image=9',
                ), ]
                ..addAll(_getOverlays(context)),
            )),
          ),
      ));
     
  }

  void _handleScaleStart(ScaleStartDetails details) {
    setState(() {
      _startingFocalPoint = details.focalPoint;
      _previousOffset = _offset;
      _previousZoom = _zoom;
    });
  }


  void _handleScaleUpdate(ScaleUpdateDetails details) {
    setState(() {
      _zoom = _previousZoom * details.scale;

      // Ensure that item under the focal point stays in the same place despite zooming
      final Offset normalizedOffset = (_startingFocalPoint - _previousOffset) / _previousZoom;
      _offset = details.focalPoint - normalizedOffset * _zoom;
    });
  }

  void _handleScaleReset() {
    setState(() {
      _zoom = 1.0;
      _offset = Offset.zero;
      widget.fractionalOffsets.clear();
    });
  }

  List<Widget> _getOverlays(BuildContext context) {
    return widget.fractionalOffsets
        .asMap() 
        .map((i, fo) => MapEntry(
            i,
            Align(
                alignment: fo,
                child: _buildIcon((i + 1).toString(), context)
            )           
        ))
        .values
        .toList();
  }

  Widget _buildIcon(String indexText, BuildContext context) {
    return FlatButton.icon(      
      icon: Icon(Icons.location_on, color: Colors.red),
      label: Text(
        indexText,
        style: TextStyle(color: Colors.red, fontWeight: FontWeight.bold),
      ),      
    );
  }

  Widget _buildCircleIcon(String indexText) {
    return Container(          
          margin: const EdgeInsets.all(8.0),
          padding: const EdgeInsets.all(8.0),
          decoration: BoxDecoration(
            color: Colors.red,
            shape: BoxShape.circle,
          ),
          child:  Text(
                indexText,
                style: TextStyle(
                  fontSize: 16,
                  color: Colors.white,
                  fontWeight: FontWeight.bold
                ),
              ));
  }

  FractionalOffset _getFractionalOffset(BuildContext context, Offset globalPosition) {    
    var renderbox = context.findRenderObject() as RenderBox;
    var localOffset = renderbox.globalToLocal(globalPosition);
    var width = renderbox.size.width;
    var height = renderbox.size.height;
    return FractionalOffset(localOffset.dx/width,localOffset.dy/height); 
  }
}

But I'm having a rough time getting the layout right. I think I have two issues:

The image likes to jump to the upper left. I'm not sure how to get it to stay centered.
When I zoom in, my _getFractionalOffset function is not returning the correct fraction. The icons in this screenshot are not showing up under my tap My matrix math is poor, as is my understanding of how Flutter does its transforms, so I would appreciate any insight.

@pskink this looks like a nice wrapper for scaling and panning, but I need to get the location from GestureDetector's TapDownDetails — chanban, Jun 07 '19 at 22:43

score 2 · Accepted Answer · answered Jun 16 '19 at 22:08

Figured it out, I had a couple issues with my code that when compounded made debugging annoying.

First, I should have been multiplying the scale/translate matrices, not adding them:

vector.Matrix4 get _transformationMatrix {
  var scale = vector.Matrix4.diagonal3Values(_zoom, _zoom, 1.0);
  var translation = vector.Matrix4.translationValues(_offset.dx, _offset.dy, 0.0);
  var transform = translation * scale;
  return transform;
}

Second, I was trying to fuss around with doing the translating from local offset to the image frame instead of letting vector_math do it for me. The new overlay offset calculation:

Offset _getOffset(BuildContext context, Offset globalPosition) {    
  var renderbox = context.findRenderObject() as RenderBox;
  var localOffset = renderbox.globalToLocal(globalPosition);    
  var localVector = vector.Vector3(localOffset.dx, localOffset.dy,0);
  var transformed =  Matrix4.inverted(_transformationMatrix).transform3(localVector);
  return Offset(transformed.x, transformed.y);
}

Since _transformationmatrix takes a point from the image frame to the screen frame, I needed to left multiply the inverse to get from the screen frame (localOffset) to the image frame (returned value).

Finally, I'm using the Positioned widget instead of Align so I can set the positioning in pixels:

double _width = 100;
double _height = 50;
List<Widget> _getOverlays(BuildContext context) {
  return widget.points
      .asMap() 
      .map((i, offset) => MapEntry(
          i,
          Positioned(
              left: offset.dx-_width/2,
              width:_width,
              top: offset.dy -_height/2,
              height:_height,
              child: _buildIcon((i + 1).toString(), context)
          )           
      ))
      .values
      .toList();
}

where widget.points is a List<Offset> of offsets returned by _getOffset above, and width/height are the width and height of my icon widget.

Hope this helps someone!

Flutter zoomable image with overlays

1 Answers1