Flutter Zoomable Graphic/Chart

Question

I'm trying to create a ... thing. It's a vertical axis with labels showing heights of famous buildings, mountains etc.

The container should have a fixed height, but the 'chart' itself should be zoomable within the container, so you can zoom out to view all or in to a selected point.

I can't seem to find any guides to building custom charts, setting coordinate systems etc and zooming in and out, and I'm not even sure if custompainter is the best way to go.

EDIT: Following the comment below I now have the following.

ScreenCap: Gif of current state

Problems are: Only the Y axis should zoom, the points should stay the same size. i.e the points get further apart but do not scale.

I think I need to use a transform rather than scale, but I don't know how to do that (transform maths confuses the hell out of me lol).

Code:

  class _TotalHeightGraphState extends State<TotalHeightGraph> {
      @override
      Widget build(BuildContext context) {
        return Container(
            height: 300,
            color: Colors.white,
            padding: EdgeInsets.all(16),
            child: Transform.scale(
              scale: -1, //transform: Matrix4.diagonal3Values(1, 1, 1.0),
              child: ZoomableWidget(
                child: Container(
                  color: Colors.amber,
                  child: CustomPaint(
                    painter: MountainAxis(),
                  ),
                ),
              ),
            ));
      }
    }

class MountainAxis extends CustomPainter {
  @override
  void paint(Canvas canvas, Size size) {
    final pointStyle = Paint()
      ..style = PaintingStyle.fill
      ..color = Colors.red;

    canvas.drawCircle(Offset(0, 0), 5, pointStyle);
    canvas.drawCircle(Offset(0, 50), 5, pointStyle);
    canvas.drawCircle(Offset(0, 100), 5, pointStyle);
  }

  @override
  bool shouldRepaint(CustomPainter oldDelegate) {
    // TODO: implement shouldRepaint
    return null;
  }
}

class ZoomableWidget extends StatefulWidget {
  final Widget child;

  const ZoomableWidget({Key key, this.child}) : super(key: key);
  @override
  _ZoomableWidgetState createState() => _ZoomableWidgetState();
}
class _ZoomableWidgetState extends State<ZoomableWidget> {
  Matrix4 matrix = Matrix4.identity();

  @override
  Widget build(BuildContext context) {
    return MatrixGestureDetector(
      shouldRotate: false,
      shouldTranslate: false,
      focalPointAlignment: Alignment.topLeft,
      onMatrixUpdate: (Matrix4 m, Matrix4 tm, Matrix4 sm, Matrix4 rm) {
        setState(() {
          matrix = m;
        });
      },
      child: Transform(
        transform: matrix,
        child: widget.child,
      ),
    );
  }
}