How Can I Optimally Draw Lines Using Flutter's CustomPainter Widget?

Question

I am attempting to figure out the most efficient way to draw lines with the CustomPainter widget.

I have everything set up and it works great as I pass in points to the custompainter and it paints all of the lines perfectly using this class:

class MyPainter extends CustomPainter {

  List<Offset?> points;
  MyPainter({required this.points});

  @override
  void paint(Canvas canvas, Size size) {
    Paint background = Paint()..color = Colors.grey.shade50;
    Rect rect = Rect.fromLTWH(0, 0, size.width, size.height);
    canvas.drawRect(rect, background);

    Paint paint = Paint()
      ..color = Colors.black
      ..strokeCap = StrokeCap.round
      ..strokeWidth = 3;

    for(int i = 0; i < points.length - 1; i++){
      if(points[i] != null && points[i + 1] != null){
        canvas.drawLine(points[i]!, points[i + 1]!, paint);
      }
      else if (points[i] != null && points[i + 1] == null){
        canvas.drawPoints(PointMode.points, [points[i]!], paint);
      }
    }
  }

  @override
  bool shouldRepaint(CustomPainter oldDelegate) {
    return true;
  }
}

My issue becomes apparent when the number of points becomes increasingly larger with time. I have 2 dials that the user spins and every frame it changes the "cursor" position based on the dial that is being spun for dx and dy respectively. Every time the dial changes it's angle and the cursor position changes this function is called:

    void cursorChange(double distanceX, double distanceY){
      setState(() {
        cursor = Offset(distanceX, distanceY);
        points.add(cursor);
        points = removeDuplicatePoints(points);
      });
    }

Which repaints the canvas appropriately based on the points list. When this list becomes bigger it lowers performance when it really shouldn't. For example if I rotated the left dial 100 degrees, it would move the cursor to the right by 100, which should draw a line from origin to where the cursor is. Logically, because it's a line you'd only need 2 points, but because I have to redraw the new line at the new cursor spot, my logic just adds another point resulting into like 200 points for one line. It is most easiest to imagine this like an etch-a-sketch game!

In Short: I need to find an algorithm that optimally draws different lines while ignoring any points that are on an existing line.

I have tried just removing all of the points in between the origin and the current cursor to make the line but the user can also move vertically creating a new line. 2 things prevent this solution:

1. The user can move backwards too and deleting the points in between would just remove the furthest point and make it smaller.
2. If the user moved the dial vertically, the line would just rotate to the cursor

I then tried to patch these issues with some conditionals to check for direction, then add a null point and treat it like a new origin. This only led me down a game of cat and mouse that seemed to never end.

Any help would be so incredibly appreciated.