Flutter animated list index issue

Question

I am making an animated list with Flutter, where you can click an item in the list to remove it. Sometimes the wrong item is removed from the list though. You can observe the described behavior in the end of this video. It seems that I'm clicking the item in the animated list before it has been rebuilt with the correct index. Thus, i'm clicking the item with the former index at that position. For example, I click the item at index 0, and then, before the list items are rebuilt, I click the item that has taken the place of the removed item before the removed item's animation completes. Thus, I'm clicking the item which has the index one, but is actually now displayed at index 0.

I've tried a few things, like tracking if an item is being removed at the moment. Sadly, without success.

Now to my question: Does anyone have a clue on how I can fix this issue?

My code can be found here.

class TodoItemWidget extends StatelessWidget {
  TodoItemWidget(this.item, this.listKey, this.index, {this.clickable = true});

  final GlobalKey<TodoItemListWidgetState> listKey;

  final TodoItem item;

  final int index;

  final bool clickable;

  void complete() {
    //int index = listKey.currentState.list.indexOf(item);
    listKey.currentState.removeItem(index);
  }

  @override
  Widget build(BuildContext context) {
    bool deadlinePassed = DateTime.now().isAfter(item.deadline);
    Color backgroundColor = deadlinePassed ? Colors.red : null;
    Color textColor = deadlinePassed ? Colors.white : Colors.black;

    return Column(
      children: <Widget>[
        Material(
          color: backgroundColor,
          child: Padding(
            padding: EdgeInsets.symmetric(horizontal: 32.0),
            child: Row(
              children: <Widget>[
                Expanded(
                  child: Padding(
                    padding: EdgeInsets.symmetric(vertical: 16.0),
                    child: Text(item.title, style: Theme.of(context).textTheme.title.copyWith(color: textColor)),
                  ),
                ),
                SizedBox(width: 16),
                Padding(
                  padding: EdgeInsets.only(top: 4.0, bottom: 12.0),
                  child: Column(
                    children: <Widget>[
                      IconButton(
                        onPressed: clickable ? complete : null,
                        icon: Icon(Icons.check),
                        color: textColor,
                      ),
                      Text(_getDateString(item.deadline), style: Theme.of(context).textTheme.body1.copyWith(fontSize: 12.0, color: textColor),),
                    ],
                  ),
                ),
              ],
            ),
          ),
        ),
        SizedBox(height: 1.0),
      ],
    );
  }

  String _getDateString(DateTime date) {
    return DateFormat("dd.MM.yyyy").format(date);
  }

}