Can I modify only the legend of a geom_point() scatter plot (in R / ggplot2) so the entries have particular shapes?

Question

This answer explains how to create a legend that is independent of the plotted data in R and ggplot2. The legend there only uses colors. I would like to also use marker shapes (and readjust the size to a bit larger in order to make them distinguishable more easily).

It was my understanding that the shapes and sizes in a ggplot2 legend could be controlled using guides(shape = guide_legend(override.aes = list(shape = [...]))) (as suggested in, e.g., this answer).

Unfortunately, I did not succeed. The modified code below unfortunately still results in the same figure as the original (from here) with no impact of shape and marker size specifications in guides(). (Note that the question is about the legend and only the legend, not the plot itself. For the plot itself, changing shape and markers is easy ans works nicely.)

Am I doing something wrong? Or am I misunderstanding how legends work (e.g., can I only have one attribute per legend) or how guide_legend modifies legends?

library(tidyverse)
the_colors <- c("#e6194b", "#3cb44b", "#ffe119", "#0082c8", "#f58231", "#911eb4", "#46f0f0", "#f032e6", 
                "#d2f53c", "#fabebe", "#008080", "#e6beff", "#aa6e28", "#fffac8", "#800000", "#aaffc3", 
                "#808000", "#ffd8b1", "#000080", "#808080", "#ffffff", "#000000")

color_df <- data.frame(the_colors, the_labels = seq_along(the_colors), the_shapes=c(1:22))

the_df <- data.frame("col1"=c(1, 2, 2, 1), "col2"=c(2, 2, 1, 1), "col3"=c(1, 2, 3, 4))

the_plot <- 
  ggplot() + 
    geom_point(data = color_df, aes(x = the_df$col1[[1]], y = the_df$col2[[1]], color = the_colors)) +
    guides(shape = guide_legend(override.aes = list(shape = color_df$the_shapes, size=5))) +
    scale_color_identity(guide = 'legend', labels = color_df$the_labels) +
    geom_point(data=the_df, aes(x=col1, y=col2), color=the_colors[[4]], shape=the_df$col3, size=5) 

print(the_plot)
ggsave("test_plot.pdf", plot=the_plot, width=10, height=8)

Note: I fully realize that this code does not use R and ggplot2 in the way it was intended. This is often met with disapproval. However, I am looking for exactly this feature, and it is is not provided as such in R / ggplot2: Control of the legend independent from the plot and without interfering with the plot.