Create fixed element that has different background color on different background elements?

Question

I am currently trying to create a floating, fixed element that will have a different color, based on the element it is currently floating above (see image below). As I scroll, the background and thus the element should slowly change.

My original idea was to create this effect using z-index and using two different floating elements but this doesn't seem to be possible (see snipped below)

body, html {
  margin: 0;
}

/* BACKGROUNDS */
.background {
  height: 90vh;
  width: 100vw;
}
.background-light {
  background-color: #EBEBEB;
  
  z-index: 1;
}
.background-dark {
  background-color: #212121;
  z-index: 3;
}

/* OVERLAY  */
.overlay {
  position: fixed;
  top: 50vh;
  left: 50vw;
  transform: translate(-50%, -50%);
  padding: 1.5rem;
  border-radius: 1rem;
}
.over-light {
  background-color: #212121;
  z-index: 2;
}
.over-dark {
  background-color: #EBEBEB;
  z-index: 4;
}

.circle {
  width: 40px;
  height: 40px;
  background-color: #6ef250;
  border-radius: 100%;
}

<div>
  <div class="background background-light">
  </div>
  <div class="background background-dark">
  </div>
</div>
<div>
  <div class="overlay over-light">
    <div class="circle"></div>
  </div>
  <div class="overlay over-dark">
    <div class="circle"></div>
  </div>
</div>

How could one achieve this effect?