Border box content changes position

Question

From my understanding, content inside of a block with box-sizing set to border-box, should not affect that block's positioning.

However, adding content to a box, appears to move that box downward. Can someone please explain what's going on here?

  div {
    display: inline-block;
    box-sizing: border-box;
    width: 64px;
    height: 64px;
    background-color: red;
  }

<div></div><div></div>
<hr />
<div>a</div><div></div>
<hr />
<div>a</div><div>a</div>
<hr />
<div></div><div>a</div>