If we check the documentation we will read this:
middle
Aligns the middle of the element with the baseline plus half the x-height of the parent.
So we need to identify these values in order to understand the alignement. If we refer to the above figure we can clearly see the baseline and we can also see the line-height (defined by 100px in our case). You can aslo notice that the middle is not the middle of the div but the middle of the text defined by the different values (font-family, font-size, etc).
To use easy words: your reference of alignment is no the div but the text inside the div.

To make it easier let keep the line-height
with the default value and define a font-size
instead (the line-height will be then equal to the font-size):
div {
border: 1px solid gray;
font-size:50px;
}
img {
height: 46px;
vertical-align: middle;
}
<div>
<img src="https://upload.wikimedia.org/wikipedia/commons/0/02/Russula_emetica_117475.jpg"> Foo
</div>
with a different font-family
<div style="font-family:arial">
<img src="https://upload.wikimedia.org/wikipedia/commons/0/02/Russula_emetica_117475.jpg"> Foo
</div>
You can cleary see that the middle
is far from the middle of the div and if we change the font-family
the alignment will also change.
In order to align content inside a div that contain text better rely on flexbox for example:
div {
border: 1px solid gray;
font-size:50px;
display:flex;
align-items:center;
}
img {
height: 46px;
}
<div>
<img src="https://upload.wikimedia.org/wikipedia/commons/0/02/Russula_emetica_117475.jpg"> Foo
</div>