Totally normal. I would try to illustrate it (using only width)
let's imagine you have (viewBox 0-24, width = 100)
| - A B - | - C D - | - E F - | - G H - |
0 6 12 18 24
When you limit (viewBox 0-12, width : 100 (unchanged)), it looks bigger, you get :
| - A B - | - C D - |
0 6 12
When you limit (viewBox : 0-12, width : 50), you get :
| - A B - | - C D - |
0 6 12
This could be see as Clipping :

If you did not want it looks bigger, you should indicate the width of your box. And if you want to display 2 times less content, you should indicate a width 2 times smaller.
Finally what you need to understand is :
- a SVG is a set of points in a 2D space
- viewBox attribute is the coordinates of the min/max displayed coordinates
On the other hand :
- you can give this rectangle (the viewBox) the size you want when you specify width/height.
Without specifying anything, you get a default width and the less you display, the bigger it is (to fill all available space).