As for your second question, not all sites are made of fixed-width centered pages even if it became very common. Even in this case, you've to have pages of very different lengths from click to click to be annoyed by this shift (BTW it shifts by half the width of the scrollbar ;) that's 8px on WinXP with a classic Win2k theme).
- At the beginning there was HTML pages without CSS : left-aligned (or right-aligned in hebrew and arabic languages)
- Then came gradually HTML tables on 14" CRT: each pixel was precious! (full width because
640kB is enough few imagined how it'd be displayed on 17" or 21" CRT)
- Then came fixed-width centered tables or divs: pixels were still precious on 1024px displays. With heights <768px, most pages had a fold (or were made with Flash).
- Now comes CSS3 Media Queries, displays with widths from 320 to 2560px (per monitor!), progressive enhancement or graceful degradation.
- Pages like Google homepage or web apps are common: no need for a vertical scrollbar there!
Good luck with that to establish a rule if you're a browser vendor, especially given the recent disappearing of menus, bars, etc on Chrome and then Firefox and maybe other browsers that may indicate that vendors are chasing a goal of fullscreen display (I just press F11 if I want to and want my addressbar back but that's another story :) ) and that they will likely be reluctant of "losing" 20px horizontally.
On a side note, maybe you as a developer of this site are the most annoyed by this horizontal shift because you've your site displayed all day long in front of your eyes and are accustomed at clicking precisely in this and that areas without even thinking about it anymore: you know your site as nobody else!
This won't be the case of your visitors, only few of them will see that more than twice and you've to be web fluent to know where this phenomenom comes from, otherwise it's just minor annoyance like on a few other sites they visited. And you've to consider what could be most annoying for visitors:
- displaying a scrollbar that indicates at first sight that they must scroll (before they see on second sight that this vertical scrollbar is disabled so in fact there's no scroll)
- a 10px horizontal shift from time to time