Glimmer does not bypass the download/parse/compile if just change the language of the template from JS or text to a binary bytecode, so download it is a lot smaller and faster to download and parse/compile is done by the Glimmer VM instead of JS VM with at simple grammar that is really fast to parse and execute, also in the future the VM can be moved to WebAssembly to make it even faster.
Glimmer VM is just the VM that executes the bytecode but it is Glimmer the one that handles the state so when it wants to render a component it takes that component template and execute the template bytecode in the VM. Rendering the template program and also generate the update program bytecode and it is stored associated to the component so when the components needs a rerender only the update program is executed in the VM.
In a Virtual Dom enviroment updates call functions that modify the virtual dom and then, instantly or when the draw system ticks (to batch updates), it will diff the virtual dom against the previous one and update the nodes that need to be updated.
Virtual DOM makes a lot of sense in react as it does not use templates, the components JSX is transpiled to JS code that use the react api to interact with the DOM (check this).
Angular Ivy (the new view engine of Angular) seems to be similar to Glimmer in that it does create some kind of Ivy code instead of full JS but it does not create the update program as Glimmer. (Not really sure that's what I get from the Ivy articles I found around).
Glimmer and Angular do something similar to Virtual DOM but at the component level, they control the changes and only rerender components that have changed their data. The main difference is that Glimmer already knows what has ben drawn and executes the update template bytecode generated previusly instead of an teardown and full render.