2

I am having issues getting TSC offsetting to work with my Hypervisor. According to the intel manuals you have to make sure the VM doesn't exit on rdtsc, so I disabled rdtsc_exiting. I also enabled the use tsc offsetting control. Setting up the VMCS this way should allow me to write to the tsc_offset control field to alter how the VM (guest OS) reads the TSC.

What I did was right before I give control back to the VM in the exit handler, I write -2000 to the TSC offset field.

__vmx_vmwrite(Vmcs::kTscOffset, -2000);

I am using this as my VmWrite implementation:

inline unsigned char __vmx_vmwrite(_In_ size_t field, _In_ size_t field_value) {
  FlagRegister flags = {};
  __asm {
    pushad
    push field_value
    mov eax, field

    _emit 0x0F
    _emit 0x79
    _emit 0x04
    _emit 0x24  // VMWRITE EAX, [ESP]

    pushfd
    pop flags.all

    add esp, 4
    popad
  }
  if (flags.fields.cf) {
    return 2;
  }
  if (flags.fields.zf) {
    return 1;
  }
  return 0;
}

According to the intel manuals, this should be enough to be able to use TSC offsetting. So to test I wrote this small test program (a CPUID is doing a VMEXIT in my case):

auto a1 = __rdtsc();
__cpuid(cpuInfo, 0);
auto a2 = __rdtsc();
result = static_cast<int>(a2 - a1);

On a barebone machine, this takes roughly 120 CPU cycles. When run from within my VM and rdtsc exiting disabled (no offsetting enabled) it takes 2200 cycles to complete.

The last test is then implementing what I described above and run the same test. Which in my case ends up with the same 2200 cycles.

Any idea why the Guest OS ignores any offsets I put in?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Steffen Brem
  • 1,738
  • 18
  • 29
  • Are you sure you're setting a scale-factor, and not just an offset? An offset would affect `a1` and `a2` equally, so it doesn't change the difference. (Hardware supports both, with scaling only in more recent hardware. See https://stackoverflow.com/questions/35137786/there-is-any-way-to-trigger-a-legacy-mode-for-rdtsc for offset vs. scaling, specifically the link to https://www-ssl.intel.com/content/www/us/en/processors/timestamp-counter-scaling-virtualization-white-paper.html). – Peter Cordes Jul 17 '17 at 03:43
  • BTW, if the code between the `__rdtsc()` functions doesn't itself trigger a vm exit (like CPUID does, I assume), you should measure the same time as on bare metal, which would be a good way to verify that `rdtsc` isn't causing VM exits. A `for(int i=0; i<1000; i++){}` loop should run in almost exactly 1000 cycles, assuming it compiles to `.loop:` `inc ecx`/`cmp ecx,1000`/`jne .loop` or similar. – Peter Cordes Jul 17 '17 at 03:44
  • In my case rdtsc does not trigger an exit. CPUID does. My goal is to adjust the TSC in a way that the guest would never think that the CPUID took more cycles than usual. I have tried doing the following at the end of my vmexit handler: `vmwrite(tscOffset, vmread(tscOffset) - vmExitCycles)`. It should always compensate for the amount of vmexit cycles and the guest would not know it took actually longer. In reality it does work, but made the guest OS flicker and it hanged few moments after. Should I be using scaling instead of offsetting in my case? – Steffen Brem Jul 17 '17 at 04:50
  • Oh, I see. Your TSC-offset code should be running in the vmexit caused by CPUID, so it looks 2000 cycles shorter than it was. That makes sense, if it works. I just didn't grok what you were trying to do the first time I read it. Maybe put your real goal at the top of the question, so people will know what it's for while reading the code. I haven't messed around with hypervisors, so I don't have any great ideas, sorry. – Peter Cordes Jul 17 '17 at 04:55

0 Answers0