Lesson 13.4Physical Memory, Machines, and Hardware

Guest Execution and Address Translation

A guest relies on virtualization support and two-stage address translation; walking one address shows both the isolation and its cost.

Lesson 13.4

Read or listen Print chapter or book

About 10 minutes

Cheatsheet for Lesson 13.4Key ideas, reference and questions

A one-page summary of “Guest Execution and Address Translation” you can keep beside your notes. Plain Markdown.

Lesson 13.4 — Guest Execution and Address Translation: cheatsheet

Remember: Translate guest virtual to guest physical, then guest physical to host physical; keep the owner of each map explicit.
Also useful: The TLB is a notebook of recent final translations, avoiding a fresh walk through every table whenever the same page is used again.

Key terms

trap and emulate: This is called trap and emulate: the guest's ordinary instructions run at full speed, and only the sensitive ones cost extra.
popf: The classic example is popf, which loads the flags register, including the interrupt flag that switches interrupts on and off.
binary translation: VMware, from 1999, examined the guest kernel's code as it ran and rewrote the problem instructions into safe sequences, a technique called binary translation.
paravirtualization: Xen, from 2003, modified the guest kernel's source code so that it asked the hypervisor directly instead of running those instructions, which is called paravirtualization.
frame: A frame is a page-sized block of physical memory.

Review

Q: Before VT-x and AMD-V, why could a hypervisor not simply run a guest kernel in ring 3 and emulate each privileged instruction when it faulted? A: Instructions like popf quietly act differently in ring 3 and never fault, so the hypervisor never sees them
Q: A guest's own page tables map guest virtual page 0x403 to guest frame 0x1F7, and the hypervisor maps guest frame 0x1F7 to host frame 0x52E3. Pages are 4 KiB. Which host physical address holds the byte at guest virtual address 0x0040_3F00? A: 0x052E_3F00

Download cheatsheet (.md)

Lesson 13.3 described what a hypervisor does: it runs a guest’s instructions natively, takes control back through VM exits, and translates every guest address a second time. This lesson looks underneath. It shows why x86 needed new hardware before a guest’s kernel could run unmodified, follows one address through both stages of translation digit by digit, counts what the second stage costs, and ends with the ways software inside a guest can tell that it is there.

Why a guest cannot simply run in user mode

The obvious way to build a hypervisor is to run the guest’s kernel in ring 3, where every privileged instruction it tries causes a fault, and have the hypervisor catch each fault and do to the virtual machine what the instruction would have done to a real one. This is called trap and emulate: the guest’s ordinary instructions run at full speed, and only the sensitive ones cost extra.

In 1974 Gerald Popek and Robert Goldberg stated the condition for trap and emulate to work: every instruction that reads or changes the machine’s control state must fault when it runs outside the most privileged level. For many years x86 did not meet it. The classic example is popf, which loads the flags register, including the interrupt flag that switches interrupts on and off. In ring 0, popf changes the interrupt flag. In ring 3 it silently leaves the flag alone and raises no fault. A guest kernel running in ring 3 that used popf to switch interrupts back on would carry on believing they were on, and the hypervisor would never find out, because nothing trapped.

Early x86 hypervisors worked around the problem. VMware, from 1999, examined the guest kernel’s code as it ran and rewrote the problem instructions into safe sequences, a technique called binary translation. Xen, from 2003, modified the guest kernel’s source code so that it asked the hypervisor directly instead of running those instructions, which is called paravirtualization. Both worked, and both were complicated. The hardware support from Lesson 13.3, VT-x and AMD-V, made both workarounds unnecessary for running an unmodified kernel: it gives the guest its own ring 0, where popf behaves exactly as the guest kernel expects.

A worked translation

Lesson 13.3 split a guest’s address translation into two stages. The guest’s own page tables turn a guest virtual address into a guest physical address, and the hypervisor’s tables turn that into a host physical address, the real location in RAM. Here is one address going through both:

Pages are 4 KiB, which is 4 × 1,024 = 4,096 bytes. Since 2¹² = 4,096, the lowest 12 bits of an address choose a byte within its page, the offset, and all the bits above them name the page. Each hex digit stands for 4 bits, so the 12 offset bits are exactly the last three hex digits; Lesson 6.5 made the same split. Take the guest virtual address 0x0040_3A2C:

guest virtual page 0x00403

offset 0xA2C

The guest virtual address 0x0040_3A2C, split before its last three hex digits.

The offset is 0xA2C = 10 × 256 + 2 × 16 + 12 = 2,560 + 32 + 12 = 2,604, safely less than 4,096. The page number is 0x403 = 4 × 256 + 3 = 1,027. As a check, page 1,027 starts at byte 1,027 × 4,096 = 4,206,592, and 4,206,592 + 2,604 = 4,209,196, which is 0x0040_3A2C written in decimal.

Stage 1. The guest kernel’s page tables map guest virtual page 0x403 to guest physical frame 0x1F7. A frame is a page-sized block of physical memory. Translation replaces the page number with the frame number and keeps the offset.

Multiplying by 0x1000, which is 4,096, shifts a number left by three hex digits, so 0x1F7 × 0x1000 = 0x1F_7000, and adding the offset gives 0x1F_7A2C. On a real machine the translation would end here, with a RAM address. In a guest, 0x001F_7A2C is only the guest’s idea of one.

Stage 2. The hypervisor’s tables map guest physical frame 0x1F7 to host physical frame 0x52E3.

0x52E3 × 0x1000 = 0x52E_3000, and adding 0xA2C gives 0x052E_3A2C, where the byte really is in RAM. The offset rode through both stages untouched; each stage only swapped the number above it.

Predict first

The guest’s tables also map page 0x404 to guest frame 0x2A0. The hypervisor’s tables map frame 0x1F7 to host frame 0x52E3 and have no entry for frame 0x2A0.

The guest reads 0x0040_4010, one page above the lesson’s address. What happens?

Answer: Stage 2 has no entry, so the processor makes a VM exit. Stage 1 works: page 0x404 becomes frame 0x2A0, so the guest physical address is 0x2A0 × 0x1000 + 0x010 = 0x002A_0010. Stage 2 finds no entry for 0x2A0 and hands control to the hypervisor.

Translate a guest address twiceGuest tables: page 0x403 → frame 0x1F7, page 0x404 → frame 0x2A0. Hypervisor tables: frame 0x1F7 → host frame 0x52E3, and no entry for 0x2A0.

Two lookups, one offset

page 0x403 (orange) · offset 0xA2C (blue)

Lookup

page 0x403

→ frame 0x1F7

keep the offset

Guest physical

0x001F_7A2C0x1F7 × 0x1000 + 0xA2C

Lookup

frame 0x1F7

→ host frame 0x52E3

keep the offset

Host physical (real RAM)

0x052E_3A2C0x52E3 × 0x1000 + 0xA2C

Offset 0xA2C is the last three hex digits and rides through unchanged. Stage 1 swaps page 0x403 for frame 0x1F7; stage 2 swaps that for host frame 0x52E3, giving 0x052E_3A2C.

Keep the page offset through two mappings. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

guest_virtual_page = 0x403
split the selected 12-bit offset
guest table maps page0x403 to frame 0x17
hypervisor maps frame 0x17 to host frame
combine host frame and unchanged offset
report all three addresses

Result: GVA 0x403A2C -> GPA 0x17A2C -> HPA 0xA4A2C. Both mapping owners matter.

Final values: frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C; guest_physical = 0x17A2C; host_frame = 0xA4; host_physical = 0xA4A2C.

Read every step of the lesson’s example

Step 1, line 1. Start with the selected inputs. Follow the highlighted operation and the changing state below.
frame = 164; offset = 2604.
Step 2, line 1. These addresses follow one small worked mapping, not a full hardware page walk.
frame = 164; offset = 2604; guest_virtual = 0x403A2C.
Step 3, line 2. The lowest 12 bits select the byte within its 4 KiB page.
frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C.
Step 4, line 3. The guest’s tables select guest physical frame 0x17.
frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C; guest_physical = 0x17A2C.
Step 5, line 4. The hypervisor supplies a separate second-stage frame.
frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C; guest_physical = 0x17A2C; host_frame = 0xA4.
Step 6, line 5. The second mapping changes the frame, not the within-page byte position.
frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C; guest_physical = 0x17A2C; host_frame = 0xA4; host_physical = 0xA4A2C.
Step 7, line 6. Result: GVA 0x403A2C -> GPA 0x17A2C -> HPA 0xA4A2C. Both mapping owners matter.
frame = 164; offset = 0xA2C; guest_virtual = 0x403A2C; guest_physical = 0x17A2C; host_frame = 0xA4; host_physical = 0xA4A2C.

What two stages cost

Real translation is not one table lookup per stage but a four-level walk (Lesson 13.1), and with two stages the walks multiply. The guest’s page tables live in guest physical memory, so each of the four entries the processor reads during the guest’s walk sits at a guest physical address that must first go through a four-level stage-2 walk. Each guest level therefore costs 4 reads to find its table plus 1 read of the entry itself, 4 + 1 = 5 reads, and 4 × 5 = 20 reads for the whole stage-1 walk. The guest physical address that comes out needs one more four-level stage-2 walk: 20 + 4 = 24 memory reads to translate one address, where a machine without virtualization needs 4.

The processor avoids paying that on every access with its translation lookaside buffer (TLB), a small cache of recent translations. The TLB keeps the final guest-virtual-to-host-physical answer, so a program that touches the same pages again, as programs mostly do, pays for the walk once.

How a program can tell it is in a virtual machine

A guest runs real instructions on a real processor, so most of what it does looks identical inside and outside a VM. A program that wants to know can still find out, in a few ways:

The processor says so. Hypervisors announce themselves on purpose. When a program runs cpuid with input 1, bit 31 of the result in ecx is the hypervisor present bit: 0 on bare hardware, set by a hypervisor. Bit 31 is worth 2³¹ = 2,147,483,648, which is 0x8000_0000, because the top hex digit 8 is binary 1000. A second input, 0x4000_0000, returns the hypervisor’s name as twelve ASCII characters, such as Microsoft Hv. This is a feature, not a leak: guests use it to find out which faster, hypervisor-aware drivers they can load.
The hardware looks virtual. Virtual disks, graphics adapters, and network cards report model names from the hypervisor’s vendor, and virtual network cards get addresses from ranges assigned to those vendors.
Some instructions are slow. Anything that causes a VM exit takes far longer than it would on bare hardware, and careful timing can notice.

Guest tools ask so they can install the right drivers. Licensing software asks. Malware asks so it can behave innocently inside the sandboxes that analysts use to study it. Some anti-cheat systems ask, and refuse to run a protected game in a VM, because a hypervisor can see and change everything a guest does. That is one more reason this book’s targets are offline, single-player builds running in your own VM.

The first check is weaker than it looks. On a Windows 11 PC with memory integrity on, Windows itself runs on the Windows hypervisor, so the hypervisor-present bit is set on an ordinary desktop too. Lesson 13.3 explains why.

References: hypervisor feature discovery and the virtual-machine extensions volume of the Intel Software Developer Manuals.

Before VT-x and AMD-V, why could a hypervisor not simply run a guest kernel in ring 3 and emulate each privileged instruction when it faulted?

Finished Lesson 13.4?

Mark it done and the lesson list shows a beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.

Chapter 13: 10 lessons

This book grew out of the original Game Hacking Academy by attilathedud and has been rewritten as a beginner-first course. Like the original repository, its source is published under the Apache License 2.0.

Guest Execution and Address Translation

Keyboard shortcuts