System Calls and Privilege Checks

Keyboard shortcuts

×

?

Open this guide

Alt + B

Show or hide the lesson list

Alt + O

Show or fold the lesson outline

Alt + N

Open or close notes

Alt + H

Toggle semantic colours

Alt + D

Toggle page brightness

Alt + C

Toggle code brightness

← →

Move through a focused code trace

Home End

First or final step in a focused trace

Escape

Close this guide or an open panel

Reader shortcuts pause while you type in a text field.

Cheatsheet for Lesson 12.6Key ideas, reference and questions

A one-page summary of “System Calls and Privilege Checks” you can keep beside your notes. Plain Markdown.

Lesson 12.6 — System Calls and Privilege Checks: cheatsheet

• Remember: An existing handle keeps its granted rights; a later operation cannot add permissions by asking more forcefully.
• Also useful: An access mask is a stencil: retain the requested bits that were granted, then confirm every required bit survived.

Key terms

• syscall: The most important of them is the syscall instruction, and the rest of this lesson follows one trip through it.
• stub: The ntdll function is a stub: a few instructions whose only job is to make the system call.
• eax: The number in eax is a system service number, an index into the kernel's table of routines.
• system service number: The number in eax is a system service number, an index into the kernel's table of routines.
• ret: The kernel places a status code in rax and executes sysret, the reverse of syscall: the level drops back to 3, and execution resumes at the address saved in rcx, which is the ret right after syscall in the stub.
• NTSTATUS: The status is an NTSTATUS, a 32-bit code whose top two bits give its severity: 00 success, 01 information, 10 warning, 11 error.
• STATUS_ACCESS_DENIED: STATUS_ACCESS_DENIED is 0xC000_0022; hex C is binary 1100, so its top two bits are 11, an error.
• C: STATUS_ACCESS_DENIED is 0xC000_0022; hex C is binary 1100, so its top two bits are 11, an error.
• STATUS_PARTIAL_COPY: STATUS_PARTIAL_COPY is 0x8000_000D; hex 8 is 1000, so its top two bits are 10, a warning, because some of the bytes did arrive.

Review

• Q: Kernel pages have their user/supervisor bit cleared, so ring-3 code faults if it touches them. Why must the kernel's NtReadVirtualMemory still check that the requested range and the caller's buffer lie in user space? A: The copy runs in ring 0, where supervisor pages are allowed, so an unchecked range could expose kernel memory
• Q: A 32-bit game running on 64-bit Windows shows cs = 0023 in x64dbg. At which privilege level is its code running? A: Ring 3, because 0x23 is 0010 0011 and its lowest two bits are 11

Download cheatsheet (.md)

Download cheatsheet (.md)

Lesson 12.4 showed the processor keeping user-mode code away from the kernel and drew one ReadProcessMemory call crossing into the kernel and back. This lesson opens both up. It reads the bits the processor actually checks, then follows that one call a layer at a time, with every value the kernel compares along the way.

The current level is two bits of a register

The processor keeps the current privilege level in the lowest two bits of the cs register, which tells the processor which code segment it is running. Two bits can hold 2 × 2 = 4 patterns, 00, 01, 10, and 11, which is exactly why there are four rings.

Open any 64-bit program in x64dbg and the register panel shows cs as 0033:

bit 70

bit 60

bit 51

bit 41

bit 30

bit 20

bit 11

bit 01

hex 3

hex 3

which code segment

level 11 = 3

cs = 0x33 in a 64-bit program. The upper bits choose a code segment; the lowest two bits are the current privilege level.

0x33 is 3 × 16 + 3 = 51. Each hex digit stands for four bits, so 0x33 is 0011 0011. Its lowest two bits are 11, which is 2 + 1 = 3: ring 3, user mode. The kernel’s own code segment is 0x10, which is 0001 0000; its lowest two bits are 00, ring 0.

A program cannot promote itself by loading 0x10 into cs. The processor refuses any change of code segment that would raise the privilege level, except through a few entry points that the kernel prepared in advance. The most important of them is the syscall instruction, and the rest of this lesson follows one trip through it.

One bit marks a page as the kernel’s

Lesson 12.4 said that a page-table entry’s user/supervisor bit, bit 2, decides whether ring 3 may touch the page. An entry combines the address of a physical page with flags describing that mapping. For the ordinary 4 KiB pages used here, the page address is aligned on a 4,096-byte boundary, leaving the low twelve bits available for flags. Chapter 12’s address-translation lesson will follow a complete walk; here we need only the permission flags in the lowest byte:

bit 70

bit 61

bit 51

bit 40

bit 30

bit 21

bit 11

bit 01

dirty

accessed

user

write

present

The lowest byte of a page-table entry for a user page, 0x67.

0x67 is 6 × 16 + 7 = 103, and 103 = 64 + 32 + 4 + 2 + 1, so bits 6, 5, 2, 1, and 0 are set. Bit 0 says the page is present, bit 1 that it may be written, and bit 2 that ring 3 may use it. The processor itself sets bit 5, accessed, the first time the page is used, and bit 6, dirty, the first time it is written. Bits 3, 4, and 7 control caching and are 0 here.

A page that holds kernel data looks almost the same:

bit 70

bit 61

bit 51

bit 40

bit 30

bit 20

bit 11

bit 01

dirty

accessed

user = 0

write

present

The same byte for a kernel page, 0x63. Only bit 2 differs, so ring 3 is refused.

0x67 − 0x63 = 4, and 4 = 2², so exactly one bit differs between these two example leaf entries: bit 2. A kernel page can be present, writable, and recently used, and still refuse access from ring 3. In a real page-table walk, the processor also considers permissions in the entries at higher levels; a single leaf byte is a teaching example, not a complete access decision.

Flip the user bit of a page-table leafA leaf byte carries permission flags. Bit 2 says whether user-mode code (ring 3) may use the page at this level.

The leaf byte and the decision

Leaf byte 0x67

Bits

bit 7bit 0

0bit 7

1bit 6

1bit 5

0bit 4

0bit 3

1user (bit 2)

1bit 1

1bit 0

Decision at this level

Ring 3 access

Bit 2 is set: user mode may use this page here

Not the whole decision

Higher levels

The page-table levels above this one also contribute to the final access decision.

0x67 has bit 2 set, so ring 3 is allowed by this level.

The same call, one layer at a time

Take a read like the ones in Lesson 6.3: a handle to the game opened with only query and read rights, and a request for 4 bytes at address 0x0050_1000. Lesson 12.4 drew the whole round trip as one sequence diagram, from your tool through kernel32.dll, kernelbase.dll, and ntdll.dll to the processor and the kernel, and back. This section follows that diagram one participant at a time.

Inside your process

ReadProcessMemory in kernel32.dll forwards to kernelbase.dll (Lesson 12.5), which calls NtReadVirtualMemory in ntdll.dll with the same arguments. The ntdll function is a stub: a few instructions whose only job is to make the system call. On 64-bit Windows its core is four instructions:

mov r10, rcx        ; the first argument moves out of rcx
mov eax, <number>   ; the system service number: which kernel routine to run
syscall             ; enter the kernel
ret                 ; back to kernelbase, with the status in eax

The first line exists because of the first thing syscall does. The Windows x64 calling convention passes a function’s first argument in rcx, but syscall overwrites rcx with the return address, so the stub moves the argument to r10 first, and the kernel knows to look for it there.

Narration: The system-call instruction overwrites rcx, so the stub preserves the first argument in r10. Missing that move would change what the kernel receives.

The number in eax is a system service number, an index into the kernel’s table of routines. The numbers are internal to Windows and change between builds, which is why this book never copies one into a program. In a debugger the stub also shows a test and a jump to an older entry instruction, int 2Eh, that some configurations use; either way, the processor arrives at an address the kernel chose.

Crossing over

The kernel’s entry code assumes that everything user mode handed it could be hostile. It switches from the program’s stack to a kernel stack that belongs to the thread, because the program’s stack pointer could point anywhere. It saves the user registers so they can be put back exactly. Then it looks the service number up in its table, rejects a number that is out of range, and calls the kernel’s own NtReadVirtualMemory.

Inside the kernel

Most of the kernel’s work is checking.

The address ranges must lie inside user space. The request covers 0x0050_1000 up to 0x0050_1000 + 4 = 0x0050_1004, and the tool’s buffer is also a user address. Both are far below the top of user space, which is a little under 0x0000_8000_0000_0000 on 64-bit Windows. A range that reached into the kernel’s half, or one long enough to run past the largest 64-bit number and wrap around to a small address, would be rejected here. The user/supervisor bit cannot do this job for the kernel: the copy runs in ring 0, and ring 0 may touch supervisor pages. So the kernel checks every range itself, and that check is why ReadProcessMemory cannot read kernel memory, whatever handle you hold.

The handle must carry the right the operation needs. The kernel finds the handle in the tool’s handle table and reads the rights granted when the handle was opened. Reading needs PROCESS_VM_READ, which is 0x0010. This handle was opened with PROCESS_QUERY_LIMITED_INFORMATION and PROCESS_VM_READ, so its rights are 0x1000 + 0x0010 = 0x1010. Written as bits, one byte at a time:

150

140

130

121

110

100

90

80

hex 1

hex 0

query

Bits 15 to 8 of the handle's rights, 0x1010. 0x1000 is 16³ = 4,096 = 2¹², so it is bit 12.

70

60

50

41

30

20

10

00

hex 1

hex 0

write

read

Bits 7 to 0. 0x0010 is 16 = 2⁴, bit 4, the read right. Bit 5, the write right, is 0.

The test is a single AND. The kernel keeps only the bits that are set in both the handle’s rights and the rights it needs, and allows the operation only if every needed bit survived. 0x1010 AND 0x0010 = 0x0010, the whole requirement, so the read is allowed.

Alternative: An access mask is a stencil: retain the requested bits that were granted, then confirm every required bit survived.

WriteProcessMemory needs PROCESS_VM_WRITE, 0x0020. That is 2 × 16 = 32 = 2⁵, bit 5, and bit 5 of 0x1010 is 0, so 0x1010 AND 0x0020 = 0: denied, whatever the tool’s author intended. The rights were fixed when the handle was created, and nothing done with the handle afterward can add to them.

TL;DR: An existing handle keeps its granted rights; a later operation cannot add permissions by asking more forcefully.

Only then does it copy. The source address belongs to the game’s address space and the destination buffer belongs to the tool’s. The kernel must interpret each in the correct process context while copying; the exact internal mapping and copy strategy is a Windows implementation detail. If a page becomes unavailable during the operation, the API may report a failed or partial read. The caller must check the returned byte count instead of assuming that asking for four bytes means four arrived.

Returning

The kernel places a status code in rax and executes sysret, the reverse of syscall: the level drops back to 3, and execution resumes at the address saved in rcx, which is the ret right after syscall in the stub.

The status is an NTSTATUS, a 32-bit code whose top two bits give its severity: 00 success, 01 information, 10 warning, 11 error. STATUS_SUCCESS is 0. STATUS_ACCESS_DENIED is 0xC000_0022; hex C is binary 1100, so its top two bits are 11, an error. STATUS_PARTIAL_COPY is 0x8000_000D; hex 8 is 1000, so its top two bits are 10, a warning, because some of the bytes did arrive. kernelbase.dll translates the status into a Win32 error code for GetLastError, 5 for access denied and 299 for a partial copy, and reports success or failure to your tool.

A 32-bit game takes one extra hop

The book’s Windows labs are 32-bit programs. A 32-bit process on 64-bit Windows uses WoW64 (“Windows 32-bit on Windows 64-bit”) to translate the call into the native 64-bit service path. That may require converting pointer-sized arguments and structures; it is not a blind widening of every number. The two modes use different code segments, which is why cs reads 0023 in a 32-bit program and 0033 in a 64-bit one. The four-instruction stub above illustrates the native 64-bit path, not every instruction a 32-bit caller runs.

Successful status and exact byte count differ. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

read the selected inputs
evaluate each stated condition
combine the required conditions
perform the effect only when accepted
otherwise retain the old state
report the result and its limits

Result: all four bytes arrived in the tool’s separate buffer.

Final values: right = yes; copied = 4; status = yes; read_granted = true; status_success = true; requested = 4; exact = true; accepted = true; decoded = complete four-byte value.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs and follow how the next operation changes the state.

   right = yes; copied = 4; status = yes.

2. Step 2, line 2. A caller must check permission, status and the count before interpreting a complete field.

   right = yes; copied = 4; status = yes; read_granted = true; status_success = true; requested = 4; exact = true.

3. Step 3, line 3. Accept only when the required conditions all hold.

   right = yes; copied = 4; status = yes; read_granted = true; status_success = true; requested = 4; exact = true; accepted = true.

4. Step 4, line 4. The model performs the accepted effect.

   right = yes; copied = 4; status = yes; read_granted = true; status_success = true; requested = 4; exact = true; accepted = true; decoded = complete four-byte value.

5. Step 5, line 6. Result: all four bytes arrived in the tool’s separate buffer.

   right = yes; copied = 4; status = yes; read_granted = true; status_success = true; requested = 4; exact = true; accepted = true; decoded = complete four-byte value.

Watch it happenFour bytes cross address spaces after the kernel checks them

A native 64-bit read request moves from user-mode wrappers into the controlled kernel entry. Windows checks the four-byte range and read rights, copies four symbolic bytes from game memory into a separate tool buffer, and returns status zero and byte count four for the caller to check.

Read the steps

1. The tool requests four bytes starting at game address 0x00501000. Its destination buffer is separate. The symbols b0 through b3 stand for those four bytes, not chosen byte values.
2. ReadProcessMemory passes through kernel32 and kernelbase to the ntdll stub. The moving request remains above the privilege boundary: every wrapper is still user-mode code.
3. syscall enters the prepared kernel path. Entry code saves the user context, switches to the kernel stack, validates the service number, and dispatches this request.
4. The four-byte range ends at 0x00501000 + 4 = 0x00501004. The existing handle passes the read-right check: 0x1010 AND 0x0010 equals the full required 0x0010.
5. Windows interprets the source in the game address space and the destination in the tool address space. Copies of all four bytes travel to the tool’s buffer; the game keeps its bytes.
6. This successful path returns NTSTATUS 0 and a byte count of 4 through sysret and the application wrappers. The tool checks both success and the returned count; a partial copy would not make a four-byte value valid.

b0–b3 identify the requested bytes without inventing their contents. Their moving copies illustrate process context; the exact Windows copy implementation is unspecified.

References: process security and access rights, using NTSTATUS values, and the Intel Software Developer Manuals for syscall, privilege levels, and page-table bits.

The process handle in this example authorized a copy between two processes. The processor supplied a controlled entrance; Windows checked the request inside it. The next lesson returns to the process view and saves a debugging snapshot of threads, modules, and registers in a dump. Chapter 12 then follows requests across DLL and driver boundaries, where the same checks on authority, arguments, and returned results still apply.

🧩

Knowledge check

Why the kernel checks addresses itself

Kernel pages have their user/supervisor bit cleared, so ring-3 code faults if it touches them. Why must the kernel's NtReadVirtualMemory still check that the requested range and the caller's buffer lie in user space?

1The copy runs in ring 0, where supervisor pages are allowed, so an unchecked range could expose kernel memory2The check is needed only for 32-bit programs that arrive through WoW64, since their shorter pointers can overflow3The user/supervisor bit is consulted only when code is fetched, never when data is read, so data copies need a check4The check speeds the copy up by letting the kernel skip page-table walks for ranges already known to be in user space

Check answer

Finished Lesson 12.6?

Mark it done and the lesson list shows a ✓ beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.

Mark lesson doneLesson done. Mark it not done

Chapter 12: 10 lessons

📝 Notes