From Win32 Calls to Kernel Services
A Windows API call crosses layers of exports, native APIs, and system calls before a kernel service acts on a request.
Cheatsheet for Lesson 12.5
A one-page summary of “From Win32 Calls to Kernel Services” you can keep beside your notes. Plain Markdown.
Lesson 12.5 — From Win32 Calls to Kernel Services: cheatsheet
- Remember: A documented Windows function can keep its contract while the DLLs and internal layers implementing it change.
- Also useful: A handle's size does not tell you who owns it. Check the API that produced it and its documented release rule; this lab's borrowed module handle must not be released with
FreeLibrary.
Key terms
- software layers: The boxes are software layers, not necessarily four separate processes.
- forwarded export: A forwarded export is not a jump instruction the loader inserts.
A: In paired string APIs,Waccepts UTF-16 code units andAaccepts 8-bit text in the API's ANSI encoding;Ais not a promise of ASCII or UTF-8.
Review
- Q: What happens conceptually when a user-mode API needs the Windows kernel to perform privileged work? A: A controlled system-call transition enters a kernel service, then returns to user mode
- Q: Which three ideas describe how a documented API call reaches the kernel? A: A contract, a library path, and a privilege transition
One function call can cross several layers
Section titled “One function call can cross several layers”The course tool can now name a process, open and own a limited handle, inspect page metadata, and observe threads. Each operation was a call to a Windows API. The remaining question is where such a call goes when the tool asks the operating system to do something it cannot do directly.
A call to a Windows function such as VirtualQueryEx does not instantly jump into the deepest part of Windows. It usually passes through layers.
Use three separate ideas: contract, implementation path, and privilege transition. Win32 documents the contract application code may rely on. User-mode DLLs implement or forward that contract. A system-call instruction performs the controlled CPU transition for work that requires kernel privilege.
The boxes are software layers, not necessarily four separate processes.
kernel32.dll and ntdll.dll are mapped into the same user-mode process as our
program. Microsoft may change the implementation path while preserving the Win32
contract, which is why “calling a deeper layer” is usually less stable rather than
more capable.
Request Call a Win32 function
Step 1: Application to Win32 DLL
The application calls an ordinary documented function.
Request Call the native stub
Step 2: Win32 DLL to ntdll stub
The DLL’s implementation calls a lower-level stub in ntdll. Still the same process, still user mode.
Encrypted Controlled system-call gate
Step 3: ntdll stub to Kernel
The stub crosses into the kernel through a gate the CPU controls. This is the only privilege change.
Alone Validate the request
Step 4: Kernel (alone)
The kernel checks the arguments and the caller’s rights before acting.
Reply Result
Step 5: Kernel to ntdll stub
The kernel returns a result or an error code.
Reply Return
Step 6: ntdll stub to Win32 DLL
The stub hands the result back up.
Reply Return to the caller
Step 7: Win32 DLL to Application
The Win32 function turns it into the value the application expects.
The application, the Win32 DLL and the ntdll stub all run in the same user-mode process. Only a controlled gate reaches the kernel.
A Win32 call such as VirtualQueryEx passes down through user-mode DLLs and the system-call gate before the kernel validates it. Calling a deeper layer is usually less stable, not more capable. The interactive version needs JavaScript; the lesson text around it explains the same idea.
User mode and kernel mode are permission levels
Section titled “User mode and kernel mode are permission levels”Most game code, application code, and Windows DLL code runs in user mode. A user-mode mistake should not be able to overwrite the kernel or another process merely by using a bad pointer.
The Windows kernel and drivers run in kernel mode, where code can manage page tables, devices, scheduling, and other system-wide resources. A bug there can affect the whole computer. That is why the course does not turn ordinary game experiments into kernel-driver experiments.
The CPU enforces this border. A system call changes privilege through a defined entry point; it is not a magic function name and it is not the same thing as calling a normal function.
This design is often described as limited direct execution. The game normally runs directly on the CPU for speed, but in user mode and within its address-space permissions. Timers and interrupts let Windows regain control for scheduling. A system call lets the game request a restricted operation through a validated entry path.
The model resolves a common confusion: Windows is not interpreting every ordinary game instruction. Most instructions execute natively. Hardware privilege, address translation, interrupts, and controlled entries provide the limits that let the OS remain in control.
Win32 is the documented application interface
Section titled “Win32 is the documented application interface”The documented Win32 API is the part application developers are normally expected to call. Examples from this course include:
ReadProcessMemoryfor a read-only inspector with query-and-read rights;VirtualQueryExfor page metadata;CreateFileWfor files and local named pipes;CreateFileMappingWfor a shared-memory object.
Microsoft can change how a supported function is implemented while keeping its documented behavior. On modern Windows, an export in kernel32.dll may forward to another DLL such as KernelBase.dll. Your source code still calls the stable Win32 name.
A forwarded export is not a jump instruction the loader inserts. The export table entry itself can hold a string such as KERNELBASE.CreateFileW in place of a code address. When GetProcAddress resolves that entry, it reads the string, locates the named DLL, and looks up the function there instead — following another forward if that entry also happens to be one. Whoever asked for kernel32!CreateFileW never sees this redirection; they get back a usable address in KernelBase.dll either way, which is exactly why Microsoft can move an implementation between DLLs across Windows versions without breaking the caller’s import:
Native APIs sit closer to the system call
Section titled “Native APIs sit closer to the system call”ntdll.dll exposes many functions whose names begin with Nt, such as NtQueryVirtualMemory. Some native functions are documented for limited uses and many are not stable application contracts.
For ordinary tools, prefer documented interfaces:
“Closer to the kernel” does not mean “better for normal application code.”
Calling an undocumented entry point or copying a system-call number from one Windows build can make a tool fragile. Numbers and internal structures can change. The course resolves one native export so you can see the layer, but it deliberately does not call it or build a direct-syscall bypass.
DLL exports are names offered to other code
Section titled “DLL exports are names offered to other code”A DLL can publish an export table. Each exported name maps to an entry that the loader can resolve. GetProcAddress asks a loaded module for one of those exports.
The returned address is a virtual address in the current process. Address Space Layout Randomization, or ASLR, means the same DLL can be mapped at a different base address in another process or after a reboot. Store the name or an RVA when that is the stable identity; do not paste a displayed absolute address into tomorrow’s program.
Build the address inspector
Section titled “Build the address inspector”The lab gets handles to DLLs already loaded in its own process, resolves three names, and prints their process-local addresses. It never opens a game process and never calls the native function:
Complete lab source: api_layers.rs
api_layers.rs
#[cfg(windows)]fn main() -> anyhow::Result<()> { use anyhow::Context; use windows::{ Win32::{ Foundation::HMODULE, System::LibraryLoader::{GetModuleHandleW, GetProcAddress}, }, core::{PCSTR, s, w}, };
fn exported_address(module: HMODULE, name: PCSTR) -> anyhow::Result<usize> { // SAFETY: module is loaded in this process and name is a terminated, // static ASCII string. We only print the returned address. let function = unsafe { GetProcAddress(module, name) } .context("the requested export was not present")?; Ok(function as usize) }
// SAFETY: these DLL names are terminated, static UTF-16 strings. Windows // loads both modules before a normal console program reaches main. let kernel32 = unsafe { GetModuleHandleW(w!("kernel32.dll")) }?; // SAFETY: ntdll.dll is likewise a terminated, static UTF-16 name for a // module that is already loaded in every normal Windows process. let ntdll = unsafe { GetModuleHandleW(w!("ntdll.dll")) }?;
let rows = [ ( "Win32", "kernel32!VirtualQueryEx", exported_address(kernel32, s!("VirtualQueryEx"))?, ), ( "Win32", "kernel32!ReadFile", exported_address(kernel32, s!("ReadFile"))?, ), ( "Native", "ntdll!NtQueryVirtualMemory", exported_address(ntdll, s!("NtQueryVirtualMemory"))?, ), ];
println!("Layer Export Address in this run"); for (layer, name, address) in rows { println!("{layer:<8} {name:<35} {address:#018x}"); } println!(); println!("These addresses belong only to this process and this run."); println!("The lab resolved names; it did not call a native API or direct syscall."); Ok(())}
#[cfg(not(windows))]fn main() { eprintln!("This Windows API-layer lab must run on Windows.");}Why GetModuleHandleW instead of LoadLibraryW?
Section titled “Why GetModuleHandleW instead of LoadLibraryW?”Both DLLs are already present in a normal Windows console process. GetModuleHandleW borrows their existing module handles; it does not request another load or add a loader reference that we would later need to release.
Why does GetProcAddress use an ASCII name?
Section titled “Why does GetProcAddress use an ASCII name?”PE export names are byte strings. The s! macro creates a terminated static PCSTR. DLL file names use Windows wide strings, so GetModuleHandleW receives values from the w! macro instead.
This original attempt mixes byte strings and wide strings. It is intentionally a non-compiling example: the Windows bindings reject the wrong parameter types before any call runs.
fn resolve_virtual_query() -> windows::core::Result<usize> { // SAFETY: static terminated strings outlive lookup; widths must match the API. let kernel32 = unsafe { GetModuleHandleW(s!("kernel32.dll")) }?;
// SAFETY: the borrowed module stays loaded during this export lookup. let address = unsafe { GetProcAddress(kernel32, w!("VirtualQueryEx")) }; address.ok_or_else(windows::core::Error::from_win32).map(|function| function as usize)}Use w! for the wide DLL filename and s! for the byte-encoded export name. Both strings are static and terminated; the DLL handle is borrowed for the lookup.
In this version, FARPROC already means an optional generic function pointer.
ok_or_else removes that Option wrapper; the helper returns its address as
usize for display. It does not call that pointer as VirtualQueryEx. The
version-pinned type
and lookup signature
explain this return-type choice.
Improved code
Optional: practise matching the full API names, module argument and address type to their jobs.
fn resolve_virtual_query() -> windows::core::Result<usize> { // SAFETY: static terminated strings outlive lookup; widths must match the API. let kernel32 = unsafe { GetModuleHandleW(w!("kernel32.dll")) }?;
// SAFETY: the borrowed module stays loaded during this export lookup. let address = unsafe { GetProcAddress(kernel32, s!("VirtualQueryEx")) }; address.ok_or_else(windows::core::Error::from_win32).map(|function| function as usize)}Show the diff
fn resolve_virtual_query() -> windows::core::Result<usize> { // SAFETY: static terminated strings outlive lookup; widths must match the API. let kernel32 = unsafe { GetModuleHandleW(s!("kernel32.dll")) }?; let kernel32 = unsafe { GetModuleHandleW(w!("kernel32.dll")) }?;
// SAFETY: the borrowed module stays loaded during this export lookup. let address = unsafe { GetProcAddress(kernel32, w!("VirtualQueryEx")) }; let address = unsafe { GetProcAddress(kernel32, s!("VirtualQueryEx")) }; address.ok_or_else(windows::core::Error::from_win32).map(|function| function as usize)}Takeaway: the API’s string type determines the encoding, even when both arguments look like ordinary names.
The types make the two encodings visible instead of silently mixing them.
Read the Windows signature before translating it
Section titled “Read the Windows signature before translating it”Microsoft’s C documentation uses names such as DWORD, BOOL, and HANDLE
to describe the API’s binary contract. Match that contract rather than choosing
a Rust type just because its name sounds similar:
| Windows spelling | What it means at the boundary | Rust reading rule |
|---|---|---|
DWORD |
Unsigned 32-bit integer, even in a 64-bit process | Usually u32; it does not grow into usize. |
BOOL |
Signed 32-bit integer Boolean value | Raw i32, or the binding’s projected type or result; Rust bool has a different representation. For Boolean truth, test nonzero rather than equality to 1. |
HANDLE |
Opaque, pointer-sized object handle | Use the binding’s handle type; it is not a Rust reference that you may dereference. |
These distinctions come from Windows coding conventions and Windows data types. Read each function’s return-value section too: its success and failure rules can be more specific than its return type.
The lab’s string-width mistake is one instance of a broader convention.
In paired string APIs, W accepts UTF-16 code units and A accepts 8-bit text
in the API’s ANSI encoding; A is not a promise of ASCII or UTF-8. C headers
may select an unsuffixed name using the UNICODE macro. Calling the explicit
W function keeps the choice visible in Rust. See Working with Strings.
The ABI, or application binary interface, also specifies how arguments and
results pass between caller and callee. WINAPI and CALLBACK in C signatures
mark a calling convention, not an access right. Generated bindings supply the
matching declarations. For a callback declared with that convention, match its
exact signature and Rust’s extern "system" ABI:
its underlying convention depends on the target, including stdcall on 32-bit
Windows for non-variadic functions.
The GetModuleHandleW contract
is the ownership rule behind that note. Likewise, a string pointer must stay
valid for as long as the receiving API uses it; a pointer wrapper alone does
not own its storage.
Which windows-rs crate does what?
Section titled “Which windows-rs crate does what?”The crate names describe different jobs in the Rust bindings. They do not change which Windows layer runs a call or what the caller may access.
| Crate | Role |
|---|---|
windows |
Generated projected APIs for Win32, COM, and WinRT, with typed values and conversions. This is the lab’s API crate. |
windows-sys |
Raw Windows ABI declarations; the caller manages pointer contracts, ownership, and error translation. |
windows-core |
Shared COM and WinRT interface types, traits, and projection support. |
windows-strings |
Windows string representations, including owning strings and borrowed pointer wrappers. |
windows-result |
Windows status and error types, including HRESULT and Result. |
The generation tools have separate jobs: windows-bindgen
turns Windows metadata into Rust bindings; windows-metadata
reads and writes that metadata; windows-clang
extracts C/C++ header declarations using libclang. The lab consumes pre-generated
bindings. Add support or generation crates only when your own task needs them;
many support types are already re-exported by the API crate.
One similarly named crate, windows-reference,
boxes a value for a WinRT signature that requires IReference<T>. That is a
Windows interface object, distinct from ordinary Rust borrowing with &T.
The address inspector needs no such boxed value.
Run it and compare two launches
Section titled “Run it and compare two launches”The program prints exported function names beside their addresses in this process. Its first run establishes a local observation; the second run tests whether those addresses are safe to treat as permanent identifiers.
cd windows-labscargo run --release --target i686-pc-windows-msvc --bin api_layersRun it again after restarting the virtual machine and compare the addresses. A difference is expected. The names and behavior are the meaningful pieces.
Then use module_inventory against wesnoth.exe, ac_client.exe, or Quake3-UrT.exe. You are looking at two sides of the same loader idea:
module_inventoryshows which DLL images the game loaded;api_layersshows how one process asks a loaded DLL for an exported name.
The complete program is api_layers.rs.
References: libraries and native-system-service entry points, using Nt and Zw routines, GetModuleHandleW, and GetProcAddress.
Follow the abstraction layers
What happens conceptually when a user-mode API needs the Windows kernel to perform privileged work?
Win32 and native API layers prepare arguments before a processor-supported privilege transition reaches a kernel service. The boundary is controlled, not a shared address space.
Finished Lesson 12.5?
Mark it done and the lesson list shows a beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.
Plain Markdown, saved in this browser.
This book grew out of the original Game Hacking Academy by attilathedud and has been rewritten as a beginner-first course. Like the original repository, its source is published under the Apache License 2.0.