Lesson 5.9Executable Maps, Instruction References, and Scans

Instruction Decoding and Disassembly

See how a disassembler finds boundaries and meanings in variable-length x86 byte streams.

Lesson 5.9

Read or listen Print chapter or book

About 22 minutes

Cheatsheet for Lesson 5.9Key ideas, reference and questions

A one-page summary of “Instruction Decoding and Disassembly” you can keep beside your notes. Plain Markdown.

Lesson 5.9 — Instruction Decoding and Disassembly: cheatsheet

Remember: Use assembly to identify operations, then observe the running target to connect those operations to game behavior.
Also useful: The opcode is only part of an instruction. Read its operand form and CPU mode before interpreting the following bytes.

Key terms

add: Hand-decoding add is a good exercise.
B8: An instruction's length is known only after its first bytes are decoded, and the next instruction starts right after it: B8 means four immediate bytes follow, so the instruction after mov eax, 25 starts at 0 + 5 = 5.
complete encodings in 32-bit mode: These are complete encodings in 32-bit mode, with hexadecimal bytes and decimal assembly operands.
C0: Here C0 is 11 000 000, while E8 is 11 101 000.
E8: Here C0 is 11 000 000, while E8 is 11 101 000.
Bytecode: Bytecode is a separate instruction format designed for a software interpreter; its numeric opcodes need not match x86.
linear sweep: The loop above is a linear sweep: decode one instruction after another.
basic block: A basic block is a straight-line run of instructions with one entry and a control transfer at the end.
cross-reference: Each hit is a cross-reference, and decoding the instructions around it leads to the function you wanted.

Review

Q: Why must a disassembler advance by the decoder's reported instruction length? A: x86 instructions use a variable number of bytes, so a fixed step would land mid-instruction
Q: A disassembler prints an instruction that loads a value from a register plus an offset of 0x30. Which fact can it NOT tell you from the bytes alone? A: That the value at that offset is a player's score

Download cheatsheet (.md)

For an unfamiliar operation, use the instruction reference: it explains operand forms, changed values and flags, with an adjustable worked trace for each instruction.

What a disassembler is for

Lesson 5.8 followed changing gold through data addresses. The next question is which machine instruction reads or writes that field. We can copy a bounded code range, but raw bytes alone do not show where one instruction ends or what operation it performs.

The CPU executes machine-code bytes. Humans prefer names such as mov, call, and jne. A disassembler decodes bytes into instruction records and then formats those records as assembly text.

It does not recover the original source. Compiler variable names, comments, generic types, and most high-level structure are normally gone. A disassembler can show that four bytes at [ebx+0x30] are subtracted, but it cannot say that ebx points at a side’s data and 0x30 is the offset of its gold. That link comes from the running game: the instruction runs exactly when that side’s gold drops.

Machine code preserves operations more reliably than it preserves intent. Two different source programs can compile to similar instructions, and one source construct can compile differently across optimization levels. Assembly shows exactly what the machine does, not why the programmer wrote it.

An instruction can be described at three levels:

mechanics: which registers, memory locations, and flags can it change;
data flow: where its inputs came from and where its outputs are used;
program meaning: which game behavior it implements.

The decoder answers the first. Control-flow and data-flow analysis answer the second. Only the running target can establish the third.

A decoder answers structural questions: instruction length, operands, registers, immediate values, memory addressing, and flow control. A formatter chooses spelling and style, such as NASM or Intel syntax. Tooling should use decoded fields for logic rather than parsing the displayed string back into facts.

Decoding is harder than formatting

A disassembler does two jobs:

decode bytes into an instruction;
format that instruction as readable assembly.

x86 instructions have prefixes, several opcode maps, ModR/M and SIB bytes, displacements, immediates, and different modes. Hand-decoding add is a good exercise. A complete decoder is a major engineering project.

Very roughly:

an opcode selects an operation;
a prefix modifies size, repetition, locking, or another property;
a ModR/M byte helps describe register and memory operands;
a SIB byte can describe scaled index addressing;
a displacement is an encoded offset;
an immediate is a value stored directly in the instruction.

Not every instruction contains every piece. Their variable length is why a detour must decode whole instructions before deciding how many bytes to replace.

mov rax, rbx — REX prefix + opcode + ModR/M

ret — opcode only

The test bytes from decodes_simple_64_bit_function later in this lesson. Here 48 89 D8 encodes mov rax, rbx in 64-bit mode, followed by C3 for ret. For comparison, Lesson 2.3's B8 19 00 00 00 encoded mov eax, 25 in 32-bit mode. An opcode selects an operation; a prefix or ModR/M byte appears only when that instruction needs it.

The opcode B8 means mov eax, imm32. Its next four bytes hold the immediate, least significant byte first. Point to or focus a byte or an assembly term to see its partner. Select one to keep the pair marked.

Instruction bytes

Assembly

eax,

Opcode ↔ mnemonicOperand bytes ↔ value

25 is 0x19. Encoding: B8 19 00 00 00.

Read the byte layout

Position	Byte	Meaning
+0	`B8`	Opcode: copy imm32 into EAX
+1	`19`	Immediate bits 0–7 (least significant)
+2	`00`	Immediate bits 8–15
+3	`00`	Immediate bits 16–23
+4	`00`	Immediate bits 24–31

An instruction’s length is known only after its first bytes are decoded, and the next instruction starts right after it: B8 means four immediate bytes follow, so the instruction after mov eax, 25 starts at 0 + 5 = 5.

Predict first

In 32-bit mode the six bytes B8 19 00 00 00 C3 are mov eax, 25 (offsets 0 to 4) and then ret (offset 5).

A disassembler is told to start at offset 2 instead of 0. What does it show?

Answer: Different instructions that look valid. The decoder treats the byte it starts on as an opcode. 00 00 decodes as add byte ptr [eax], al and 00 C3 as add bl, al, so even the ret vanishes into a wrong instruction.

Start decoding at a different byteB8 19 00 00 00 C3 is mov eax, 25 then ret in 32-bit mode. Choose where the decoder starts; nothing in the bytes marks the real starts.

Decoded from offset 2

Real split (from offset 0)

mov eax, 25offsets 0 to 4

ret5

Split from offset 2

offset 0offset 5

B8+0

19+1

00+2

00+3

00+4

C3+5

at 2, 2 bytes

add byte ptr [eax], al

at 4, 2 bytes

add bl, al

Offset 2 is inside mov eax, 25. The bytes still decode, but as add byte ptr [eax], al, then add bl, al. The C3 of ret is now part of add bl, al, so the ret is gone.

A small opcode lookup table

These are complete encodings in 32-bit mode, with hexadecimal bytes and decimal assembly operands. They are examples, not every form of each instruction:

Machine-code bytes	Assembly	How to read the bytes
`B8 19 00 00 00`	`mov eax, 25`	`B8` selects EAX; the next four bytes hold 25, little-endian
`83 C0 01`	`add eax, 1`	`83` plus ModR/M `C0` selects addition to EAX; `01` is the immediate
`83 E8 01`	`sub eax, 1`	The same `83`, with ModR/M `E8`, selects subtraction
`31 C0`	`xor eax, eax`	`31` selects XOR; ModR/M names EAX as both operands, clearing it
`39 D8`	`cmp eax, ebx`	Compare EAX with EBX by setting flags; neither register is changed
`74 02`	`je next_ip + 2`	Jump two bytes past the next instruction address when ZF is 1
`50`	`push eax`	In this mode, decrease ESP by four and store EAX on the stack
`58`	`pop eax`	Read four stack bytes into EAX and increase ESP by four
`C3`	`ret`	Pop a return address into EIP and resume there

Notice that 83 alone does not distinguish add from sub. A ModR/M byte has three fields: mod, reg, and r/m, containing 2, 3, and 3 bits. Here C0 is 11 000 000, while E8 is 11 101 000. For this opcode, 11 chooses a register operand, the middle field selects addition (000) or subtraction (101), and the final 000 names EAX. In other instructions, the middle field can name a second register instead.

The decoder below takes any ModR/M byte apart the same way. Try C0 and E8 from the table, then 45, which turns the r/m field into a memory address.

Take a ModR/M byte apartOne byte holds three fields. Pick or type a byte and see what each field says about the instruction's operands.

ModR/M byte 45 = binary 01 000 101

mod = 01: A memory operand plus an 8-bit displacement (one byte) after this byte.
reg = 000: As a register operand: AL (8-bit), AX (16-bit) or EAX (32-bit); the opcode decides the width. Or, for opcodes such as 83, an opcode extension: /0, which selects ADD.
r/m = 101: The address is EBP plus the signed 8-bit displacement that follows.

Operand form from Table 2-2: [EBP]+disp8
After this byte come an 8-bit displacement (1 byte): 1 byte.

Table 2-2, the r/m rows for mod = 01
r/m	Operand	Byte if reg = 000
000	[EAX]+disp8	40
001	[ECX]+disp8	41
010	[EDX]+disp8	42
011	[EBX]+disp8	43
100	[--][--]+disp8	44
101	[EBP]+disp8	45
110	[ESI]+disp8	46
111	[EDI]+disp8	47

Table 2-2, the reg rows
reg	r8	r16	r32	83 /digit
000	AL	AX	EAX	/0 ADD
001	CL	CX	ECX	/1 OR
010	DL	DX	EDX	/2 ADC
011	BL	BX	EBX	/3 SBB
100	AH	SP	ESP	/4 AND
101	CH	BP	EBP	/5 SUB
110	DH	SI	ESI	/6 XOR
111	BH	DI	EDI	/7 CMP

Six real instructions, decoded

Bytes	Assembly	How the bytes read
8B 45 08	`mov eax, [ebp+8]`	8B is MOV r32, r/m32: reg is the destination and r/m is the source. ModR/M 45 = 01 000 101: mod 01 (memory), reg 000 = EAX or /0, r/m 101 gives [EBP]+disp8.
89 D8	`mov eax, ebx`	89 is MOV r/m32, r32: r/m is the destination and reg is the source. ModR/M D8 = 11 011 000: mod 11 (register), reg 011 = EBX or /3, r/m 000 names EAX.
83 C0 01	`add eax, 1`	83 /digit imm8: the reg field picks the operation, so /0 is ADD. ModR/M C0 = 11 000 000: mod 11 (register), reg 000 = EAX or /0, r/m 000 names EAX.
83 E8 01	`sub eax, 1`	83 /digit imm8: /5 is SUB. ModR/M E8 = 11 101 000: mod 11 (register), reg 101 = EBP or /5, r/m 000 names EAX.
8B 04 24	`mov eax, [esp]`	8B is MOV r32, r/m32. The SIB byte 24 supplies the base. ModR/M 04 = 00 000 100: mod 00 (memory), reg 000 = EAX or /0, r/m 100 gives [--][--]. SIB 24 = 00 100 100: scale 1, no index, base ESP, so the address is [ESP].
8B 05 78 56 34 12	`mov eax, [0x12345678]`	8B is MOV r32, r/m32. The four displacement bytes are little-endian: 78 56 34 12 is 0x12345678. ModR/M 05 = 00 000 101: mod 00 (memory), reg 000 = EAX or /0, r/m 101 gives disp32.

The table reproduces the 32-bit rows of Intel's Table 2-2 (Intel SDM Volume 2, section 2.1.5). In 64-bit mode a REX prefix can extend each field to four bits, and mod 00 with r/m 101 means RIP-relative addressing instead of an absolute address. The MMX and XMM register columns of the Intel table are left out.

For complete forms and flag effects, consult Intel’s instruction manuals. Bytecode is a separate instruction format designed for a software interpreter; its numeric opcodes need not match x86. The toy CPU’s encoding table shows that distinction with seven deliberately simple operations.

For our tool, use iced-x86.

[dependencies]
iced-x86 = "1"

Decode a byte slice

The decoder needs the byte slice, the target’s CPU bitness, and the address where those bytes execute. That address lets it print meaningful destinations for relative branches; it does not change the copied bytes. The function below returns one formatted line per decoded instruction:

use iced_x86::{
    Decoder, DecoderOptions, Formatter, Instruction, NasmFormatter,
};

fn disassemble(bytes: &[u8], bitness: u32, start_ip: u64) -> anyhow::Result<Vec<String>> {
    // 🛡️ The same bytes decode differently in different CPU modes. Refuse a
    // guessed value before the decoder can produce convincing nonsense.
    anyhow::ensure!(matches!(bitness, 16 | 32 | 64), "invalid bitness");

    // 🧭 `start_ip` gives relative branches their real live destination; it does
    // not say where this local byte slice is stored in the analysis tool.
    let mut decoder = Decoder::with_ip(
        bitness,
        bytes,
        start_ip,
        DecoderOptions::NONE,
    );
    let mut formatter = NasmFormatter::new();
    formatter.options_mut().set_digit_separator("`");
    formatter.options_mut().set_first_operand_char_index(10);

    // ♻️ Reuse these allocations inside the loop; a large code section may
    // contain thousands of instructions.
    let mut instruction = Instruction::default();
    let mut formatted = String::new();
    let mut lines = Vec::new();

    while decoder.can_decode() {
        decoder.decode_out(&mut instruction);
        formatted.clear();
        formatter.format(&instruction, &mut formatted);

        // 📏 Convert the decoder's virtual IP back into an index only after the
        // decoder has advanced, then validate the complete instruction range.
        let offset = usize::try_from(instruction.ip() - start_ip)?;
        let end = offset.checked_add(instruction.len())
            .context("instruction range overflowed")?;
        let raw = bytes.get(offset..end)
            .context("decoder produced an out-of-range instruction")?;
        let raw_hex = raw.iter()
            .map(|byte| format!("{byte:02X}"))
            .collect::<String>();

        lines.push(format!(
            "{:016X} {:<24} {}",
            instruction.ip(),
            raw_hex,
            formatted,
        ));
    }

    Ok(lines)
}

Decoder::with_ip makes relative branches and instruction addresses meaningful.

The instruction pointer passed as start_ip does not change the input bytes. It gives them a location so a relative branch can be reported as its real destination. Decoding the same bytes at a different address can therefore produce a different displayed branch target.

Use the target’s bitness

Decode 32-bit process code as 32-bit and 64-bit process code as 64-bit. The same bytes can mean different things in different modes.

Bitness changes default operand/address sizes and which encodings are valid. It is a property of the code being decoded, not necessarily of the computer or the analysis tool. A 64-bit Windows system can run the 32-bit Wesnoth course build.

Read an executable section

Use your process wrapper and PE parser to:

find the target module;
locate an executable section such as .text;
copy a bounded byte range;
pass its real virtual start address as start_ip.

A small disassembly listing: addresses, bytes, and instructions.

Do not decode unreadable gaps as though they were code.

Instructions are data

iced-x86 exposes more than formatted text:

use iced_x86::FlowControl;

fn describe_flow(instruction: &Instruction) -> &'static str {
    match instruction.flow_control() {
        FlowControl::Next => "continues",
        FlowControl::ConditionalBranch => "conditional branch",
        FlowControl::UnconditionalBranch => "unconditional branch",
        FlowControl::Call => "call",
        FlowControl::Return => "return",
        _ => "other flow",
    }
}

This is more reliable than searching formatted strings for jmp.

Linear sweep versus control flow

The loop above is a linear sweep: decode one instruction after another. It may decode data embedded in a code section as nonsense instructions.

A control-flow disassembler begins at a known entry point and follows reachable branches and calls. That needs:

a work queue of addresses;
a set of visited addresses;
range checks;
branch-target extraction;
an explicit choice to enter callees or continue after calls.

Start linear. Add control flow only after the basic decoder and bounds are solid.

A basic block is a straight-line run of instructions with one entry and a control transfer at the end. Connecting blocks by possible jumps creates a control-flow graph. Loops appear as edges that return to an earlier block; if statements often appear as a branch whose paths later meet.

The graph still contains uncertainty. An indirect jump may obtain its destination from a table or register, exception handling can create non-obvious edges, and code may be reached only through callbacks. Mark unknown successors instead of pretending the graph is complete.

Invalid bytes

Decoders can return an invalid instruction. Show it clearly and decide whether to stop or advance by the decoder’s reported length.

Never enter an infinite loop at one bad byte.

Test known examples

These four bytes have a known 64-bit interpretation: mov rax, rbx followed by ret. Passing them with start address 0x1000 should produce two formatted instructions in that order, so the test checks both instruction boundaries and basic decoding:

#[test]
fn decodes_simple_64_bit_function() {
    let bytes = [0x48, 0x89, 0xD8, 0xC3]; // mov rax, rbx; ret
    let lines = disassemble(&bytes, 64, 0x1000).unwrap();
    assert_eq!(lines.len(), 2);
    assert!(lines[0].contains("mov"));
    assert!(lines[1].contains("ret"));
}

Add tests for truncated instructions, invalid bitness, a relative branch, 32-bit bytes, and a start address near overflow.

This simulation needs JavaScript. The lesson text around it works through the same numbers.

What a disassembler can and cannot prove

Use a mature decoder for instruction truth. Spend your effort on the tool around it: safe process reads, section selection, navigation, symbols, and clear error messages.

Start from a string and follow what refers to it

An unfamiliar executable has thousands of functions, and nothing labels the one behind a message such as Not enough gold. The message is easier to find: its bytes sit in read-only data, where a text search finds them. The code that shows it must use the string’s address, so the next search is for code that refers to that address. Each hit is a cross-reference, and decoding the instructions around it leads to the function you wanted.

How the address appears in code depends on bitness. 32-bit code, such as this course’s Wesnoth build, usually stores the whole address inside the instruction: push 0x00C4A2F0 is the bytes 68 F0 A2 C4 00, so a byte search of .text for F0 A2 C4 00 finds it. Decode around each hit, because data can contain the same four bytes by chance. If the loader moved the module, it has already rewritten such addresses, so search the live code for the live address. 64-bit code usually stores a distance instead, measured from the end of the instruction. The CPU adds it to RIP, the 64-bit instruction pointer, which by then holds the address of the next instruction.

Predict first

A 64-bit game keeps the string at 0x1_4002_A4F0. A byte search of its code for the eight address bytes F0 A4 02 40 01 00 00 00 finds nothing.

What does the empty search show?

Answer: Nothing yet: decode each instruction and work out where its distance points. Only decoding gives each instruction its length, and so the address of the next instruction that the distance is measured from. Then every target can be compared with 0x1_4002_A4F0.

Where does this RIP-relative operand point?A 64-bit instruction stores a signed four-byte distance, lowest byte first, measured from the end of the whole instruction. Add it to the next instruction’s address and compare the result with the string.

From the stored distance to an address

Stored distance

E9lowest

00highest

The four bytes, read lowest byte first

0x000294E9 = +169193top bit clear, so positive

RIP holds this while the instruction runs

Next instruction = address + length 7

0x1_4000_1000 + 7 = 0x1_4000_1007

add

Target = next instruction + distance

0x1_4002_A4F00x1_4000_1007 + 0x294E9

Verdict

Lands exactly on the string at 0x1_4002_A4F0: a reference

0x1_4000_1007 + 0x294E9 = 0x1_4002_A4F0, the string’s first byte: this instruction refers to the string.

Disassemble the real Wesnoth function

The completed binary opens the live 32-bit Wesnoth process, reads 0x50 bytes beginning at the verified lesson address 0x007CCD91, decodes with a 32-bit iced-x86 decoder, and prints the real address, raw bytes, and NASM-style instruction text:

.\target\i686-pc-windows-msvc\release\disassembler.exe

It stops on invalid bytes or an out-of-range decode instead of silently inventing output. See the entire tool in disassembler.rs and the checked cross-process read in process.rs.

disassembler.rs

windows-labs/src/bin/ · Rust, 47 lines

View on GitHub

Show the codeHide the code

#[cfg(windows)]
fn main() -> anyhow::Result<()> {
    use anyhow::Context;
    use gha_windows_labs::Process;
    use iced_x86::{Decoder, DecoderOptions, Formatter, Instruction, NasmFormatter};

    const START: usize = 0x007C_CD91;
    const COUNT: usize = 0x50;
    let process = Process::open_by_name("wesnoth.exe", false)?;
    anyhow::ensure!(process.is_32_bit()?, "this address profile is 32-bit");
    let bytes = process.read_bytes(START, COUNT)?;

    let mut decoder = Decoder::with_ip(32, &bytes, START as u64, DecoderOptions::NONE);
    let mut formatter = NasmFormatter::new();
    let mut instruction = Instruction::default();
    let mut text = String::new();

    while decoder.can_decode() {
        decoder.decode_out(&mut instruction);
        anyhow::ensure!(
            !instruction.is_invalid(),
            "invalid x86 bytes at {:#010x}",
            instruction.ip()
        );
        text.clear();
        formatter.format(&instruction, &mut text);
        let start = usize::try_from(instruction.ip() - START as u64)?;
        let end = start
            .checked_add(instruction.len())
            .context("decoded instruction range overflowed")?;
        let raw = bytes
            .get(start..end)
            .context("decoder left the input range")?;
        let raw = raw
            .iter()
            .map(|byte| format!("{byte:02X}"))
            .collect::<Vec<_>>()
            .join(" ");
        println!("{:08X}  {:<24} {text}", instruction.ip(), raw);
    }
    Ok(())
}

#[cfg(not(windows))]
fn main() {
    eprintln!("This live disassembler must run on Windows.");
}

process.rs

windows-labs/src/windows_impl/ · Rust, 350 lines

View on GitHub

Show the codeHide the code

use std::{ffi::c_void, mem::size_of};

use anyhow::{Context, Result};
use windows::Win32::{
    Foundation::ERROR_NO_MORE_FILES,
    System::{
        Diagnostics::{
            Debug::{FlushInstructionCache, ReadProcessMemory, WriteProcessMemory},
            ToolHelp::{
                CreateToolhelp32Snapshot, MODULEENTRY32W, Module32FirstW, Module32NextW,
                PROCESSENTRY32W, Process32FirstW, Process32NextW, TH32CS_SNAPMODULE,
                TH32CS_SNAPMODULE32, TH32CS_SNAPPROCESS,
            },
        },
        Memory::{
            MEM_COMMIT, MEMORY_BASIC_INFORMATION, PAGE_EXECUTE, PAGE_EXECUTE_READ,
            PAGE_EXECUTE_READWRITE, PAGE_EXECUTE_WRITECOPY, PAGE_GUARD, PAGE_NOACCESS,
            PAGE_PROTECTION_FLAGS, PAGE_READONLY, PAGE_READWRITE, PAGE_WRITECOPY, VirtualProtectEx,
            VirtualQueryEx,
        },
        SystemInformation::{
            IMAGE_FILE_MACHINE, IMAGE_FILE_MACHINE_I386, IMAGE_FILE_MACHINE_UNKNOWN,
        },
        Threading::{
            IsWow64Process2, OpenProcess, PROCESS_ACCESS_RIGHTS, PROCESS_QUERY_INFORMATION,
            PROCESS_VM_OPERATION, PROCESS_VM_READ, PROCESS_VM_WRITE,
        },
    },
};

use super::OwnedHandle;

#[derive(Clone, Debug)]
pub struct ProcessEntry {
    pub id: u32,
    pub name: String,
}

#[derive(Clone, Copy, Debug)]
pub struct MemoryRegion {
    pub base: usize,
    pub size: usize,
    pub readable: bool,
    pub writable: bool,
    pub executable: bool,
}

#[derive(Debug)]
pub struct Process {
    handle: OwnedHandle,
    id: u32,
    name: String,
}

fn wide_text(buffer: &[u16]) -> String {
    let end = buffer
        .iter()
        .position(|unit| *unit == 0)
        .unwrap_or(buffer.len());
    String::from_utf16_lossy(&buffer[..end])
}

fn no_more_files(error: &windows::core::Error) -> bool {
    error.code() == ERROR_NO_MORE_FILES.to_hresult()
}

impl Process {
    /// Lists every running process with ToolHelp.
    pub fn list() -> Result<Vec<ProcessEntry>> {
        // SAFETY: no borrowed pointers are passed; the returned snapshot is owned below.
        let snapshot = unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0) }?;
        let snapshot = OwnedHandle::from_raw(snapshot)?;

        let mut raw = PROCESSENTRY32W {
            dwSize: size_of::<PROCESSENTRY32W>() as u32,
            ..Default::default()
        };

        // SAFETY: `raw` is correctly sized and writable for the duration of the call.
        unsafe { Process32FirstW(snapshot.raw(), &mut raw) }?;

        let mut processes = Vec::new();
        loop {
            processes.push(ProcessEntry {
                id: raw.th32ProcessID,
                name: wide_text(&raw.szExeFile),
            });

            // SAFETY: `raw` remains a valid PROCESSENTRY32W output buffer.
            match unsafe { Process32NextW(snapshot.raw(), &mut raw) } {
                Ok(()) => {}
                Err(error) if no_more_files(&error) => break,
                Err(error) => return Err(error.into()),
            }
        }
        Ok(processes)
    }

    pub fn find(name: &str) -> Result<ProcessEntry> {
        Self::list()?
            .into_iter()
            .find(|entry| entry.name.eq_ignore_ascii_case(name))
            .with_context(|| format!("process {name:?} is not running"))
    }

    pub fn open_by_name(name: &str, allow_write: bool) -> Result<Self> {
        let entry = Self::find(name)?;
        Self::open(entry, allow_write)
    }

    pub fn open(entry: ProcessEntry, allow_write: bool) -> Result<Self> {
        let mut access: PROCESS_ACCESS_RIGHTS = PROCESS_QUERY_INFORMATION | PROCESS_VM_READ;
        if allow_write {
            access |= PROCESS_VM_OPERATION | PROCESS_VM_WRITE;
        }

        Self::open_with_access(entry, access)
    }

    pub fn open_with_access(entry: ProcessEntry, access: PROCESS_ACCESS_RIGHTS) -> Result<Self> {
        // SAFETY: the PID is a value; inheritance is disabled; `OwnedHandle` owns the result.
        let handle = unsafe { OpenProcess(access, false, entry.id) }?;
        Ok(Self {
            handle: OwnedHandle::from_raw(handle)?,
            id: entry.id,
            name: entry.name,
        })
    }

    pub fn is_32_bit(&self) -> Result<bool> {
        let mut process_machine = IMAGE_FILE_MACHINE::default();
        let mut native_machine = IMAGE_FILE_MACHINE::default();
        // SAFETY: both output pointers remain valid for the call.
        unsafe {
            IsWow64Process2(
                self.handle.raw(),
                &mut process_machine,
                Some(&mut native_machine),
            )?;
        }
        Ok(process_machine == IMAGE_FILE_MACHINE_I386
            || (process_machine == IMAGE_FILE_MACHINE_UNKNOWN
                && native_machine == IMAGE_FILE_MACHINE_I386))
    }

    pub fn id(&self) -> u32 {
        self.id
    }

    pub fn name(&self) -> &str {
        &self.name
    }

    pub fn raw_handle(&self) -> windows::Win32::Foundation::HANDLE {
        self.handle.raw()
    }

    pub fn read_exact(&self, address: usize, output: &mut [u8]) -> Result<()> {
        let mut bytes_read = 0_usize;
        // SAFETY: `output` is writable for its length. Windows validates the remote range.
        unsafe {
            ReadProcessMemory(
                self.handle.raw(),
                address as *const c_void,
                output.as_mut_ptr().cast(),
                output.len(),
                Some(&mut bytes_read),
            )?;
        }
        anyhow::ensure!(bytes_read == output.len(), "short read at {address:#x}");
        Ok(())
    }

    pub fn read_bytes(&self, address: usize, count: usize) -> Result<Vec<u8>> {
        let mut bytes = vec![0_u8; count];
        self.read_exact(address, &mut bytes)?;
        Ok(bytes)
    }

    pub fn read_u32(&self, address: usize) -> Result<u32> {
        let bytes: [u8; 4] = self
            .read_bytes(address, 4)?
            .try_into()
            .expect("the read length is exactly four");
        Ok(u32::from_le_bytes(bytes))
    }

    pub fn read_f32(&self, address: usize) -> Result<f32> {
        Ok(f32::from_bits(self.read_u32(address)?))
    }

    pub fn write_exact(&self, address: usize, bytes: &[u8]) -> Result<()> {
        let mut bytes_written = 0_usize;
        // SAFETY: `bytes` is readable for its length. Windows validates the remote range.
        unsafe {
            WriteProcessMemory(
                self.handle.raw(),
                address as *const c_void,
                bytes.as_ptr().cast(),
                bytes.len(),
                Some(&mut bytes_written),
            )?;
        }
        anyhow::ensure!(bytes_written == bytes.len(), "short write at {address:#x}");
        Ok(())
    }

    pub fn write_u32(&self, address: usize, value: u32) -> Result<()> {
        self.write_exact(address, &value.to_le_bytes())
    }

    /// Writes machine code, flushes the CPU instruction cache, and restores protection.
    pub fn write_code(&self, address: usize, bytes: &[u8]) -> Result<()> {
        let mut old = PAGE_PROTECTION_FLAGS::default();
        // SAFETY: Windows validates the target range and writes `old` before returning.
        unsafe {
            VirtualProtectEx(
                self.handle.raw(),
                address as *const c_void,
                bytes.len(),
                PAGE_EXECUTE_READWRITE,
                &mut old,
            )?;
        }

        let write_result = self.write_exact(address, bytes);
        if write_result.is_ok() {
            // SAFETY: the process handle is live and the written range is the one above.
            unsafe {
                FlushInstructionCache(
                    self.handle.raw(),
                    Some(address as *const c_void),
                    bytes.len(),
                )?;
            }
        }

        let mut ignored = PAGE_PROTECTION_FLAGS::default();
        // SAFETY: this restores the protection returned by the first call.
        let restore_result = unsafe {
            VirtualProtectEx(
                self.handle.raw(),
                address as *const c_void,
                bytes.len(),
                old,
                &mut ignored,
            )
        };

        write_result?;
        restore_result?;
        Ok(())
    }

    pub fn module(&self, wanted_name: &str) -> Result<(usize, usize)> {
        // Include SNAPMODULE32 so a 64-bit tool can list a 32-bit game module.
        // SAFETY: the PID is valid and the returned snapshot is owned below.
        let snapshot =
            unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPMODULE | TH32CS_SNAPMODULE32, self.id) }?;
        let snapshot = OwnedHandle::from_raw(snapshot)?;
        let mut raw = MODULEENTRY32W {
            dwSize: size_of::<MODULEENTRY32W>() as u32,
            ..Default::default()
        };

        // SAFETY: `raw` is correctly sized and writable.
        unsafe { Module32FirstW(snapshot.raw(), &mut raw) }?;
        loop {
            if wide_text(&raw.szModule).eq_ignore_ascii_case(wanted_name) {
                return Ok((raw.modBaseAddr as usize, raw.modBaseSize as usize));
            }
            // SAFETY: `raw` remains a valid output buffer.
            match unsafe { Module32NextW(snapshot.raw(), &mut raw) } {
                Ok(()) => {}
                Err(error) if no_more_files(&error) => break,
                Err(error) => return Err(error.into()),
            }
        }
        anyhow::bail!("module {wanted_name:?} was not found in {}", self.name)
    }

    pub fn regions(&self, start: usize, end: usize) -> Result<Vec<MemoryRegion>> {
        let mut current = start;
        let mut regions = Vec::new();

        while current < end {
            let mut raw = MEMORY_BASIC_INFORMATION::default();
            // SAFETY: `raw` is a valid output structure; Windows validates the address.
            let returned = unsafe {
                VirtualQueryEx(
                    self.handle.raw(),
                    Some(current as *const c_void),
                    &mut raw,
                    size_of::<MEMORY_BASIC_INFORMATION>(),
                )
            };
            if returned == 0 {
                break;
            }

            let protection = raw.Protect;
            let blocked = protection.contains(PAGE_GUARD) || protection.contains(PAGE_NOACCESS);
            let readable = !blocked
                && raw.State == MEM_COMMIT
                && [
                    PAGE_READONLY,
                    PAGE_READWRITE,
                    PAGE_WRITECOPY,
                    PAGE_EXECUTE_READ,
                    PAGE_EXECUTE_READWRITE,
                    PAGE_EXECUTE_WRITECOPY,
                ]
                .iter()
                .any(|flag| protection.contains(*flag));
            let writable = readable
                && [
                    PAGE_READWRITE,
                    PAGE_WRITECOPY,
                    PAGE_EXECUTE_READWRITE,
                    PAGE_EXECUTE_WRITECOPY,
                ]
                .iter()
                .any(|flag| protection.contains(*flag));
            let executable = !blocked
                && [
                    PAGE_EXECUTE,
                    PAGE_EXECUTE_READ,
                    PAGE_EXECUTE_READWRITE,
                    PAGE_EXECUTE_WRITECOPY,
                ]
                .iter()
                .any(|flag| protection.contains(*flag));

            let base = raw.BaseAddress as usize;
            let size = raw.RegionSize;
            regions.push(MemoryRegion {
                base,
                size,
                readable,
                writable,
                executable,
            });

            let next = base.checked_add(size).context("memory map overflowed")?;
            anyhow::ensure!(next > current, "memory map did not advance");
            current = next;
        }
        Ok(regions)
    }
}

Why must a disassembler advance by the decoder's reported instruction length?

Finished Lesson 5.9?

Mark it done and the lesson list shows a beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.

Chapter 5: 11 lessons

This book grew out of the original Game Hacking Academy by attilathedud and has been rewritten as a beginner-first course. Like the original repository, its source is published under the Apache License 2.0.