Hooks as Live Control-Flow Changes

Keyboard shortcuts

×

?

Open this guide

Alt + B

Show or hide the lesson list

Alt + O

Show or fold the lesson outline

Alt + N

Open or close notes

Alt + H

Toggle semantic colours

Alt + D

Toggle page brightness

Alt + C

Toggle code brightness

← →

Move through a focused code trace

Home End

First or final step in a focused trace

Escape

Close this guide or an open panel

Reader shortcuts pause while you type in a text field.

Cheatsheet for Lesson 8.10Key ideas, reference and questions

A one-page summary of “Hooks as Live Control-Flow Changes” you can keep beside your notes. Plain Markdown.

Lesson 8.10 — Hooks as Live Control-Flow Changes: cheatsheet

• Remember: Preserve the observer's live inputs, replay displaced work once per call, and restore original patch bytes during safe removal. None substitutes for the other two.

Key terms

• Replay: Replay means executing the displaced addition exactly once before continuation.

Review

• Q: Why enter a Draining phase before restoring bytes and freeing a trampoline? A: It stops new callback work and lets active callbacks finish while their code and state are still alive
• Q: While other threads keep running, what rule should a live hook installation satisfy? A: Any thread sees it either completely absent or completely working, never half-written

Download cheatsheet (.md)

Download cheatsheet (.md)

The tool now has an owner for its commands, features, and cleanup. A hook adds a harder lifetime problem: it changes a path that the game itself can enter while the tool is starting or stopping.

A detour is installed into code that other threads may already be executing. Its correctness depends on whole x86-64 instructions, the calling convention, executable-page protections, a valid trampoline, and ownership that can undo the change. We will treat those as one live contract.

A hook changes a live control-flow graph

Lesson 2.9 replaced a verified span of instructions with a jump to a code cave, then resumed the original path. Writing that jump is the easy part.

The hard part is that you are editing a program while it runs. Another thread may be executing the very bytes you are half-way through rewriting. A call may already be inside your replacement at the moment you decide to remove it. The game may throw away the object you hooked and build a fresh one.

In the vocabulary of Chapter 2, a detour replaces one edge in the game’s control-flow graph — it changes where a single instruction leads. Every difficulty in this lesson comes from doing that to a graph other threads are walking at the same time.

The installation invariant is:

The hook is either completely absent or completely working. There is never a moment when a call can find half of it.

“Completely working” means four things are true at once, for every call: the instructions it lands on are whole, the registers and stack are as the caller expects, everything the hook touches is still alive, and there is a correct way back.

Model installation and removal explicitly:

Watch it happenA live call must finish before its hook can be released

The original route remains until the trampoline and callback state are prepared. Peer threads are held for complete publication. One call enters owned callback work; draining closes new work and waits for that call to leave. Active users reaches zero before original bytes are restored and resources released.

Read the steps

1. The exact supported build and original bytes are verified. Calls still follow the original path. Preparation failure leaves that path unchanged.
2. Preparation makes a trampoline from whole instructions, relocates address-dependent operands, and validates the route back. Its owner also retains the original bytes and callback state.
3. The declared rendezvous holds peer threads while the complete detour is published. The cache is flushed and protections are restored before peers resume. There is no usable half patch.
4. One pictured call enters the callback, so active users becomes 1. The callback’s bounded work uses its still-owned state; forwarding uses the validated trampoline.
5. Draining refuses new callback work, while this existing call keeps its state and trampoline alive. It finishes its work, forwards through the trampoline, and returns; active users reaches 0.
6. With no remaining users, the same thread-safety mechanism restores and verifies the exact original bytes. Only then are the trampoline and callback state released. Calls take the original route again.

One moving dot is one pictured active call, giving the temporary count 1. The guarantee is that all users reach zero before resources are released.

There should be no HalfPatched state visible to executing threads.

Resources stay alive until the active callbacks reach zeroA separate drain example counts 2, 1, 0 active callbacks. Removal then restores the exact original bytes.

RemoverHook callbacks

1. 1

   Control Request removal: block new callbacks

   Step 1: Remover to Hook callbacks

   New callback work is refused from this moment, so the count can only go down.

2. 2

   Alone Active callbacks: 2, then 1, then 0

   Step 2: Hook callbacks (alone)

   While two calls are still running inside the hook, its code and data must stay alive. Each caller that returns lowers the count by one.

3. 3

   Data Restore the exact original site bytes

   Step 3: Remover to Hook callbacks

   At 0, nothing is running inside the hook, so it is now safe to put the original bytes back.

4. 4

   Alone Release the hook’s resources

   Step 4: Remover (alone)

   The hook’s memory is freed last, after nothing can reach it.

A separate drain example counts 2, 1, 0 active callbacks. Removal then restores the exact original bytes.

Preserve five contracts

Contract	What must remain true	Typical failure
Instruction	Whole instructions are displaced and relocated correctly	Return lands inside an instruction
ABI	Required registers, stack alignment, unwind expectations, and return values survive	Caller receives corrupted state
Identity	Expected module build and original bytes match	Patch targets a different function version
Concurrency	No thread executes a partly written or already freed path	Intermittent crash during install/remove
Lifetime	Callback state and trampoline outlive every user	Removal frees memory while a callback runs

A successful single-threaded test proves almost none of these under load.

Restoring state and replaying instructions are separate work

A detour can replace the bytes of an operation that the original path still needs to perform. Its trampoline must execute the displaced work with its original meaning, then continue at the first whole instruction after the replaced span. Saving and restoring a register does not perform that work.

Consider a toy path that loads gold, adds a bonus, and stores the result. The detour replaces the addition with a jump. Its observer temporarily uses EAX, so preserving the incoming value prevents that scratch calculation from changing the bonus’s input. Replay means executing the displaced addition exactly once before continuation. Try the three deliberate mistakes: no replay, two replays, or leaving the temporary register change in place.

Preserve the input, replay the displaced work once. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

mov eax, [gold]
jmp observer
observer: push eax
add eax, 10 ; temporary observer work
pop eax
add eax, bonus ; displaced operation
; no second replay
jmp continuation
continuation: mov [gold], eax

Store 105. This matches the original result.

Final values: gold = 105; bonus = 5; EAX = 105; expected = 105; saved = released; strategy = correct; matches = true.

Read every step of the lesson’s example

1. Step 1, line 1. The original path would load 100, add 5 once, and store 105.

   gold = 100; bonus = 5; EAX = 100; expected = 105; saved = none; strategy = correct.

2. Step 2, line 3. Save the incoming EAX value, 100, before temporary observer work.

   gold = 100; bonus = 5; EAX = 100; expected = 105; saved = 100; strategy = correct.

3. Step 3, line 4. Temporary observer work changes EAX to 110. That value must not leak into the original calculation.

   gold = 100; bonus = 5; EAX = 110; expected = 105; saved = 100; strategy = correct.

4. Step 4, line 5. Restore EAX to 100. Restoring this register has not yet performed the original bonus addition.

   gold = 100; bonus = 5; EAX = 100; expected = 105; saved = released; strategy = correct.

5. Step 5, line 6. Replay the displaced addition once: EAX becomes 105.

   gold = 100; bonus = 5; EAX = 105; expected = 105; saved = released; strategy = correct.

6. Step 6, line 9. Store 105. This matches the original result.

   gold = 105; bonus = 5; EAX = 105; expected = 105; saved = released; strategy = correct; matches = true.

The successful case preserves one register and replays one addition. It is only a computation model. A real trampoline also has to preserve live flags and required machine state, relocate address-dependent operands, satisfy the calling convention, and survive concurrent entry. The five contracts above remain necessary even if the toy’s final number matches.

There is a third operation at removal: restore the original patch-site bytes after callers can no longer enter or remain in the detour. That puts the original code path back. Replaying displaced work during a call, preserving machine state during observer work, and restoring bytes during removal happen at different times and solve different problems.

TL;DR: Preserve the observer’s live inputs, replay displaced work once per call, and restore original patch bytes during safe removal. None substitutes for the other two.

Restoration cheatsheet: preserve live register/flag inputs; replay displaced work exactly once; restore original patch bytes only during safe removal. These are three separate jobs.

Decode before deciding patch length

x86-64 instructions have variable length. Decode complete instructions until their total size can hold the chosen branch encoding. Every displaced instruction has to be kept, because the trampoline runs it later, and any operand that depends on the instruction’s own address has to be identified:

• RIP-relative memory operands;
• relative calls and jumps;
• conditional branches;
• instructions whose behavior depends on the original address.

Copying such bytes to a trampoline without relocation changes their target.

You can watch it happen with a RIP-relative load. At the original site, the instruction means “read the global sitting 0x2F1A bytes past the end of this instruction”:

Copy those seven bytes into a trampoline at a different address. The displacement is unchanged, but the address it is measured from is not:

The bytes are byte-for-byte identical and the instruction is entirely valid. It just reads somewhere else now. Relative calls and conditional branches fail in exactly the same way, and the symptom is rarely a crash at the hook — it is a wrong value, or a jump into unrelated code, surfacing much later. The trampoline builder must either rewrite the operand correctly or reject the site. “Unsupported instruction” is a valid, reliable outcome.

Make installation a transaction

Prepare everything before publishing the detour:

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum HookPhase {
    Absent,
    Preparing,
    Installed,
    Draining,
}

struct PreparedHook {
    target: usize,
    expected: Vec<u8>,
    original: Vec<u8>,
    patch: Vec<u8>,
    trampoline: ExecutableBlock,
}

ExecutableBlock stands for the owning wrapper around the trampoline’s executable allocation; its implementation depends on the allocation strategy used by the detour tool. This sketch describes the state that must stay alive, rather than supplying another complete hook library. The saved original bytes belong to the same owner, so restoration cannot outlive its evidence.

The preparation stage can verify:

1. exact target build and module identity;
2. current bytes equal expected;
3. all displaced instructions decode completely;
4. relative operands can be relocated;
5. trampoline and return address are reachable;
6. callback ABI matches the target function;
7. a dry-run decode of the final patch succeeds.

If a precondition fails, publish nothing: release unused prepared resources and report the failed check. Only a verified preparation may reach the owner that publishes the change. If verification fails after that owner has changed the bytes, roll back its change under the same thread-safety mechanism used for installation. As a flow:

Publication must have a real thread-safety mechanism

A multi-byte detour is not magically atomic. Another thread must not fetch the first half of the new jump while the second half still contains old bytes. Pick one publication mechanism and name it in the design:

Mechanism	What makes the patch indivisible to observers	Important limit
cooperative safe point	game-owned worker threads park at a known boundary before publication	only works when you control or can instrument those workers
thread rendezvous	peer threads are paused, their instruction pointers are checked or moved away from the patch span, and they resume only after verification	naïve suspension can deadlock when a stopped thread owns a lock the installer needs
framework-managed transaction	a mature detour library relocates affected instruction pointers and publishes through its documented transaction API	the library’s guarantees still need to match the target architecture and process

With peer threads unable to execute the patch span, the publisher changes the page protection, writes the complete detour, calls Windows FlushInstructionCache for the modified range, restores the original page protection, rereads the bytes, and only then releases the rendezvous. The same sequence applies in reverse during removal. VirtualProtect only changes page permissions; it does not make a five-byte jump atomic or synchronize other cores.

Caution

Do not claim that an ordinary byte copy is atomic. A single aligned store is a valid shortcut only when the architecture and exact patch encoding explicitly guarantee atomicity for that width. If those facts are not proven, use a real rendezvous or a transaction-capable hook library.

Re-entry is part of the contract

A callback can indirectly call the hooked function again. That is re-entry. Choose and document a policy:

Policy	Behavior	Suitable use
Allow	Nested calls execute callback again	Callback is pure and recursion is expected
Bypass nested	Nested call uses original path	Logging or observation that may call related code
Reject	Nested entry returns a defined error	Fixture where recursion is invalid

A thread-local depth counter can implement “bypass nested” in a local fixture. It is not a substitute for process-wide lifetime tracking during removal.

Keep callback work bounded

The hooked thread may be the render thread or simulation thread. A callback should capture a small owned event and return. Do not perform blocking I/O, wait for another thread that may need the current thread, hold a global lock through the original function, or let a panic cross the callback boundary.

A bounded queue makes overload visible. Count dropped events rather than silently allocating without limit.

Drain before removing

Removal needs two counters or states:

• whether new callback work is accepted;
• how many callbacks are currently active.

Set the state to Draining, stop publishing new work, wait for the active count to reach zero using a bounded policy, restore the exact original bytes, verify them, then release trampoline and callback state. If draining times out, keep the still-valid resources and report the incomplete shutdown; freeing them is not recovery.

Test more than the happy path

Use a local fixture function and cover:

• expected bytes match and installation succeeds;
• one expected byte differs and nothing is written;
• a displaced RIP-relative instruction is relocated or cleanly rejected;
• nested callback entry follows the documented policy;
• 16 or more threads call the fixture while events are captured;
• the event queue fills and reports drops;
• removal begins while callbacks are active;
• install/remove repeated hundreds of times leaves original bytes intact;
• a callback failure returns a defined result without unwinding across the ABI.

Glossary terms introduced here

• Detour: a deliberate replacement of one control-flow edge.
• Trampoline: relocated displaced instructions followed by a return edge.
• Relocation: adjusting address-dependent instructions for a new location.
• Re-entry: the hooked path is entered again before an earlier call returns.
• Draining: refusing new work while existing users finish.
• Publish: make prepared state visible to other threads.

A hook works only while its instruction, ABI, identity, concurrency, and lifetime contracts all hold. Installation prepares and validates before publication; removal drains active calls before freeing anything. Rollback and the stress cases above test the transitions where those contracts are easiest to break.

With the hook’s lifetime established, we can use it to observe rendering. The next chapter connects the game’s coordinate frames to the draw calls and pixels that a rendering tool works with.

🧩

Knowledge check

Remove a hook without a lifetime race

Why enter a Draining phase before restoring bytes and freeing a trampoline?

1It stops new callback work and lets active callbacks finish while their code and state are still alive2It rewrites every displaced instruction to one byte, so the trampoline can be freed immediately3It suspends all process threads for good, so nothing can enter the hook after restoration begins4It lets the tool skip saving the original bytes, since the callbacks will restore them themselves

Check answer

Finished Lesson 8.10?

Mark it done and the lesson list shows a ✓ beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.

Mark lesson doneLesson done. Mark it not done

Chapter 8: 10 lessons

📝 Notes