9. Network Messages, Authority, and IPC — print chapter

Game Hacking Academy

Chapter 9: Network Messages, Authority, and IPC

00000000  47 61 6D 65 20 48 61 63  6B 69 6E 67 20 41 63 61  |Game Hacking Aca|
00000010  64 65 6D 79                                       |demy|

The title as it sits in memory: one byte per letter, written in hexadecimal.

10 lessons from a book of 155. Printed from mustcodeal.github.io/rust-game-hacking-book, built 2026-10-18.

Grew out of the original Game Hacking Academy by attilathedud (gamehacking.academy). Published under the Apache License 2.0.

This page contains only Chapter 9. The browser’s Print dialog lets you save it as a PDF.

Print this chapter or save as PDFOther print options

Chapter 9 contents

1. 9.1 — Messages, Streams, and Multiplayer
2. 9.2 — What the Server Can See
3. 9.3 — Protocol Capture and Message Framing
4. 9.4 — Message Parsing and Serialization
5. 9.5 — Protocol State Machines and Replay
6. 9.6 — Clients, Handshakes, and Chat Automation
7. 9.7 — Local Proxies and Bidirectional Streams
8. 9.8 — Shared Memory Between Processes
9. 9.9 — Named Pipes and Local Message Framing
10. 9.10 — How the Source Engine Works

Lesson 9.1

Messages, Streams, and Multiplayer

Client and server

A client is the game program running on a player’s device. It reads the player’s keyboard or controller, draws the world, and exchanges messages with another program over the network.

A server is a program that receives requests from clients and provides a service to them. A game server might keep track of who has joined a match, calculate player movement and damage, and send the resulting positions and health to the connected players. The computer running that program is also often called “the server.” The program can run on a rented machine, on one player’s computer, or beside your client on the same computer during a local test.

Imagine a match where the server calculates movement. It stores a player at (10, 15). The player’s client sends “move north.” The server checks whether the destination is inside the map and free of a wall. If the move is possible, it changes its stored position to (10, 14) and sends that position to the clients. Each client updates the character it draws. If a wall blocks the move, the server keeps (10, 15) and reports that position instead.

In this example	Client program	Server program
Input	Notices the north key press and sends a move request.	Receives the request and identifies the player who sent it.
Simulation	May predict movement to make the controls feel responsive.	Checks the destination and updates the match’s stored position.
Display	Draws the player at the latest known or predicted position.	Sends position updates; a dedicated server usually draws no game window.

Clarification: Client and server name the programs’ jobs. They do not require two computers: a local test can run both programs on one machine and connect them through loopback.

This is one common arrangement, called a server-authoritative simulation: the server’s accepted position is the one the clients eventually use. Other games give the server a smaller job, such as hosting a lobby or forwarding players’ inputs. A server does not necessarily simulate every part of a game, and the clients do not necessarily hold identical or complete copies of its data.

This chapter follows that message from intent to bytes, from captured bytes to a parsed event, and from one event to a legal session transition. A local chat client and proxy put the pieces together. The final lessons ask how two programs on one computer can exchange equally well-defined messages without using a network socket.

The client’s “move north” is an intent: a request for a change, not the final truth. The reply reports what the server actually accepted. If a wall blocked the move, the reply might still say (10, 15), and the client would correct its display.

These programs have separate memory. Changing a position in the client’s memory does not directly change the server’s stored position. They communicate by sending bytes. A client may display a predicted move while it waits, then correct that display when the server’s reply arrives. The next question is how the reply’s bytes become one complete message at the other end.

Peer-to-peer describes who communicates with whom

In a peer-to-peer arrangement, players’ programs can exchange messages directly. A lobby service may still introduce them, and a relay may carry bytes when a direct connection is unavailable. Neither service must calculate the match.

A player who hosts a server on their own computer still runs a client/server arrangement if that server accepts movement and sends the final positions. The location of the program does not change its job.

Example arrangement	Message path	Who calculates the accepted position here?
Dedicated game server	Each client talks to the server program.	The server program.
Player hosts the game server	Clients talk to the server on a player’s computer.	That server program.
Peers exchange inputs	Players exchange the same ordered input history.	Each peer runs the agreed deterministic simulation.

The last row describes one particular design, often called lockstep. Peer-to-peer does not automatically mean lockstep: another design can give one peer responsibility for an object or use different ways to resolve conflicts. Always ask two separate questions: where do messages go, and whose result is accepted? Direct communication alone does not prove which copy owns a gold value.

The same movement request, three arrangements. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

start at (10, 15)
request move north
deliver input to the selected participants
check destination against the shared map
accept (10, 14) or keep (10, 15)
draw the accepted position

Each player draws (10, 14). The server’s accepted result determines the final position, even when that program runs on a player’s computer.

Final values: arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

Read every step of the lesson’s example

1. Step 1, line 1. The worked match begins at the same position in each arrangement.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

2. Step 2, line 2. The player requests north; this request does not itself establish the final position.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

3. Step 3, line 3. The client sends the request to the dedicated server program.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

4. Step 4, line 4. dedicated server program checks the destination. A wall keeps the player in place.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program; blocked = false.

5. Step 5, line 5. The calculation accepts (10, 14).

   arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

6. Step 6, line 6. Each player draws (10, 14). The server’s accepted result determines the final position, even when that program runs on a player’s computer.

   arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

Clarification: A server is a program with a job. A peer is another participant. Hosting on a player’s computer and exchanging directly between peers describe different choices.

Topology cheatsheet: client/server identifies service roles; peer-to-peer describes communication between participants; authority identifies the accepted result. A topology label alone cannot answer all three questions.

TCP is a stream

TCP is a common transport for a connection between the two programs. It delivers an ordered stream of bytes. A program calls read to take some bytes that have arrived from that stream. TCP does not preserve the game’s message boundaries for that call.

One call to read may return:

• half of one message;
• exactly one message;
• one message plus part of the next;
• several messages.

Concretely: if the sender writes two length-prefixed messages back to back, the receiver has no say in how they turn up.

sent:       [00 00 00 02][41 42]  [00 00 00 03][43 44 45]

read #1 ->  00 00 00 02 41 42 00 00      one message, plus half a header
read #2 ->  00 03 43 44 45               the remainder

000

100

200

302

441

542

600

700

800

903

1043

1144

1245

length 2

AB

length 3

CDE

read #1: 8 bytes

read #2: 5 bytes

The 4 + 2 + 4 + 3 = 13 bytes on the wire, numbered from 0. In ASCII, 41 42 is AB and 43 44 45 is CDE. Read #1 stops after box 7, halfway through the second length field, so the receiver must keep boxes 6 and 7 until read #2 brings boxes 8 and 9.

Neither read lands on a message boundary. The receiver has to hold what it has in a buffer, remove complete messages when enough bytes have arrived, and keep the leftover for next time. A parser that assumes one read equals one message works flawlessly over a fast loopback connection while you are testing, then breaks the moment timing changes.

This is why a protocol needs framing.

use std::io::{self, Read};

fn read_frame(stream: &mut impl Read, max_size: usize) -> io::Result<Vec<u8>> {
    let mut length_bytes = [0_u8; 4];
    // Wait for the whole header, even if it arrives across several reads.
    stream.read_exact(&mut length_bytes)?;
    let length = u32::from_be_bytes(length_bytes) as usize;

    // Check an untrusted size before allocating its payload buffer.
    if length > max_size {
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            "frame is too large",
        ));
    }

    let mut payload = vec![0_u8; length];
    // A successful result contains one complete payload, without the header.
    stream.read_exact(&mut payload)?;
    Ok(payload)
}

stream supplies incoming bytes and max_size is the caller’s permitted payload limit. A successful call returns one payload as a Vec<u8>; a truncated header or payload produces an I/O error through ?. For a header 00 00 00 02 followed by 41 42, the returned vector contains just 41 42. The maximum size prevents a bad length field from requesting absurd amounts of memory. This blocking helper also needs an appropriate timeout on a real socket so a peer cannot make it wait indefinitely.

TCP reliability ends at delivering an ordered byte stream to the receiving socket. It does not prove the application parsed a complete message, accepted the request, saved the state, or sent a meaningful reply. The application must provide those guarantees. An acknowledgment can report acceptance, a request ID can identify a repeated request, and a timeout can tell the client when to stop waiting. Decide whether a retry repeats the same request or starts a new one before implementing it.

Byte order is part of the protocol

The same integer has more than one possible byte order. Decimal 258 is 1 × 256 + 2, so as a four-byte number it has one byte worth 256 (01), one byte worth 1 (02), and two zero bytes. Wesnoth’s four-byte frame length is big endian, which puts the byte worth the most first:

+000

+100

+201

+302

× 16,777,216

× 65,536

× 256

× 1

1 × 256 + 2 × 1 = 258

Big endian, the order Wesnoth sends on the wire.

On little-endian x86 memory, a native u32 with the same value would usually appear in the opposite order:

+002

+101

+200

+300

× 1

× 256

× 65,536

× 16,777,216

2 × 1 + 1 × 256 = 258

Little endian, the order of a native u32 in x86 memory: the same value with its four bytes reversed.

Read with the wrong order, the wire bytes 00 00 01 02 become 0x02010000, which is 2 × 16,777,216 + 1 × 65,536 = 33,619,968: a frame length of more than 33 million bytes. Network code must follow the protocol, not the host CPU’s default. That is why the reader uses u32::from_be_bytes rather than a pointer cast.

Packets and protocols

A packet is a chunk of bytes sent across a network. That part is straightforward.

A protocol specifies how programs communicate: how to separate messages, decode their fields, order the conversation, and respond to invalid input. Both programs need matching answers to these four questions:

1. Where does one message end and the next begin? A TCP connection is one long stream of bytes with no natural gaps, so something has to mark the boundaries.
2. What is each part of a message? Which bytes are the length, which are the message type, which are the payload, and how each of those is encoded.
3. Which messages are allowed right now? A login reply makes no sense before a login request. People routinely forget this is part of the protocol at all.
4. What happens when something is wrong? An unknown message type, a length that does not fit, a message arriving out of turn.

None of that can be worked out by staring at the bytes, because none of it is written in them. Both programs have to know it in advance. That is precisely why reverse engineering a protocol means recovering those four answers — and why a byte layout on its own only ever answers the second one.

Note

📦 Watch for: the same bytes can be perfectly valid at one point in a conversation and meaningless at another. A protocol model has to check when a message is legal, not only what it contains.

A simple framed message might contain:

[4-byte length][1-byte kind][payload bytes...]

A typed message models the decoded meaning:

enum Message {
    Chat { sender: String, text: String },
    Move { unit_id: u32, x: i32, y: i32 },
    Ping,
}

This enum is a declaration fragment, not a network reader. After framing and validation, a parser can turn a move’s fields into Message::Move with a unit ID and two coordinates; the game logic receives named values instead of raw byte offsets. The bytes are untrusted until parsing succeeds.

One message is wrapped in several layers

Several layers cooperate when the bot sends one chat message:

chat meaning:      sender and text
serialization:     WML fields and tags
compression:       gzip bytes
framing:           four-byte length plus payload
transport:         ordered TCP byte stream
network:           loopback IP packets

WML, Wesnoth Markup Language, is Wesnoth’s own tagged text format for structured data — chat messages, maps, units, and more all get written as bracketed tags such as [message] and [/message] wrapping quoted key="value" fields. It is the serialization format this chapter keeps reversing.

When something fails, ask which layer broke. A correct TCP connection does not prove the WML is valid. A valid gzip stream does not prove the length prefix used the right byte order.

The receiver undoes the layers in the opposite order, and each step has its own way to fail:

Serialization turns structured values into bytes. Framing tells the receiver where one serialized message ends. Transport moves bytes between endpoints. Keeping those jobs separate makes both the explanation and the code easier to test.

Watch it happenRecover bytes before accepting a message

Two TCP reads deliver thirteen bytes. Eight bytes supply one complete AB frame and half the next header. The remaining five bytes complete the CDE frame. A separate Wesnoth example then expands gzip into WML, validates a typed chat message, and checks session rules before accepting it.

Read the steps

1. Read #1 supplies eight bytes. Copy them into the buffer; do not treat that read as one game message.
2. The first four bytes name length 2. Remove that header and the two payload bytes, 41 42, which spell AB. Keep the two leftover zeros: they are only half of the next header.
3. Read #2 supplies five more bytes. Append them after the saved pair. The header is now 00 00 00 03 and all three payload bytes have arrived.
4. Remove the second header and its three-byte payload, 43 44 45, which spells CDE. Both frames are recovered and the receiver buffer is empty.
5. Framing is only the first layer. For a separate Wesnoth message, the complete payload contains gzip bytes. Decompression expands those bytes into WML text.
6. The WML parser validates tags and fields. It creates Message::Chat with named sender and text values; malformed data cannot cross this step.
7. The session checks whether this chat message is allowed now. Only a permitted typed message becomes an accepted event for the game logic.

Read boundaries and message boundaries differ. The later gzip example is separate from the plain ASCII AB/CDE frames.

Messages can feed a data pipeline

A multiplayer service handles more than packets. It may keep durable account and inventory records in a database, reuse expensive results in a cache, and build a search index for queries such as finding a match by region and mode. Those are different access patterns: keep data across restarts, avoid repeating work, and find selected records efficiently. They are not three names for the authoritative state of a running match.

Processing is asynchronous when the sender can continue before the work finishes. A request can enter a queue, then a worker handles it later. A queue acknowledgment might mean “received”; it does not by itself mean “validated, applied, and saved.” The protocol must say which stage each reply confirms.

There are also two common ways to process a sequence of events:

Processing style	Game-service example	What it must account for
Stream processing	Update a match’s statistics as accepted events arrive	Ordering, late events, duplicates, and how quickly the result is needed
Batch processing	Recompute a leaderboard from a collected set of finished-match results	Which results belong to the batch, when it runs, and how to repeat it safely

“Stream processing” here means processing ongoing events; it does not mean the TCP byte stream discussed earlier. A service can obtain events from many connections, files, or queues. A batch deliberately waits for a group of data and processes that group together. Neither approach implies that a game can wait for a leaderboard job before processing a player’s movement or damage.

For example, a match event might carry match_id, event_id, and a score change. If a sender retries it after a lost acknowledgment, the receiver must recognise the same event rather than add the score twice. Event identity and retry behaviour belong to the model and protocol, not to whether TCP delivered the underlying bytes correctly.

Bound queues as well as messages: an unbounded queue consumes memory when input arrives faster than it can be processed, and its results grow increasingly stale. Whether a full queue may discard something depends on what the message means, which Lesson 9.7 works through for a relay.

Design the backend for one match

A player logs in, joins match 42, and buys an item for 300 coins. The game backend is the set of services behind those actions. Separating their jobs explains why a working socket alone cannot prove a successful purchase.

From login to the live world

Authentication establishes the player’s identity. A session lets later requests refer to that authenticated login; it can outlive one TCP connection. A matchmaker groups compatible players and arranges a match location. It does not calculate their movement. Microsoft’s PlayFab matchmaking overview illustrates that service role.

An API gateway routes requests to the appropriate service. Middleware performs shared checks along that route: validate the session, bound the request, enforce its permissions, and record its outcome. A load balancer can spread these entry requests across equivalent gateway instances. The authoritative match server owns match 42’s accepted positions and damage; route its commands to that owner instead of a randomly chosen empty worker. These are logical jobs, which a small project can combine in one program.

Login, match allocation and asset delivery have different owners. These are logical service roles, not a requirement to deploy every box separately. Choose a tab and step through it to see who answers each request:

Log in: the database owns the accountIdentity comes from the durable account record. After that, every request carries a session instead of the password.

ClientGatewayDatabase

1. 1

   Request Log in: name and password

   Step 1: Client to Gateway

   The client talks to the gateway, the single public entry point. It never talks to the database itself.

2. 2

   Alone Middleware checks the request

   Step 2: Gateway (alone)

   Shared checks run before any service sees it: bound the size, enforce permissions, record the outcome.

3. 3

   Request Look up the account

   Step 3: Gateway to Database

   The database is the durable owner of account records and balances. The gateway asks it; it does not keep a copy it would trust more.

4. 4

   Reply Account found, password matches

   Step 4: Database to Gateway

   The stored (hashed) password agrees with what was sent, so the player is authenticated.

5. 5

   Reply Session token

   Step 5: Gateway to Client

   A session lets later requests refer to this login. It can outlive one TCP connection, so a dropped socket does not mean logging in again.

Identity comes from the durable account record. After that, every request carries a session instead of the password.

Join match 42: one match server owns the worldThe matchmaker only arranges a place to play. The authoritative match server that owns match 42 decides positions and damage.

ClientGatewayMatch-makerOwner recordMatch server

1. 1

   Request Join match 42 with the session

   Step 1: Client to Gateway

   The request names the match and carries the token from the login.

2. 2

   Alone Middleware validates the session

   Step 2: Gateway (alone)

   An unknown or expired session stops here, before any matchmaking work.

3. 3

   Request Find or allocate match 42

   Step 3: Gateway to Match-maker

   The matchmaker groups compatible players and arranges a match location. It does not calculate their movement.

4. 4

   Control Which worker owns match 42?

   Step 4: Match-maker to Owner record

   A coordination record says which worker owns the match. If that worker fails, a replacement needs a new ownership generation so the old one cannot write late.

5. 5

   Reply Owner: this match server

   Step 5: Owner record to Match-maker

   The answer names one owner. Commands must go to it, not to a random empty worker.

6. 6

   Reply Match 42 runs on that server

   Step 6: Match-maker to Gateway

   The gateway learns where the match lives.

7. 7

   Reply Address of the match server

   Step 7: Gateway to Client

   The client now knows where to send its game commands.

8. 8

   Request Game command: move north

   Step 8: Client to Match server

   Commands go to the owner of match 42, which checks them against the rules.

9. 9

   Reply Accepted position (10, 14)

   Step 9: Match server to Client

   The server keeps the accepted state; each client draws what the server accepted. A wall would have left the position at (10, 15).

The matchmaker only arranges a place to play. The authoritative match server that owns match 42 decides positions and damage.

Download a patch: assets come from the CDNLarge files that are the same for every player come from a content delivery network, not from the match server or the database.

ClientCDN edgeObject storage

1. 1

   Request Get patch file v12

   Step 1: Client to CDN edge

   The patch is a versioned file, so any nearby copy is as good as the original.

2. 2

   Alone Cache lookup: no copy yet

   Step 2: CDN edge (alone)

   This edge has not served the file before, so it must go to the origin once.

3. 3

   Request Fetch the file by its key

   Step 3: CDN edge to Object storage

   Object storage keeps larger files addressed by keys, such as versioned patches and replay recordings. It is the origin, the owner of the original bytes.

4. 4

   Data File bytes

   Step 4: Object storage to CDN edge

   The origin returns the file.

5. 5

   Data File bytes, copy kept at the edge

   Step 5: CDN edge to Client

   The edge stores a copy for later requests that are eligible for caching.

6. 6

   Request The next player asks for v12

   Step 6: Client to CDN edge

   A different player, near the same edge, asks for the same file.

7. 7

   Data Served from the edge

   Step 7: CDN edge to Client

   A cache hit avoids the origin fetch. Personal account responses follow separate rules: an old cached balance must never authorize a new purchase.

Large files that are the same for every player come from a content delivery network, not from the match server or the database.

Three questions, three ownersEach kind of data has one place that is allowed to answer authoritatively. The gateway routes; it owns none of them.

ClientGatewayMatch serverDatabaseCache

1. 1

   Request Ask: move north

   Step 1: Client to Gateway

   A request to change the world. The gateway checks it and routes it, but does not decide the result.

2. 2

   Request Route to the match’s owner

   Step 2: Gateway to Match server

   Positions and damage belong to the authoritative match server for that match.

3. 3

   Reply Accepted position (10, 14)

   Step 3: Match server to Client

   Owner of positions: the match server. Its answer is the truth the clients show.

4. 4

   Request Ask: what is my balance?

   Step 4: Client to Gateway

   A question about money, which must be exactly right.

5. 5

   Request Read the balance

   Step 5: Gateway to Database

   Owner of balances and inventory: the database, in protected, durable storage.

6. 6

   Reply Balance: 1,000 coins

   Step 6: Database to Client

   A current, authoritative answer. A cached value could be old, so it is never used to approve spending.

7. 7

   Request Ask: how many players in the lobby?

   Step 7: Client to Gateway

   A question where a slightly old answer is harmless.

8. 8

   Request Read the cached count

   Step 8: Gateway to Cache

   A cache holds a reusable result that can be rebuilt. It owns no original data.

9. 9

   Reply Count, possibly a few seconds old

   Step 9: Cache to Client

   Fast and cheap, and good enough for a label. If it is lost, it is rebuilt from the real data.

Each kind of data has one place that is allowed to answer authoritatively. The gateway routes; it owns none of them.

Put each value in the right store

A database keeps structured, durable account balances and inventory. Object storage keeps larger files addressed by keys, such as versioned patches and replay recordings. A cache reuses a result that can be rebuilt; a CDN, or content delivery network, brings eligible asset copies closer to players. An asset cache hit can avoid an origin fetch, as Cloudflare’s request lifecycle explains. Personal account responses need separate authorization and cache rules. A cached lobby count is useful; its old balance must not authorize a new purchase.

One purchase, one effect

The player has 1,000 coins. Request P7 asks for the 300-coin item. Validate the server’s price and current balance, debit to 700, grant the item, and record P7’s result inside one transaction: either that whole change commits or none does. Concurrent requests also need appropriate isolation or locking; merely surrounding unsafe read-then-write logic with a transaction is insufficient. PostgreSQL’s isolation guide explains these concurrency guarantees and possible transaction retries.

If the reply is lost, retry the same request ID. Idempotency means that repeat has the same effect: return the recorded result, without another debit or item. Scope the ID to its account, reject a changed payload under the same ID, and save the receipt atomically with the effect.

Purchase P7 commits its debit, item and receipt together; retrying P7 returns that receipt. The separate leaderboard example may display an older score until its derived view catches up. The order of the steps is the point, so step through each tab:

Buy an item: one transaction, three changesThe player has 1,000 coins and asks for a 300-coin item with request P7. The debit, the item and the receipt commit together or not at all.

ClientGatewayGame serviceDatabase

1. 1

   Request Buy the item for 300 coins (P7)

   Step 1: Client to Gateway

   P7 is the request ID. The client states what it wants, but not what it costs: the server knows the price.

2. 2

   Request Forward P7 for this account

   Step 2: Gateway to Game service

   After the middleware checks, the request reaches the service that owns purchases.

3. 3

   Control Begin transaction

   Step 3: Game service to Database

   From here to the commit, the changes are all-or-nothing.

4. 4

   Request Read the price (300) and the balance (1,000)

   Step 4: Game service to Database

   The server’s own price decides the cost. The read is protected by isolation or a lock, so a second request cannot spend the same coins in between.

5. 5

   Alone Debit to 700, add item, record P7

   Step 5: Database (alone)

   1,000 − 300 = 700. Three changes: the coins, the item, and the receipt for P7. None of them is visible to anyone else yet.

6. 6

   Control Commit

   Step 6: Game service to Database

   Only now do all three changes become visible together. A crash before this point would have undone every one of them.

7. 7

   Reply Committed

   Step 7: Database to Game service

   The purchase is durable.

8. 8

   Reply Item granted, balance 700 (P7)

   Step 8: Game service to Gateway

   The reply is sent only after the commit, so the player never sees an effect that could still vanish.

9. 9

   Reply Item granted, balance 700 (P7)

   Step 9: Gateway to Client

   The client shows the new balance.

The player has 1,000 coins and asks for a 300-coin item with request P7. The debit, the item and the receipt commit together or not at all.

Retry after a lost reply: the receipt answersThe reply to P7 never arrives, so the client sends P7 again. Idempotency means the repeat has the same effect as one request: no second debit, no second item.

ClientGatewayGame serviceDatabase

1. 1

   Alone The reply to P7 never arrives

   Step 1: Client (alone)

   The purchase did commit, but the client cannot know that: the network lost the answer.

2. 2

   Request Retry the same request ID: P7

   Step 2: Client to Gateway

   Because the ID is the same, the server can tell this is a repeat and not a second purchase.

3. 3

   Request Forward P7

   Step 3: Gateway to Game service

   The retry takes the same route as the original.

4. 4

   Request Look up the receipt for P7

   Step 4: Game service to Database

   The ID is scoped to the account. A changed payload under the same ID would be rejected, not merged.

5. 5

   Reply Receipt found: item granted, balance 700

   Step 5: Database to Game service

   The receipt was saved in the same transaction as the effect, so it exists exactly when the purchase does.

6. 6

   Reply The recorded result of P7

   Step 6: Game service to Gateway

   The service returns the stored result. It does not debit again and does not grant a second item.

7. 7

   Reply Item granted, balance 700 (P7)

   Step 7: Gateway to Client

   The client finally gets its answer. The balance is still 700, not 400.

The reply to P7 never arrives, so the client sends P7 again. Idempotency means the repeat has the same effect as one request: no second debit, no second item.

Leaderboard: allowed to lag, in a known wayAn accepted score is durable at once, but the leaderboard is a derived view. It may show the old score until it catches up, which is fine for a ranking and wrong for a balance.

Match serverDatabaseCachePlayer

1. 1

   Request Save the accepted score: 10 to 12

   Step 1: Match server to Database

   The score is a fact the match server has accepted. The database stores it durably.

2. 2

   Reply Committed: the score is 12

   Step 2: Database to Match server

   Anyone who reads the database now sees 12.

3. 3

   Request Open the leaderboard

   Step 3: Player to Cache

   The page reads a derived view held in a cache, not the database, so the database stays free for important work.

4. 4

   Data Shows 10: an older view

   Step 4: Cache to Player

   The view has not heard about the 12 yet. Showing a clearly dated value is acceptable here.

5. 5

   Control Rebuild the view from the new score

   Step 5: Database to Cache

   Eventual consistency promises that the view converges once updates stop and propagation succeeds. It does not promise how long that takes.

6. 6

   Request Refresh the leaderboard

   Step 6: Player to Cache

   The player asks again, some time later.

7. 7

   Data Shows 12

   Step 7: Cache to Player

   The view caught up. The same lag would not be acceptable for authorizing a purchase, which needs a current protected decision.

An accepted score is durable at once, but the leaderboard is a derived view. It may show the old score until it catches up, which is fine for a ranking and wrong for a balance.

Consistency describes which writes a read can observe. A strongly consistent read respects completed writes within its promised scope. A weaker read can return an older replica or cache value. Eventual consistency promises convergence when updates stop and propagation succeeds, not a fixed maximum delay. Currency authorization needs a current protected decision; a leaderboard can tolerate a clearly dated older view. Cosmos DB’s consistency definitions show why these guarantees must be stated explicitly.

Optional: explore the smaller concurrency problem below. Starting at 1,000, a reward of 500 and purchase of 300 should leave 1,200. Overlapping reads and writes can lose one change. Its local lock illustrates ordering; a backend must enforce its database transaction rules across all participating processes.

This simulation needs JavaScript. The lesson text around it works through the same numbers.

More matches, with explicit ownership

Sharding partitions work or records by a key: independent matches can run on different workers. A shared world crossing shard boundaries additionally needs ownership handoffs and ordered cross-boundary actions. Moving a job to another process also adds a message boundary, possible failures, and delay, so start with a small correct local service and distribute work when measurements justify it. Azure’s sharding pattern describes the partitioning tradeoff.

Coordination records which worker owns match 42. On failure, a replacement needs recoverable state and a new ownership generation; an expired worker must be prevented from committing late writes. Electing a replacement alone cannot reconstruct lost match state. Leader-election guidance explains leases and their failure limits. Retried result events also need persistent event IDs to avoid awarding the same win twice.

Spend the latency budget where it matters

A toy 60-tick-per-second server has about 16.7 milliseconds per simulation step. That budget is distinct from network round-trip delay. Keep slow replay uploads and leaderboard work outside the movement path. Measure queue depth, bytes per player, and slow-tail latency before optimizing. Spatial indexes can reduce repeated full-world searches; smaller state deltas reduce bandwidth, while compression spends CPU time. Extra workers cannot repair an undefined ownership rule or an overloaded shared database.

UDP is made of datagrams

UDP preserves datagram boundaries but does not guarantee delivery, order, or uniqueness. Fast games often tolerate those tradeoffs for time-sensitive state.

A later position update can make an older one useless, so freshness may matter more than retransmission. In contrast, a one-time inventory change may need explicit ordering and acknowledgment. Choose behavior per message kind instead of labeling an entire game “reliable” or “unreliable.”

Do not assume a game uses only one protocol. It might use TCP for login/chat and UDP for movement.

Ports and sockets

A socket represents a network endpoint. This complete helper opens a TCP connection to our local test server, or returns an error:

use std::{io, net::{SocketAddr, TcpStream}};
use std::time::Duration;

fn connect_local() -> io::Result<TcpStream> {
    let address = SocketAddr::from(([127, 0, 0, 1], 15000));
    // This bounds connection establishment, not later read/write operations.
    TcpStream::connect_timeout(&address, Duration::from_secs(3))
}

127.0.0.1 means the same computer; 15000 selects the server’s listening port. We use loopback and a local test server for this section. A successful result is a connected stream; framing and protocol checks still remain.

Encryption changes what you can see

If a protocol uses modern encryption correctly, a packet capture shows encrypted bytes, addresses, sizes, and timing—not readable messages. Do not try to defeat encryption or certificate checks.

For learning, use:

• your own toy protocol;
• a local open-source server;
• captured bytes provided by a challenge;
• application logs you control.

Captures in this chapter stay local

The capture, replay, and proxy examples use loopback or a local test server. That keeps timing, state changes, and failures observable while you learn what each protocol layer contributes.

Now that the message boundary and its byte order have a purpose, Lesson 9.3 records a real local exchange and asks which captured bytes correspond to one controlled action.

Lesson 9.2

What the Server Can See

The server receives requests and runs the shared simulation. It can see the accepted position, inventory, cooldowns, and command history. It does not directly see what a player was thinking or every program on their computer. Build checks around what the server actually knows.

Validate an action in the current world

Validation checks types, ranges, and current game state before data can change the world. For a movement command, check the input’s finite numeric values and allowed range. Then simulate movement using server-owned speed, collision, and time. A client-reported final position cannot replace those steps.

The permitted range must match the movement being simulated. A knockback, vehicle, or teleport ability changes what movement is possible. Checking only a single speed ceiling can mistake a valid ability for cheating.

A purchase needs the player’s identity, the item’s current price, and their server-owned balance. Apply the debit and reward together. Otherwise, two concurrent requests could both spend the same old balance, like the lost-update problem in the threading lessons.

Bound how much work a client causes

A rate limit bounds work or requests within a period. Limit expensive queries, parser allocation, and accepted actions separately. Receiving a burst after network delay does not mean the player earned extra simulation time.

Lesson 9.1 separated framing, parsing, and permitted messages in each session state. Keep that order here. Reject an invalid length before allocating its claimed buffer. Reject an action in the wrong session state before changing the world. Handle duplicates according to the action’s documented retry behavior.

Record the original server timing as well as the command’s claimed timing. Source’s lag-compensation checks in Lesson 9.10 illustrate why the server must bound and compare those claims rather than accepting arbitrary history.

Withhold information the client does not need

Fog of war limits what a client learns: hide game information until the player is entitled to observe it. A server that sends every hidden enemy position and merely hides the corresponding drawing has already disclosed the data.

In the toy picture, a wall blocks the observer’s sightline. The server knows the enemy’s position but sends none to this client. The enemy moves above the wall. Only after the new sightline clears the wall does its position enter the client’s observation.

The server withholds what the observer cannot seeIn the toy wall example the server knows the enemy position, but a wall blocks the sightline, so it sends nothing about the enemy.

ServerThis client

1. 1

   Alone Enemy position is known here

   Step 1: Server (alone)

   The server simulates the whole world, including the enemy.

2. 2

   Alone Sightline crosses the wall

   Step 2: Server (alone)

   The observer’s line of sight to the enemy is blocked.

3. 3

   Control Send nothing about the enemy

   Step 3: Server to This client

   A hack on this client cannot reveal what the client was never given.

In the toy wall example the server knows the enemy position, but a wall blocks the sightline, so it sends nothing about the enemy.

The position is disclosed once the view is clearWhen the enemy moves above the wall and the sightline clears, the position becomes part of this client’s observation.

ServerThis client

1. 1

   Alone Enemy moves above the wall

   Step 1: Server (alone)

   The world changes on the server.

2. 2

   Alone Sightline is clear

   Step 2: Server (alone)

   Nothing blocks the observer’s view of the enemy now.

3. 3

   Data Enemy position enters the observation

   Step 3: Server to This client

   Only now does the server send the position to this client.

When the enemy moves above the wall and the sightline clears, the position becomes part of this client’s observation.

Filter state before sending the message. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

read the selected inputs
evaluate each stated condition
combine the required conditions
perform the effect only when accepted
otherwise retain the old state
report the result and its limits

Result: the outgoing record has no hidden-enemy position. Client drawing cannot recover an absent field.

Final values: position = 8; wall = yes; line_clear = false; accepted = false; sent_position = absent.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs and follow how the next operation changes the state.

   position = 8; wall = yes.

2. Step 2, line 2. The server can retain an entity while omitting its position from this client’s record.

   position = 8; wall = yes; line_clear = false.

3. Step 3, line 3. Accept only when the required conditions all hold.

   position = 8; wall = yes; line_clear = false; accepted = false.

4. Step 4, line 5. The model retains its previous state.

   position = 8; wall = yes; line_clear = false; accepted = false; sent_position = absent.

5. Step 5, line 6. Result: the outgoing record has no hidden-enemy position. Client drawing cannot recover an absent field.

   position = 8; wall = yes; line_clear = false; accepted = false; sent_position = absent.

Watch it happenWithhold the hidden position

A wall hides an enemy until the server finds a clear sightline.

Read the steps

1. The server knows where the enemy is. A wall blocks this observer, so the client has no enemy position.
2. The enemy moves above the wall. The server updates its geometric visibility test.
3. The new sightline clears the wall. This position now qualifies for disclosure.
4. Only now does a position update cross into the client observation.
5. The client draws the received enemy. Hiding a label would not have removed information; withholding the position did.

This toy sightline shows the policy boundary. Production visibility also handles latency and geometry.

Real visibility systems must account for more than this one line. Animation, sound, team information, latency, and other players’ perspectives can make an entity relevant. Reveal it early enough to avoid visible pop-in, but do not disclose unrelated hidden state. Lesson 7.8 applies the same server-owned visibility decision to radar markers.

Alternative: Ask what the client needs to draw now. Construct that observation on the server before sending it.

Replays preserve useful context

A replay records enough accepted state and input to inspect a past sequence. Store the game version and the timing used to apply commands alongside it. A video from one client can omit information available to the server and include prediction or interpolation differences.

Replay review helps distinguish an impossible state from an unusual but legal tactic. It also reveals what a check was unable to observe. A replay that omits raw input edges cannot prove the timing of those omitted edges.

Machine learning fits a model to examples instead of using only hand- written checks. It can rank large numbers of replay segments for review. Its training labels can contain mistakes, and a new game update can change the input distribution. Test on held-out legitimate matches, different devices, and the updated game. A model score is a signal with measured limits, not an explanation of guilt by itself.

Lesson 9.3

Protocol Capture and Message Framing

Separate network layers before capturing

Lesson 9.1 showed that one game message can arrive across two TCP reads. A packet capture lets us inspect the bytes on the way, but its packet rows are not automatically game-message rows. We will send one distinctive lobby message, then follow its bytes in both directions.

An application message is not the same thing as an IP packet or TCP segment. Each layer has a different job:

game protocol message   login, chat, lobby update
TCP byte stream         ordered, reliable bytes between two sockets
IP packet               routed data between host addresses
link/loopback capture   what the capture interface records

TCP may split one game message across several segments or combine bytes from several messages in one segment. Retransmissions can repeat packet data. The receiving application sees an ordered stream, so protocol parsing needs its own framing rule such as a length prefix or delimiter.

This board needs JavaScript to move things around. Here is the lesson’s sorting:

Game protocol message

• A login message. Login, chat and lobby updates are game protocol messages. That is the level your parser works at.
• A length prefix saying where a message ends. The receiver sees an ordered stream, so protocol parsing needs its own framing rule such as a length prefix or a delimiter.

TCP byte stream

• Bytes arrive in order, none missing. That is the TCP promise: an ordered, reliable stream of bytes between two sockets.
• The four-tuple: source address and port, destination address and port. A TCP connection is identified by these four values, which is why one connection reads differently in each direction.
• One game message split across several segments. TCP may split one message across several segments or combine bytes from several messages into one. A segment boundary is not a message boundary.

IP packet

• Routing data between host addresses. Moving data between hosts by address is the IP packet’s job.

Capture

• The row you see in the capture tool. A capture records bytes and timing at one point. A row there is not automatically a game message, and it cannot say why it was sent.

A socket is an operating-system object representing one endpoint. A TCP connection is identified by source address, source port, destination address, and destination port—the four-tuple used below.

Follow a join request beyond loopback

Our capture stays local. To understand the wider journey, imagine a client joining match 42 and downloading its map from assets.example. Configuration, name lookup, delivery, and game permission solve different parts of that trip.

Find a destination before connecting

On a typical IPv4 LAN, DHCP gives a device an address lease and network configuration; options can supply a router and DNS servers. It configures the device, not the remote game’s identity. DHCP and its options specify that exchange. Static configuration and other address-assignment systems are also possible.

DNS resolves a name such as assets.example to records including IP addresses. A resolver can reuse an answer until its cache lifetime expires; name resolution does not deliver the map. The client then connects to the chosen endpoint. DNS concepts explain this separate lookup.

HTTP is the web’s request/response protocol. HTTPS carries that exchange over an authenticated, encrypted connection. A CDN can serve a cached map from an edge location. A cache miss may require an origin fetch; DNS answer caching and content caching have different lifetimes. Account requests still need their own authentication and cache policy. Cloudflare’s request journey shows these boundaries.

DHCP: a device joins the LANA device with no address asks the network for a lease. The reply also names the router and the DNS servers.

New deviceDHCP server

1. 1

   Control DISCOVER (broadcast)

   Step 1: New device to DHCP server

   The device has no address yet, so it shouts to everyone on the LAN: is there a DHCP server?

2. 2

   Control OFFER: 192.168.1.40

   Step 2: DHCP server to New device

   The server proposes an address and a lease time, plus the router and DNS server addresses.

3. 3

   Control REQUEST: I accept 192.168.1.40

   Step 3: New device to DHCP server

   The device accepts one offer. If several servers answered, this tells the others to withdraw.

4. 4

   Control ACK: lease confirmed

   Step 4: DHCP server to New device

   The lease is now recorded. The device configures itself; it has learned nothing about any game server yet.

A device with no address asks the network for a lease. The reply also names the router and the DNS servers.

DNS: from a name to an addressA resolver follows the name to the CDN and returns one address. Nothing is downloaded yet.

ClientResolverSite's DNSCDN's DNS

1. 1

   Request assets.example?

   Step 1: Client to Resolver

   The client asks the resolver it was configured with (often learned through DHCP).

2. 2

   Alone Check cache: nothing fresh

   Step 2: Resolver (alone)

   A cached answer that has not expired would end the lookup here.

3. 3

   Request assets.example?

   Step 3: Resolver to Site's DNS

   The resolver asks the name servers responsible for the name. Root and top-level referrals are left out to keep the picture short.

4. 4

   Reply CNAME assets.example.cdn.example

   Step 4: Site's DNS to Resolver

   The site points its name at the CDN’s name. The site’s DNS does not hold the file or the final address.

5. 5

   Request assets.example.cdn.example?

   Step 5: Resolver to CDN's DNS

   The resolver now asks the CDN’s name servers about the CDN’s own name.

6. 6

   Reply A 203.0.113.7 · TTL 30 s

   Step 6: CDN's DNS to Resolver

   The CDN chooses an edge near the resolver and sets a short lifetime, so it can steer players elsewhere soon.

7. 7

   Reply 203.0.113.7

   Step 7: Resolver to Client

   The resolver remembers the answer for 30 seconds and returns it. The client has an address, not the map.

A resolver follows the name to the CDN and returns one address. Nothing is downloaded yet.

CDN edge with a fresh copyAfter the connection and TLS handshake, a nearby edge answers from its cache.

ClientCDN edge

1. 1

   Request HTTPS: GET /maps/42.bin

   Step 1: Client to CDN edge

   The client connects to 203.0.113.7, completes the TLS handshake, then asks for the map.

2. 2

   Alone Cache lookup: fresh copy found

   Step 2: CDN edge (alone)

   The edge may reuse a stored copy while the origin’s cache rules say it is still fresh.

3. 3

   Reply 200 OK + map bytes

   Step 3: CDN edge to Client

   The map arrives from a nearby machine; the origin never saw this request.

After the connection and TLS handshake, a nearby edge answers from its cache.

CDN edge without a copyA miss climbs the cache hierarchy toward the origin, and each cache may keep the answer on the way back.

ClientCDN edgeRegional cacheOrigin

1. 1

   Request HTTPS: GET /maps/42.bin

   Step 1: Client to CDN edge

   The request is the same as before; the client cannot tell hit from miss.

2. 2

   Alone Cache lookup: miss

   Step 2: CDN edge (alone)

   The edge has no usable copy.

3. 3

   Request GET /maps/42.bin

   Step 3: CDN edge to Regional cache

   A larger regional cache is asked before the distant origin.

4. 4

   Request GET /maps/42.bin (miss too)

   Step 4: Regional cache to Origin

   The regional cache also misses, so it fetches from the origin storage.

5. 5

   Reply 200 OK + map bytes

   Step 5: Origin to Regional cache

   The origin supplies the file. The regional cache stores a copy if the response headers allow it.

6. 6

   Reply 200 OK + map bytes

   Step 6: Regional cache to CDN edge

   The edge receives the file and can store it too.

7. 7

   Reply 200 OK + map bytes

   Step 7: CDN edge to Client

   The player waits for the whole chain once. The next nearby request is a hit until the copy expires.

A miss climbs the cache hierarchy toward the origin, and each cache may keep the answer on the way back.

An IPv4 lease configures the client. DNS returns an address; only the later HTTPS connection retrieves the asset from the CDN, with an origin fetch on a cache miss.

Wrap the bytes across a LAN

A LAN, or local area network, joins nearby devices through links such as Ethernet or Wi-Fi. In this example, an Ethernet frame carries an IP packet, which carries a UDP datagram, which carries game bytes. This wrapping is encapsulation. A switch forwards link frames; a router forwards the IP packet onto another link with a new link frame. Physical signaling carries the bits on each link.

The OSI model names seven conceptual layers: application, presentation, session, transport, network, data link, and physical. The Internet’s TCP/IP model commonly groups application work, transport, Internet/IP, and link work. These models describe responsibilities, not seven mandatory headers. Our existing layer board remains useful: identify the job before assigning a name. Internet host requirements organize the protocol suite by these responsibilities.

The journey of one game messageHeaders are added on the way down, then removed on the way up. Play it step by step, or change the message size.

Ethernet header14 BIP header20 BUDP header8 BGame message24 BCheck sequence4 B

A 24-byte message becomes a 32-byte datagram, a 52-byte IP packet and a 70-byte frame: 46 of 70 bytes (66%) are wrapping.

Sending computer

1. 1

   User space Game code calls send()

   The message sits in the program’s own memory. The program hands it to the socket API; it builds no headers itself.

2. 2

   Kernel The kernel copies it into the socket’s send buffer

   The operating system copies the bytes out of the program’s memory into a buffer owned by the socket. send() can return once the copy is made; that says nothing about arrival.

3. 3

   Kernel The transport layer adds its header

   UDP adds 8 bytes: ports, length, checksum. TCP adds 20: ports, sequence and acknowledgement numbers, flags, window, checksum. The result is a datagram (UDP) or a segment (TCP).

4. 4

   Kernel The IP layer adds the IP header

   Twenty bytes with the source and destination addresses, a time-to-live and the protocol number. The kernel also consults its routing table to choose the next hop and the outgoing interface. The result is a packet.

5. 5

   Kernel The packet waits in an output queue

   When several packets are ready, a queue decides which one the card gets next. Linux calls this the queueing discipline, or qdisc. Other systems have comparable queues under other names.

6. 6

   Kernel The link layer wraps it in an Ethernet frame

   A 14-byte header holds the next hop’s MAC address, this card’s MAC address and the EtherType (0x0800 means IPv4). A 4-byte check sequence goes at the end, usually computed by the card. A frame shorter than 64 bytes is padded.

7. 7

   Network card The driver hands the frame to the card, which sends bits

   The driver places a descriptor for the frame in a ring buffer in memory. The card reads the frame by direct memory access and turns each byte into signals on the cable or radio: bits.

Between the computers

8. 8

   On the wire A switch forwards the frame

   A switch reads only the Ethernet header and sends the frame out of the port where the destination MAC address was last seen. The IP packet inside is untouched.

9. 9

   On the wire A router moves the packet onto the next link

   The router throws away the old Ethernet header and check sequence, lowers the IP time-to-live by one, recalculates the IP header checksum and wraps the same packet in a new frame. The transport header and the data are not touched.

Receiving computer

10. 10

    Network card The receiving card accepts the frame

    The card checks the destination MAC address and the check sequence; a damaged frame is dropped. A good frame is written into a ring buffer in memory by direct memory access, and the card raises an interrupt.

11. 11

    Kernel An interrupt, then deferred processing

    The CPU pauses its work to run a short interrupt handler, which schedules the longer work for later. On Linux that is a softirq run by ksoftirqd; on Windows it is a deferred procedure call. That work empties the ring buffer.

12. 12

    Kernel The Ethernet header is removed

    The EtherType says the payload is IPv4, so the packet goes to the IP layer. The link header and trailer have done their one-hop job.

13. 13

    Kernel The IP header is checked and removed

    The kernel verifies the header checksum and that the destination address is this computer, then reads the protocol number (17 means UDP, 6 means TCP) to choose the transport layer.

14. 14

    Kernel The transport layer finds the socket

    The destination port (for TCP, the whole four-tuple) identifies the socket. TCP also acknowledges, reorders and discards duplicates. The data is placed in the socket’s receive buffer.

15. 15

    User space Game code calls recv()

    The kernel copies the bytes from the receive buffer into the program’s memory. Only now does the program see them. With TCP, one recv() need not return exactly what one send() sent, so the game must mark its own message boundaries.

Complete journey shown.

The same layers, named two ways. The OSI model is a teaching map; real stacks follow the TCP/IP grouping.

OSI layer	TCP/IP model	Unit of data
7. ApplicationHTTP, FTP, your game protocol	Application	Data
6. PresentationEncoding, compression, TLS encryption
5. SessionKeeping a conversation open
4. TransportTCP, UDP	Transport	Segment (TCP) or datagram (UDP)
3. NetworkIP; routers	Internet	Packet
2. Data linkEthernet, Wi-Fi; switches	Link	Frame
1. PhysicalCables, radio, the network card		Bit

Maximum data in one Ethernet frame: 1,500 bytes of IP packet, so 1460 bytes of TCP data or 1,472 bytes of UDP data without fragmentation.

The same game bytes are wrapped inside a UDP datagram or TCP segment, an IP packet and an Ethernet frame. A router replaces the link wrapper for the next hop. The OSI/TCP-IP mapping is conceptual, not a literal count of wire headers.

Choose the delivery contract, then the application protocol

If middle bytes are lost, TCP recovers them before exposing later bytes in its ordered stream. UDP can deliver a later datagram without the missing one; the game decides whether to replace stale state or add its own recovery. Neither transport acknowledgment proves the server accepted a purchase. TCP and UDP specify those different contracts.

One lost packet: TCP waits, UDP moves onFive messages leave one per tick and need one tick to arrive. Packet 3 is lost on its first try. Choose other packets, or move through time.

TCP Gives the game bytes in order, so one missing packet holds back everything after it.

TCP timeline. Columns are ticks of time; messages are numbered from 1.

Row	0	1	2	3	4	5	6	7
Sent	1	2	3✕	4	5		3↻
Arrives		1	2		4	5		3
Held back						4	45
Game receives		1	2					345

UDP Gives the game each datagram as it arrives, in whatever order, and never resends.

UDP timeline. Columns are ticks of time; messages are numbered from 1.

Row	0	1	2	3	4	5	6	7
Sent	1	2	3✕	4	5
Arrives		1	2		4	5
Game receives		1	2		4	5

• 1 sent
• 1✕ lost on the way
• 1↻ sent again by TCP
• 1 reaches the computer
• 1 held back
• 1 handed to the game

TCP resends message 3 and gives the game all 5 in order, finishing at tick 7. Messages that had already arrived were held back: message 4 waits 3 ticks; message 5 waits 2 ticks. UDP gives the game messages 1, 2, 4 and 5 as each arrives and never delivers 3 on its own: the game decides whether to ask again or move on to newer data.

The timer is a simplification: real TCP can resend sooner when later packets arrive and the receiver keeps repeating the same acknowledgement. The ordering rule is real. TCP also needs a three-packet handshake before its first data; UDP can send at tick 0.

What each wrapper costs: a 8-byte UDP header or a 20-byte TCP header

UDP · 8 bytes

Source port16 bitsDestination port16 bits

Length16 bitsChecksum16 bits

TCP · 20 bytes without options

Source port16 bitsDestination port16 bits

Sequence number32 bits

Acknowledgement number32 bits

Offset4 bitsReserved + flags12 bitsWindow16 bits

Checksum16 bitsUrgent pointer16 bits

Options0 to 40 more bytes, only when needed

Source port

Identifies the sending socket on its computer.

Destination port

Identifies the receiving socket on its computer.

Length

Header plus data, in bytes.

Checksum

Detects corrupted bits in the header and data.

Sequence number

Position in the byte stream of the first data byte in this segment.

Acknowledgement number

The next byte the sender of this segment expects to receive.

Offset

Header length in 32-bit words; 5 means 20 bytes.

Reserved + flags

Flag bits such as SYN, ACK, FIN and RST say what kind of segment this is.

Window

How many more bytes the sender of this segment will accept (flow control).

Urgent pointer

Used only with the URG flag, which is rarely used.

TCP application messages still need framing: a lost packet changes timing, not message boundaries.

A socket is the local operating-system endpoint already introduced above. A WebSocket is a two-way framed-message protocol, useful for browser lobby chat. The RFC 6455 handshake uses HTTP/1.1 Upgrade; wss protects the exchange with TLS, Transport Layer Security, which authenticates the server and protects data in transit. HTTP/3 also has a WebSocket mapping, so an Upgrade handshake is not universal. Game authorization remains separate.

QUIC v1 runs over UDP but supplies its own reliable streams and loss recovery, using TLS 1.3 for connection security. HTTP/3 maps HTTP requests and responses onto QUIC. Loss affecting one stream need not block unrelated stream delivery, although streams share connection congestion limits. QUIC is not “unreliable HTTP,” and HTTP/3 is not a synonym for every game’s UDP protocol. The standards are QUIC v1, QUIC with TLS, and HTTP/3.

Two message paths, two socket typesBoth end at a socket, but the layers above it differ. Point at a layer to read what it does.

Secure WebSocket (RFC 6455)

1. Game message
2. WebSocket frame
3. TLS 1.3 record
4. TCP connection
5. TCP socket
6. IP packets and link frames

HTTP/3 (RFC 9114)

1. Game request
2. HTTP/3 frames
3. QUIC streams
4. QUIC packets with TLS 1.3
5. UDP socket
6. IP packets and link frames

Game message

The text or bytes your game sends, such as "join lobby 7". Your own rules decide what it means and who may send it.

WebSocket frame

A small header (opcode, length, a mask on frames the client sends) followed by the message. Frames keep message boundaries that raw TCP does not have.

TLS 1.3 record

Encrypts and authenticates the frames. The server’s certificate proves who it is.

TCP connection

Delivers the TLS bytes reliably and in order. One lost packet holds back everything after it on this connection.

TCP socket

The program’s endpoint for this connection, named by local and remote address and port. A socket is an operating-system object, not a message format.

IP packets and link frames

Routers move the packets; each link wraps them in its own frame.

Game request

An HTTP request such as a map download or an account call.

HTTP/3 frames

HEADERS and DATA frames carried on one request stream.

QUIC streams

Independent ordered byte streams inside one QUIC connection. Loss on one stream need not stall the others.

QUIC packets with TLS 1.3

QUIC does its own loss recovery, congestion control and encryption. TLS 1.3 is part of its handshake.

UDP socket

A datagram endpoint. Each QUIC packet travels inside one UDP datagram.

IP packets and link frames

The same delivery path as before.

• Application meaning
• Protection
• Delivery
• Routing and links

Inside one QUIC datagram (after the handshake)

UDP header8 bytesVisible to the networkFlags byte1 byteHeader-protectedDestination connection ID0 to 20 bytesVisible to the networkPacket number1 to 4 bytesHeader-protectedSTREAM framevariesEncrypted and authenticatedOther framesvariesEncrypted and authenticatedAuthentication tag16 bytesEncrypted and authenticated

UDP header Visible to the network

Source port, destination port, length and checksum. Routers, firewalls and address translators read these, so they cannot be hidden.

Flags byte Header-protected

Says this is a short-header packet and holds the key phase and packet-number length. Some of its bits are masked by header protection.

Destination connection ID Visible to the network

Lets the receiver find the connection even if the sender’s address changed, for example when a phone moves from Wi-Fi to mobile data.

Packet number Header-protected

Counts packets so the receiver can acknowledge them. Header protection hides it from observers.

STREAM frame Encrypted and authenticated

Stream ID, offset, length and a slice of that stream’s data. The offset lets the receiver put slices back in order even if packets arrive out of order.

Other frames Encrypted and authenticated

Acknowledgements, flow-control updates and more can share the same packet with STREAM frames.

Authentication tag Encrypted and authenticated

A checksum only the two endpoints can compute. If any protected byte changed on the way, the packet is rejected.

Handshake packets use a longer header with version and both connection IDs; this is the layout most packets have afterwards. Sizes follow QUIC version 1 (RFC 9000 and RFC 9001).

Clarification: A socket names an endpoint; WebSocket names a message protocol. QUIC builds reliable streams over UDP. An encrypted channel still needs game-specific identity and action checks.

To compare how these protocols begin, step through one conversation for each.

HTTP/1.1 over TCPA connection first, then one request and one response.

ClientServer

1. 1

   Control TCP handshake: SYN

   Step 1: Client to Server

   HTTP/1.1 normally runs on TCP, so the connection is set up first. The three handshake packets are drawn as one step here.

2. 2

   Request GET /lobby HTTP/1.1

   Step 2: Client to Server

   The request names a method, a path and headers. HTTP itself is readable text.

3. 3

   Reply 200 OK + body

   Step 3: Server to Client

   The response carries a status code, headers and the content. The connection can stay open for the next request.

A connection first, then one request and one response.

HTTPS: HTTP inside TLS 1.3The same request, after both sides have agreed on keys.

ClientServer

1. 1

   Control TCP handshake first

   Step 1: Client to Server

   TLS rides on a TCP connection, so TCP is set up before any TLS message.

2. 2

   Encrypted ClientHello + key share

   Step 2: Client to Server

   The client offers its supported algorithms and a public key share.

3. 3

   Encrypted ServerHello + key share + certificate

   Step 3: Server to Client

   Both sides combine the key shares into the same session keys. The certificate proves the server’s identity, and it already travels encrypted.

4. 4

   Encrypted Finished

   Step 4: Client to Server

   The client checks the certificate and confirms the handshake. Now encrypted application data can flow.

5. 5

   Encrypted Encrypted: GET /lobby

   Step 5: Client to Server

   The HTTP request is the same text as before, but an observer on the path sees only encrypted bytes.

6. 6

   Encrypted Encrypted: 200 OK + body

   Step 6: Server to Client

   The response is protected in the same way.

The same request, after both sides have agreed on keys.

HTTP/3 over QUIC and UDPQUIC joins the transport handshake and TLS 1.3, then carries independent streams.

ClientServer

1. 1

   Encrypted Initial: ClientHello + key share

   Step 1: Client to Server

   QUIC runs over UDP datagrams and includes TLS 1.3 in its own handshake, so transport and security start together.

2. 2

   Encrypted Initial/Handshake + certificate

   Step 2: Server to Client

   The server answers with its key share and certificate, protected as the handshake proceeds.

3. 3

   Encrypted Finished + first request

   Step 3: Client to Server

   The client can send its HTTP/3 request on a new stream as soon as the keys are ready.

4. 4

   Encrypted Stream 0: HTTP response

   Step 4: Server to Client

   Each request uses its own stream, identified by a stream ID.

5. 5

   Encrypted Stream 4: another request

   Step 5: Client to Server

   Streams are independent. A lost packet carrying one stream need not stall the other stream, although both share the connection’s congestion limits.

QUIC joins the transport handshake and TLS 1.3, then carries independent streams.

WebSocket: upgrade, then both waysAn HTTP request turns the connection into a two-way message channel.

ClientServer

1. 1

   Request GET /chat + Upgrade: websocket

   Step 1: Client to Server

   The handshake is an ordinary HTTP/1.1 request that asks to switch protocols. With wss it runs inside TLS.

2. 2

   Reply 101 Switching Protocols

   Step 2: Server to Client

   The server agrees. From now on the same connection carries WebSocket frames, not HTTP messages.

3. 3

   Data Frame: join lobby 7

   Step 3: Client to Server

   The client sends a framed message whenever it likes.

4. 4

   Data Frame: lobby 7 roster

   Step 4: Server to Client

   The server can reply at any moment; no new request is needed.

5. 5

   Data Frame: player joined

   Step 5: Server to Client

   The server can also push a message nobody asked for. That is why it suits live chat.

An HTTP request turns the connection into a two-way message channel.

TCP: connect, send, closeReliable ordered bytes need a handshake, acknowledgements and a teardown.

ClientServer

1. 1

   Control SYN

   Step 1: Client to Server

   The client proposes a connection and its starting sequence number.

2. 2

   Control SYN + ACK

   Step 2: Server to Client

   The server agrees, proposes its own starting number and acknowledges the client’s.

3. 3

   Control ACK

   Step 3: Client to Server

   The client acknowledges the server’s number. Both sides now agree on the connection.

4. 4

   Data Data: 100 bytes (seq 1–100)

   Step 4: Client to Server

   Every byte has a sequence number so the receiver can put bytes in order and spot gaps.

5. 5

   Control ACK 101

   Step 5: Server to Client

   The acknowledgement names the next byte the server expects, which confirms bytes 1 to 100.

6. 6

   Control FIN

   Step 6: Client to Server

   The client has nothing more to send and starts closing its direction.

7. 7

   Control ACK + FIN

   Step 7: Server to Client

   The server acknowledges and closes its own direction too.

8. 8

   Control ACK

   Step 8: Client to Server

   The final acknowledgement finishes the close. The client keeps the connection record briefly to catch stray late packets.

Reliable ordered bytes need a handshake, acknowledgements and a teardown.

UDP: just send a datagramNo setup, no acknowledgements, no automatic resend.

ClientServer

1. 1

   Data Datagram: position update #41

   Step 1: Client to Server

   There is no handshake. The first packet already carries game data.

2. 2

   Data Datagram: world snapshot

   Step 2: Server to Client

   Any reply is another independent datagram, sent when the server decides to send it.

3. 3

   Data Datagram: position update #42

   Step 3: Client to Server

   If a datagram vanishes nothing is resent for you. The game’s own sequence numbers can show the gap.

No setup, no acknowledgements, no automatic resend.

SMTP: mail moves between serversA short text conversation hands the message from one mail server to the next.

Sender’s appSender’s serverRecipient’s server

1. 1

   Request EHLO, MAIL FROM, RCPT TO

   Step 1: Sender’s app to Sender’s server

   SMTP is a text conversation. The client names itself, the sender and the recipient before any message is sent.

2. 2

   Data DATA: headers + body

   Step 2: Sender’s app to Sender’s server

   Then the message itself, ending with a line that holds only a dot.

3. 3

   Reply 250 OK, queued

   Step 3: Sender’s server to Sender’s app

   The server accepts responsibility for delivering the message.

4. 4

   Request Relay: the same SMTP commands

   Step 4: Sender’s server to Recipient’s server

   The server finds the recipient domain’s mail server through DNS and repeats the conversation with it.

5. 5

   Reply 250 OK, accepted

   Step 5: Recipient’s server to Sender’s server

   The receiving server takes the message.

6. 6

   Alone Store in the recipient’s mailbox

   Step 6: Recipient’s server (alone)

   Reading mail later uses other protocols such as IMAP. SMTP only moves the message along.

A short text conversation hands the message from one mail server to the next.

FTP: commands and files on separate connectionsOne connection carries commands; a second connection carries each file.

ClientServer

1. 1

   Control Control: USER, PASS

   Step 1: Client to Server

   Commands travel on the control connection (port 21). Plain FTP sends the password unencrypted; SFTP and FTPS are the protected options.

2. 2

   Control Control: 230 logged in

   Step 2: Server to Client

   Numeric reply codes say what happened.

3. 3

   Control Control: PASV

   Step 3: Client to Server

   The client asks the server to open a port for a data connection.

4. 4

   Control Control: 227 use this data port

   Step 4: Server to Client

   The reply holds the address and port for the data connection.

5. 5

   Control Control: RETR map42.bin

   Step 5: Client to Server

   The client names the file it wants.

6. 6

   Data Data connection: file bytes

   Step 6: Server to Client

   The file travels on the second connection, separate from the commands.

7. 7

   Control Control: 226 transfer complete

   Step 7: Server to Client

   The control connection reports the end of the transfer.

One connection carries commands; a second connection carries each file.

Other names belong at other boundaries

Publishing the map might use FTP, which separates control and data connections, or SFTP, which transfers files over encrypted SSH. SFTP is different from FTPS, FTP secured with TLS. A web server might use FastCGI to pass requests to a persistent application worker; the player’s side still uses HTTP. Web3 applications may use blockchain-backed records, but a ledger is not a replacement for sockets or real-time match transport. Likewise, Base64 only encodes bytes: an encoded login value has not thereby been encrypted.

With those boundaries separate, return to the controlled local capture. We can now ask which interface, direction, and protocol layer produced each row.

Capture answers “what crossed,” not “why”

A network trace records bytes and timing at one capture point. It does not contain the sender’s variable names, the receiver’s interpretation, or any record of which UI action caused which packet. That link comes from changing one thing at a time:

1. capture a quiet baseline;
2. perform exactly one distinctive local action;
3. repeat the action with one changed input;
4. compare both directions and several repetitions;
5. confirm the candidate fields in the open-source client or server.

TL;DR: A capture records traffic. Controlled actions and repeat comparisons connect those bytes to meaning.

Build a controlled capture

Use the course’s local Wesnoth server setup or your own toy client and server. Keep both endpoints inside the lab VM.

A local multiplayer lobby: client and server on the same machine.

Start Wireshark, choose the loopback adapter (Tip: On Windows the loopback adapter comes from Npcap, which the Wireshark installer offers. Wireshark lists it as “Adapter for loopback traffic capture”; older versions called it “Npcap Loopback Adapter”.) when client and server share the machine, and begin capture before connecting.

Capturing on the wrong interface produces an empty-looking experiment. Traffic to 127.0.0.1 stays on the loopback path rather than crossing the physical Wi-Fi or Ethernet adapter.

The Wireshark interface, where the capture starts.

Reduce noise

Filter by the known local port. Wireshark has two filter boxes, and each has its own syntax for the same job:

Display filter (hides rows afterwards)

tcp.port == 15000

Capture filter (never records the rest)

tcp port 15000

A display filter can be changed at any time, because everything was recorded. A capture filter keeps the file small, but a packet it dropped cannot be recovered, so use the display filter while you are still learning what to look for.

For the Wesnoth 1.14.9 lab, keep 15000: that is wesnothd’s default listening port. Launch wesnothd.exe, choose Connect to Server in the game, and enter localhost:15000.

Perform the actions at known times, so each burst of packets can be matched to the action just before it:

00:00 connect
00:02 enter username
00:05 join lobby
00:08 send "hello-17"
00:12 disconnect

A distinctive message such as hello-17 is easier to find than “hi.”

Direction matters

One connection has one four-tuple, and each direction reads it the other way round:

client IP : client port → server IP : server port
server IP : server port → client IP : client port

The client port is often temporary. The server port is the stable listening port.

The server first owns a listening socket on port 15000. Accepting a client creates a separate connected socket while the listener remains available for other clients. The client normally receives an ephemeral local port selected by Windows.

Follow the stream

Wireshark’s “Follow TCP Stream” view reassembles TCP bytes for easier inspection (Tip: To open it, right-click any packet of the connection and choose Follow, then TCP Stream. The shortcut is Ctrl+Alt+Shift+T.).

Alternative: Picture TCP as one long ribbon: reads cut pieces from it, while your parser identifies complete messages.

Follow TCP Stream shows the reassembled bytes of one connection.

Remember: this is still a stream, not necessarily a list of messages. Look for:

• fixed-size headers;
• length fields;
• magic bytes;
• repeated separators;
• readable text;
• compressed-looking regions;
• request/response pairs.

“Follow TCP Stream” performs reassembly for inspection. Your client still has to handle partial reads: read() may return fewer bytes than requested even when more bytes will arrive later. read_exact is appropriate only after a validated frame length tells you exactly how many bytes belong to the next piece.

Reassembly also chooses an order from sequence numbers and removes retransmitted overlap. Preserve the original packet capture alongside any exported stream fixture so later questions about direction, timing, or retransmission can still be answered.

Export a tiny fixture

Save only the bytes for one clearly labeled action. A small binary fixture is excellent for parser tests.

#[test]
fn parses_recorded_local_chat_frame() {
    let bytes = include_bytes!("fixtures/local-chat-frame.bin");
    let message = parse_frame(bytes).expect("fixture should parse");
    assert!(matches!(message, Message::Chat { .. }));
}

Do not store secrets, account tokens, or other people’s traffic in fixtures.

Compare two captures

Repeat the same action with exactly one changed input:

Capture A: send "red"
Capture B: send "blue"

Compare lengths and differing byte ranges. If only one region changes, it may contain the text. If the whole payload changes, compression or encryption may be involved.

Narration: Changing only one input gives differences a clearer meaning than comparing two sessions where everything changed.

A server message identified in the stream.

Identify the real Wesnoth handshake

Add this display filter so Wireshark shows client/server data rather than every TCP acknowledgement:

tcp.port == 15000 && tcp.flags.push == 1

The first client payload in a Wesnoth 1.14.9 connection is exactly four zero bytes:

00 00 00 00

Replay only those four bytes from a tiny TCP client. The local server responds with 41 bytes across the observed replies, proving that the zeros begin protocol negotiation. The next client frame identifies version 1.14.9; the following frame logs in with the chosen username.

After the four-byte frame length, both of those later frames begin with gzip magic bytes 1F 8B. For example, a captured version frame begins:

00 00 00 2F  1F 8B 08 00 ...
└─ length 47 ┘  └─ gzip data

Do the same controlled capture with usernames IEUser and FFFAAAKKKEEE. The second name makes repeated-character compression easier to notice. Put together, the Wesnoth 1.14.9 connection works like this:

target: Wesnoth 1.14.9 + local wesnothd
transport: TCP
server: 127.0.0.1:15000
negotiation: 00 00 00 00
normal frame: u32 big-endian compressed length + gzip(Simple WML)
login order: negotiation → version → username → lobby

Capture facts the parser needs

A useful frame fixture records:

• the start of one frame;
• the field that gives its size, or another reliable delimiter;
• its direction;
• one input and the bytes that changed because of it.

Lesson 9.4

Message Parsing and Serialization

Parse the real Wesnoth frame

The controlled captures from Wesnoth 1.14.9 reveal this frame:

u32 big-endian compressed length
gzip-compressed Simple WML payload

The first four bytes do not include themselves in the length. If the header ends in 4E, the following gzip member is 78 bytes and the complete TCP payload is 4 + 78 = 82 bytes. The length is big endian, so 78, hexadecimal 0x0000004E, appears on the wire as 00 00 00 4E; u32::from_be_bytes states that order directly.

Lesson 9.1 showed why a receiver needs this prefix: it accumulates bytes until it has the four-byte header, then exactly the advertised number of payload bytes, however TCP splits them across reads.

The header is a claim from the sender, not permission to allocate whatever it asks for:

Predict first

This receiver refuses any length above its limit, MAX_FRAME = 1,048,576 bytes. Its buffer holds the header FF FF FF FF and then four payload bytes.

What should it do now?

Answer: Refuse the frame now. FF FF FF FF is 255 × 16,777,216 + 255 × 65,536 + 255 × 256 + 255 = 4,294,967,295, which is above 1,048,576. The header alone decides it, so the receiver refuses before allocating or waiting.

What a receiver does with a length-prefixed frameThe first four bytes are a big-endian length; the limit is MAX_FRAME, 1,048,576 bytes. Change the claim and how many payload bytes have arrived.

Header, payload, decision

Receive buffer

Header = 5

byte 0byte 3

00×256³

00×256²

00×256

05×1

Payload bytes

48'H'

65'e'

6C'l'

6C'l'

6F'o'

Receiver’s decision

Take 4 + 5 = 9 bytes: one complete framenothing is left over

The header says 5 and 5 payload bytes are here, so 4 + 5 = 9 bytes form one complete frame.

Code that allocates from that claim could request nearly four gigabytes. First reject any length above the configured maximum; only then wait for the advertised number of payload bytes, with a bounded buffer and a way to stop a stalled connection. Having only some payload bytes so far does not by itself prove a network frame is false, because TCP may deliver the rest later. After a complete frame is assembled, the parser checks its exact length before decoding it. This separates byte validation from the later question of whether the decoded chat message makes sense.

Keep five layers separate

A protocol message has several nested layers. Each layer decodes the representation produced by the one outside it:

TCP byte stream
  -> frame: one length plus exactly that many bytes
  -> compressed payload: one gzip member
  -> document: Simple WML text and tree structure
  -> game meaning: a lobby chat message

Lesson 9.1 introduced these layers; here each one becomes its own parsing step. Only the last, the game-level schema, says that sender, room, and message form a chat event.

TL;DR: Framing, compression, text encoding, syntax, and session meaning can each fail independently.

This separation is a debugging tool. If the first four bytes advertise an impossible size, inspect framing. If decompression fails, inspect the wrapper and payload. If WML is malformed, inspect serialization. If valid WML is ignored by the game, inspect the expected message fields and connection state. Calling every failure a “packet problem” hides the layer that actually broke.

Several controlled chat messages in the capture.

Read with a cursor

The four-byte length at the front of the captured frame must be read before the compressed body, and every read must stop at the captured buffer’s end. A cursor pairs the borrowed bytes with the next unread position. If a read fails, the position stays where it was, so an error cannot leave a parser halfway through a claimed field:

#[derive(Debug)]
enum ParseError {
    Truncated,
    TooLarge(usize),
    TrailingBytes,
    InvalidGzip(std::io::Error),
    InvalidUtf8,
}

struct Cursor<'a> {
    bytes: &'a [u8],
    position: usize,
}

impl<'a> Cursor<'a> {
    fn take(&mut self, count: usize) -> Result<&'a [u8], ParseError> {
        // 📏 Calculate and validate the complete range before changing position.
        // A failed read therefore cannot leave the parser half-advanced.
        let end = self.position.checked_add(count).ok_or(ParseError::Truncated)?;
        let slice = self.bytes.get(self.position..end).ok_or(ParseError::Truncated)?;
        self.position = end;
        Ok(slice)
    }

    fn u32_be(&mut self) -> Result<u32, ParseError> {
        // 📦 The protocol says network byte order (big-endian), regardless of
        // the little-endian x86 CPU running the game.
        Ok(u32::from_be_bytes(
            self.take(4)?.try_into().map_err(|_| ParseError::Truncated)?,
        ))
    }
}

Alternative: The cursor is a bookmark: every accepted field advances it only after its bytes fit.

This cursor replaces direct indexing and hopeful conversion:

The original parser takes four header bytes, trusts their length, and indexes the payload. This is a failure example: a short header or a length beyond the captured buffer can panic; extra bytes are silently accepted.

Before: index directly into an untrusted frame

fn parse_payload(bytes: &[u8]) -> Result<&[u8], ParseError> {
    let length = u32::from_be_bytes(bytes[0..4].try_into().unwrap()); // Short header panics.
    let payload = &bytes[4..4 + length as usize]; // Trusts the claimed boundary.
    Ok(payload)
}

The bounded cursor makes every read fallible and advances only after a valid range has been obtained. Requiring the cursor to finish at the end rejects trailing bytes in this one-frame format.

Improved code

fn parse_payload(bytes: &[u8]) -> Result<&[u8], ParseError> {
    let mut cursor = Cursor { bytes, position: 0 };
    let length = cursor.u32_be()? as usize;
    let payload = cursor.take(length)?;
    if cursor.position != bytes.len() {
        return Err(ParseError::TrailingBytes);
    }
    Ok(payload)
}

Show the diff

fn parse_payload(bytes: &[u8]) -> Result<&[u8], ParseError> {
    let length = u32::from_be_bytes(bytes[0..4].try_into().unwrap()); // Short header panics.
    let payload = &bytes[4..4 + length as usize]; // Trusts the claimed boundary.
    let mut cursor = Cursor { bytes, position: 0 };
    let length = cursor.u32_be()? as usize;
    let payload = cursor.take(length)?;
    if cursor.position != bytes.len() {
        return Err(ParseError::TrailingBytes);
    }
    Ok(payload)
}

Takeaway: treat malformed input as a parse error, and check the complete frame boundary.

Why this version?

Network input is allowed to be short, oversized, or malformed. Indexing panics when a boundary is wrong, and unwrap() erases the difference between bad input and a programmer bug. The cursor makes failure a normal Result, advances only after a successful bounded read, and lets the caller reject hidden trailing data after one complete frame.

The lifetime 'a says every slice returned by the cursor borrows from the original input. The cursor owns only a position number; it does not copy payloads or create references that outlive the frame.

take is the one place that advances parsing state. checked_add stops a hostile count from wrapping the end to a small position, and get returns None for a range past the buffer. Later field readers get a simple promise: success consumes exactly the requested bytes; failure consumes nothing and returns an error.

try_into() converts a four-byte slice into [u8; 4], the fixed-size array required by from_be_bytes. The conversion can only succeed after take(4), but keeping the error path explicit makes the invariant visible instead of using unwrap().

Every read checks its boundary. Now decompress only the advertised bytes and cap the output:

use std::io::Read;
use flate2::read::GzDecoder;

const MAX_FRAME: usize = 1_048_576;

fn decode_wesnoth_frame(bytes: &[u8]) -> Result<String, ParseError> {
    let mut cursor = Cursor { bytes, position: 0 };
    let compressed_len = cursor.u32_be()? as usize;
    // 🛡️ Reject the advertised allocation before slicing or decompressing.
    if compressed_len > MAX_FRAME {
        return Err(ParseError::TooLarge(compressed_len));
    }

    let compressed = cursor.take(compressed_len)?;
    if cursor.position != bytes.len() {
        return Err(ParseError::TrailingBytes);
    }

    let mut decoder = GzDecoder::new(compressed);
    let mut plain = Vec::new();
    // 📏 Read one byte beyond the limit. Reaching that extra byte proves the
    // decompressed message is oversized without allocating without bound.
    decoder.by_ref().take((MAX_FRAME + 1) as u64)
        .read_to_end(&mut plain)
        .map_err(ParseError::InvalidGzip)?;
    if plain.len() > MAX_FRAME {
        return Err(ParseError::TooLarge(plain.len()));
    }

    String::from_utf8(plain).map_err(|_| ParseError::InvalidUtf8)
}

There are two separate size limits here:

• compressed_len limits how many bytes the parser accepts from the frame;
• the take(MAX_FRAME + 1) wrapper limits how many bytes decompression may produce.

Both matter because a small compressed input can expand into a very large output. Reading one byte beyond the limit lets the parser distinguish “exactly at the limit” from “too large” without allocating an unbounded vector.

There is a third limit outside this pure function: the connection’s accumulated input buffer. A peer could advertise a legal one-megabyte frame and then send one byte every few seconds. The stream reader needs a maximum buffered size, read timeouts or cancellation, and a policy for a connection that never finishes its frame. Parsing bounds protect completed inputs; transport bounds protect the time and memory spent waiting for them.

The trailing-byte check makes this function parse one complete frame. Without it, a valid prefix could hide extra bytes that the caller mistakenly ignores. A streaming connection reader should split the stream into exact frame slices first and call this parser once per slice.

Decompressing a captured chat frame reveals readable Simple WML:

[message]
  message="a"
  room="lobby"
  sender="FFFAAAKKKEEE"
[/message]

That is the actual structure we will rebuild.

Simple WML nests these bracketed tags and quoted fields as a tree. This lesson handles the one chat-message shape needed by our local Wesnoth proxy. If you extend the tool to inspect maps, units, or arbitrary add-on messages, use a real tokenizer that handles escaping, nested nodes, repeated fields, and comments instead of adding more substring searches.

The receiver also has an expected schema, even when no formal schema file exists. That schema includes field names, allowed repetitions, value types, and which fields are required. The sender writes one version of that agreement; the receiver reads according to the version it understands.

When a game update adds a field, ask two different compatibility questions:

• Can the new reader still understand old messages?
• Can the old reader safely ignore or default the new field?

Those questions are not the same. A tolerant tree format may ignore an unknown optional field, but renaming a required field can break both directions. For a versioned research tool, keep captured fixtures from every supported game build and record which message shape each fixture uses instead of guessing that one parser is universal.

Compress and frame a real chat message

Use flate2:

[dependencies]
flate2 = "1"

With the gzip encoder available, the next function accepts a chat field as text and rejects quotes or line breaks that would change the simple WML structure. Only validated fields go into the message before compression and length framing:

use flate2::{Compression, write::GzEncoder};
use std::io::Write;

fn simple_field(value: &str) -> Result<&str, &'static str> {
    if value.chars().any(|ch| matches!(ch, '"' | '\n' | '\r')) {
        return Err("quotes and newlines are not allowed in this lab field");
    }
    Ok(value)
}

fn encode_wesnoth_frame(wml: &str) -> Result<Vec<u8>, &'static str> {
    let mut encoder = GzEncoder::new(Vec::new(), Compression::default());
    encoder.write_all(wml.as_bytes()).map_err(|_| "gzip write failed")?;
    let compressed = encoder.finish().map_err(|_| "gzip finish failed")?;
    let length = u32::try_from(compressed.len()).map_err(|_| "frame too large")?;

    let mut frame = Vec::with_capacity(4 + compressed.len());
    frame.extend_from_slice(&length.to_be_bytes());
    frame.extend_from_slice(&compressed);
    Ok(frame)
}

fn encode_chat(sender: &str, message: &str) -> Result<Vec<u8>, &'static str> {
    let sender = simple_field(sender)?;
    let message = simple_field(message)?;
    encode_wesnoth_frame(&format!(
        "[message]\nmessage=\"{message}\"\nroom=\"lobby\"\nsender=\"{sender}\"\n[/message]"
    ))
}

Encoding runs the decoding pipeline in reverse:

1. validate values before placing them inside quoted WML fields;
2. serialize the typed message into UTF-8 text;
3. finish the gzip stream so its footer is present;
4. convert the compressed size to u32 without truncation;
5. reserve one vector for header plus payload;
6. append the big-endian length and compressed bytes.

Vec::with_capacity is an optimization, not a safety requirement. It avoids reallocating as the two pieces are appended. u32::try_from is a correctness requirement because as u32 could silently discard high bits for an oversized buffer.

GzEncoder produces the correct gzip wrapper; a plain zlib stream is not interchangeable here. The first two compressed bytes should be 1F 8B.

Gzip and zlib can both use the DEFLATE compression algorithm but wrap it in different headers and checksums. “Same compression algorithm” does not mean “same wire format.” Matching the observed wrapper is part of implementing the protocol.

Round-trip tests

Feed the serializer one sender and one message, then give its frame to the decoder. If the resulting WML still contains both values, the length, compression wrapper, UTF-8 text, and field quoting agree between these two functions for this case:

#[test]
fn chat_round_trip() {
    let frame = encode_chat("lab-user", "hello-17").unwrap();
    let decoded = decode_wesnoth_frame(&frame).unwrap();
    assert!(decoded.contains("message=\"hello-17\""));
    assert!(decoded.contains("sender=\"lab-user\""));
}

Also test an empty input, a truncated gzip member, an oversized length, invalid UTF-8 after decompression, extra trailing bytes, and rejected quote/newline characters.

Narration: Encoding then decoding your own message checks consistency; captured fixtures also check whether both directions share the same mistaken assumption.

A round trip proves that the encoder produces something this decoder accepts. It does not prove compatibility by itself, because two functions could share the same mistake. Keep at least one captured fixture from the real local session and verify that the decoder accepts it. Then send an encoded message to the controlled server and confirm that another capture has the expected four-byte prefix and gzip header.

Use a small failure matrix:

Test input	Expected failure
Fewer than four bytes	Truncated while reading the length
Length larger than remaining bytes	Truncated while taking the payload
Length above the cap	TooLarge before decompression
Valid frame plus one extra byte	TrailingBytes
Broken gzip checksum	InvalidGzip
Decompressed non-UTF-8 bytes	InvalidUtf8
Quote inside a simple field	Rejected before serialization

Comparing one changed message field.

The goal is not merely to make one capture parse. It is to define exactly which byte strings are accepted and how malformed data fails.

The buildable protocol module is wesnoth_protocol.rs. It contains bounded frame reads, gzip encode/decode, output-size caps, strict trailing-byte checks, WML field sanitation, send_wml, encode_chat, and round-trip/error tests. Both the real chatbot and proxy import this same implementation instead of copying a shortened lesson version.

wesnoth_protocol.rs

windows-labs/src/ · Rust, 107 lines

View on GitHub ↗

Show the codeHide the code

//! Bounds-checked Wesnoth 1.14.9 gzip/WML framing used by the local network labs.

use std::io::{self, Read, Write};

use anyhow::{Context, Result};
use flate2::{Compression, read::GzDecoder, write::GzEncoder};

pub const MAX_FRAME: usize = 1_048_576;

pub fn simple_field(value: &str) -> Result<&str> {
    anyhow::ensure!(
        !value
            .chars()
            .any(|character| matches!(character, '"' | '\n' | '\r')),
        "WML fields in this lab cannot contain quotes or newlines"
    );
    anyhow::ensure!(value.len() <= 512, "WML field is too long");
    Ok(value)
}

pub fn encode_frame(wml: &str) -> Result<Vec<u8>> {
    anyhow::ensure!(wml.len() <= MAX_FRAME, "plain WML frame is too large");
    let mut encoder = GzEncoder::new(Vec::new(), Compression::default());
    encoder
        .write_all(wml.as_bytes())
        .context("could not gzip WML")?;
    let compressed = encoder.finish().context("could not finish gzip member")?;
    let length = u32::try_from(compressed.len()).context("compressed frame is too large")?;
    let mut frame = Vec::with_capacity(4 + compressed.len());
    frame.extend_from_slice(&length.to_be_bytes());
    frame.extend_from_slice(&compressed);
    Ok(frame)
}

pub fn decode_frame(frame: &[u8]) -> Result<String> {
    anyhow::ensure!(frame.len() >= 4, "frame has no four-byte length header");
    let length = u32::from_be_bytes(frame[..4].try_into().expect("slice is four bytes")) as usize;
    anyhow::ensure!(length <= MAX_FRAME, "compressed frame is too large");
    anyhow::ensure!(
        frame.len() == 4 + length,
        "frame length does not match header"
    );

    let mut decoder = GzDecoder::new(&frame[4..]);
    let mut plain = Vec::new();
    decoder
        .by_ref()
        .take((MAX_FRAME + 1) as u64)
        .read_to_end(&mut plain)
        .context("invalid gzip payload")?;
    anyhow::ensure!(plain.len() <= MAX_FRAME, "decompressed WML is too large");
    String::from_utf8(plain).context("WML payload is not UTF-8")
}

pub fn read_frame(reader: &mut impl Read) -> io::Result<Vec<u8>> {
    let mut header = [0_u8; 4];
    reader.read_exact(&mut header)?;
    let length = u32::from_be_bytes(header) as usize;
    if length > MAX_FRAME {
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            "frame is too large",
        ));
    }
    let mut frame = Vec::with_capacity(4 + length);
    frame.extend_from_slice(&header);
    frame.resize(4 + length, 0);
    reader.read_exact(&mut frame[4..])?;
    Ok(frame)
}

pub fn send_wml(writer: &mut impl Write, wml: &str) -> Result<()> {
    let frame = encode_frame(wml)?;
    writer
        .write_all(&frame)
        .context("could not send complete frame")?;
    writer.flush().context("could not flush frame")?;
    Ok(())
}

pub fn encode_chat(sender: &str, message: &str) -> Result<Vec<u8>> {
    let sender = simple_field(sender)?;
    let message = simple_field(message)?;
    encode_frame(&format!(
        "[message]\nmessage=\"{message}\"\nroom=\"lobby\"\nsender=\"{sender}\"\n[/message]"
    ))
}

#[cfg(test)]
mod tests {
    use super::{decode_frame, encode_chat, encode_frame};

    #[test]
    fn real_chat_shape_round_trips() {
        let frame = encode_chat("ChatBot", "Hello!").unwrap();
        assert_eq!(&frame[4..6], &[0x1F, 0x8B]);
        let plain = decode_frame(&frame).unwrap();
        assert!(plain.contains("message=\"Hello!\""));
    }

    #[test]
    fn rejects_trailing_bytes() {
        let mut frame = encode_frame("[version]\n[/version]").unwrap();
        frame.push(0);
        assert!(decode_frame(&frame).is_err());
    }
}

Read the module by responsibility

Start with the pure functions that accept and return byte vectors; they are easiest to test. Next read the exact stream functions that repeatedly call read_exact for a header and payload. Finally read send_wml, which owns the complete serialization-and-write transaction.

This order keeps three kinds of failure distinct:

• transport failure: the socket closed or timed out;
• framing failure: the advertised message length was impossible;
• content failure: gzip, UTF-8, or WML data was invalid.

Preserving those categories in error messages is part of the explanation. If everything becomes “chat failed,” a learner cannot tell whether to inspect Wireshark, the frame cursor, or the message serializer.

This interactive lab needs JavaScript. The lesson text around it still explains the same idea.

Lesson 9.5

Protocol State Machines and Replay

A packet parser is only the first layer

Parsing one message tells you what its bytes mean. A protocol also defines when that message is legal.

TL;DR: Valid bytes can still request a transition that makes no sense in the current session.

A local Wesnoth test client might move through:

Disconnected -> Connected -> Greeted -> InLobby -> InMatch -> Closed

A chat message before the greeting may be invalid even when its length and UTF-8 text parse perfectly. That makes protocol reversing a state-machine problem as well as a byte-parsing problem.

Replay processes the saved frames in order. A record changes the modeled session only after both its bytes and its transition have passed validation:

If any stage fails, the record and old state remain available for diagnosis; the replay never needs to invent a replacement state.

A message log and a session state are different things

The capture is an ordered log of the frames that crossed the connection. The session state is a model produced by applying their messages in order:

initial state + message 1 + message 2 + ... -> current modeled state

For a five-frame capture of one short session, starting from Connected:

frame 0Greeting v1

frame 1LobbyJoined

frame 2Chat

frame 3MatchStarted

frame 4Goodbye

→ Greeted

→ InLobby

stays InLobby

→ InMatch

→ Closed

The boxes are the log; the labels under them are the state after each frame is applied. The log never stores a state, so a corrected rule can rebuild every state from the same five frames.

Keep the raw log immutable and rebuild the model from it. If a parser or transition rule changes, you can replay the same log and see exactly which frame now produces a different result. Editing the stored “current state” by hand loses that history.

Alternative: Think of the log as receipts and the session state as the current balance. Replay rebuilds the balance from those receipts.

Not every message is an event that happened exactly once: a snapshot can repeat a whole state, and a retransmitted request may appear twice. Store direction, sequence identifiers, and timing with each frame; without them, duplicate bytes cannot be told apart from duplicate actions. The real-time section below defines those message kinds and identifiers.

Model states and messages separately

Receiving a well-formed Chat message is different from being ready to accept it. A connection begins in Connected, reaches a lobby only after greeting and room messages, and can then accept chat. These enums keep the connection state and the parsed message as separate facts:

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum SessionState {
    Connected,
    Greeted,
    InLobby,
    InMatch,
    Closed,
}

#[derive(Clone, Debug, Eq, PartialEq)]
enum Message {
    Greeting { version: u16 },
    LobbyJoined { room: String },
    MatchStarted { scenario: String },
    Chat { text: String },
    Goodbye,
}

#[derive(Debug)]
enum ProtocolError {
    IllegalTransition { state: SessionState, message: &'static str },
    UnsupportedVersion(u16),
    EmptyName,
}

The parser creates Message values. The session machine decides whether each value makes sense now.

A valid Chat arrives while ConnectedThe bytes parse into a Chat, but this state only accepts a greeting, so the gate refuses it and the state stays Connected.

ParserSession state

1. 1

   Data Chat { text: "hello" }

   Step 1: Parser to Session state

   Parsing already succeeded: the bytes are a well-formed Chat. The parser says what the message is, not whether it is allowed now.

2. 2

   Alone (Connected, Chat): no accepting arm

   Step 2: Session state (alone)

   The session machine pairs the current state with the message. Nothing in the match lets a Chat through before the lobby is reached.

3. 3

   Control Err(IllegalTransition)

   Step 3: Session state to Parser

   The caller gets an error naming the state and the message. The session is still Connected and nothing was delivered.

The bytes parse into a Chat, but this state only accepts a greeting, so the gate refuses it and the state stays Connected.

The same Chat arrives while InLobbyIdentical bytes, identical parsed value. The current state is the only difference, and this state accepts a Chat without moving.

ParserSession state

1. 1

   Data Chat { text: "hello" }

   Step 1: Parser to Session state

   The same parsed message as before. Nothing about the bytes changed.

2. 2

   Alone (InLobby, Chat): accepted

   Step 2: Session state (alone)

   This state has an arm for a Chat, so the pairing is legal. The message is allowed because of where the session is.

3. 3

   Reply Ok(()) · state stays InLobby

   Step 3: Session state to Parser

   A chat line does not move the session anywhere. It stays InLobby until a message that triggers a transition arrives.

Identical bytes, identical parsed value. The current state is the only difference, and this state accepts a Chat without moving.

Make transitions explicit

Now pair the current state with one parsed message. A version-1 greeting moves Connected to Greeted; an unsupported version returns a precise error; a chat message in the lobby leaves the state unchanged. The match below makes every accepted transition visible in one place:

fn apply(state: &mut SessionState, message: &Message) -> Result<(), ProtocolError> {
    match (*state, message) {
        (SessionState::Connected, Message::Greeting { version: 1 }) => {
            *state = SessionState::Greeted;
            Ok(())
        }
        (SessionState::Connected, Message::Greeting { version }) => {
            Err(ProtocolError::UnsupportedVersion(*version))
        }
        (SessionState::Greeted, Message::LobbyJoined { room }) if !room.is_empty() => {
            *state = SessionState::InLobby;
            Ok(())
        }
        (SessionState::InLobby, Message::MatchStarted { scenario })
            if !scenario.is_empty() =>
        {
            *state = SessionState::InMatch;
            Ok(())
        }
        (SessionState::InLobby | SessionState::InMatch, Message::Chat { .. }) => Ok(()),
        (_, Message::Goodbye) => {
            *state = SessionState::Closed;
            Ok(())
        }
        (state, message) => Err(ProtocolError::IllegalTransition {
            state,
            message: match message {
                Message::Greeting { .. } => "greeting",
                Message::LobbyJoined { .. } => "lobby_joined",
                Message::MatchStarted { .. } => "match_started",
                Message::Chat { .. } => "chat",
                Message::Goodbye => "goodbye",
            },
        }),
    }
}

Drawn as a diagram, apply is this state machine:

Each arrow is one arm of the match. A message with no arrow out of the current state, such as Chat while Greeted, falls through to the last arm and returns IllegalTransition. A Greeting with any version other than 1 is refused by its own arm with UnsupportedVersion, and Goodbye is accepted in every state, including Closed itself.

This is longer than a chain of if statements, but every state and message variant has an explicit case. Adding a variant makes an exhaustive match stop compiling until you decide how to handle it.

Unchanged bytes can be invalid in a new session phase. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

state = Connected
read the next recorded event
check which events are allowed in this state
update state for an accepted transition
deliver Chat only from InLobby
report the stopping phase

Result: Closed, 1 delivered chat event(s), sequence accepted. Replaying bytes does not recreate their original session context.

Final values: order = normal; event = Goodbye; state = Closed; delivered = 1; accepted = true.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs and follow how the next operation changes the state.

   order = normal.

2. Step 2, line 2. Read Greeting while Connected.

   order = normal; event = Greeting; state = Connected.

3. Step 3, line 4. Accept the event and retain its resulting phase.

   order = normal; event = Greeting; state = Greeted; delivered = 0.

4. Step 4, line 2. Read Join while Greeted.

   order = normal; event = Join; state = Greeted; delivered = 0.

5. Step 5, line 4. Accept the event and retain its resulting phase.

   order = normal; event = Join; state = InLobby; delivered = 0.

6. Step 6, line 2. Read Chat while InLobby.

   order = normal; event = Chat; state = InLobby; delivered = 0.

7. Step 7, line 4. Accept the event and retain its resulting phase.

   order = normal; event = Chat; state = InLobby; delivered = 1.

8. Step 8, line 2. Read Match while InLobby.

   order = normal; event = Match; state = InLobby; delivered = 1.

9. Step 9, line 4. Accept the event and retain its resulting phase.

   order = normal; event = Match; state = InMatch; delivered = 1.

10. Step 10, line 2. Read Goodbye while InMatch.

    order = normal; event = Goodbye; state = InMatch; delivered = 1.

11. Step 11, line 4. Accept the event and retain its resulting phase.

    order = normal; event = Goodbye; state = Closed; delivered = 1.

12. Step 12, line 6. Result: Closed, 1 delivered chat event(s), sequence accepted. Replaying bytes does not recreate their original session context.

    order = normal; event = Goodbye; state = Closed; delivered = 1; accepted = true.

Watch it happenReplay five frames into a session state

Five unchanged frames are Greeting v1, LobbyJoined, Chat, MatchStarted and Goodbye. A separate state marker moves from Connected to Greeted, InLobby, InMatch and Closed. The Chat frame is delivered without moving the state marker out of InLobby.

Read the steps

1. Keep the five captured frames unchanged. The replay starts in Connected; the state marker is separate from the log.
2. Frame 0: version 1 is supported, so Connected becomes Greeted.
3. Frame 1: the lobby message is legal after the greeting. Move from Greeted to InLobby.
4. Frame 2: chat is legal in the lobby. Deliver the chat, but keep the session in InLobby. A message need not change the state.
5. Frame 3: the match-start message is legal in the lobby. Move from InLobby to InMatch.
6. Frame 4: Goodbye closes the session. The state moves to Closed while every original frame remains in the log.
7. The result is a model built from the log. Changing a rule means replaying these same five frames, rather than editing a stored state by hand.

The log records messages. Applying legal messages produces state; chat can be accepted without a state transition.

Store captures as direction plus bytes

Do not save only the decoded text. A replay fixture should preserve:

• client-to-server or server-to-client direction;
• time relative to the previous frame;
• exact frame bytes;
• the game and protocol version;
• the experiment that produced the capture.

#[derive(Clone, Debug)]
struct CapturedFrame {
    after_ms: u32,
    from_server: bool,
    bytes: Vec<u8>,
}

Redact account names or chat content before committing fixtures. A local test capture should contain only invented data.

Replay without a network

Replay takes recorded frames and the parser used for this protocol version. It starts from Connected, validates each frame’s size and delay, parses its bytes, and applies the resulting message to the session state. The returned state is the result of those messages under the current transition function; the error names the frame where interpretation stopped:

use anyhow::Context;

fn replay(
    frames: &[CapturedFrame],
    mut parse: impl FnMut(&[u8]) -> anyhow::Result<Message>,
) -> anyhow::Result<SessionState> {
    let mut state = SessionState::Connected;

    for (index, frame) in frames.iter().enumerate() {
        anyhow::ensure!(frame.bytes.len() <= 64 * 1024, "frame {index} is too large");
        anyhow::ensure!(frame.after_ms <= 30_000, "frame {index} has an absurd delay");

        let message = parse(&frame.bytes)
            .with_context(|| format!("could not parse frame {index}"))?;
        apply(&mut state, &message)
            .map_err(|error| anyhow::anyhow!("frame {index}: {error:?}"))?;
    }

    Ok(state)
}

The test does not sleep for after_ms. It validates timing metadata deterministically. A separate integration test can use a virtual clock to exercise timeouts.

Deterministic replay requires controlling every input that affects a transition: message order, clock readings, random choices, configuration, and initial state. If the same fixture sometimes reaches different states, one of those inputs is still hidden. Pass it into the model instead of reading a real clock or global setting from inside apply.

For example, imagine a lobby rule that closes a connection if no greeting arrives within five seconds. The same captured Greeting should be accepted at 4,900 ms and rejected at 5,100 ms when a test supplies those times to a virtual clock. Reading the wall clock inside apply would make the result depend on how fast the test machine runs, so that timeout test needs a clock-aware version of the model rather than the small replay above.

Mutate one fact at a time

Useful negative replays include:

• truncate one length prefix;
• make a length larger than the available payload;
• send chat before greeting;
• repeat a one-time greeting;
• remove the final frame;
• insert an unknown message type;
• exceed the maximum accepted text length.

The expected outcome is a precise error at a precise frame, never an out-of-bounds read or an infinite wait. Replaying Greeting twice, for example, stops at frame 1 with an IllegalTransition error for state Greeted, because the state machine has no Greeting arrow out of Greeted.

Narration: A single changed field lets a rejected replay identify one failed assumption instead of several competing explanations.

A length larger than the available payload has to fail before any message is decoded:

000

100

200

309

443

544

645

7??

8??

9??

10??

11??

12??

length field: 9

3 payload bytes present

6 bytes the frame claims but does not have

The big-endian length 00 00 00 09 promises 9 payload bytes, but only 3 follow, so 9 − 3 = 6 are missing. A parser that trusted the length would read 6 bytes past the end of the frame; a checked slice turns that into an error for this frame instead.

For messages that may be retried, test idempotence explicitly. Applying the same acknowledgment twice should either preserve the same state or return a clear duplicate error; it should not create a second match, grant a reward twice, or move backward in the session.

Lesson 9.1’s purchase retry showed the server side: it remembers each request identifier and returns the earlier result, or rejects the repeat as a duplicate. In a replay, that identifier is what makes “same request” testable. A repeated Chat with identical text could instead be a new message, so matching bytes alone is not a safe duplicate rule.

Why replay beats manual clicking

Clicking through a live session tests one run once. A replay sends the same frames through a changed parser, framing code, or session rule again in milliseconds. Keep live-network code at the edge and feed the bytes it receives into the same parser and state machine tested by replay. That design keeps transport separate from parsing and makes the protocol model reproducible. 🌐

From session replay to real-time state

The Wesnoth lobby replay above checked whether each message was legal in one conversation. A fast action game adds a second question: which version of the world should the client draw while messages are still travelling? The same capture, framing, and state-machine discipline applies, but ticks and sequence numbers become part of the state.

Real-time games replicate a simulation, not just variables

Most action games advance the world in discrete ticks. If a server runs at 60 ticks per second, one simulation step is approximately:

tick duration = 1 second / 60 = 16.67 milliseconds

The client samples input, labels it, and sends a command. The server processes commands during its own ticks and periodically sends a snapshot or delta. The client then draws frames between confirmed server states:

A field’s meaning depends on the kind of message carrying it. In Lesson 9.1’s move north, (10, 14) in the request is a desired destination, an intent; the same (10, 14) in the server’s reply is an accepted position. An acknowledgment confirms that a command arrived or was applied. A snapshot reports enough state to stand alone, while a delta reports only changes from an identified earlier state, its baseline. Equal-looking bytes can have different authority.

Several similar-looking states can therefore coexist:

State	Purpose	Important label
Last confirmed state	What the server has accepted	Server tick or snapshot ID
Predicted state	Immediate response to local input	Last locally simulated command
Interpolated state	Smooth display of remote objects	Render timestamp between snapshots
Historical state	Rewind or lag-compensation query	Past server tick or time

This explains why a coordinate found in memory can be correct yet appear to “snap back.” You may have found the predicted or rendered copy while a later server snapshot replaced it. Three facts tell the copies apart: which thread writes the field, whether its updates follow input or incoming packets, and which tick or sequence number travels beside it.

Ordering has to be carried in the message

Packet arrival order is not automatically simulation order. UDP packets can arrive late or twice; even over TCP, one stream can contain application events whose own timestamps refer to different moments. Protocols commonly carry:

• a monotonically increasing sequence number;
• a server tick or timestamp;
• an acknowledgment of the newest processed command;
• a bit mask acknowledging several earlier packets;
• a baseline ID telling which snapshot a delta modifies.

Sequence numbers often wrap. For an unsigned N-bit counter, comparisons must use modular arithmetic and accept only a bounded forward window, not a plain incoming > current test.

Work through one 16-bit wraparound. The last accepted sequence number is 65534, and the next two packets carry 65535 and then 0:

current  incoming   naive: incoming > current    (incoming - current) mod 65536
65534    65535      65535 > 65534  -> accept     1   -> accept
65535    0          0 > 65535      -> reject     1   -> accept

The naive test gets the first packet right and the second one wrong. 0 really is one step newer than 65535, but as a plain integer it is smaller, so the packet is thrown away. It gets worse from there: current never advances past 65535, so every sequence number after the wrap looks old in the same way, and the connection appears to stop receiving updates entirely.

The modular test asks a different question — how far forward is incoming from current, counting the wrap? (0 - 65535) mod 65536 is 1, a single step ahead, so the packet is accepted and current moves on. Accept only while that distance stays inside an agreed window such as 1..=32767. A distance of 32768 is exactly halfway around a 16-bit counter and cannot be classified as newer or older from the sequence number alone, so this rule refuses it. A genuinely old packet computes a distance near 65536, which also falls outside the window. The exact window is part of the protocol contract and deserves boundary tests; RFC 1982’s serial-number arithmetic explains the half-range limit.

Applying a delta to the wrong baseline may parse perfectly and still create nonsense. Robust tooling keeps the baseline ID with the bytes, refuses missing dependencies, and bounds how long incomplete state may wait.

The live bot in Lesson 9.6 returns to the simpler Wesnoth lobby conversation. It uses the same separation between framed bytes, decoded messages, and legal session actions that replay made visible.

Lesson 9.6

Clients, Handshakes, and Chat Automation

What we are building

Run wesnothd.exe on your own computer, connect a client named ChatBot, and make it answer \wave with Hello! in the lobby.

This is not a toy protocol. It uses the exact flow we reversed from Wesnoth 1.14.9:

Keep transport, decoding, session meaning, and behavior as separate layers. One incoming message should travel through the pipeline before any reply is built:

A parsed message still needs session context. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

read the selected inputs
evaluate each stated condition
combine the required conditions
perform the effect only when accepted
otherwise retain the old state
report the result and its limits

Result: one greeting reply is prepared for serialization, compression and framing.

Final values: phase = lobby; message = wave; in_lobby = true; text_matches = true; accepted = true; reply = Hello! from ChatBot.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs and follow how the next operation changes the state.

   phase = lobby; message = wave.

2. Step 2, line 2. Valid text alone does not establish that a chat action belongs in this phase.

   phase = lobby; message = wave; in_lobby = true; text_matches = true.

3. Step 3, line 3. Accept only when the required conditions all hold.

   phase = lobby; message = wave; in_lobby = true; text_matches = true; accepted = true.

4. Step 4, line 4. The model performs the accepted effect.

   phase = lobby; message = wave; in_lobby = true; text_matches = true; accepted = true; reply = Hello! from ChatBot.

5. Step 5, line 6. Result: one greeting reply is prepared for serialization, compression and framing.

   phase = lobby; message = wave; in_lobby = true; text_matches = true; accepted = true; reply = Hello! from ChatBot.

Watch it happenTurn a lobby command into a framed reply

A bounded gzip frame expands into a WML message whose text is backslash wave and room is lobby. The validated chat event reaches a handler, which creates Hello! from ChatBot. The reply is serialized, compressed and framed with a four-byte length before entering TCP.

Read the steps

1. Read the four-byte big-endian length, enforce the frame limit, and keep collecting until the whole gzip payload is available.
2. Decompress the complete payload. The WML text now has message and room fields; compressed bytes are no longer passed to the behavior layer.
3. Parse and validate the WML. The client is already in the lobby, where this chat message is permitted. Build an owned Chat event containing the text \wave.
4. Pass the event to the command handler. Its text matches \wave; ordinary chat would finish here without producing a reply.
5. The bot constructs Hello! with room lobby and sender ChatBot. These fields are new outbound data; the received frame is not copied back unchanged.
6. Serialize the reply as WML, compress it with gzip, and place its byte count in a four-byte big-endian header. The encoder supplies the actual compressed length.
7. Write all bytes of that new frame to the TCP stream. The header and complete compressed reply leave together; transport can still split them across reads.

This shows the layered client design. The compact prototype below recognizes its command with a text test.

This separation lets a malformed frame fail before it reaches chat behavior, and it lets the command handler work with events instead of raw network bytes:

A well-formed command reaches the handlerEach layer passes on only what it has checked: a complete frame, then named fields, then an event.

TCP streamread_frameDecoderCommand handler

1. 1

   Data u32 length, then that many bytes

   Step 1: TCP stream to read_frame

   read_frame reads the four-byte big-endian length, checks it against MAX_FRAME (1,048,576), then reads exactly that many bytes.

2. 2

   Data one complete gzip payload

   Step 2: read_frame to Decoder

   Only a complete frame is passed on; a partial read never reaches the decoder.

3. 3

   Data Chat event: text "\wave"

   Step 3: Decoder to Command handler

   gunzip and the WML parser turn bytes into named fields, so the handler receives an event, not bytes.

4. 4

   Reply new framed reply: Hello!

   Step 4: Command handler to TCP stream

   The text matches \wave, so the handler builds new reply fields; they are serialized, compressed and framed before writing.

Each layer passes on only what it has checked: a complete frame, then named fields, then an event.

A malformed payload stops at the decoderThe frame is complete, but its bytes are not valid gzip. The error ends the chain before any behavior runs.

TCP streamread_frameDecoderCommand handler

1. 1

   Data u32 length, then that many bytes

   Step 1: TCP stream to read_frame

   The length is within MAX_FRAME and all its bytes arrive, so framing succeeds.

2. 2

   Data one complete payload

   Step 2: read_frame to Decoder

   The decoder receives the whole payload.

3. 3

   Alone gunzip fails: return an error

   Step 3: Decoder (alone)

   No Chat event is built, so the command handler never runs and no reply is sent.

The frame is complete, but its bytes are not valid gzip. The error ends the chain before any behavior runs.

Connect with timeouts

Before reading protocol bytes, the client needs a connection that cannot silently drift to another host or wait forever for a stalled local server. connect_local accepts one socket address, checks that it is the course loopback endpoint, then sets finite connect, read, and write waits:

use std::{
    io::{self, Read, Write},
    net::{SocketAddr, TcpStream},
    time::Duration,
};

const MAX_FRAME: usize = 1_048_576;

fn connect_local(address: SocketAddr) -> io::Result<TcpStream> {
    if !address.ip().is_loopback() || address.port() != 15_000 {
        return Err(io::Error::new(
            io::ErrorKind::PermissionDenied,
            "this Wesnoth lab connects only to loopback port 15000",
        ));
    }

    let stream = TcpStream::connect_timeout(&address, Duration::from_secs(3))?;
    stream.set_read_timeout(Some(Duration::from_secs(5)))?;
    stream.set_write_timeout(Some(Duration::from_secs(5)))?;
    Ok(stream)
}

TL;DR: A timeout bounds how long the bot can wait; a verified endpoint bounds whom it talks to.

The timeout prevents a broken server or parser from hanging the bot forever.

Read and write exact frames

Lesson 9.4’s encode_wesnoth_frame performs the gzip compression and adds the four-byte big-endian length. write_all matters because one write may send only part of a buffer.

byte 0-3`u32` length, big-endian

byte 4..gzip-compressed WML, that many bytes

The frame read_frame parses below. It trusts only the four-byte header at first, then reads exactly length more bytes before decompressing.

The two helpers implement the strip in opposite directions. send_wml frames text and writes all bytes; read_frame reads the four-byte length, checks the cap, then reads that many payload bytes before a decoder sees them. A short TCP read is normal, so both paths use the exact-read/write operations:

fn send_wml(stream: &mut TcpStream, wml: &str) -> anyhow::Result<()> {
    let frame = encode_wesnoth_frame(wml)?;
    stream.write_all(&frame)?;
    stream.flush()?;
    Ok(())
}

fn read_frame(stream: &mut TcpStream) -> io::Result<Vec<u8>> {
    let mut header = [0_u8; 4];
    stream.read_exact(&mut header)?;
    let length = u32::from_be_bytes(header) as usize;
    if length > MAX_FRAME {
        return Err(io::Error::new(io::ErrorKind::InvalidData, "frame too large"));
    }

    let mut frame = Vec::with_capacity(4 + length);
    frame.extend_from_slice(&header);
    frame.resize(4 + length, 0);
    stream.read_exact(&mut frame[4..])?;
    Ok(frame)
}

read_exact handles TCP fragmentation. It does not assume one receive call equals one Wesnoth message.

Handle early server replies

The opening negotiation is special, so collect its replies until a short quiet period. The historical capture received 41 bytes.

Alternative: Treat the connection like a conversation: inspect the reply before deciding which question is appropriate next.

This quiet-period reader is only for the unframed negotiation reply. It collects successive TCP reads until the server closes or pauses briefly, while the size cap prevents an unexpected reply from growing forever. After login, read_frame uses the protocol’s length field instead:

fn read_server_batch(stream: &mut TcpStream) -> io::Result<Vec<u8>> {
    stream.set_read_timeout(Some(Duration::from_millis(250)))?;
    let mut collected = Vec::new();
    let mut chunk = [0_u8; 512];

    loop {
        match stream.read(&mut chunk) {
            Ok(0) => break,
            Ok(count) => {
                collected.extend_from_slice(&chunk[..count]);
                if collected.len() > MAX_FRAME {
                    return Err(io::Error::new(io::ErrorKind::InvalidData, "reply too large"));
                }
            }
            Err(error)
                if matches!(
                    error.kind(),
                    io::ErrorKind::WouldBlock | io::ErrorKind::TimedOut
                ) => break,
            Err(error) => return Err(error),
        }
    }

    stream.set_read_timeout(Some(Duration::from_secs(5)))?;
    Ok(collected)
}

Keep this helper for negotiation only. Once the session is in the lobby, use the frame reader.

Perform the Wesnoth 1.14.9 login

Windows VM

The parser and frame writer now have a concrete job. The course server first expects four zero negotiation bytes, then a version message and a login message in compressed WML frames. log_in sends them in that order and reads each reply before advancing; a nickname is validated before it enters WML:

fn log_in(stream: &mut TcpStream, nickname: &str) -> anyhow::Result<()> {
    let nickname = simple_field(nickname)?;

    // The first message is not compressed or length-prefixed.
    stream.write_all(&[0, 0, 0, 0])?;
    let reply = read_server_batch(stream)?;
    anyhow::ensure!(!reply.is_empty(), "server did not negotiate");

    send_wml(
        stream,
        "[version]\nversion=\"1.14.9\"\n[/version]",
    )?;
    let _version_reply = read_server_batch(stream)?;

    send_wml(
        stream,
        &format!("[login]\nusername=\"{nickname}\"\n[/login]"),
    )?;
    let _login_reply = read_server_batch(stream)?;
    Ok(())
}

If the server rejects the version, compare the decompressed frame with the one produced by the official 1.14.9 client. Do not randomly change compressed bytes; fix the WML and recompress it.

Answer \wave

Once the login succeeds, the client announces itself and loops over framed server messages. A decoded \wave event will produce the reply described above; ordinary messages leave the connection running:

fn run(address: SocketAddr, nickname: String) -> anyhow::Result<()> {
    let mut stream = connect_local(address)?;
    log_in(&mut stream, &nickname)?;

    let nickname = simple_field(&nickname)?;
    send_wml(
        &mut stream,
        &format!(
            "[message]\nmessage=\"ChatBot connected\"\nroom=\"lobby\"\nsender=\"{nickname}\"\n[/message]"
        ),
    )?;

    loop {
        let frame = read_frame(&mut stream)?;
        let wml = decode_wesnoth_frame(&frame)?;
        println!("{wml}");

        if wml.contains("message=\"\\wave\"") {
            send_wml(
                &mut stream,
                &format!(
                    "[message]\nmessage=\"Hello!\"\nroom=\"lobby\"\nsender=\"{nickname}\"\n[/message]"
                ),
            )?;
        }
    }
}

Prove the bot works

1. Start wesnothd.exe.
2. Run the client with address 127.0.0.1:15000 and nickname ChatBot.
3. Join localhost:15000 from the normal Wesnoth client.
4. Type \wave in the lobby.
5. Confirm Hello! appears from ChatBot.

That visible lobby response proves the real negotiation, version, login, gzip framing, Simple WML, receive loop, and command logic all work together.

Keep it well behaved

Limit message length, respond at most once per command frame, add a short send rate limit, and exit cleanly when the local server closes. A parser error should print the frame size and stop; it should not enter an automatic reconnect storm.

Narration: A reply rate limit prevents one repeated command from filling the lobby with messages.

Run the complete bot

After building the workspace, start the course wesnothd.exe on port 15000 and run:

.\target\i686-pc-windows-msvc\release\wesnoth_chatbot.exe ChatBot

The full client is wesnoth_chatbot.rs. The framing/parser it imports is wesnoth_protocol.rs. Together they implement the real loopback connection, timeouts, four-zero negotiation, version frame, login frame, lobby receive loop, \wave recognition, and Hello! reply.

Complete lab source: wesnoth_chatbot.rs

wesnoth_chatbot.rs

windows-labs/src/bin/ · Rust, 100 lines

View on GitHub ↗

use std::{
    io::{self, Read, Write},
    net::{IpAddr, Ipv4Addr, SocketAddr, TcpStream},
    time::Duration,
};

use anyhow::{Context, Result};
use gha_windows_labs::wesnoth_protocol::{
    MAX_FRAME, decode_frame, read_frame, send_wml, simple_field,
};

fn read_server_batch(stream: &mut TcpStream) -> io::Result<Vec<u8>> {
    stream.set_read_timeout(Some(Duration::from_millis(250)))?;
    let mut collected = Vec::new();
    let mut chunk = [0_u8; 512];
    loop {
        match stream.read(&mut chunk) {
            Ok(0) => break,
            Ok(count) => {
                collected.extend_from_slice(&chunk[..count]);
                if collected.len() > MAX_FRAME {
                    return Err(io::Error::new(
                        io::ErrorKind::InvalidData,
                        "reply is too large",
                    ));
                }
            }
            Err(error)
                if matches!(
                    error.kind(),
                    io::ErrorKind::WouldBlock | io::ErrorKind::TimedOut
                ) =>
            {
                break;
            }
            Err(error) => return Err(error),
        }
    }
    stream.set_read_timeout(Some(Duration::from_secs(5)))?;
    Ok(collected)
}

fn log_in(stream: &mut TcpStream, nickname: &str) -> Result<()> {
    let nickname = simple_field(nickname)?;
    stream.write_all(&[0, 0, 0, 0])?;
    anyhow::ensure!(
        !read_server_batch(stream)?.is_empty(),
        "server did not negotiate"
    );

    send_wml(stream, "[version]\nversion=\"1.14.9\"\n[/version]")?;
    anyhow::ensure!(
        !read_server_batch(stream)?.is_empty(),
        "server did not answer version"
    );

    send_wml(
        stream,
        &format!("[login]\nusername=\"{nickname}\"\n[/login]"),
    )?;
    let _login_reply = read_server_batch(stream)?;
    Ok(())
}

fn main() -> Result<()> {
    let nickname = std::env::args().nth(1).unwrap_or_else(|| "ChatBot".into());
    simple_field(&nickname)?;
    let address = SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 15_000);
    let mut stream = TcpStream::connect_timeout(&address, Duration::from_secs(3))
        .context("start local wesnothd.exe on port 15000 first")?;
    stream.set_read_timeout(Some(Duration::from_secs(5)))?;
    stream.set_write_timeout(Some(Duration::from_secs(5)))?;
    log_in(&mut stream, &nickname)?;
    send_wml(
        &mut stream,
        &format!(
            "[message]\nmessage=\"ChatBot connected\"\nroom=\"lobby\"\nsender=\"{nickname}\"\n[/message]"
        ),
    )?;
    println!("{nickname} joined the local Wesnoth lobby. Type \\wave from a normal client.");

    loop {
        let frame = match read_frame(&mut stream) {
            Ok(frame) => frame,
            Err(error) if error.kind() == io::ErrorKind::UnexpectedEof => break,
            Err(error) => return Err(error.into()),
        };
        let wml = decode_frame(&frame)?;
        println!("{wml}");
        if wml.contains("message=\"\\wave\"") {
            send_wml(
                &mut stream,
                &format!(
                    "[message]\nmessage=\"Hello!\"\nroom=\"lobby\"\nsender=\"{nickname}\"\n[/message]"
                ),
            )?;
        }
    }
    Ok(())
}

The reusable gzip/WML framing stays in one tested protocol module; this file owns the session state. That split is intentional: byte framing should not know what \wave means, and chat behavior should not reimplement gzip.

An external client answering Wesnoth chatSilent demonstration · 11 seconds

The original external-client demonstration shows a chat command receiving a reply. The visible exchange depends on the earlier connection, negotiation, login, frame decoding, and command handling; a valid message alone is not a complete client.

Download An external client answering Wesnoth chat video

This bot owns its own server connection. Lesson 9.7 puts a relay between the official client and the same local server, so we can observe an existing conversation before choosing one bounded change.

Lesson 9.7

Local Proxies and Bidirectional Streams

What a proxy does

A local TCP proxy accepts a client connection, opens a connection to your local server, and relays bytes both ways.

TL;DR: A relay preserves both directions of the conversation while providing a place to observe their bytes.

Begin as a transparent relay. If the protocol fails before you modify anything, fix the relay first. This first version treats bytes as bytes. Only after it passes the normal Wesnoth login should the proxy switch to the framed, message-aware upstream path below. The two modes have different failure behavior: a transparent relay does not need to decode; a message-aware relay must decide what to do when decoding fails.

Restrict both endpoints

A proxy is easy to connect to the wrong service if an address is copied from a different test. Validate both endpoints as loopback addresses before opening either connection; this helper returns the accepted address so the caller cannot forget which one passed the check:

use std::net::{IpAddr, SocketAddr};

fn require_loopback(address: SocketAddr) -> anyhow::Result<SocketAddr> {
    anyhow::ensure!(
        address.ip() == IpAddr::V4(std::net::Ipv4Addr::LOCALHOST)
            || address.ip() == IpAddr::V6(std::net::Ipv6Addr::LOCALHOST),
        "proxy endpoints must be loopback addresses"
    );
    Ok(address)
}

This course proxy is intentionally not a general remote interception tool.

This board needs JavaScript to move things around. Here is the lesson’s sorting:

Accepted

• 127.0.0.1:15000 (the local Wesnoth server). 127.0.0.1 is the IPv4 loopback address, so this endpoint stays on the same computer.
• 127.0.0.1:27015 (the proxy’s own address). Same loopback address. The check looks at the IP address, not the port.
• [::1]:15000 (IPv6 localhost). The check accepts the IPv6 localhost address too, so this one passes as well.

Refused with an error

• 192.168.1.20:15000 (another computer on the network). It is not loopback. This proxy is intentionally not a general remote interception tool, so the check refuses before opening any connection.
• 8.8.8.8:15000 (a public internet address). Not loopback, so it is refused before anything connects.
• 0.0.0.0:27015 (every network interface). The code compares with the localhost addresses exactly, and 0.0.0.0 is not one of them, so it is refused.

Relay one direction

The standard library can copy a stream until EOF:

use std::{io, net::TcpStream};

fn relay(mut source: TcpStream, mut destination: TcpStream) -> io::Result<u64> {
    io::copy(&mut source, &mut destination)
}

TCP is full duplex, so run two relays:

fn proxy_pair(client: TcpStream, server: TcpStream) -> anyhow::Result<()> {
    let client_to_server = client.try_clone()?;
    let server_to_client = server.try_clone()?;

    let upstream = std::thread::spawn(move || relay(client_to_server, server));
    let downstream = std::thread::spawn(move || relay(server_to_client, client));

    upstream.join().map_err(|_| anyhow::anyhow!("upstream relay panicked"))??;
    downstream.join().map_err(|_| anyhow::anyhow!("downstream relay panicked"))??;
    Ok(())
}

For many connections, an async runtime such as Tokio scales better. Threads keep the first lab easy to see.

Half-closes matter

One direction can reach EOF while the other still has data to send. A reliable local Wesnoth proxy should use shutdown(Write) on the destination after that relay finishes, then let the server-to-client relay deliver any remaining game messages before both sockets close.

Alternative: One side can stop speaking while still listening. Closing both directions immediately discards that distinction.

Log metadata, not unlimited payloads

Useful bounded log lines include:

connection opened
client → server: 142 bytes
server → client: 68 bytes
connection closed

If you parse frame previews, cap their length and avoid logging secrets or full chat histories by default.

Diagnose handshake failures

A failed local handshake.

When a transparent proxy fails, compare direct and proxied captures:

• Did both directions start?
• Were bytes changed or dropped?
• Did the client send the server’s port or address inside the protocol?
• Is there TLS (Transport Layer Security, the encryption behind HTTPS) or a certificate bound to a hostname?
• Did a relay block because it waited for EOF?
• Did one side expect a UDP companion channel?

Do not bypass certificate validation. For encrypted protocols, create a development configuration with certificates you control.

Parse without changing

The safest next step is a tee:

read bytes
→ feed a copy to a bounded parser/log channel
→ forward the original bytes unchanged

Keep parsing off the relay’s critical path. If the parser cannot keep up, drop log entries rather than blocking the connection. This tee is for observation. It cannot decide where to insert a new message, because a TCP read does not identify a frame boundary. The insertion path therefore waits for a complete bounded frame and handles parse errors explicitly.

Bound the parser’s queue as well as each frame. What a full queue may discard depends on the message meaning: a newer position snapshot may replace an older one under a documented policy, but discarding an unprocessed item grant loses the item. The tee can drop its log copies only because the relay still forwards every original frame; the forwarding path itself never drops game bytes to keep up.

Narration: Observing first provides a baseline: later edits can be compared against traffic that the relay already handled correctly.

Inject a real local Wesnoth chat response

The original lab used proxy port 27015 and server port 15000. Start wesnothd.exe, start the local proxy, then tell Wesnoth to connect to localhost:27015. Reaching the normal lobby through the proxy proves both directions work.

For injection, one task must own the upstream writer so two threads cannot interleave bytes. After the special four-zero negotiation has passed, have the client-to-server path read one complete Wesnoth frame at a time. Forward the original bytes unchanged, but inspect a decoded copy:

use std::io::Write;

fn forward_with_chat_hook(
    client: &mut TcpStream,
    server: &mut TcpStream,
) -> anyhow::Result<()> {
    loop {
        // 📦 `read_frame` returns one complete length-prefixed message. A plain
        // `read` here could split one frame or combine several frames.
        let original = read_frame(client)?;
        // A decode error stops this modified upstream path before sending this frame.
        // The transparent relay above has no such dependency on parsing.
        let decoded = decode_wesnoth_frame(&original)?;

        // 📦 Preserve the legitimate game packet exactly as captured and send
        // it first; inspecting a copy must not silently replace normal traffic.
        server.write_all(&original)?;

        if decoded.contains("message=\"\\wave\"") {
            // 🔒 This task is the sole owner of `server`, so the original and
            // injected frames cannot be interleaved by the reverse relay.
            let injected = encode_chat("ChatBot", "Hello!")?;
            server.write_all(&injected)?;
        }
        server.flush()?;
    }
}

Keep the server-to-client direction as an ordinary transparent relay. The handshake needs one small state flag because its first four zero bytes are not a gzip frame; forward those four bytes directly before switching the upstream parser into framed mode.

The contains check is enough only for this controlled one-command fixture. It could match that text inside an unrelated WML field. A general proxy should parse the WML tree from Lesson 9.4 and inspect the actual chat-message field and session state before inserting a response. The complete lab below uses the same narrow shortcut, so do not treat it as a general message router.

Now send \wave from the real Wesnoth client. The server receives both the original command and the injected Hello! message on the already authenticated connection. This is the practical advantage of the proxy: the official client performs its own login while the proxy observes and modifies the loopback traffic.

Verify the proxy

The local proxy is correct when:

• direct and proxied sessions behave the same;
• byte counts agree with a packet capture;
• either side closing ends the session cleanly;
• an undecodable frame in the message-aware mode returns a clear error before that frame is forwarded; proxy_pair then shuts down both sockets before waiting for the reverse relay. A separate transparent mode could relay bytes without decoding them;
• non-loopback endpoints are rejected;
• the only modification is the documented \wave response;
• removing both decoding and injection returns the proxy to byte-for-byte transparent behavior.

Run the complete proxy

Windows VM

Start wesnothd.exe on its normal local port, then run:

.\target\i686-pc-windows-msvc\release\wesnoth_proxy.exe

Connect the official Wesnoth 1.14.9 client to localhost:27015. Type \wave after login and confirm the injected Hello! appears. The complete two-direction relay is wesnoth_proxy.rs. It binds only to loopback, forwards the unframed negotiation, parses bounded upstream frames, preserves each original frame, injects through the single upstream writer, half-closes both directions, and keeps the server-to-client stream transparent.

Complete lab source: wesnoth_proxy.rs

wesnoth_proxy.rs

windows-labs/src/bin/ · Rust, 94 lines

View on GitHub ↗

Show the codeHide the code

use std::{
    io::{self, Read, Write},
    net::{IpAddr, Ipv4Addr, Shutdown, SocketAddr, TcpListener, TcpStream},
    thread,
    time::Duration,
};

use anyhow::{Context, Result};
use gha_windows_labs::wesnoth_protocol::{decode_frame, encode_chat, read_frame};

fn loopback(port: u16) -> SocketAddr {
    SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), port)
}

fn upstream(mut client: TcpStream, mut server: TcpStream) -> Result<()> {
    // The negotiation request is the protocol's one unframed client message.
    let mut negotiation = [0_u8; 4];
    client.read_exact(&mut negotiation)?;
    anyhow::ensure!(negotiation == [0, 0, 0, 0], "unexpected negotiation bytes");
    server.write_all(&negotiation)?;
    server.flush()?;

    loop {
        let frame = match read_frame(&mut client) {
            Ok(frame) => frame,
            Err(error) if error.kind() == io::ErrorKind::UnexpectedEof => break,
            Err(error) => return Err(error.into()),
        };
        let decoded = decode_frame(&frame)?;
        server.write_all(&frame)?;
        if decoded.contains("message=\"\\wave\"") {
            server.write_all(&encode_chat("ChatBot", "Hello!")?)?;
        }
        server.flush()?;
    }
    server.shutdown(Shutdown::Write)?;
    Ok(())
}

fn downstream(mut server: TcpStream, mut client: TcpStream) -> Result<()> {
    io::copy(&mut server, &mut client)?;
    client.shutdown(Shutdown::Write)?;
    Ok(())
}

fn proxy_pair(client: TcpStream) -> Result<()> {
    let server = TcpStream::connect_timeout(&loopback(15_000), Duration::from_secs(3))
        .context("start local wesnothd.exe on port 15000 first")?;
    client.set_read_timeout(Some(Duration::from_secs(30)))?;
    client.set_write_timeout(Some(Duration::from_secs(5)))?;
    server.set_read_timeout(Some(Duration::from_secs(30)))?;
    server.set_write_timeout(Some(Duration::from_secs(5)))?;

    // Keep control handles so a decoding failure in the upstream worker can
    // close both directions before we wait for the downstream worker.
    let client_control = client.try_clone()?;
    let server_control = server.try_clone()?;
    let upstream_client = client.try_clone()?;
    let downstream_server = server.try_clone()?;
    let up = thread::spawn(move || upstream(upstream_client, server));
    let down = thread::spawn(move || downstream(downstream_server, client));
    let upstream_result = up
        .join()
        .map_err(|_| anyhow::anyhow!("upstream thread panicked"))
        .and_then(|result| result);
    if upstream_result.is_err() {
        let _ = client_control.shutdown(Shutdown::Both);
        let _ = server_control.shutdown(Shutdown::Both);
    }
    let downstream_result = down
        .join()
        .map_err(|_| anyhow::anyhow!("downstream thread panicked"))
        .and_then(|result| result);
    upstream_result?;
    downstream_result?;
    Ok(())
}

fn main() -> Result<()> {
    let listener = TcpListener::bind(loopback(27_015))?;
    println!("Wesnoth proxy listening on 127.0.0.1:27015 -> 127.0.0.1:15000");
    println!("Connect the official 1.14.9 client to localhost:27015.");
    for connection in listener.incoming() {
        let client = connection?;
        anyhow::ensure!(
            client.peer_addr()?.ip().is_loopback(),
            "client is not loopback"
        );
        if let Err(error) = proxy_pair(client) {
            eprintln!("session ended: {error:#}");
        }
    }
    Ok(())
}

use std::{
    io::{self, Read, Write},
    net::{IpAddr, Ipv4Addr, Shutdown, SocketAddr, TcpListener, TcpStream},
    thread,
    time::Duration,
};

use anyhow::{Context, Result};
use gha_windows_labs::wesnoth_protocol::{decode_frame, encode_chat, read_frame};

fn loopback(port: u16) -> SocketAddr {
    SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), port)
}

fn upstream(mut client: TcpStream, mut server: TcpStream) -> Result<()> {
    // ⚠️ The negotiation request is the protocol's one unframed client message.
    let mut negotiation = [0_u8; 4];
    client.read_exact(&mut negotiation)?;
    anyhow::ensure!(negotiation == [0, 0, 0, 0], "unexpected negotiation bytes");
    server.write_all(&negotiation)?;
    server.flush()?;

    loop {
        let frame = match read_frame(&mut client) {
            Ok(frame) => frame,
            // ✅ EOF stops this direction before any incomplete frame is forwarded.
            Err(error) if error.kind() == io::ErrorKind::UnexpectedEof => break,
            Err(error) => return Err(error.into()),
        };
        let decoded = decode_frame(&frame)?;
        // 🔁 Forward the authentic frame even when it does not trigger the lab.
        server.write_all(&frame)?;
        if decoded.contains("message=\"\\wave\"") {
            server.write_all(&encode_chat("ChatBot", "Hello!")?)?;
        }
        server.flush()?;
    }
    // 🧹 Signal “no more client bytes” while leaving the read half available
    // for the downstream task to drain the server's final response.
    server.shutdown(Shutdown::Write)?;
    Ok(())
}

fn downstream(mut server: TcpStream, mut client: TcpStream) -> Result<()> {
    io::copy(&mut server, &mut client)?;
    client.shutdown(Shutdown::Write)?;
    Ok(())
}

fn proxy_pair(client: TcpStream) -> Result<()> {
    let server = TcpStream::connect_timeout(&loopback(15_000), Duration::from_secs(3))
        .context("start local wesnothd.exe on port 15000 first")?;
    client.set_read_timeout(Some(Duration::from_secs(30)))?;
    client.set_write_timeout(Some(Duration::from_secs(5)))?;
    server.set_read_timeout(Some(Duration::from_secs(30)))?;
    server.set_write_timeout(Some(Duration::from_secs(5)))?;

    // 🔀 Clones share each socket but each thread owns one traffic direction.
    // Only `upstream` ever writes to the game server, preserving frame order.
    let client_control = client.try_clone()?;
    let server_control = server.try_clone()?;
    let upstream_client = client.try_clone()?;
    let downstream_server = server.try_clone()?;
    let up = thread::spawn(move || upstream(upstream_client, server));
    let down = thread::spawn(move || downstream(downstream_server, client));
    let upstream_result = up
        .join()
        .map_err(|_| anyhow::anyhow!("upstream thread panicked"))
        .and_then(|result| result);
    if upstream_result.is_err() {
        let _ = client_control.shutdown(Shutdown::Both);
        let _ = server_control.shutdown(Shutdown::Both);
    }
    let downstream_result = down
        .join()
        .map_err(|_| anyhow::anyhow!("downstream thread panicked"))
        .and_then(|result| result);
    upstream_result?;
    downstream_result?;
    Ok(())
}

fn main() -> Result<()> {
    let listener = TcpListener::bind(loopback(27_015))?;
    println!("Wesnoth proxy listening on 127.0.0.1:27015 -> 127.0.0.1:15000");
    println!("Connect the official 1.14.9 client to localhost:27015.");
    for connection in listener.incoming() {
        let client = connection?;
        anyhow::ensure!(
            client.peer_addr()?.ip().is_loopback(),
            "client is not loopback"
        );
        if let Err(error) = proxy_pair(client) {
            eprintln!("session ended: {error:#}");
        }
    }
    Ok(())
}

Only the upstream task writes to the server. That single-writer rule prevents the original frame and injected frame from being interleaved. The downstream task stays byte-for-byte transparent, so one experimental hook cannot accidentally rewrite both directions.

The proxy needs TCP because its two endpoints are network clients. When two cooperating programs run on the same computer, they can exchange the same kind of bounded message through shared memory or a named pipe instead. Lessons 9.8 and 9.9 compare those local transports.

Lesson 9.8

Shared Memory Between Processes

Processes normally have separate address spaces

The proxy copied messages between sockets. A supported game add-on and a separate visualizer could instead agree on a local shared byte region. The message still needs a layout and a lifetime; only the transport changes.

Two running programs can both use the address 0x00400000, yet those numbers can refer to completely different physical storage. Each process has its own virtual address space.

That separation is a safety feature. A map helper should not need to patch a game process merely to receive a supported event from a mod. Windows provides interprocess communication, or IPC, for programs that intentionally cooperate.

Named shared memory is one IPC option:

The two views do not need the same virtual address. Windows connects both views to the same file-mapping object.

The programs share bytes through one kernel-managed object, not pointers to each other’s private address spaces:

The views may have different addresses, but they expose the same mapped bytes. The small protocol is what turns those bytes back into a trustworthy value.

A mapping object is not the view

These terms describe different resources:

• A file-mapping object tells Windows what storage is shared, how large it is, and which protections are allowed.
• A view maps some or all of that object into one process’s virtual address space.
• A handle names the mapping object.
• A pointer from MapViewOfFile names the beginning of one process’s view.

Closing the handle and unmapping the view are separate cleanup operations. The lab uses OwnedHandle for the first and a small MappedView type with Drop for the second.

TL;DR: The handle names the shared mapping; the view is the address where this process can access its bytes.

Paging-file-backed means no ordinary file is required

CreateFileMappingW normally accepts a file handle. Passing INVALID_HANDLE_VALUE asks Windows for storage backed by the system paging file instead. We must provide a nonzero maximum size—in this lab, 4096 bytes.

The name starts with Local\, so it lives in the current logon session’s local namespace. We do not use Global\, which can require extra privilege and makes the scope wider than this lab needs.

Bytes need a tiny protocol

Memory has no built-in String type. Both programs must agree on a layout:

0..4little-endian `u32` text length

4..4+lenUTF-8 text

4+len..zeroed

The shared-memory wire format. HEADER_BYTES is 4; everything from 4 + len onward is left as zero padding.

The four-byte length makes embedded zero bytes unambiguous and lets the reader reject impossible sizes before creating a string slice.

Alternative: Sharing storage is like sharing a notebook: agree on the page layout and on when a complete entry may be read.

This is the beginning of serialization: turning structured information into an agreed byte representation.

The original reader treats the first zero byte as the end of UTF-8 text. A legitimate embedded zero therefore shortens the message. The improved format uses the four-byte length header shown above; the writer must use that same format too.

Before: stop text at the first zero byte

fn read_text(bytes: &[u8]) -> anyhow::Result<&str> {
    let end = bytes.iter().position(|byte| *byte == 0)
        .unwrap_or(bytes.len()); // A zero inside the text looks like its end.
    Ok(std::str::from_utf8(&bytes[..end])?)
}

Validate that a complete header exists before slicing, then check its declared length against the remaining mapping. The length describes payload bytes, not the mapping’s zero padding.

Improved code

fn read_text(bytes: &[u8]) -> anyhow::Result<&str> {
    anyhow::ensure!(bytes.len() >= HEADER_BYTES, "shared-memory header is truncated");
    let length = u32::from_le_bytes(bytes[..HEADER_BYTES].try_into()?) as usize;
    anyhow::ensure!(length <= bytes.len() - HEADER_BYTES,
        "shared-memory length is larger than its mapping");
    let payload = &bytes[HEADER_BYTES..HEADER_BYTES + length];
    Ok(std::str::from_utf8(payload)?)
}

Show the diff

fn read_text(bytes: &[u8]) -> anyhow::Result<&str> {
    let end = bytes.iter().position(|byte| *byte == 0)
        .unwrap_or(bytes.len()); // A zero inside the text looks like its end.
    Ok(std::str::from_utf8(&bytes[..end])?)
    anyhow::ensure!(bytes.len() >= HEADER_BYTES, "shared-memory header is truncated");
    let length = u32::from_le_bytes(bytes[..HEADER_BYTES].try_into()?) as usize;
    anyhow::ensure!(length <= bytes.len() - HEADER_BYTES,
        "shared-memory length is larger than its mapping");
    let payload = &bytes[HEADER_BYTES..HEADER_BYTES + length];
    Ok(std::str::from_utf8(payload)?)
}

Takeaway: agree on an explicit length format at both ends, and validate the header before trusting it.

One hazard is worth naming here, even though this one-shot lab sidesteps it by writing only once. Shared memory gives two processes access to the same bytes. It gives them no agreement about when. Nothing prevents a reader from arriving in the middle of a write:

The new length arrives before the new textThe mapping holds READY (5 bytes). The writer is replacing it with RUNNING (7 bytes) and stores the new length first.

1. 1 READY is published

   Shared mapping

   Bytes 0 to 3: length

   05= 5

   00

   00

   00

   Bytes 4 to 10: text

   R52

   E45

   A41

   D44

   Y59

   00zero

   00zero

   A little-endian u32 length of 5, then the five text bytes. The rest of the mapping is zero.

2. 2 Length 7 stored, text not yet

   Shared mapping

   Bytes 0 to 3: length

   07= 7

   00

   00

   00

   Bytes 4 to 10: text

   R52

   E45

   A41

   D44

   Y59

   00zero

   00zero

   What the reader returns

   read_text

   READY + two zero bytesneither the old nor the new message

   If the reader copies now, 7 fits in the mapping and zero bytes are valid UTF-8, so both checks in read_text pass.

3. 3 Both sides take one named mutex

   Shared mapping

   Bytes 0 to 3: length

   07= 7

   00

   00

   00

   Bytes 4 to 10: text

   R52

   U55

   N4E

   N4E

   I49

   N4E

   G47

   Order

   Writer

   lock

   length and text

   unlock

   Reader

   lock: waits

   copy 11 bytes

   unlock

   The reader waits while the writer holds the mutex, so it copies all of READY or all of RUNNING, never a mix.

The mapping holds READY (5 bytes). The writer is replacing it with RUNNING (7 bytes) and stores the new length first.

The reader’s length check passes, the UTF-8 decode very likely succeeds, and the text returned is simply wrong — or is half of the previous message. Writing the payload before the length narrows that window without closing it, since the CPU and the compiler may reorder the two stores anyway. Closing it takes a sequence number the reader checks before and after copying, or a real synchronization object.

The figure shows the smallest case: old payload READY (five bytes), new payload RUNNING (seven bytes), and a reader that copies in the gap between publishing the length and finishing the text. Both checks in read_text succeed, yet the message is not one the writer ever meant to publish. Seeing the same bytes does not give either program an agreed moment to read them.

A wake-up does not protect a shared record. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

record = {health: 100, gold: 80}
reader attempts to copy health
writer updates health and gold together
reader attempts to copy gold
release the shared mutex if used
publish only the copied observation

Result: copied health 100 and gold 80 belong to the same protected record.

Final values: coordination = mutex; copied_health = 100; copied_gold = 80; consistent = true.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs. Follow the highlighted operation and the changing state below.

   coordination = mutex.

2. Step 2, line 1. Begin with an intact old record.

   coordination = mutex; copied_health = not copied; copied_gold = not copied.

   Memory: health = 100; gold = 80.

3. Step 3, line 2. The reader holds the mutex while copying both fields; the writer waits.

   coordination = mutex; copied_health = 100; copied_gold = not copied.

   Memory: health = 100; gold = 80.

4. Step 4, line 4. Copy gold before releasing the same lock.

   coordination = mutex; copied_health = 100; copied_gold = 80.

   Memory: health = 100; gold = 80.

5. Step 5, line 5. The reader releases the mutex; the writer can now proceed.

   coordination = mutex; copied_health = 100; copied_gold = 80.

   Memory: health = 100; gold = 80.

6. Step 6, line 3. The waiting writer publishes its complete newer record.

   coordination = mutex; copied_health = 100; copied_gold = 80.

   Memory: health = 80; gold = 60.

7. Step 7, line 6. Result: copied health 100 and gold 80 belong to the same protected record.

   coordination = mutex; copied_health = 100; copied_gold = 80; consistent = true.

   Memory: health = 80; gold = 60.

Build the writer and reader

Complete lab source: shared_memory_lab.rs

shared_memory_lab.rs

windows-labs/src/bin/ · Rust, 149 lines

View on GitHub ↗

Show the codeHide the code

#[cfg(windows)]
mod app {
    use std::{io, slice};

    use anyhow::{Context, ensure};
    use gha_windows_labs::OwnedHandle;
    use windows::{
        Win32::{
            Foundation::INVALID_HANDLE_VALUE,
            System::Memory::{
                CreateFileMappingW, FILE_MAP, FILE_MAP_READ, FILE_MAP_WRITE,
                MEMORY_MAPPED_VIEW_ADDRESS, MapViewOfFile, OpenFileMappingW, PAGE_READWRITE,
                UnmapViewOfFile,
            },
        },
        core::w,
    };

    const MAPPING_BYTES: usize = 4_096;
    const HEADER_BYTES: usize = 4;

    struct MappedView {
        address: MEMORY_MAPPED_VIEW_ADDRESS,
        length: usize,
    }

    impl MappedView {
        fn new(handle: &OwnedHandle, access: FILE_MAP) -> windows::core::Result<Self> {
            // SAFETY: handle refers to a file-mapping object. A null address is
            // checked before any slice is created, and the view is unmapped in Drop.
            let address = unsafe { MapViewOfFile(handle.raw(), access, 0, 0, MAPPING_BYTES) };
            if address.Value.is_null() {
                return Err(windows::core::Error::from_thread());
            }
            Ok(Self {
                address,
                length: MAPPING_BYTES,
            })
        }

        fn bytes(&self) -> &[u8] {
            // SAFETY: MapViewOfFile returned a non-null view of length MAPPING_BYTES,
            // and self keeps the view mapped for the slice's entire lifetime.
            unsafe { slice::from_raw_parts(self.address.Value.cast(), self.length) }
        }

        fn bytes_mut(&mut self) -> &mut [u8] {
            // SAFETY: the writer requested FILE_MAP_WRITE, owns this mutable borrow,
            // and the mapped range remains valid until Drop.
            unsafe { slice::from_raw_parts_mut(self.address.Value.cast(), self.length) }
        }
    }

    impl Drop for MappedView {
        fn drop(&mut self) {
            // SAFETY: this object owns one live mapped view and unmaps it once.
            let _ = unsafe { UnmapViewOfFile(self.address) };
        }
    }

    fn writer(message: &str) -> anyhow::Result<()> {
        ensure!(
            message.len() <= MAPPING_BYTES - HEADER_BYTES,
            "message is longer than {} UTF-8 bytes",
            MAPPING_BYTES - HEADER_BYTES
        );
        // SAFETY: INVALID_HANDLE_VALUE requests paging-file-backed memory. The
        // mapping name is static and the size is nonzero.
        let mapping = unsafe {
            CreateFileMappingW(
                INVALID_HANDLE_VALUE,
                None,
                PAGE_READWRITE,
                0,
                MAPPING_BYTES as u32,
                w!("Local\\GhaSharedMemoryV1"),
            )
        }?;
        let mapping = OwnedHandle::from_raw(mapping)?;
        let mut view = MappedView::new(&mapping, FILE_MAP_WRITE)?;
        let bytes = view.bytes_mut();
        bytes.fill(0);
        bytes[..HEADER_BYTES].copy_from_slice(&(message.len() as u32).to_le_bytes());
        bytes[HEADER_BYTES..HEADER_BYTES + message.len()].copy_from_slice(message.as_bytes());

        println!(
            "Published {} UTF-8 bytes to a Local\\ mapping.",
            message.len()
        );
        println!("Run `shared_memory_lab read` in another window.");
        println!("Press Enter here after the reader finishes.");
        let mut line = String::new();
        io::stdin().read_line(&mut line)?;
        Ok(())
    }

    fn reader() -> anyhow::Result<()> {
        // SAFETY: the name is static, inheritance is disabled, and the reader
        // requests read-only access.
        let mapping = unsafe {
            OpenFileMappingW(
                FILE_MAP_READ.0,
                false,
                w!("Local\\GhaSharedMemoryV1"),
            )
        }
        .context("open the writer first so the named mapping exists")?;
        let mapping = OwnedHandle::from_raw(mapping)?;
        let view = MappedView::new(&mapping, FILE_MAP_READ)?;
        let bytes = view.bytes();
        let length = u32::from_le_bytes(bytes[..HEADER_BYTES].try_into()?) as usize;
        ensure!(
            length <= MAPPING_BYTES - HEADER_BYTES,
            "shared-memory length header is out of range"
        );
        let message = std::str::from_utf8(&bytes[HEADER_BYTES..HEADER_BYTES + length])
            .context("shared message was not valid UTF-8")?;
        println!("Read {length} bytes: {message}");
        println!("The reader requested FILE_MAP_READ and changed nothing.");
        Ok(())
    }

    pub fn run() -> anyhow::Result<()> {
        let mut arguments = std::env::args().skip(1);
        match arguments.next().as_deref() {
            Some("write") => {
                let message = arguments.collect::<Vec<_>>().join(" ");
                let message = if message.is_empty() {
                    "Wesnoth turn 12: local lab ready"
                } else {
                    &message
                };
                writer(message)
            }
            Some("read") => reader(),
            _ => anyhow::bail!("usage: shared_memory_lab write <message> | shared_memory_lab read"),
        }
    }
}

#[cfg(windows)]
fn main() -> anyhow::Result<()> {
    app::run()
}

#[cfg(not(windows))]
fn main() {
    eprintln!("This named shared-memory lab must run on Windows.");
}

#[cfg(windows)]
mod app {
    use std::{io, slice};

    use anyhow::{Context, ensure};
    use gha_windows_labs::OwnedHandle;
    use windows::{
        Win32::{
            Foundation::INVALID_HANDLE_VALUE,
            System::Memory::{
                CreateFileMappingW, FILE_MAP, FILE_MAP_READ, FILE_MAP_WRITE,
                MEMORY_MAPPED_VIEW_ADDRESS, MapViewOfFile, OpenFileMappingW,
                PAGE_READWRITE, UnmapViewOfFile,
            },
        },
        core::w,
    };

    const MAPPING_BYTES: usize = 4_096;
    const HEADER_BYTES: usize = 4;

    struct MappedView {
        address: MEMORY_MAPPED_VIEW_ADDRESS,
        length: usize,
    }

    impl MappedView {
        fn new(handle: &OwnedHandle, access: FILE_MAP) -> windows::core::Result<Self> {
            // 🛡️ SAFETY: handle refers to a file-mapping object. A null address is
            // checked before any slice is created, and the view is unmapped in Drop.
            let address = unsafe {
                MapViewOfFile(handle.raw(), access, 0, 0, MAPPING_BYTES)
            };
            if address.Value.is_null() {
                return Err(windows::core::Error::from_thread());
            }
            Ok(Self { address, length: MAPPING_BYTES })
        }

        fn bytes(&self) -> &[u8] {
            // ✅ SAFETY: MapViewOfFile returned a non-null view of length MAPPING_BYTES,
            // and self keeps the view mapped for the slice's entire lifetime.
            unsafe { slice::from_raw_parts(self.address.Value.cast(), self.length) }
        }

        fn bytes_mut(&mut self) -> &mut [u8] {
            // 🔒 SAFETY: the writer requested FILE_MAP_WRITE, owns this mutable borrow,
            // and the mapped range remains valid until Drop.
            unsafe { slice::from_raw_parts_mut(self.address.Value.cast(), self.length) }
        }
    }

    impl Drop for MappedView {
        fn drop(&mut self) {
            // 🧹 SAFETY: this object owns one live mapped view and unmaps it once.
            let _ = unsafe { UnmapViewOfFile(self.address) };
        }
    }

    fn writer(message: &str) -> anyhow::Result<()> {
        ensure!(
            message.len() <= MAPPING_BYTES - HEADER_BYTES,
            "message is longer than {} UTF-8 bytes",
            MAPPING_BYTES - HEADER_BYTES
        );
        // 🛡️ SAFETY: INVALID_HANDLE_VALUE requests paging-file-backed memory. The
        // mapping name is static and the size is nonzero.
        let mapping = unsafe {
            CreateFileMappingW(
                INVALID_HANDLE_VALUE,
                None,
                PAGE_READWRITE,
                0,
                MAPPING_BYTES as u32,
                w!("Local\\GhaSharedMemoryV1"),
            )
        }?;
        let mapping = OwnedHandle::from_raw(mapping)?;
        let mut view = MappedView::new(&mapping, FILE_MAP_WRITE)?;
        let bytes = view.bytes_mut();
        // 📦 Zeroing removes bytes left by an older, longer message. The first
        // four bytes are the only trusted boundary for the current payload.
        bytes.fill(0);
        bytes[..HEADER_BYTES]
            .copy_from_slice(&(message.len() as u32).to_le_bytes());
        bytes[HEADER_BYTES..HEADER_BYTES + message.len()]
            .copy_from_slice(message.as_bytes());

        println!("Published {} UTF-8 bytes to a Local\\ mapping.", message.len());
        println!("Run `shared_memory_lab read` in another window.");
        println!("Press Enter here after the reader finishes.");
        // 🧹 Keep both the mapping handle and its view alive until the reader has
        // opened the named object; dropping the last handle destroys its name.
        let mut line = String::new();
        io::stdin().read_line(&mut line)?;
        Ok(())
    }

    fn reader() -> anyhow::Result<()> {
        // 🔍 SAFETY: the name is static, inheritance is disabled, and the reader
        // requests read-only access.
        let mapping = unsafe {
            OpenFileMappingW(
                FILE_MAP_READ.0,
                false,
                w!("Local\\GhaSharedMemoryV1"),
            )
        }
        .context("open the writer first so the named mapping exists")?;
        let mapping = OwnedHandle::from_raw(mapping)?;
        let view = MappedView::new(&mapping, FILE_MAP_READ)?;
        let bytes = view.bytes();
        // 📏 The writer stores a little-endian byte length, not a character count.
        let length = u32::from_le_bytes(bytes[..HEADER_BYTES].try_into()?) as usize;
        ensure!(
            length <= MAPPING_BYTES - HEADER_BYTES,
            "shared-memory length header is out of range"
        );
        // 🛡️ Validate the length before slicing, then validate UTF-8 before
        // treating arbitrary shared bytes as text.
        let message = std::str::from_utf8(
            &bytes[HEADER_BYTES..HEADER_BYTES + length]
        ).context("shared message was not valid UTF-8")?;
        println!("Read {length} bytes: {message}");
        println!("The reader requested FILE_MAP_READ and changed nothing.");
        Ok(())
    }

    pub fn run() -> anyhow::Result<()> {
        let mut arguments = std::env::args().skip(1);
        match arguments.next().as_deref() {
            Some("write") => {
                let message = arguments.collect::<Vec<_>>().join(" ");
                let message = if message.is_empty() {
                    "Wesnoth turn 12: local lab ready"
                } else {
                    &message
                };
                writer(message)
            }
            Some("read") => reader(),
            _ => anyhow::bail!(
                "usage: shared_memory_lab write <message> | shared_memory_lab read"
            ),
        }
    }
}

#[cfg(windows)]
fn main() -> anyhow::Result<()> {
    app::run()
}

#[cfg(not(windows))]
fn main() {
    eprintln!("This named shared-memory lab must run on Windows.");
}

Why is this unsafe code contained?

MapViewOfFile returns a raw pointer. The compiler cannot prove how many bytes it covers, whether the view is writable, or when Windows unmaps it. MappedView establishes those facts once, then ordinary methods return bounded slices.

The writer receives &mut [u8]; the reader receives &[u8]. That is semantic highlighting at the type-system level: mutability shows which side is allowed to change data.

Run two copies

Windows VM

In PowerShell window one:

cd windows-labs
cargo run --release --target i686-pc-windows-msvc `
  --bin shared_memory_lab -- write "AssaultCube local observer ready"

Leave it waiting. In window two:

cd windows-labs
cargo run --release --target i686-pc-windows-msvc `
  --bin shared_memory_lab -- read

Return to window one and press Enter. The mapping object disappears after the last handle closes, so another reader should then receive a clear “open the writer first” error.

Narration: Both the reader and writer must take the same mutex; signalling an event does not exclude a partial read.

Where shared memory fits

A Wesnoth add-on or game build could publish a tiny turn/event record. A separate visualizer could read it. The producer defines the shared-memory contract, so both sides agree on its layout and lifetime.

The one-shot lab avoids concurrent writes. A concurrent design could hold the same named mutex while the writer updates a record and while the reader copies it; the reader would then avoid mixing old and new fields. Both sides also need a shared format version and an agreed sequence-number meaning. A notification event alone announces readiness; it does not by itself protect the copy. Never put passwords or secret tokens in a mapping protected only by a convenient name.

The buildable source is shared_memory_lab.rs.

References: sharing files and memory, creating named shared memory, CreateFileMappingW, and MapViewOfFile.

Lesson 9.9 uses a named pipe when the producer and consumer would rather exchange one request and one reply than coordinate reads of a shared region.

Lesson 9.9

Named Pipes and Local Message Framing

A named pipe is a local client/server channel

Shared memory in Lesson 9.8 gave two programs views of the same bytes, but a concurrent writer needed an additional timing agreement. For one request followed by one reply, a named pipe gives us a connection with an explicit read/write order.

A named pipe server creates a known pipe name and waits. A client opens that name. After they connect, both sides can exchange bytes through ordinary Windows read and write calls.

This resembles a tiny network service, but the lab keeps both programs on the same Windows machine:

named_pipe_lab server
        ⇅ one bounded request and response
named_pipe_lab client

Named pipes are useful when a supported game plug-in, launcher, capture helper, or your own engine build needs to talk to a separate tool without sharing raw pointers.

One bounded exchange has a clear beginning, two validated messages, and a clear end:

The pipe transports bytes. Both programs still need to agree on the length field, message kinds, payload encoding, and which requests to reject.

Note

🔌 IPC idea: define a small message format instead of sharing addresses. Each process keeps ownership of its own memory and exchanges bounded copies.

Message mode preserves write boundaries

Windows pipes can operate in byte mode or message mode.

TL;DR: Message mode remembers write boundaries, but a small read buffer can still return only part of one message.

• Byte mode is a stream. One read can receive part of a write or data from several writes.
• Message mode preserves message boundaries. A read receives one message, provided the buffer is large enough.

The lab selects PIPE_TYPE_MESSAGE when creating the server and PIPE_READMODE_MESSAGE for both ends.

It still adds a four-byte length header. Why use both? Message mode keeps transport boundaries, while the length header validates the application protocol. The receiver can reject a damaged or unexpected frame instead of trusting every byte Windows delivered.

byte 0-3`u32` length, little-endian

byte 4..UTF-8 text, up to 1024 bytes

The pipe frame read_frame validates below. FRAME_BYTES (1028) reserves room for the header plus the maximum payload.

Treat IPC data like untrusted input

“Local” does not mean “automatically trustworthy.” Another program in the same account or session might know or guess the pipe name.

Alternative: A local sender is another program. Validate its lengths and contents just as you would a network sender.

The receiver checks all of these facts before making a String:

1. At least four header bytes arrived.
2. The declared text length is no more than 1024 bytes.
3. The received frame size equals 4 + declared length.
4. The text bytes are valid UTF-8.

The first check that fails rejects the frame, and the later ones never run. Try the edge of the limit: 1,028 bytes whose header says 1,024 pass, because the limit is on the text length.

The four checks on one pipe frameA frame is a little-endian u32 text length, then the text. Change what one read returned; the first failed check rejects the frame.

The frame and the checks

What one read returned

Header = 5

byte 0byte 3

05×1

00×256

00×256²

00×256³

Text bytes

5 bytes after the header

Checks, in order

1. bytes_read ≥ 4

9 ≥ 4: the header is complete

2. length ≤ 1,024

5 ≤ 1,024

3. bytes_read = 4 + length

9 = 4 + 5

4. valid UTF-8

valid: the text can become a String

Result

Accept: return the 5-byte text

All four checks pass: 9 bytes arrived, the header says 5, and 9 = 4 + 5. The limit applies to the text length, so 5 ≤ 1,024 is allowed.

The server accepts one client, handles one request, sends one fixed reply, and exits. Small limits make the behavior easy to reason about.

This byte-stream counterexample shows why a single successful read need not contain a complete payload. The Windows lab below chooses message mode and still checks lengths.

One byte-stream message can arrive in pieces. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

promised_payload_bytes = 8
copied = read(first_fragment)
remaining = promised_payload_bytes - copied
read again until remaining == 0
parse only the complete bounded payload
report read calls and assembled bytes

Result: eight bytes assembled in 2 read call(s). A disconnected or malformed sender must fail instead of inventing missing bytes.

Final values: first = 3; expected = 8; copied = 8; read_calls = 2; remaining = 0; parsed = true.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs. Follow the highlighted operation and the changing state below.

   first = 3.

2. Step 2, line 1. The length prefix already promised eight payload bytes.

   first = 3; expected = 8; copied = 0; read_calls = 0.

3. Step 3, line 2. The first successful read supplies 3 bytes.

   first = 3; expected = 8; copied = 3; read_calls = 1.

4. Step 4, line 3. Keep track of the exact unfilled amount.

   first = 3; expected = 8; copied = 3; read_calls = 1; remaining = 5.

5. Step 5, line 4. In this illustration the next read fills the remaining 5 bytes. Real code loops until full or an error.

   first = 3; expected = 8; copied = 8; read_calls = 2; remaining = 0.

6. Step 6, line 5. The parser receives the complete eight-byte payload, not its first fragment.

   first = 3; expected = 8; copied = 8; read_calls = 2; remaining = 0; parsed = true.

7. Step 7, line 6. Result: eight bytes assembled in 2 read call(s). A disconnected or malformed sender must fail instead of inventing missing bytes.

   first = 3; expected = 8; copied = 8; read_calls = 2; remaining = 0; parsed = true.

Keep the pipe local and never impersonate

Named pipes can support remote clients. This lab includes PIPE_REJECT_REMOTE_CLIENTS, so Windows rejects them automatically.

This original creation helper chooses message mode and duplex access, but does not explicitly reject remote clients. This helper extracts the creation step from the complete lab below, using its pipe name and FRAME_BYTES buffer limit; OwnedHandle closes the created pipe. Accepting and parsing the request happens later.

Before: create a pipe without rejecting remote clients

fn create_server_pipe() -> anyhow::Result<OwnedHandle> {
    let mode = PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT; // No remote check.
    // SAFETY: The pipe name is static and terminated; no security pointer is supplied.
    let pipe = unsafe {
        CreateNamedPipeW(
            windows::core::w!(r"\\.\pipe\gha-local-game-ipc-v1"), PIPE_ACCESS_DUPLEX, mode,
            1, FRAME_BYTES as u32, FRAME_BYTES as u32, 0, None,
        )
    };
    let pipe = OwnedHandle::from_raw(pipe)?;
    Ok(pipe)
}

Add the local-only capability boundary to the creation mode. It rejects remote clients at the Windows pipe boundary; it does not replace the normal access check for local clients.

Improved code

fn create_server_pipe() -> anyhow::Result<OwnedHandle> {
    let mode = PIPE_TYPE_MESSAGE
        | PIPE_READMODE_MESSAGE
        | PIPE_WAIT
        | PIPE_REJECT_REMOTE_CLIENTS;
    // SAFETY: The pipe name is static and terminated; no security pointer is supplied.
    let pipe = unsafe {
        CreateNamedPipeW(
            windows::core::w!(r"\\.\pipe\gha-local-game-ipc-v1"), PIPE_ACCESS_DUPLEX, mode,
            1, FRAME_BYTES as u32, FRAME_BYTES as u32, 0, None,
        )
    };
    let pipe = OwnedHandle::from_raw(pipe)?;
    Ok(pipe)
}

Show the diff

fn create_server_pipe() -> anyhow::Result<OwnedHandle> {
    let mode = PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT; // No remote check.
    let mode = PIPE_TYPE_MESSAGE
        | PIPE_READMODE_MESSAGE
        | PIPE_WAIT
        | PIPE_REJECT_REMOTE_CLIENTS;
    // SAFETY: The pipe name is static and terminated; no security pointer is supplied.
    let pipe = unsafe {
        CreateNamedPipeW(
            windows::core::w!(r"\\.\pipe\gha-local-game-ipc-v1"), PIPE_ACCESS_DUPLEX, mode,
            1, FRAME_BYTES as u32, FRAME_BYTES as u32, 0, None,
        )
    };
    let pipe = OwnedHandle::from_raw(pipe)?;
    Ok(pipe)
}

Takeaway: restrict who can reach a channel when creating it, then validate each received message.

The program does not call ImpersonateNamedPipeClient, duplicate tokens, start a process as another identity, or change privileges. Those operations solve different identity and access-control problems; they are not part of this message channel.

Build both modes in one program

Complete lab source: named_pipe_lab.rs

named_pipe_lab.rs

windows-labs/src/bin/ · Rust, 161 lines

View on GitHub ↗

Show the codeHide the code

#[cfg(windows)]
mod app {
    use anyhow::{Context, ensure};
    use gha_windows_labs::OwnedHandle;
    use windows::{
        Win32::{
            Foundation::{ERROR_PIPE_CONNECTED, GENERIC_READ, GENERIC_WRITE},
            Storage::FileSystem::{
                CreateFileW, FILE_ATTRIBUTE_NORMAL, FILE_SHARE_MODE, OPEN_EXISTING,
                PIPE_ACCESS_DUPLEX, ReadFile, WriteFile,
            },
            System::Pipes::{
                ConnectNamedPipe, CreateNamedPipeW, PIPE_READMODE_MESSAGE,
                PIPE_REJECT_REMOTE_CLIENTS, PIPE_TYPE_MESSAGE, PIPE_WAIT, SetNamedPipeHandleState,
            },
        },
        core::w,
    };

    const MAX_TEXT_BYTES: usize = 1_024;
    const FRAME_BYTES: usize = MAX_TEXT_BYTES + 4;

    fn frame(text: &str) -> anyhow::Result<Vec<u8>> {
        ensure!(
            text.len() <= MAX_TEXT_BYTES,
            "message is longer than {MAX_TEXT_BYTES} UTF-8 bytes"
        );
        let mut bytes = Vec::with_capacity(text.len() + 4);
        bytes.extend_from_slice(&(text.len() as u32).to_le_bytes());
        bytes.extend_from_slice(text.as_bytes());
        Ok(bytes)
    }

    fn read_frame(handle: &OwnedHandle) -> anyhow::Result<String> {
        let mut buffer = [0_u8; FRAME_BYTES];
        let mut bytes_read = 0_u32;
        // SAFETY: buffer is writable, bytes_read is a valid output pointer,
        // and this synchronous lab does not supply an OVERLAPPED structure.
        unsafe { ReadFile(handle.raw(), Some(&mut buffer), Some(&mut bytes_read), None) }?;
        let bytes_read = bytes_read as usize;
        ensure!(
            bytes_read >= 4,
            "pipe message did not contain a length header"
        );
        let declared = u32::from_le_bytes(buffer[..4].try_into()?) as usize;
        ensure!(
            declared <= MAX_TEXT_BYTES,
            "pipe length header is too large"
        );
        ensure!(
            bytes_read == declared + 4,
            "pipe frame length did not match its header"
        );
        Ok(std::str::from_utf8(&buffer[4..bytes_read])
            .context("pipe message was not valid UTF-8")?
            .to_owned())
    }

    fn write_frame(handle: &OwnedHandle, text: &str) -> anyhow::Result<()> {
        let bytes = frame(text)?;
        let mut bytes_written = 0_u32;
        // SAFETY: bytes remains readable during the synchronous call and
        // bytes_written is a valid output pointer.
        unsafe { WriteFile(handle.raw(), Some(&bytes), Some(&mut bytes_written), None) }?;
        ensure!(
            bytes_written as usize == bytes.len(),
            "Windows wrote only part of the pipe frame"
        );
        Ok(())
    }

    fn server() -> anyhow::Result<()> {
        let mode =
            PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE | PIPE_WAIT | PIPE_REJECT_REMOTE_CLIENTS;
        // SAFETY: the pipe name is static. The server allows one synchronous,
        // message-mode connection and supplies no custom security pointer.
        let pipe = unsafe {
            CreateNamedPipeW(
                w!(r"\\.\pipe\gha-local-game-ipc-v1"),
                PIPE_ACCESS_DUPLEX,
                mode,
                1,
                FRAME_BYTES as u32,
                FRAME_BYTES as u32,
                0,
                None,
            )
        };
        let pipe = OwnedHandle::from_raw(pipe)?;
        println!("Server is waiting for one local client...");

        // A client can connect between CreateNamedPipeW and ConnectNamedPipe.
        // In that harmless race, Windows reports ERROR_PIPE_CONNECTED.
        // SAFETY: pipe owns a valid server-pipe handle and this synchronous
        // call deliberately supplies no OVERLAPPED structure.
        match unsafe { ConnectNamedPipe(pipe.raw(), None) } {
            Ok(()) => {}
            Err(error) if error.code() == ERROR_PIPE_CONNECTED.to_hresult() => {}
            Err(error) => return Err(error.into()),
        }

        let message = read_frame(&pipe)?;
        println!("Client said: {message}");
        write_frame(&pipe, "ACK: the local game-tool message arrived")?;
        println!("Replied once; the server will now close the pipe.");
        Ok(())
    }

    fn client(message: &str) -> anyhow::Result<()> {
        // SAFETY: the path is static. This opens only the local course pipe,
        // requests read/write data access, and does not inherit the handle.
        let pipe = unsafe {
            CreateFileW(
                w!(r"\\.\pipe\gha-local-game-ipc-v1"),
                GENERIC_READ.0 | GENERIC_WRITE.0,
                FILE_SHARE_MODE(0),
                None,
                OPEN_EXISTING,
                FILE_ATTRIBUTE_NORMAL,
                None,
            )
        }
        .context("start `named_pipe_lab server` in another window first")?;
        let pipe = OwnedHandle::from_raw(pipe)?;
        let read_mode = PIPE_READMODE_MESSAGE;
        // SAFETY: pipe is a connected named-pipe handle and read_mode remains
        // valid for the duration of the call.
        unsafe { SetNamedPipeHandleState(pipe.raw(), Some(&read_mode), None, None) }?;

        write_frame(&pipe, message)?;
        println!("Server replied: {}", read_frame(&pipe)?);
        Ok(())
    }

    pub fn run() -> anyhow::Result<()> {
        let mut arguments = std::env::args().skip(1);
        match arguments.next().as_deref() {
            Some("server") => server(),
            Some("client") => {
                let message = arguments.collect::<Vec<_>>().join(" ");
                let message = if message.is_empty() {
                    "Wesnoth helper connected"
                } else {
                    &message
                };
                client(message)
            }
            _ => anyhow::bail!("usage: named_pipe_lab server | named_pipe_lab client <message>"),
        }
    }
}

#[cfg(windows)]
fn main() -> anyhow::Result<()> {
    app::run()
}

#[cfg(not(windows))]
fn main() {
    eprintln!("This local named-pipe lab must run on Windows.");
}

#[cfg(windows)]
mod app {
    use anyhow::{Context, ensure};
    use gha_windows_labs::OwnedHandle;
    use windows::{
        Win32::{
            Foundation::{ERROR_PIPE_CONNECTED, GENERIC_READ, GENERIC_WRITE},
            Storage::FileSystem::{
                CreateFileW, FILE_ATTRIBUTE_NORMAL, FILE_SHARE_MODE,
                OPEN_EXISTING, PIPE_ACCESS_DUPLEX, ReadFile, WriteFile,
            },
            System::Pipes::{
                ConnectNamedPipe, CreateNamedPipeW, PIPE_READMODE_MESSAGE,
                PIPE_REJECT_REMOTE_CLIENTS, PIPE_TYPE_MESSAGE, PIPE_WAIT,
                SetNamedPipeHandleState,
            },
        },
        core::w,
    };

    const MAX_TEXT_BYTES: usize = 1_024;
    const FRAME_BYTES: usize = MAX_TEXT_BYTES + 4;

    fn frame(text: &str) -> anyhow::Result<Vec<u8>> {
        // 📏 `str::len` is UTF-8 bytes, exactly what the wire header measures;
        // visible character count would be wrong for emoji or accented text.
        ensure!(text.len() <= MAX_TEXT_BYTES,
            "message is longer than {MAX_TEXT_BYTES} UTF-8 bytes");
        let mut bytes = Vec::with_capacity(text.len() + 4);
        // 📦 Prefixing the length makes the receiver reject truncated or joined
        // data instead of guessing where the message ends.
        bytes.extend_from_slice(&(text.len() as u32).to_le_bytes());
        bytes.extend_from_slice(text.as_bytes());
        Ok(bytes)
    }

    fn read_frame(handle: &OwnedHandle) -> anyhow::Result<String> {
        let mut buffer = [0_u8; FRAME_BYTES];
        let mut bytes_read = 0_u32;
        // 🛡️ SAFETY: buffer is writable, bytes_read is a valid output pointer,
        // and this synchronous lab does not supply an OVERLAPPED structure.
        unsafe {
            ReadFile(handle.raw(), Some(&mut buffer), Some(&mut bytes_read), None)
        }?;
        let bytes_read = bytes_read as usize;
        // 🛡️ Validate in dependency order: header exists, declared payload fits,
        // and the one message-mode read agrees exactly with that declaration.
        ensure!(bytes_read >= 4, "pipe message did not contain a length header");
        let declared = u32::from_le_bytes(buffer[..4].try_into()?) as usize;
        ensure!(declared <= MAX_TEXT_BYTES, "pipe length header is too large");
        ensure!(bytes_read == declared + 4,
            "pipe frame length did not match its header");
        // ✅ Only the declared, completely received payload is decoded as UTF-8.
        Ok(std::str::from_utf8(&buffer[4..bytes_read])
            .context("pipe message was not valid UTF-8")?
            .to_owned())
    }

    fn write_frame(handle: &OwnedHandle, text: &str) -> anyhow::Result<()> {
        let bytes = frame(text)?;
        let mut bytes_written = 0_u32;
        // 📦 SAFETY: bytes remains readable during the synchronous call and
        // bytes_written is a valid output pointer.
        unsafe {
            WriteFile(handle.raw(), Some(&bytes), Some(&mut bytes_written), None)
        }?;
        // ⚠️ A successful synchronous call still reports how much it wrote;
        // accepting a short write would send a frame whose header lies.
        ensure!(bytes_written as usize == bytes.len(),
            "Windows wrote only part of the pipe frame");
        Ok(())
    }

    fn server() -> anyhow::Result<()> {
        let mode = PIPE_TYPE_MESSAGE
            | PIPE_READMODE_MESSAGE
            | PIPE_WAIT
            | PIPE_REJECT_REMOTE_CLIENTS;
        // 🔒 SAFETY: the pipe name is static. The server allows one synchronous,
        // message-mode connection and supplies no custom security pointer.
        let pipe = unsafe {
            CreateNamedPipeW(
                w!(r"\\.\pipe\gha-local-game-ipc-v1"),
                PIPE_ACCESS_DUPLEX,
                mode,
                1,
                FRAME_BYTES as u32,
                FRAME_BYTES as u32,
                0,
                None,
            )
        };
        let pipe = OwnedHandle::from_raw(pipe)?;
        println!("Server is waiting for one local client...");

        // ⚠️ A client can connect between CreateNamedPipeW and ConnectNamedPipe.
        // In that harmless race, Windows reports ERROR_PIPE_CONNECTED.
        // 🛡️ SAFETY: pipe owns a valid server-pipe handle and this synchronous
        // call deliberately supplies no OVERLAPPED structure.
        match unsafe { ConnectNamedPipe(pipe.raw(), None) } {
            Ok(()) => {}
            Err(error) if error.code() == ERROR_PIPE_CONNECTED.to_hresult() => {}
            Err(error) => return Err(error.into()),
        }

        let message = read_frame(&pipe)?;
        println!("Client said: {message}");
        write_frame(&pipe, "ACK: the local game-tool message arrived")?;
        println!("Replied once; the server will now close the pipe.");
        Ok(())
    }

    fn client(message: &str) -> anyhow::Result<()> {
        // 🧭 SAFETY: the path is static. This opens only the local course pipe,
        // requests read/write data access, and does not inherit the handle.
        let pipe = unsafe {
            CreateFileW(
                w!(r"\\.\pipe\gha-local-game-ipc-v1"),
                GENERIC_READ.0 | GENERIC_WRITE.0,
                FILE_SHARE_MODE(0),
                None,
                OPEN_EXISTING,
                FILE_ATTRIBUTE_NORMAL,
                None,
            )
        }
        .context("start `named_pipe_lab server` in another window first")?;
        let pipe = OwnedHandle::from_raw(pipe)?;
        let read_mode = PIPE_READMODE_MESSAGE;
        // ✅ SAFETY: pipe is a connected named-pipe handle and read_mode remains
        // valid for the duration of the call.
        unsafe {
            SetNamedPipeHandleState(pipe.raw(), Some(&read_mode), None, None)
        }?;

        write_frame(&pipe, message)?;
        println!("Server replied: {}", read_frame(&pipe)?);
        Ok(())
    }

    pub fn run() -> anyhow::Result<()> {
        let mut arguments = std::env::args().skip(1);
        match arguments.next().as_deref() {
            Some("server") => server(),
            Some("client") => {
                let message = arguments.collect::<Vec<_>>().join(" ");
                let message = if message.is_empty() {
                    "Wesnoth helper connected"
                } else {
                    &message
                };
                client(message)
            }
            _ => anyhow::bail!(
                "usage: named_pipe_lab server | named_pipe_lab client <message>"
            ),
        }
    }
}

#[cfg(windows)]
fn main() -> anyhow::Result<()> {
    app::run()
}

#[cfg(not(windows))]
fn main() {
    eprintln!("This local named-pipe lab must run on Windows.");
}

Understand the connection race

A client can open the pipe after CreateNamedPipeW creates it but just before the server calls ConnectNamedPipe. Windows then returns ERROR_PIPE_CONNECTED. That does not mean the connection failed—it means the client already won the race.

Narration: A client can connect before the server starts waiting; that successful connection must not be mistaken for a failed session.

The match arm accepts only that documented condition and returns every other error:

The client may connect in the narrow interval before the server calls ConnectNamedPipe. In that case Windows reports ERROR_PIPE_CONNECTED, which means the connection already exists rather than that it failed. The match below accepts both successful outcomes and propagates other errors:

match unsafe { ConnectNamedPipe(pipe.raw(), None) } {
    Ok(()) => {}
    Err(error) if error.code() == ERROR_PIPE_CONNECTED.to_hresult() => {}
    Err(error) => return Err(error.into()),
}

Run the request and reply

Windows VM

In window one:

cd windows-labs
cargo run --release --target i686-pc-windows-msvc `
  --bin named_pipe_lab -- server

In window two:

cd windows-labs
cargo run --release --target i686-pc-windows-msvc `
  --bin named_pipe_lab -- client "Urban Terror local replay marker"

The client prints the acknowledgement and both programs exit. Start the server again for another message.

How to grow this design safely

A larger tool could add a version number and an enum of allowed messages such as TurnStarted, MapLoaded, or PlayerPositionFromSupportedMod. It should authenticate sensitive clients, set an explicit security descriptor, rate-limit messages, keep bounded queues, and separate parsing from actions.

Do not accept a command string and pass it to PowerShell or cmd.exe. Define a small message enum and reject unknown variants. Data should select one defined action, not become code.

The buildable source is named_pipe_lab.rs.

References: named-pipe operations, CreateNamedPipeW, ConnectNamedPipe, and SetNamedPipeHandleState.

Lesson 9.10

How the Source Engine Works

You press forward. Your player moves immediately, yet the server has not received the input. Later, a reply may adjust that movement. To understand why, follow the input across the pieces of the game that own each decision. Lesson 9.1 introduced client and server messages. Source gives us a concrete engine in which those messages connect to objects and time.

Why study Source?

Source is Valve’s game-engine family behind Half-Life 2, Counter-Strike: Source, Team Fortress 2, Left 4 Dead, and the original Counter-Strike: Global Offensive. They share architectural ideas, but their engine branches differ. Counter-Strike 2 uses Source 2, a later engine family. An interface name or memory layout from Source 1 is not a promise about Counter-Strike 2.

Valve’s public Source SDK 2013 supplies game code and interfaces for Half-Life 2, Half-Life 2: Deathmatch, and Team Fortress 2. An SDK, or software development kit, gives developers code and tools for building compatible software. This repository lets us inspect the game side of the boundary. It does not supply every implementation inside a shipping engine.

This lesson follows the SDK revision beginning b8cfb12. Treat its named interfaces and defaults as facts about that revision. Keep experiments in your own offline SDK mod. A mod supplies changed game content or behavior while using the engine’s supported loading contracts.

The engine connects client and server game code

A launcher starts the engine and selects the game to load. The engine provides common services such as files, rendering, sound, and networking. Game-specific code defines weapons and entities, calculates movement and damage, and decides how those objects respond to input and contact.

On Windows, this game code commonly comes in client.dll and server.dll. A DLL is a library loaded into a process while it runs, as introduced in Lesson 8.1. Other operating systems use their own shared-library formats for the same architectural division.

The client library handles the player’s input, local presentation, and prediction. The server library calculates movement, collisions, damage, and other changes to the match state; it sends the accepted results to clients. Some movement code is shared so both sides can compute comparable results. Rendering a player and deciding whether that player can pass a wall are separate responsibilities.

A listen server hosts the simulation inside a game that also has a local player. A dedicated server runs the hosting side without that local player’s view. Even a local game can retain the client/server division. The boundary describes who owns a decision, whether its messages cross the Internet or stay on one machine.

TL;DR: The client presents and predicts. The server decides the game state. The engine connects their game libraries.

Modules ask for an interface by name

The engine cannot call an arbitrary function inside a loaded library and hope its arguments fit. The two sides need an agreed set of operations. That set is an interface: a contract describing which methods an object provides and how to call them.

Source modules expose a factory named CreateInterface. A factory is a function that supplies an object through an agreed interface. The caller passes the requested name. The module searches its registered interfaces and returns a pointer to the matching object, or a null pointer if it cannot provide one. A separate, optional status output receives IFACE_OK or IFACE_FAILED.

Each name carries a version suffix. In this revision, VClient017 names the client game interface and VEngineServer023 the engine services offered to server code. The suffix is part of the requested contract: the returned object’s virtual methods follow that version’s C++ calling contract, which a neighbouring version can change. CreateInterface itself is an ordinary export, found by name the way Lesson 8.1’s injector finds gha_start.

Alternative: Picture a shared menu of operations. The version name selects the menu both modules agree to use.

This explains a startup failure that otherwise looks mysterious: a library loads successfully, but it cannot provide the interface the engine asks for. Loading bytes and agreeing on their callable contract are different checks.

Entity state becomes selected network properties

Once the libraries connect, the server can create an entity, an object the game tracks: a player, door, projectile, or another piece of the world. Its full object contains more information than a client needs. Some fields are server bookkeeping. Others describe what the client must display.

A SendTable declares properties that server game code makes available for network encoding. A corresponding RecvTable describes how client game code receives those properties. These are descriptions of fields and their encoding, rather than a copy of the whole C++ object. Nested tables let one class reuse another class’s networked fields.

Think of a door whose server object stores its position, opening state, and internal scheduling data. The client needs enough state to place and draw the door. It does not need every scheduling field. The property declarations make that selection explicit. The engine can also limit which entities and updates are relevant to a particular client.

Only the selected door properties cross the networkThe server door holds position, opening state and scheduling. Position and opening state are sent; scheduling stays on the server.

Server doorSendTableRecvTableClient door

1. 1

   Alone Door holds position, open state, scheduling

   Step 1: Server door (alone)

   The server’s door object has more state than any client needs.

2. 2

   Data Position and opening state

   Step 2: Server door to SendTable

   The SendTable names which fields become network properties. These two are on the list.

3. 3

   Alone Scheduling stays here

   Step 3: Server door (alone)

   Fields that are not on the list never leave the server.

4. 4

   Data Network properties

   Step 4: SendTable to RecvTable

   The chosen values are sent to the client in a compact form.

5. 5

   Data Update the client’s door

   Step 5: RecvTable to Client door

   The RecvTable describes how those values land in the client’s copy of the door.

6. 6

   Alone Draw the door

   Step 6: Client door (alone)

   The client draws what it was told, with no knowledge of the server’s scheduling.

The server door holds position, opening state and scheduling. Position and opening state are sent; scheduling stays on the server.

Floating-point properties can trade precision for bandwidth through quantization: representing a range with a limited set of codes. As an illustration, four bits give 2⁴ = 16 codes. Spread those codes across a chosen range from 0 to 15, including both ends. The spacing is (15 − 0) ÷ (16 − 1) = 1. Values between the available steps need an encoding rule that chooses a representable value.

The SDK’s SendPropFloat accepts a bit count, range, and flags. Some settings use an unscaled float instead. The four-bit example explains the trade; it does not describe every Source float on the wire. A client-side value can therefore differ slightly from the server’s value even when both sides work correctly.

Input becomes a command record

Now the player presses a movement key. The client game interface’s CreateMove produces a user command, one record of player input for simulation. Source names that record CUserCmd.

Its command_number identifies a command in the sequence. Its tick_count records the tick when the client created it. A tick is one step of the simulation clock. Command numbers and tick numbers have different jobs, even when a small example gives them matching values.

The record includes view angles and the intended movement values forwardmove, sidemove, and upmove. It also includes weapon selection and mouse movement. These inputs describe what the player requests. They do not establish the final position, damage, or inventory result.

The buttons field packs several requests into one integer. In this SDK, attack uses bit 0 and jump uses bit 1. Their values are 2⁰ = 1 and 2¹ = 2. Requesting both sets the combined mask 1 | 2 = 3, whose two lowest bits are on. The server can test each bit separately.

This memory object is not the packet format. Command encoding records changes relative to another command, and decoding rebuilds the record. Fields used for local prediction bookkeeping do not all become raw packet bytes. Keeping object layout and wire encoding separate matters just as it did for the message parsers earlier in this chapter.

A tick advances the world; prediction advances the local estimate

The SDK’s default tick interval is 0.015 seconds. Multiply by 1000 to get 15 milliseconds per tick. Divide one second by the interval to get 1 ÷ 0.015 ≈ 66.67 simulation steps per second. A game can supply its own interval, so this default is not every Source game’s rate.

The server game interface exposes GameFrame for simulation work and GetTickInterval for its interval. Drawing frames happens on another schedule. A faster renderer can draw additional views between simulation ticks; it does not give the player additional accepted movement.

Waiting for a network reply before showing every step would make movement feel delayed. Prediction lets the client run its local player’s movement using input commands immediately. The server still runs its own simulation, and its results are authoritative in the sense of Lesson 9.1: they are the state the client must reconcile with.

Use a deliberately small movement model to see the reconciliation. Start at x = 0. Each accepted forward command adds one position unit. Commands 1 and 2 request +1, giving 0 + 1 = 1, then 1 + 1 = 2. These are toy displacements, not Source’s velocity and collision formulas. The scene’s 15-millisecond clock comes from the SDK default.

A barrier now closes on the server and prevents movement past x = 2. The client has not received that update. Command 3 requests another +1. The client predicts 2 + 1 = 3; server collision keeps the position at 2. Before the reply arrives, command 4 requests −1. The client’s current prediction becomes 3 − 1 = 2.

Narration: The server’s barrier has closed. The client’s earlier prediction has not heard about it yet.

Acknowledge commands, then replay only the pending ones. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

client predicts all four commands
server executes the acknowledged prefix
receive server position and barrier
drop commands through the acknowledgement
replay the remaining commands from server state
report the corrected predicted position

Result: server snapshot 2, 1 pending command(s), corrected prediction 1. Replaying an already acknowledged command would apply its effect twice.

Final values: ack = 3; barrier = yes; old_prediction = 2; server_position = 2; pending = 1; prediction = 1.

Read every step of the lesson’s example

1. Step 1, line 1. Start with the selected inputs and follow how the next operation changes the state.

   ack = 3; barrier = yes.

2. Step 2, line 2. The old client prediction ends at two.

   ack = 3; barrier = yes; old_prediction = 2.

3. Step 3, line 3. The snapshot acknowledges a prefix, not every command the client has sent.

   ack = 3; barrier = yes; old_prediction = 2; server_position = 2.

4. Step 4, line 4. Discard only the acknowledged prefix.

   ack = 3; barrier = yes; old_prediction = 2; server_position = 2; pending = 1.

5. Step 5, line 5. Replay pending commands using the newly known barrier.

   ack = 3; barrier = yes; old_prediction = 2; server_position = 2; pending = 1; prediction = 1.

6. Step 6, line 6. Result: server snapshot 2, 1 pending command(s), corrected prediction 1. Replaying an already acknowledged command would apply its effect twice.

   ack = 3; barrier = yes; old_prediction = 2; server_position = 2; pending = 1; prediction = 1.

Watch it happenCorrect prediction without discarding pending input

A toy client starts at position zero and predicts three positive moves to positions one, two, and three. A newly closed server barrier blocks the third move at two. A fourth negative move is still pending. A snapshot acknowledges command three at position two, removes the first three pending commands, updates the barrier, and replays the fourth move to position one.

Read the steps

1. This is a toy one-axis track starting at x = 0. Each positive command requests one position unit; it is not a literal Source movement field. The SDK default tick is 0.015 seconds, or 15 milliseconds.
2. Command 1 requests +1. The client predicts 0 + 1 = 1 immediately. The numbered command travels to the server, which accepts position 1. Keep the command until it is acknowledged.
3. Command 2 requests +1. Both copies reach 1 + 1 = 2. The pending queue now contains commands 1 and 2 because the client has not received their acknowledgment yet.
4. The server barrier closes, limiting this toy position to 2. The client still has the old open copy and predicts 2 + 1 = 3 for command 3. The server blocks that move and stays at 2.
5. Before the snapshot arrives, command 4 requests −1. The client predicts 3 − 1 = 2 and keeps all four commands. The server snapshot reports x = 2 and acknowledges commands through 3.
6. Apply the authoritative state at command 3: x = 2, with the closed barrier. Remove acknowledged commands 1, 2, and 3 from the pending queue. Command 4 still needs to be replayed.
7. Replay the remaining command 4 on that restored state. Its −1 move gives 2 − 1 = 1. The corrected client ends at 1; the outline marks the old prediction at 2. Keep command 4 pending until its own acknowledgment.

The positions and steps are a toy fixture. Motion is slowed. Source movement also uses velocity and collision; its 15 ms default tick does not imply one-unit displacement.

The server reply acknowledges command 3 and reports position 2. An acknowledgment identifies the input already covered by the returned state. The client starts from that authoritative position and drops commands 1 through 3 from its pending history. Command 4 is still pending, so it replays that −1: 2 − 1 = 1.

That replay preserves the player’s newer input while correcting the earlier prediction. Simply replacing the displayed position would lose pending movement. Replaying commands already acknowledged would repeat work the server’s state already contains.

Snapshots smooth other players; lag compensation checks past positions

A snapshot describes server state at a point in simulation time. It may be encoded as changes from an earlier known snapshot. Snapshot delivery, tick rate, and rendering rate are separate schedules.

For another player’s movement, the client usually has received states rather than that player’s newest input. Interpolation draws intermediate positions between received states. Showing a slightly older timeline gives the renderer states on both sides of the time it wants to draw. That makes motion smooth instead of jumping whenever a message arrives.

Prediction estimates the local player’s response to input. Interpolation fills the visible gaps between other entities’ received states. Neither makes a client observation newer than the information actually received.

This delay also matters when a player fires. The target they saw may occupy an earlier position than the server’s current position. Lag compensation temporarily uses recorded earlier player states for the relevant server-side hit check, then restores the current simulation. The server still performs the hit check, and the world does not permanently travel backward because a packet arrived.

In this SDK, the initial target tick is the command’s tick_count minus the interpolation delay expressed in ticks. The server also checks that time against its latency estimate. A discrepancy greater than 0.2 seconds uses a latency-based target instead. The inspected sv_maxunlag default is 1 second and bounds the allowed correction. This is a bound on the decision; it is not a promise that every past state remains available for exactly one second.

Console variables have permissions as well as values

Games need settings without recompiling each change. A console variable, or ConVar, is a named setting with a value and flags describing how it may be used. The developer console exposes these settings and named commands.

FCVAR_CHEAT restricts certain development controls to permitted cheat contexts, including the server’s sv_cheats setting. FCVAR_REPLICATED marks a server-controlled setting enforced on connected clients. Flags describe permissions and ownership as well as storage behavior.

A server can ask for a client’s variable through StartQueryCvarValue. The request returns a cookie, an identifier for matching a later reply to that request. It does not return the variable’s value immediately. OnQueryCvarValueFinished supplies the later status and value. The FCVAR_SERVER_CANNOT_QUERY flag forbids that query for the marked variable.

For a defender, a client-reported setting is evidence about what the client reported, not a replacement for the server-side movement, inventory, and damage checks of Lesson 9.2. A timeout or failed query is not automatically proof of cheating either.

The same engine loads packages and compiled maps

Those entities need geometry, models, textures, and sounds. VPK, short for Valve PacK, bundles resource files behind a directory tree. An entry identifies where its bytes live, possibly in a companion archive. Some bytes can be stored with the directory as preload data. This lets the file system locate a resource without treating every small asset as a separate loose file.

A BSP is a compiled map. The name comes from binary space partitioning, which divides space to organize geometry queries. A Source BSP contains sections called lumps, holding map data such as geometry, visibility, and starting entity definitions. It can also contain embedded resources.

The inspected header starts with VBSP, then a version and a directory of 64 lump descriptors. Each descriptor provides an offset, length, and version-related metadata. Different game branches can interpret sections differently. Reading a file therefore starts with identifying its format and validating its bounded sections, not treating its bytes as a live entity object. Chapter 9 develops that file-validation work.

Use each boundary for the question it can answer

An offline observer built with supported interfaces can record command numbers, simulation ticks, received properties, and prediction corrections. Those records answer different questions. A rendered location describes a view; a server result describes an accepted simulation step; a package entry locates stored content.

For your own mod, keep module versions explicit and record which side owns each value. Start work after DLL loading has finished, stop workers before unloading, and release their handles when stopping tools. For defenders, validate player requests against server-owned state and keep work per client bounded. Interpolation, quantization, and correction explain some ordinary discrepancies, so one mismatch needs context. Chapter 15 builds on this separation between requests, observations, and decisions.