Messages, Streams, and Multiplayer

Keyboard shortcuts

×

?

Open this guide

Alt + B

Show or hide the lesson list

Alt + O

Show or fold the lesson outline

Alt + N

Open or close notes

Alt + H

Toggle semantic colours

Alt + D

Toggle page brightness

Alt + C

Toggle code brightness

← →

Move through a focused code trace

Home End

First or final step in a focused trace

Escape

Close this guide or an open panel

Reader shortcuts pause while you type in a text field.

Cheatsheet for Lesson 9.1Key ideas, reference and questions

A one-page summary of “Messages, Streams, and Multiplayer” you can keep beside your notes. Plain Markdown.

Lesson 9.1 — Messages, Streams, and Multiplayer: cheatsheet

• Remember: A client is the game program running on a player's device. It reads the player's keyboard or controller, draws the world, and exchanges messages with another program over the network.

Key terms

• client: A client is the game program running on a player's device.
• server-authoritative simulation: This is one common arrangement, called a server-authoritative simulation: the server's accepted position is the one the clients eventually use.
• intent: The client's “move north” is an intent: a request for a change, not the final truth.
• max_size: stream supplies incoming bytes and max_size is the caller's permitted payload limit.
• asynchronous: Processing is asynchronous when the sender can continue before the work finishes.
• game backend: The game backend is the set of services behind those actions.
• Consistency: Consistency describes which writes a read can observe.

Formulas and patterns

• sent: [00 00 00 02][41 42] [00 00 00 03][43 44 45]; read #1 -> 00 00 00 02 41 42 00 00 one message, plus half a header; read #2 -> 00 03 43 44 45 the remainder

Review

• Q: In a typical multiplayer design, why can a client-side visual value disagree with the final result? A: The server owns the authoritative shared state and resolves outcomes
• Q: Why can one call to read not be assumed to return one complete application message? A: TCP is a byte stream and does not keep the sender's write boundaries

Download cheatsheet (.md)

Download cheatsheet (.md)

Client and server

A client is the game program running on a player’s device. It reads the player’s keyboard or controller, draws the world, and exchanges messages with another program over the network.

A server is a program that receives requests from clients and provides a service to them. A game server might keep track of who has joined a match, calculate player movement and damage, and send the resulting positions and health to the connected players. The computer running that program is also often called “the server.” The program can run on a rented machine, on one player’s computer, or beside your client on the same computer during a local test.

Imagine a match where the server calculates movement. It stores a player at (10, 15). The player’s client sends “move north.” The server checks whether the destination is inside the map and free of a wall. If the move is possible, it changes its stored position to (10, 14) and sends that position to the clients. Each client updates the character it draws. If a wall blocks the move, the server keeps (10, 15) and reports that position instead.

In this example	Client program	Server program
Input	Notices the north key press and sends a move request.	Receives the request and identifies the player who sent it.
Simulation	May predict movement to make the controls feel responsive.	Checks the destination and updates the match’s stored position.
Display	Draws the player at the latest known or predicted position.	Sends position updates; a dedicated server usually draws no game window.

Clarification: Client and server name the programs’ jobs. They do not require two computers: a local test can run both programs on one machine and connect them through loopback.

This is one common arrangement, called a server-authoritative simulation: the server’s accepted position is the one the clients eventually use. Other games give the server a smaller job, such as hosting a lobby or forwarding players’ inputs. A server does not necessarily simulate every part of a game, and the clients do not necessarily hold identical or complete copies of its data.

This chapter follows that message from intent to bytes, from captured bytes to a parsed event, and from one event to a legal session transition. A local chat client and proxy put the pieces together. The final lessons ask how two programs on one computer can exchange equally well-defined messages without using a network socket.

The client’s “move north” is an intent: a request for a change, not the final truth. The reply reports what the server actually accepted. If a wall blocked the move, the reply might still say (10, 15), and the client would correct its display.

These programs have separate memory. Changing a position in the client’s memory does not directly change the server’s stored position. They communicate by sending bytes. A client may display a predicted move while it waits, then correct that display when the server’s reply arrives. The next question is how the reply’s bytes become one complete message at the other end.

Peer-to-peer describes who communicates with whom

In a peer-to-peer arrangement, players’ programs can exchange messages directly. A lobby service may still introduce them, and a relay may carry bytes when a direct connection is unavailable. Neither service must calculate the match.

A player who hosts a server on their own computer still runs a client/server arrangement if that server accepts movement and sends the final positions. The location of the program does not change its job.

Example arrangement	Message path	Who calculates the accepted position here?
Dedicated game server	Each client talks to the server program.	The server program.
Player hosts the game server	Clients talk to the server on a player’s computer.	That server program.
Peers exchange inputs	Players exchange the same ordered input history.	Each peer runs the agreed deterministic simulation.

The last row describes one particular design, often called lockstep. Peer-to-peer does not automatically mean lockstep: another design can give one peer responsibility for an object or use different ways to resolve conflicts. Always ask two separate questions: where do messages go, and whose result is accepted? Direct communication alone does not prove which copy owns a gold value.

The same movement request, three arrangements. This step-by-step trace needs JavaScript. Here is the code and where the run ends.

start at (10, 15)
request move north
deliver input to the selected participants
check destination against the shared map
accept (10, 14) or keep (10, 15)
draw the accepted position

Each player draws (10, 14). The server’s accepted result determines the final position, even when that program runs on a player’s computer.

Final values: arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

Read every step of the lesson’s example

1. Step 1, line 1. The worked match begins at the same position in each arrangement.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

2. Step 2, line 2. The player requests north; this request does not itself establish the final position.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

3. Step 3, line 3. The client sends the request to the dedicated server program.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program.

4. Step 4, line 4. dedicated server program checks the destination. A wall keeps the player in place.

   arrangement = dedicated; destination = clear; position = (10, 15); calculator = dedicated server program; blocked = false.

5. Step 5, line 5. The calculation accepts (10, 14).

   arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

6. Step 6, line 6. Each player draws (10, 14). The server’s accepted result determines the final position, even when that program runs on a player’s computer.

   arrangement = dedicated; destination = clear; position = (10, 14); calculator = dedicated server program; blocked = false.

Clarification: A server is a program with a job. A peer is another participant. Hosting on a player’s computer and exchanging directly between peers describe different choices.

Topology cheatsheet: client/server identifies service roles; peer-to-peer describes communication between participants; authority identifies the accepted result. A topology label alone cannot answer all three questions.

TCP is a stream

TCP is a common transport for a connection between the two programs. It delivers an ordered stream of bytes. A program calls read to take some bytes that have arrived from that stream. TCP does not preserve the game’s message boundaries for that call.

One call to read may return:

• half of one message;
• exactly one message;
• one message plus part of the next;
• several messages.

Concretely: if the sender writes two length-prefixed messages back to back, the receiver has no say in how they turn up.

sent:       [00 00 00 02][41 42]  [00 00 00 03][43 44 45]

read #1 ->  00 00 00 02 41 42 00 00      one message, plus half a header
read #2 ->  00 03 43 44 45               the remainder

000

100

200

302

441

542

600

700

800

903

1043

1144

1245

length 2

AB

length 3

CDE

read #1: 8 bytes

read #2: 5 bytes

The 4 + 2 + 4 + 3 = 13 bytes on the wire, numbered from 0. In ASCII, 41 42 is AB and 43 44 45 is CDE. Read #1 stops after box 7, halfway through the second length field, so the receiver must keep boxes 6 and 7 until read #2 brings boxes 8 and 9.

Neither read lands on a message boundary. The receiver has to hold what it has in a buffer, remove complete messages when enough bytes have arrived, and keep the leftover for next time. A parser that assumes one read equals one message works flawlessly over a fast loopback connection while you are testing, then breaks the moment timing changes.

This is why a protocol needs framing.

use std::io::{self, Read};

fn read_frame(stream: &mut impl Read, max_size: usize) -> io::Result<Vec<u8>> {
    let mut length_bytes = [0_u8; 4];
    // Wait for the whole header, even if it arrives across several reads.
    stream.read_exact(&mut length_bytes)?;
    let length = u32::from_be_bytes(length_bytes) as usize;

    // Check an untrusted size before allocating its payload buffer.
    if length > max_size {
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            "frame is too large",
        ));
    }

    let mut payload = vec![0_u8; length];
    // A successful result contains one complete payload, without the header.
    stream.read_exact(&mut payload)?;
    Ok(payload)
}

stream supplies incoming bytes and max_size is the caller’s permitted payload limit. A successful call returns one payload as a Vec<u8>; a truncated header or payload produces an I/O error through ?. For a header 00 00 00 02 followed by 41 42, the returned vector contains just 41 42. The maximum size prevents a bad length field from requesting absurd amounts of memory. This blocking helper also needs an appropriate timeout on a real socket so a peer cannot make it wait indefinitely.

TCP reliability ends at delivering an ordered byte stream to the receiving socket. It does not prove the application parsed a complete message, accepted the request, saved the state, or sent a meaningful reply. The application must provide those guarantees. An acknowledgment can report acceptance, a request ID can identify a repeated request, and a timeout can tell the client when to stop waiting. Decide whether a retry repeats the same request or starts a new one before implementing it.

Byte order is part of the protocol

The same integer has more than one possible byte order. Decimal 258 is 1 × 256 + 2, so as a four-byte number it has one byte worth 256 (01), one byte worth 1 (02), and two zero bytes. Wesnoth’s four-byte frame length is big endian, which puts the byte worth the most first:

+000

+100

+201

+302

× 16,777,216

× 65,536

× 256

× 1

1 × 256 + 2 × 1 = 258

Big endian, the order Wesnoth sends on the wire.

On little-endian x86 memory, a native u32 with the same value would usually appear in the opposite order:

+002

+101

+200

+300

× 1

× 256

× 65,536

× 16,777,216

2 × 1 + 1 × 256 = 258

Little endian, the order of a native u32 in x86 memory: the same value with its four bytes reversed.

Read with the wrong order, the wire bytes 00 00 01 02 become 0x02010000, which is 2 × 16,777,216 + 1 × 65,536 = 33,619,968: a frame length of more than 33 million bytes. Network code must follow the protocol, not the host CPU’s default. That is why the reader uses u32::from_be_bytes rather than a pointer cast.

Packets and protocols

A packet is a chunk of bytes sent across a network. That part is straightforward.

A protocol specifies how programs communicate: how to separate messages, decode their fields, order the conversation, and respond to invalid input. Both programs need matching answers to these four questions:

1. Where does one message end and the next begin? A TCP connection is one long stream of bytes with no natural gaps, so something has to mark the boundaries.
2. What is each part of a message? Which bytes are the length, which are the message type, which are the payload, and how each of those is encoded.
3. Which messages are allowed right now? A login reply makes no sense before a login request. People routinely forget this is part of the protocol at all.
4. What happens when something is wrong? An unknown message type, a length that does not fit, a message arriving out of turn.

None of that can be worked out by staring at the bytes, because none of it is written in them. Both programs have to know it in advance. That is precisely why reverse engineering a protocol means recovering those four answers — and why a byte layout on its own only ever answers the second one.

Note

📦 Watch for: the same bytes can be perfectly valid at one point in a conversation and meaningless at another. A protocol model has to check when a message is legal, not only what it contains.

A simple framed message might contain:

[4-byte length][1-byte kind][payload bytes...]

A typed message models the decoded meaning:

enum Message {
    Chat { sender: String, text: String },
    Move { unit_id: u32, x: i32, y: i32 },
    Ping,
}

This enum is a declaration fragment, not a network reader. After framing and validation, a parser can turn a move’s fields into Message::Move with a unit ID and two coordinates; the game logic receives named values instead of raw byte offsets. The bytes are untrusted until parsing succeeds.

One message is wrapped in several layers

Several layers cooperate when the bot sends one chat message:

chat meaning:      sender and text
serialization:     WML fields and tags
compression:       gzip bytes
framing:           four-byte length plus payload
transport:         ordered TCP byte stream
network:           loopback IP packets

WML, Wesnoth Markup Language, is Wesnoth’s own tagged text format for structured data — chat messages, maps, units, and more all get written as bracketed tags such as [message] and [/message] wrapping quoted key="value" fields. It is the serialization format this chapter keeps reversing.

When something fails, ask which layer broke. A correct TCP connection does not prove the WML is valid. A valid gzip stream does not prove the length prefix used the right byte order.

The receiver undoes the layers in the opposite order, and each step has its own way to fail:

Serialization turns structured values into bytes. Framing tells the receiver where one serialized message ends. Transport moves bytes between endpoints. Keeping those jobs separate makes both the explanation and the code easier to test.

Watch it happenRecover bytes before accepting a message

Two TCP reads deliver thirteen bytes. Eight bytes supply one complete AB frame and half the next header. The remaining five bytes complete the CDE frame. A separate Wesnoth example then expands gzip into WML, validates a typed chat message, and checks session rules before accepting it.

Read the steps

1. Read #1 supplies eight bytes. Copy them into the buffer; do not treat that read as one game message.
2. The first four bytes name length 2. Remove that header and the two payload bytes, 41 42, which spell AB. Keep the two leftover zeros: they are only half of the next header.
3. Read #2 supplies five more bytes. Append them after the saved pair. The header is now 00 00 00 03 and all three payload bytes have arrived.
4. Remove the second header and its three-byte payload, 43 44 45, which spells CDE. Both frames are recovered and the receiver buffer is empty.
5. Framing is only the first layer. For a separate Wesnoth message, the complete payload contains gzip bytes. Decompression expands those bytes into WML text.
6. The WML parser validates tags and fields. It creates Message::Chat with named sender and text values; malformed data cannot cross this step.
7. The session checks whether this chat message is allowed now. Only a permitted typed message becomes an accepted event for the game logic.

Read boundaries and message boundaries differ. The later gzip example is separate from the plain ASCII AB/CDE frames.

Messages can feed a data pipeline

A multiplayer service handles more than packets. It may keep durable account and inventory records in a database, reuse expensive results in a cache, and build a search index for queries such as finding a match by region and mode. Those are different access patterns: keep data across restarts, avoid repeating work, and find selected records efficiently. They are not three names for the authoritative state of a running match.

Processing is asynchronous when the sender can continue before the work finishes. A request can enter a queue, then a worker handles it later. A queue acknowledgment might mean “received”; it does not by itself mean “validated, applied, and saved.” The protocol must say which stage each reply confirms.

There are also two common ways to process a sequence of events:

Processing style	Game-service example	What it must account for
Stream processing	Update a match’s statistics as accepted events arrive	Ordering, late events, duplicates, and how quickly the result is needed
Batch processing	Recompute a leaderboard from a collected set of finished-match results	Which results belong to the batch, when it runs, and how to repeat it safely

“Stream processing” here means processing ongoing events; it does not mean the TCP byte stream discussed earlier. A service can obtain events from many connections, files, or queues. A batch deliberately waits for a group of data and processes that group together. Neither approach implies that a game can wait for a leaderboard job before processing a player’s movement or damage.

For example, a match event might carry match_id, event_id, and a score change. If a sender retries it after a lost acknowledgment, the receiver must recognise the same event rather than add the score twice. Event identity and retry behaviour belong to the model and protocol, not to whether TCP delivered the underlying bytes correctly.

Bound queues as well as messages: an unbounded queue consumes memory when input arrives faster than it can be processed, and its results grow increasingly stale. Whether a full queue may discard something depends on what the message means, which Lesson 9.7 works through for a relay.

Design the backend for one match

A player logs in, joins match 42, and buys an item for 300 coins. The game backend is the set of services behind those actions. Separating their jobs explains why a working socket alone cannot prove a successful purchase.

From login to the live world

Authentication establishes the player’s identity. A session lets later requests refer to that authenticated login; it can outlive one TCP connection. A matchmaker groups compatible players and arranges a match location. It does not calculate their movement. Microsoft’s PlayFab matchmaking overview illustrates that service role.

An API gateway routes requests to the appropriate service. Middleware performs shared checks along that route: validate the session, bound the request, enforce its permissions, and record its outcome. A load balancer can spread these entry requests across equivalent gateway instances. The authoritative match server owns match 42’s accepted positions and damage; route its commands to that owner instead of a randomly chosen empty worker. These are logical jobs, which a small project can combine in one program.

Login, match allocation and asset delivery have different owners. These are logical service roles, not a requirement to deploy every box separately. Choose a tab and step through it to see who answers each request:

Log in: the database owns the accountIdentity comes from the durable account record. After that, every request carries a session instead of the password.

ClientGatewayDatabase

1. 1

   Request Log in: name and password

   Step 1: Client to Gateway

   The client talks to the gateway, the single public entry point. It never talks to the database itself.

2. 2

   Alone Middleware checks the request

   Step 2: Gateway (alone)

   Shared checks run before any service sees it: bound the size, enforce permissions, record the outcome.

3. 3

   Request Look up the account

   Step 3: Gateway to Database

   The database is the durable owner of account records and balances. The gateway asks it; it does not keep a copy it would trust more.

4. 4

   Reply Account found, password matches

   Step 4: Database to Gateway

   The stored (hashed) password agrees with what was sent, so the player is authenticated.

5. 5

   Reply Session token

   Step 5: Gateway to Client

   A session lets later requests refer to this login. It can outlive one TCP connection, so a dropped socket does not mean logging in again.

Identity comes from the durable account record. After that, every request carries a session instead of the password.

Join match 42: one match server owns the worldThe matchmaker only arranges a place to play. The authoritative match server that owns match 42 decides positions and damage.

ClientGatewayMatch-makerOwner recordMatch server

1. 1

   Request Join match 42 with the session

   Step 1: Client to Gateway

   The request names the match and carries the token from the login.

2. 2

   Alone Middleware validates the session

   Step 2: Gateway (alone)

   An unknown or expired session stops here, before any matchmaking work.

3. 3

   Request Find or allocate match 42

   Step 3: Gateway to Match-maker

   The matchmaker groups compatible players and arranges a match location. It does not calculate their movement.

4. 4

   Control Which worker owns match 42?

   Step 4: Match-maker to Owner record

   A coordination record says which worker owns the match. If that worker fails, a replacement needs a new ownership generation so the old one cannot write late.

5. 5

   Reply Owner: this match server

   Step 5: Owner record to Match-maker

   The answer names one owner. Commands must go to it, not to a random empty worker.

6. 6

   Reply Match 42 runs on that server

   Step 6: Match-maker to Gateway

   The gateway learns where the match lives.

7. 7

   Reply Address of the match server

   Step 7: Gateway to Client

   The client now knows where to send its game commands.

8. 8

   Request Game command: move north

   Step 8: Client to Match server

   Commands go to the owner of match 42, which checks them against the rules.

9. 9

   Reply Accepted position (10, 14)

   Step 9: Match server to Client

   The server keeps the accepted state; each client draws what the server accepted. A wall would have left the position at (10, 15).

The matchmaker only arranges a place to play. The authoritative match server that owns match 42 decides positions and damage.

Download a patch: assets come from the CDNLarge files that are the same for every player come from a content delivery network, not from the match server or the database.

ClientCDN edgeObject storage

1. 1

   Request Get patch file v12

   Step 1: Client to CDN edge

   The patch is a versioned file, so any nearby copy is as good as the original.

2. 2

   Alone Cache lookup: no copy yet

   Step 2: CDN edge (alone)

   This edge has not served the file before, so it must go to the origin once.

3. 3

   Request Fetch the file by its key

   Step 3: CDN edge to Object storage

   Object storage keeps larger files addressed by keys, such as versioned patches and replay recordings. It is the origin, the owner of the original bytes.

4. 4

   Data File bytes

   Step 4: Object storage to CDN edge

   The origin returns the file.

5. 5

   Data File bytes, copy kept at the edge

   Step 5: CDN edge to Client

   The edge stores a copy for later requests that are eligible for caching.

6. 6

   Request The next player asks for v12

   Step 6: Client to CDN edge

   A different player, near the same edge, asks for the same file.

7. 7

   Data Served from the edge

   Step 7: CDN edge to Client

   A cache hit avoids the origin fetch. Personal account responses follow separate rules: an old cached balance must never authorize a new purchase.

Large files that are the same for every player come from a content delivery network, not from the match server or the database.

Three questions, three ownersEach kind of data has one place that is allowed to answer authoritatively. The gateway routes; it owns none of them.

ClientGatewayMatch serverDatabaseCache

1. 1

   Request Ask: move north

   Step 1: Client to Gateway

   A request to change the world. The gateway checks it and routes it, but does not decide the result.

2. 2

   Request Route to the match’s owner

   Step 2: Gateway to Match server

   Positions and damage belong to the authoritative match server for that match.

3. 3

   Reply Accepted position (10, 14)

   Step 3: Match server to Client

   Owner of positions: the match server. Its answer is the truth the clients show.

4. 4

   Request Ask: what is my balance?

   Step 4: Client to Gateway

   A question about money, which must be exactly right.

5. 5

   Request Read the balance

   Step 5: Gateway to Database

   Owner of balances and inventory: the database, in protected, durable storage.

6. 6

   Reply Balance: 1,000 coins

   Step 6: Database to Client

   A current, authoritative answer. A cached value could be old, so it is never used to approve spending.

7. 7

   Request Ask: how many players in the lobby?

   Step 7: Client to Gateway

   A question where a slightly old answer is harmless.

8. 8

   Request Read the cached count

   Step 8: Gateway to Cache

   A cache holds a reusable result that can be rebuilt. It owns no original data.

9. 9

   Reply Count, possibly a few seconds old

   Step 9: Cache to Client

   Fast and cheap, and good enough for a label. If it is lost, it is rebuilt from the real data.

Each kind of data has one place that is allowed to answer authoritatively. The gateway routes; it owns none of them.

Put each value in the right store

A database keeps structured, durable account balances and inventory. Object storage keeps larger files addressed by keys, such as versioned patches and replay recordings. A cache reuses a result that can be rebuilt; a CDN, or content delivery network, brings eligible asset copies closer to players. An asset cache hit can avoid an origin fetch, as Cloudflare’s request lifecycle explains. Personal account responses need separate authorization and cache rules. A cached lobby count is useful; its old balance must not authorize a new purchase.

One purchase, one effect

The player has 1,000 coins. Request P7 asks for the 300-coin item. Validate the server’s price and current balance, debit to 700, grant the item, and record P7’s result inside one transaction: either that whole change commits or none does. Concurrent requests also need appropriate isolation or locking; merely surrounding unsafe read-then-write logic with a transaction is insufficient. PostgreSQL’s isolation guide explains these concurrency guarantees and possible transaction retries.

If the reply is lost, retry the same request ID. Idempotency means that repeat has the same effect: return the recorded result, without another debit or item. Scope the ID to its account, reject a changed payload under the same ID, and save the receipt atomically with the effect.

Purchase P7 commits its debit, item and receipt together; retrying P7 returns that receipt. The separate leaderboard example may display an older score until its derived view catches up. The order of the steps is the point, so step through each tab:

Buy an item: one transaction, three changesThe player has 1,000 coins and asks for a 300-coin item with request P7. The debit, the item and the receipt commit together or not at all.

ClientGatewayGame serviceDatabase

1. 1

   Request Buy the item for 300 coins (P7)

   Step 1: Client to Gateway

   P7 is the request ID. The client states what it wants, but not what it costs: the server knows the price.

2. 2

   Request Forward P7 for this account

   Step 2: Gateway to Game service

   After the middleware checks, the request reaches the service that owns purchases.

3. 3

   Control Begin transaction

   Step 3: Game service to Database

   From here to the commit, the changes are all-or-nothing.

4. 4

   Request Read the price (300) and the balance (1,000)

   Step 4: Game service to Database

   The server’s own price decides the cost. The read is protected by isolation or a lock, so a second request cannot spend the same coins in between.

5. 5

   Alone Debit to 700, add item, record P7

   Step 5: Database (alone)

   1,000 − 300 = 700. Three changes: the coins, the item, and the receipt for P7. None of them is visible to anyone else yet.

6. 6

   Control Commit

   Step 6: Game service to Database

   Only now do all three changes become visible together. A crash before this point would have undone every one of them.

7. 7

   Reply Committed

   Step 7: Database to Game service

   The purchase is durable.

8. 8

   Reply Item granted, balance 700 (P7)

   Step 8: Game service to Gateway

   The reply is sent only after the commit, so the player never sees an effect that could still vanish.

9. 9

   Reply Item granted, balance 700 (P7)

   Step 9: Gateway to Client

   The client shows the new balance.

The player has 1,000 coins and asks for a 300-coin item with request P7. The debit, the item and the receipt commit together or not at all.

Retry after a lost reply: the receipt answersThe reply to P7 never arrives, so the client sends P7 again. Idempotency means the repeat has the same effect as one request: no second debit, no second item.

ClientGatewayGame serviceDatabase

1. 1

   Alone The reply to P7 never arrives

   Step 1: Client (alone)

   The purchase did commit, but the client cannot know that: the network lost the answer.

2. 2

   Request Retry the same request ID: P7

   Step 2: Client to Gateway

   Because the ID is the same, the server can tell this is a repeat and not a second purchase.

3. 3

   Request Forward P7

   Step 3: Gateway to Game service

   The retry takes the same route as the original.

4. 4

   Request Look up the receipt for P7

   Step 4: Game service to Database

   The ID is scoped to the account. A changed payload under the same ID would be rejected, not merged.

5. 5

   Reply Receipt found: item granted, balance 700

   Step 5: Database to Game service

   The receipt was saved in the same transaction as the effect, so it exists exactly when the purchase does.

6. 6

   Reply The recorded result of P7

   Step 6: Game service to Gateway

   The service returns the stored result. It does not debit again and does not grant a second item.

7. 7

   Reply Item granted, balance 700 (P7)

   Step 7: Gateway to Client

   The client finally gets its answer. The balance is still 700, not 400.

The reply to P7 never arrives, so the client sends P7 again. Idempotency means the repeat has the same effect as one request: no second debit, no second item.

Leaderboard: allowed to lag, in a known wayAn accepted score is durable at once, but the leaderboard is a derived view. It may show the old score until it catches up, which is fine for a ranking and wrong for a balance.

Match serverDatabaseCachePlayer

1. 1

   Request Save the accepted score: 10 to 12

   Step 1: Match server to Database

   The score is a fact the match server has accepted. The database stores it durably.

2. 2

   Reply Committed: the score is 12

   Step 2: Database to Match server

   Anyone who reads the database now sees 12.

3. 3

   Request Open the leaderboard

   Step 3: Player to Cache

   The page reads a derived view held in a cache, not the database, so the database stays free for important work.

4. 4

   Data Shows 10: an older view

   Step 4: Cache to Player

   The view has not heard about the 12 yet. Showing a clearly dated value is acceptable here.

5. 5

   Control Rebuild the view from the new score

   Step 5: Database to Cache

   Eventual consistency promises that the view converges once updates stop and propagation succeeds. It does not promise how long that takes.

6. 6

   Request Refresh the leaderboard

   Step 6: Player to Cache

   The player asks again, some time later.

7. 7

   Data Shows 12

   Step 7: Cache to Player

   The view caught up. The same lag would not be acceptable for authorizing a purchase, which needs a current protected decision.

An accepted score is durable at once, but the leaderboard is a derived view. It may show the old score until it catches up, which is fine for a ranking and wrong for a balance.

Consistency describes which writes a read can observe. A strongly consistent read respects completed writes within its promised scope. A weaker read can return an older replica or cache value. Eventual consistency promises convergence when updates stop and propagation succeeds, not a fixed maximum delay. Currency authorization needs a current protected decision; a leaderboard can tolerate a clearly dated older view. Cosmos DB’s consistency definitions show why these guarantees must be stated explicitly.

Optional: explore the smaller concurrency problem below. Starting at 1,000, a reward of 500 and purchase of 300 should leave 1,200. Overlapping reads and writes can lose one change. Its local lock illustrates ordering; a backend must enforce its database transaction rules across all participating processes.

This simulation needs JavaScript. The lesson text around it works through the same numbers.

More matches, with explicit ownership

Sharding partitions work or records by a key: independent matches can run on different workers. A shared world crossing shard boundaries additionally needs ownership handoffs and ordered cross-boundary actions. Moving a job to another process also adds a message boundary, possible failures, and delay, so start with a small correct local service and distribute work when measurements justify it. Azure’s sharding pattern describes the partitioning tradeoff.

Coordination records which worker owns match 42. On failure, a replacement needs recoverable state and a new ownership generation; an expired worker must be prevented from committing late writes. Electing a replacement alone cannot reconstruct lost match state. Leader-election guidance explains leases and their failure limits. Retried result events also need persistent event IDs to avoid awarding the same win twice.

Spend the latency budget where it matters

A toy 60-tick-per-second server has about 16.7 milliseconds per simulation step. That budget is distinct from network round-trip delay. Keep slow replay uploads and leaderboard work outside the movement path. Measure queue depth, bytes per player, and slow-tail latency before optimizing. Spatial indexes can reduce repeated full-world searches; smaller state deltas reduce bandwidth, while compression spends CPU time. Extra workers cannot repair an undefined ownership rule or an overloaded shared database.

UDP is made of datagrams

UDP preserves datagram boundaries but does not guarantee delivery, order, or uniqueness. Fast games often tolerate those tradeoffs for time-sensitive state.

A later position update can make an older one useless, so freshness may matter more than retransmission. In contrast, a one-time inventory change may need explicit ordering and acknowledgment. Choose behavior per message kind instead of labeling an entire game “reliable” or “unreliable.”

Do not assume a game uses only one protocol. It might use TCP for login/chat and UDP for movement.

Ports and sockets

A socket represents a network endpoint. This complete helper opens a TCP connection to our local test server, or returns an error:

use std::{io, net::{SocketAddr, TcpStream}};
use std::time::Duration;

fn connect_local() -> io::Result<TcpStream> {
    let address = SocketAddr::from(([127, 0, 0, 1], 15000));
    // This bounds connection establishment, not later read/write operations.
    TcpStream::connect_timeout(&address, Duration::from_secs(3))
}

127.0.0.1 means the same computer; 15000 selects the server’s listening port. We use loopback and a local test server for this section. A successful result is a connected stream; framing and protocol checks still remain.

Encryption changes what you can see

If a protocol uses modern encryption correctly, a packet capture shows encrypted bytes, addresses, sizes, and timing—not readable messages. Do not try to defeat encryption or certificate checks.

For learning, use:

• your own toy protocol;
• a local open-source server;
• captured bytes provided by a challenge;
• application logs you control.

Captures in this chapter stay local

The capture, replay, and proxy examples use loopback or a local test server. That keeps timing, state changes, and failures observable while you learn what each protocol layer contributes.

Now that the message boundary and its byte order have a purpose, Lesson 9.3 records a real local exchange and asks which captured bytes correspond to one controlled action.

🧩

Knowledge check

Find the official state

In a typical multiplayer design, why can a client-side visual value disagree with the final result?

1The server owns the authoritative shared state and resolves outcomes2Packets carry bare timestamps, so each client rebuilds values itself3The GPU applies the game rules before the server sends a reply4The clients share one address space and overwrite each other's values

Check answer

Finished Lesson 9.1?

Mark it done and the lesson list shows a ✓ beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.

Mark lesson doneLesson done. Mark it not done

Chapter 9: 10 lessons

📝 Notes