How Detections Work
Compare detector signals and count the real people behind a false-positive rate.
Cheatsheet for Lesson 15.4
A one-page summary of “How Detections Work” you can keep beside your notes. Plain Markdown.
Lesson 15.4 — How Detections Work: cheatsheet
- Remember: A small error rate applied to a large honest population can produce more false flags than true ones.
Key terms
- heuristic: A heuristic is a rule based on an expected pattern rather than an exact identity.
- base rate: The base rate is how common a condition is in the population being tested.
- Detection precision: Detection precision is the fraction of flags that are correct.
- Detection recall: Detection recall is the fraction of cheaters caught: here
99 ÷ 100 = 99%.
Review
- Q: In the hypothetical population, 99 cheaters and 999 honest players are flagged. About what fraction of the flagged group cheats? A: 9 percent of the flagged group
- Q: What does a file hash identify? A: An exact sequence of file bytes
A detector turns observations into a signal. To assess that signal, ask what was observed, what rule matched, and how often it makes mistakes in the population where it runs. Its output is not automatically a verdict.
Different signals measure different things
Section titled “Different signals measure different things”A signature detector matches a known pattern. A byte signature can identify a known code sequence, as in Lesson 5.7. A file hash can identify exact bytes. Matching a pattern is only as meaningful as its uniqueness and the trust in the observation.
A heuristic is a rule based on an expected pattern rather than an exact identity. Repeated input timing or an unlikely view change might trigger one. Legitimate devices, accessibility controls, or gameplay changes can also produce that pattern.
A behavioral detector studies actions over time. It may compare input edges, aiming transitions, or movement consistency. A statistical model describes the distribution of these observations. A machine-learning model learns relationships from labeled examples. Both need evaluation on legitimate cases outside the data used to build the model.
An integrity signal compares observed state with an expected reference. Hardware and account signals describe another part of the environment or its history. A new account or private profile is not itself proof of an unfair game action. Combining signals requires attention to what each actually measures.
Count a rare event before trusting a flag
Section titled “Count a rare event before trusting a flag”The base rate is how common a condition is in the population being
tested. Use a hypothetical population of 100,000 players. Assume one in
1,000 cheats. Then 100,000 ÷ 1,000 = 100 cheat, and
100,000 − 100 = 99,900 are honest.
Now assume the detector catches 99 percent of cheaters and wrongly flags 1 percent of honest players. These are chosen teaching assumptions, not claims about a real anti-cheat product.
Among the 100 cheaters, 0.99 × 100 = 99 are caught. The remaining
100 − 99 = 1 is missed. Among the 99,900 honest players,
0.01 × 99,900 = 999 are wrongly flagged. The remaining
99,900 − 999 = 98,901 are correctly left unflagged.
The flagged group therefore contains 99 + 999 = 1,098 people. Only
99 ÷ 1,098 ≈ 0.0902, or about 9 percent, are cheating.
Dots and the share of real flags
Each dot is 9 players
Counts
Chance that a flag is real
99 of 1098 flagged players are cheating: 9.0%. With 9 players per dot that is 11 filled and 111 open dots.
Most flagged players are honest in this hypothetical rare-cheating population.
Read the steps
- One in a thousand of 100,000 players gives 100 cheaters and 99,900 honest players.
- Catching 99 percent of 100 gives 99. Wrongly flagging 1 percent of 99,900 gives 999.
- The first eleven dots represent 99 caught cheaters: eleven times nine.
- Another 111 dots represent 999 honest players: 111 times nine. They join the same flagged population.
- There are 122 dots, representing 1,098 flags. Only eleven of those dots represent cheaters.
- 99 divided by 1,098 is about 9 percent. A flag needs further evidence before an accusation.
The rates are chosen assumptions, not estimates for a real game. The lesson derives every count.
The picture groups nine flagged people into each dot. Nine is chosen
because it divides both flagged counts exactly: 99 ÷ 9 = 11 dots for cheating,
and 999 ÷ 9 = 111 dots for honest players. Together they make 122 dots. It depicts
the flagged group, rather than every person in the original population.
This simulation needs JavaScript. The lesson text around it works through the same numbers.
Detection precision is the fraction of flags that are correct. In this example
it is about 9 percent, even though the detector catches almost all cheaters.
Detection recall is the fraction of cheaters caught: here 99 ÷ 100 = 99%.
The two numbers answer different questions.
Match the response to the evidence
Section titled “Match the response to the evidence”A hard rule violation, such as an impossible purchase against server-owned inventory, can be prevented immediately. A weak behavioral flag can send a trace for review without publicly naming the player.
Rechecking with another observation can help, but its errors may be correlated with the first check. Two models trained on the same mislabeled examples are not two independent witnesses. Measure the combined decision on real legitimate cases instead of multiplying error rates by assumption.
Delaying enforcement can allow further validation and group related cases. It also leaves time for harm, so prevention and mitigation must continue. Human reviewers need the trace and its context, clear decision criteria, and a way to correct mistakes. A human verdict is not automatically error-free either.
Count the flagged group
In the hypothetical population, 99 cheaters and 999 honest players are flagged. About what fraction of the flagged group cheats?
The group has 99 plus 999, or 1,098 people. Dividing 99 by 1,098 gives about 0.0902, or 9 percent. Catching 99 percent of cheaters measures recall, not precision.
Finished Lesson 15.4?
Mark it done and the lesson list shows a beside it. A chapter whose lessons are all done is marked done too. Progress is saved in this browser only.
Plain Markdown, saved in this browser.
This book grew out of the original Game Hacking Academy by attilathedud and has been rewritten as a beginner-first course. Like the original repository, its source is published under the Apache License 2.0.