When is a representation mistaken for the thing it represents?

The problem isn't always that the picture is wrong. It's that we believe the picture is complete.

A representation is mistaken for reality when we treat a measurement, description, image, or model as a complete account of the thing it represents. We forget what has been selected, simplified, or omitted and begin making judgments as though nothing important is missing.

A school reports an average test score of 85. The number is accurate, but it does not tell us whether every student scored near 85 or half scored 100 while the other half scored 70. Both classrooms produce the same average. They do not present the same educational challenges.

The number is correct. Our conclusion about what it means is not.

Every representation leaves something out

A representation is useful precisely because it simplifies reality.

A road map preserves locations, distances, and connections. It leaves out the weather, traffic, noise, and countless other features of traveling through a city. A map that captured everything about a city would be almost as complicated as the city itself.

Philosopher and scientist Alfred Korzybski popularized the observation that the map is not the territory. Belgian artist René Magritte explored a related distinction in his 1929 painting The Treachery of Images, which depicts a pipe beneath the words Ceci n'est pas une pipe, meaning This is not a pipe. The image resembles a pipe, but you cannot fill it with tobacco or smoke it.

The distinction matters when we use simplified descriptions to draw broader conclusions.

Revenue is taken to indicate business health. Test scores stand in for intelligence. Job titles suggest competence. Online followers imply influence. Each measure captures something useful while leaving other dimensions unexplored.

We make these substitutions because representations are easier to compare, communicate, and remember than the complicated realities behind them. A single number settles an argument more quickly than an explanation of what produced it.

Convenience encourages conclusions the evidence does not support.

When the measurement changes reality

In September 2016, the Consumer Financial Protection Bureau penalized Wells Fargo for widespread illegal sales practices. Employees had opened deposit and credit card accounts without customer authorization, encouraged by sales targets and compensation incentives.

An initial review identified approximately 2.1 million potentially unauthorized accounts. In August 2017, Wells Fargo reported that an expanded review had identified approximately 3.5 million, while acknowledging that some flagged accounts were properly authorized.

The bank emphasized selling additional financial products as a measure of success. Higher account counts appeared to demonstrate stronger customer relationships and greater business performance.

Employees faced incentives to increase those numbers, including by creating accounts customers had never requested.

This illustrates Goodhart's law, commonly understood as the tendency of a measure to lose its value as an indicator when people begin optimizing for the measure itself.

The account counts could be numerically accurate while failing to reflect genuine customer demand or satisfaction. The measure was incomplete, and the incentives tied to it encouraged behavior that widened the gap between what was measured and what actually mattered.

The organization had a number it could improve without improving the outcome the number was supposed to indicate.

Accurate is not the same as sufficient

We often evaluate a representation by asking whether it is accurate. Does the number add up? Does the photograph depict what was there? Does the model predict the outcome correctly?

Accuracy answers only part of the question.

A representation can accurately describe everything it includes and still be insufficient for the decision we want to make.

A company reports 25 consecutive years of revenue growth. The statistic is accurate and sounds impressive. But its expenses have grown faster than revenue, its debt has become difficult to service, and its largest customers are leaving.

The historical growth remains true. It tells us little about whether the company is financially healthy today.

Or return to the school reporting an average test score of 85. If the goal is to compare overall test performance, the average provides useful information. If the goal is to identify students who need additional support, the same number is inadequate.

The underlying information has not changed. The question has.

This gives us two separate tests for any representation:

• Is it accurate? Does it correctly describe what it claims to measure or depict?
• Is it sufficient? Does it contain the information needed to support the conclusion we are drawing?

The first tests the representation against reality. The second tests its usefulness for a particular question.

Representations shape what we notice

A manager describes an employee as unreliable after three missed deadlines.

The label summarizes observed behavior, but it excludes deadlines met, circumstances behind the delays, and evidence of improvement.

Once the manager accepts that label as a complete description, subsequent behavior is interpreted through it. Another late assignment confirms the judgment. Completed assignments receive less attention.

The representation now influences which observations the manager considers meaningful. Evidence is no longer evaluated independently of the original judgment.

The danger extends beyond overlooking information. Decisions based on an incomplete representation influence future behavior and opportunities. A manager who assumes an employee is unreliable might stop assigning important projects, limiting the employee's chances to demonstrate otherwise.

The original description has begun influencing the conditions against which its accuracy will be judged.

Artificial intelligence makes the problem harder

Artificial intelligence introduces two distinct risks. A representation can be false but convincing, or accurate but incomplete.

In 2023, lawyers representing a plaintiff in Mata v. Avianca submitted a court filing containing nonexistent judicial decisions and fabricated quotations produced with ChatGPT.

The citations looked like ordinary legal references, complete with case names and supposed judicial reasoning. But the decisions did not exist. Even after the court questioned the authorities, the lawyers continued defending the fabricated material. In June 2023, a federal judge imposed a $5,000 sanction.

The familiar appearance of legal authority substituted for verification against actual court records.

The second risk persists even when an AI system produces factually correct information.

An AI system reviewing résumés might accurately extract education, employment history, and technical skills. Yet those fields do not fully represent an applicant's judgment, creativity, adaptability, or ability to collaborate.

Improving extraction accuracy does not resolve the problem of missing information. The system still evaluates people through a restricted set of characteristics.

A more detailed model is not automatically a more useful one. Greater realism, precision, and fluency do little to improve a judgment if the information that matters most remains outside the representation.

How do we recognize the substitution?

Start by identifying what the representation actually captures. A financial report records defined financial measures. A test score measures performance on particular tasks. A photograph captures a particular view at a particular moment.

Then identify what your conclusion requires. Understanding a company's health demands more than revenue. Evaluating someone's abilities demands more than a résumé. Understanding an event demands more than a single image.

Finally, look for evidence outside the representation. If direct observations contradict a model, investigate the discrepancy. If a metric improves while the underlying experience deteriorates, examine what the metric rewards. If a label prevents us from noticing contradictory behavior, reconsider its usefulness.

Representations are indispensable. Without them, we would struggle to communicate, calculate, plan, or understand things beyond our immediate experience. Their value comes from simplifying reality while preserving enough of it to answer the questions we ask.

A representation is mistaken for reality when we stop asking what is missing.

📚Bookmarked for You

Thinking in Systems by Donella H. Meadows: Explains how models, boundaries, measurements, and feedback influence our understanding of complex systems.

The Image: A Guide to Pseudo-events in America by Daniel J. Boorstin: Examines how manufactured images and events shape our perceptions of reality.

🧬 QuestionStrings to Practice: Beyond the Model

Use this sequence when a statistic, label, report, or model is influencing an important decision. Let each answer inform the next.

What am I using to represent the situation? → What does it accurately capture? → What relevant information does it leave out? → Could that missing information change my conclusion? → What additional evidence should I examine before deciding?

Every day, QuestionClass offers another question worth considering. Read, reflect, and practice recognizing the question a situation demands.

← Home