Renner Complex Systems

Starting Point

Rodney Brooks’s 1991 paper “Intelligence without representation” is one of the foundational challenges to classical symbolic AI. Brooks is not merely criticizing a particular knowledge representation scheme or planning technique. His argument is more fundamental: intelligence should not be understood as a centralized process that first constructs a complete internal description of the world, reasons over that description, and then converts its conclusions into action. Instead, Brooks proposes an architecture in which multiple relatively autonomous behavioral layers operate in parallel and connect perception directly to action. [1]

From today’s perspective, the paper is striking because many of its objections to 1980s AI have reappeared in contemporary debates about AGI: embodied intelligence, agent architectures, world models, multimodal systems, representation learning, and the question of whether language models genuinely understand the world or merely operate on abstractions that humans have already produced.

The central RCS interpretation is:

Brooks’s lasting contribution is not the strong claim that intelligent systems need no representations at all. It is the warning that we should not prematurely impose the state variables, abstractions, and interfaces that an intelligent system is supposed to use.

1. Brooks’s Challenge to the Classical AI Architecture

Classical AI of the 1970s and 1980s can be summarized schematically as a processing pipeline:

$$
\text{Perception}
\rightarrow
\text{World Representation}
\rightarrow
\text{Reasoning / Planning}
\rightarrow
\text{Action}
$$

Sensory data are first translated into a symbolic description of the world. A centralized reasoner or planner then operates on that representation. Finally, the result is translated back into actions.

Brooks sees a structural problem in this design. Perception, representation, planning, and action are treated as largely separable modules. That forces researchers to decide in advance:

which entities exist,
which properties matter,
which relationships should be represented,
which information must pass between modules,
and which level of abstraction is appropriate for the task.

Yet those decisions may themselves constitute a major part of what intelligence actually is.

2. The Hidden Intelligence of the Experimenter

One of the paper’s strongest observations concerns the preprocessing of problems.

A classical planning system does not receive raw reality. It does not, for example, receive an image containing a person, a chair, and a banana. Instead, it may receive statements such as:

PERSON(A)
CHAIR(B)
BANANA(C)
SITTING_ON(A,B)
ABOVE(C,A)

At that point, the problem has already been reduced to a compact, discrete state space.

But the critical transformation

$$
\text{raw sensory data}
\rightarrow
\text{relevant entities and relations}
$$

has been performed by a human.

Brooks’s objection is essentially this:

If the researcher has already decided what counts as an object, which properties matter, and which relations connect those objects, the researcher may already have solved the hardest part of the intelligence problem. [1]

What remains for the AI system may be little more than search, optimization, or symbolic inference.

This criticism remains highly relevant to modern AI benchmarks.

3. Brooks’s Alternative: Decomposition by Activity

Brooks replaces the functional decomposition

$$
\text{Perception}
\rightarrow
\text{Cognition}
\rightarrow
\text{Action}
$$

with a decomposition by activity or behavior.

Each layer connects perception directly to action:

$$
S_t
\rightarrow
B_i(S_t)
\rightarrow
A_t
$$

Several such systems operate in parallel:

$$
B_1,\; B_2,\; \ldots,\; B_n
$$

Typical behaviors include:

obstacle avoidance,
wandering,
exploration,
moving toward distant targets,
grasping particular objects.

Higher-level layers can suppress or inhibit lower-level ones without requiring a complete model of how those lower-level behaviors work internally.

This became known as the subsumption architecture. [1]

The key point is that there is no single location in the system where a complete representation of the world exists.

4. Intelligence as a Coupled Dynamical System

From an RCS perspective, Brooks is more naturally expressed in the language of dynamical systems than in the language of classical symbolic computation.

Each behavioral layer can be interpreted as a dynamical subsystem:

$$
\dot{x}_i=f_i(x_i,y)
$$

with sensory coupling

$$
y=h(E)
$$

to an environment $E$.

The agent’s actions, in turn, modify that environment:

$$
\dot{E}=G(E,A)
$$

so the overall coupled dynamics can be written schematically as

$$
\dot{X}=F(X,E)
$$

$$
\dot{E}=G(E,A(X,E)).
$$

Under this view, intelligence is not simply a property of an isolated internal algorithm. It emerges through the feedback loop

$$
\boxed{\text{Agent} \leftrightarrow \text{Environment}}
$$

The overall behavior can appear coherent and goal-directed even when there is no centralized controller.

5. “The World Is Its Own Best Model”

One of Brooks’s best-known ideas is to use the world itself as its own model. [1]

A conventional model-based agent might construct an internal state

$$
z_t=E(o_t)
$$

and then simulate

$$
\hat{z}_{t+1}=F(z_t,a_t).
$$

Brooks asks a different question: why construct a complete internal replica of the world when the agent can simply observe the world again?

After an action, reality itself produces the next state:

$$
o_{t+1}=\operatorname{Sensor}(World_{t+1}).
$$

From a control-theoretic perspective, this is a powerful idea.

An internal model

$$
\dot{\hat{x}}=f(\hat{x},u)
$$

is always potentially wrong.

A closed perception-action loop, by contrast, can continuously use new measurements

$$
y(t)=h(x(t))+\epsilon(t)
$$

to correct model error.

Brooks therefore shifts the emphasis from open-loop cognition toward closed-loop cognition.

6. Emergence Instead of Central Control

Brooks’s architecture contains no single process that represents the system’s “true goal” and then controls everything else.

Instead, several behavioral systems interact.

The macroscopic behavior

$$
B(t)
$$

emerges from

$$
B(t)=\Phi(B_1(t),B_2(t),\ldots,B_n(t),E(t)).
$$

To an external observer, the resulting behavior may look coherent and strategic.

Brooks explicitly notes that an observer can easily attribute a centralized plan or representation to the system even when no such physical locus exists. [1]

This creates a clear connection to Minsky’s Society of Mind and, more broadly, to distributed approaches to cognition.

7. The Moravec Paradox

Brooks makes an evolutionary argument.

Basic sensorimotor capabilities took extremely long evolutionary timescales to develop. Abstract language, formal reasoning, expert knowledge, and complex symbolic culture appeared much later. [1]

This suggests that

$$
\text{seeing, moving, manipulating}
$$

may be computationally more fundamental and difficult than

$$
\text{chess, logic, language, formal problem solving}.
$$

This is closely related to what later became known as the Moravec paradox.

Modern AI provides at least one striking indication that Brooks was onto something. Systems can now write sophisticated text, produce software, and solve difficult mathematical problems, while robust general-purpose physical manipulation remains much harder.

Current multimodal robotics research treats this gap as a central challenge. Google DeepMind’s Gemini Robotics work explicitly combines perception, physical reasoning, planning, and real-world action. [4]

8. Where Brooks Looks Remarkably Modern

Several of the paper’s core ideas reappear in modern form.

8.1 End-to-End Learning

Brooks criticizes artificially imposed interfaces between perception, cognition, and action.

Modern neural systems increasingly try to learn useful intermediate representations directly:

$$
o
\rightarrow
z
\rightarrow
a.
$$

The latent representation

$$
z=f_\theta(o)
$$

is not specified in advance as a hand-built symbolic ontology.

In that sense, deep learning has absorbed a major part of Brooks’s critique.

8.2 Embodied AI

Brooks insists on real perception-action loops.

Modern embodied AI again emphasizes exactly this structure:

$$
\text{Perception}
\rightarrow
\text{Reasoning}
\rightarrow
\text{Action}
\rightarrow
\text{New Perception}.
$$

Current robotics systems increasingly combine multimodal models, planning, and physical control in real or realistic environments. [4]

8.3 Agent Systems

Modern agent architectures also decompose complex behavior into specialized components:

planner
researcher
critic
executor
memory
tool user
monitor

Anthropic has explored both practical agent architectures and the increasing autonomy of AI systems. [5][6]

The similarity to Brooks lies in the move away from a single monolithic control process.

The crucial difference is that modern agent components often contain powerful foundation models with rich internal representations.

9. Where Brooks Probably Went Too Far

The strong version of Brooks’s thesis is roughly

$$
\boxed{\text{Intelligence does not require representation}}
$$

and that is difficult to defend today.

Modern neural systems create internal latent states

$$
z\in\mathbb{R}^d
$$

that systematically carry information about inputs, objects, relationships, states, and possible actions.

From an information-theoretic perspective, an internal state $z$ can already be treated as a representation if

$$
I(z;E)>0,
$$

meaning that the state contains information about the environment $E$.

Brooks himself acknowledges that one could interpret the internal states of his systems as implicit representations. He mainly rejects the term because those states do not resemble the explicit symbolic representations common in AI at the time. [1]

A more defensible modern version of his argument is therefore

$$
\boxed{
\text{Intelligence does not require a single explicit centralized symbolic world model}
}
$$

That claim remains much stronger.

10. World Models: A Partial Countermovement to Brooks

Contemporary research is explicitly returning to world models.

Meta describes V-JEPA 2 as a self-supervised foundation world model that learns abstract states and physical dynamics from video and can use them for prediction, planning, and robotics. [2][3]

Schematically:

$$
z_t=E(o_t)
$$

$$
\hat{z}_{t+1}=F(z_t,a_t).
$$

The agent can therefore simulate possible developments before acting.

This appears to contradict the radical reading of Brooks:

$$
\text{the world is its own best model}.
$$

But the contradiction is only partial.

For immediate reactive control, the real world can indeed serve as the relevant “simulator.”

For long-horizon planning, that is not enough.

11. Brooks’s Limit: Hypothetical Futures

Suppose an agent must reason about what might happen tomorrow.

That future state does not yet exist and therefore cannot be directly observed.

The agent needs an estimate such as

$$
P(s_{t+k}\mid s_t,a_t,\ldots,a_{t+k-1}).
$$

For counterfactual reasoning, it may need something like

$$
P(Y\mid do(A)).
$$

The system must therefore reason about states that are not part of the currently observable world.

At that point, the world cannot simply serve as its own immediate model.

Some form of internal simulation is required:

$$
\hat{F}:(s_t,a_t)\rightarrow s_{t+1}.
$$

The longer the planning horizon, the more important this becomes.

12. Two Cognitive Loops Instead of an Either-Or Choice

A modern synthesis can therefore be described as two coupled loops.

Fast real-world loop

$$z_t\rightarrow
a_t
\rightarrow
World
\rightarrow
z_{t+1}
$$

This is the Brooks side.

Slower hypothetical loop

z_t \rightarrow \hat{F}(z_t,a) \rightarrow \widehat{z}_{t+1} \rightarrow \widehat{z}_{t+2} \rightarrow \cdots

This is the world-model side.

A general intelligent architecture will probably need both.

13. Brooks and LLMs

A conventional LLM initially looks almost like the opposite of Brooks’s proposal.

Its basic training objective can be simplified as

$$
P(x_{t+1}\mid x_{\leq t}).
$$

In its basic form, such a model has

no body,
no continuous sensors,
no direct perception-action loop,
no persistent physical existence,
no autonomous experimental interaction with its environment.

From Brooks’s perspective, a language model would therefore probably not count as a complete “Creature.”

Yet the success of LLMs demonstrates something Brooks may have underestimated: language contains an extraordinary amount of compressed world structure.

14. Delegated Embodiment

Text does not arise independently of the physical world.

The information chain is closer to

$$
World
\rightarrow
Human\ Perception
\rightarrow
Human\ Cognition
\rightarrow
Language
\rightarrow
LLM.
$$

The LLM therefore receives, indirectly, the accumulated results of human perception and abstraction.

This can be described as delegated embodiment.

It also explains why Brooks’s critique remains relevant.

The success of language models demonstrates

$$
\text{human abstractions}
\rightarrow
\text{powerful learned cognition}
$$

but it does not by itself demonstrate

$$
\text{raw world}
\rightarrow
\text{autonomous discovery of useful abstractions}.
$$

That second transformation is much closer to the problem Brooks cared about.

15. A Harder AGI Test

Many current benchmarks already operate inside highly structured problem spaces.

A Brooks-style test would be more demanding.

An intelligent system would enter an unfamiliar dynamic environment and have to determine for itself:

which entities persist,
which variables matter,
which degrees of freedom exist,
which actions are possible,
which quantities are controllable,
which regularities govern the environment,
which causal relationships matter,
and which representation is useful for the task.

Formally:

$$
(o_0,o_1,\ldots)
\rightarrow
\text{discover state space}
\rightarrow
\text{discover dynamics}
\rightarrow
\text{discover affordances}
\rightarrow
\text{act}.
$$

The AGI question then changes from

How well can a model reason within a representation that has already been given?

Can the system discover which representation of the world is useful in the first place?

That is a much deeper requirement.

16. Brooks’s Scaling Problem

Brooks explicitly recognizes a possible limitation of his architecture: how many layers can be added before their interactions become too complex to manage? [1]

With $N$ interacting modules, the number of possible pairwise relationships can in the worst case grow roughly as

$$
O(N^2).
$$

A small number of layers can be designed and debugged manually.

A general intelligence, however, may require a very large number of behavioral modes.

This is one plausible reason why hand-built subsumption architectures did not scale directly into AGI.

17. Deep Learning as a Possible Solution to Brooks’s Scaling Problem

One useful historical interpretation is that deep learning did not simply invalidate Brooks. It automated part of his program.

Instead of manually constructing thousands of behavioral modules and their interfaces, we learn a policy

$$
\pi_\theta(a\mid s)
$$

or a latent representation

$$
z=f_\theta(o)
$$

followed by

$$
a=\pi_\theta(z).
$$

The system’s important internal variables are therefore not specified entirely by the developer.

This points toward a softened Brooks principle:

The state variables that matter for intelligent behavior should emerge from data and interaction rather than being fully prescribed by a fixed ontology.

18. A Modern Reformulation of Brooks

The historical thesis

$$
\boxed{\text{Intelligence without representation}}
$$

is better reformulated today as

$$
\boxed{\text{Intelligence without predefined representation}}
$$

The agent learns from high-dimensional observations

$$
o_t\in\mathbb{R}^N
$$

a useful latent state

$$
z_t=E_\theta(o_{\leq t})
$$

such that this state supports the relevant tasks:

$$z_t
\arg\min_z
\left[
L_{\text{prediction}}
+
L_{\text{control}}
+
L_{\text{planning}}
\right].
$$

Representation is not eliminated.

It is endogenized.

The key question is no longer

Does the system have a representation?

but rather

Where does that representation come from, what determines it, and for which dynamics is it sufficient?

19. The Connection to Synergetics

From a synergetic perspective, Brooks is especially interesting.

His architecture consists of many local processes

$$
\dot{x}_i=f_i(x_i,E)
$$

whose interactions produce ordered macroscopic behavior.

That behavior can be represented abstractly as

$$
B=\Phi(x_1,\ldots,x_N,E).
$$

This immediately raises synergetic questions:

Are there a small number of macroscopic order parameters?
Which microscopic degrees of freedom become slaved?
Which activity modes compete?
Which control parameters trigger qualitative regime changes?
Which attractors correspond to stable behavioral patterns?
Under what conditions do new macroscopic states emerge?

Brooks’s architecture can therefore be read as an early version of a view in which intelligence is not primarily an algorithm but an emergent dynamic of a coupled agent-environment system.

20. A Possible RCS Architecture for AGI

Combining Brooks, modern representation learning, world models, and synergetic systems theory suggests a useful synthesis.[7][8]

Layer 1 — Sensory state formation

$$
o_t
\rightarrow
z_t
$$

Multimodal observations are mapped into learned latent states.

Layer 2 — Fast perception-action loops

$$
z_t
\rightarrow
a_t
$$

Robust reactive control handles short-timescale behavior.

Layer 3 — Dynamic world model

$$
(z_t,a_t)
\rightarrow
\hat{z}_{t+1}
$$

The system learns local and global dynamics.

Layer 4 — Macroscopic order parameters

A small set of relevant macrovariables

$$
q_1,\ldots,q_k
$$

is extracted from the high-dimensional state.

Their dynamics may be much lower dimensional:

$$
\dot{q}=G(q,\alpha).
$$

The control parameters $\alpha$ determine the qualitative regime of the system.

Layer 5 — Semantic reasoning

LLM-like systems interpret states, formulate hypotheses, analyze goals, and translate semantic requirements into model parameters.

Layer 6 — Internal simulation

Possible future developments are simulated not only linguistically, but through appropriate specialized models:

q_t \rightarrow \widehat{q}_{t+1} \rightarrow \widehat{q}_{t+2} \rightarrow \cdots

Layer 7 — Feedback to reality

Simulation generates candidate actions.

Reality then provides new measurements and corrects the model.

The resulting loop is

$$
\boxed{
\text{Observation}
\rightarrow
\text{Representation}
\rightarrow
\text{Dynamics}
\rightarrow
\text{Simulation}
\rightarrow
\text{Action}
\rightarrow
\text{Observation}
}
$$

21. Multiple Timescales of Intelligence

Such an architecture would not be a single global reasoner. It would be a multiscale system.

For example,

$$
\tau_1\sim \text{ms}
$$

for reflexes and motor control,

$$
\tau_2\sim \text{s}
$$

for local perception-action loops,

$$
\tau_3\sim \text{minutes}
$$

for task planning,

$$
\tau_4\sim \text{days}
$$

for strategic objectives,

and potentially

$$
\tau_5\sim \text{years}
$$

for long-term development of knowledge, models, and goals.

AGI, in this view, is less a single universal algorithm than a hierarchically coupled dynamical system operating across multiple timescales and levels of abstraction.

22. Critical Assessment

Brooks was remarkably prescient on several points.

What still looks strong

Classical AI abstracted away too much of the real world.
Perception is not a trivial preprocessing module.
Perception, action, and reasoning cannot be developed independently without cost.
Closed perception-action loops are fundamental.
Centralized control is not necessarily required.
Complex behavior can emerge from interacting subsystems.
Real-world robustness is not the same thing as benchmark performance.
Classical AI significantly underestimated sensorimotor intelligence.

What was overstated

Representations in general are probably not dispensable.
Long-horizon planning requires internal models or functionally equivalent mechanisms.
Abstract cognition is not obviously “easy” merely because it appeared late in evolution.
Hand-built subsumption layers do not scale gracefully to highly complex systems.

What matters most today

The central question is no longer

$$
\text{representation}
\quad\text{vs.}\quad
\text{no representation}.
$$

A more important distinction is

$$
\boxed{
\text{predefined representation}
\quad\text{vs.}\quad
\text{learned task- and dynamics-dependent representation}
}
$$

23. The RCS Core Thesis

The most important consequence from an RCS perspective is this:

Intelligence does not begin with reasoning over state variables. It begins with discovering which state variables, order parameters, timescales, and model classes are relevant to the system in the first place.

That creates a natural connection among

complex systems,
system identification,
representation learning,
model reduction,
synergetics,
simulation,
agents,
world models,
and decision intelligence.

An intelligent system should not merely compute

$$
f(x).
$$

It should first discover

$$
\boxed{\text{What is }x\text{?}}
$$

Then:

$$
\boxed{\text{What dynamics govern }x\text{?}}
$$

Then:

$$
\boxed{\text{What low-dimensional structure actually matters?}}
$$

And finally:

$$
\boxed{\text{How will an intervention change the future dynamics?}}
$$

That is a fundamentally different view of AGI from the idea that ever-larger language models simply need to become progressively better at reasoning.

Brooks’s most durable lesson may therefore be summarized this way:

Do not confuse intelligence with reasoning over abstractions that someone else has already produced.

The deeper problem is to discover the abstractions themselves.

Sources

[1] Rodney A. Brooks, Intelligence without representation, Artificial Intelligence 47 (1991), 139–159.
Brooks paper

[2] Meta AI, V-JEPA 2: A self-supervised foundation world model.
V-JEPA 2

[3] Meta AI, Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning.
Meta AI — V-JEPA 2 benchmarks

[4] Google DeepMind, Gemini Robotics 2 brings whole-body intelligence to robots.
Google DeepMind — Gemini Robotics 2

[5] Anthropic, Building Effective AI Agents.
Anthropic — Building Effective AI Agents

[6] Anthropic, Measuring AI agent autonomy in practice.
Anthropic — Measuring Agent Autonomy

[7] Anna Jordanous, Intelligence without Representation: A Historical Perspective, Systems 8(3), 2020.
Historical Perspective

[8] Vincent C. Müller, Is there a future for AI without representation?, 2025.
Müller — AI without representation

Intelligence Without Representation – Brooks, AGI, and an RCS Perspective

Starting Point

1. Brooks’s Challenge to the Classical AI Architecture

2. The Hidden Intelligence of the Experimenter

3. Brooks’s Alternative: Decomposition by Activity

4. Intelligence as a Coupled Dynamical System

5. “The World Is Its Own Best Model”

6. Emergence Instead of Central Control

7. The Moravec Paradox

8. Where Brooks Looks Remarkably Modern

8.1 End-to-End Learning

8.2 Embodied AI

8.3 Agent Systems

9. Where Brooks Probably Went Too Far

10. World Models: A Partial Countermovement to Brooks

11. Brooks’s Limit: Hypothetical Futures

12. Two Cognitive Loops Instead of an Either-Or Choice

Fast real-world loop

Slower hypothetical loop

13. Brooks and LLMs

14. Delegated Embodiment

15. A Harder AGI Test

16. Brooks’s Scaling Problem

17. Deep Learning as a Possible Solution to Brooks’s Scaling Problem

18. A Modern Reformulation of Brooks

19. The Connection to Synergetics

20. A Possible RCS Architecture for AGI

Layer 1 — Sensory state formation

Layer 2 — Fast perception-action loops

Layer 3 — Dynamic world model

Layer 4 — Macroscopic order parameters

Layer 5 — Semantic reasoning

Layer 6 — Internal simulation

Layer 7 — Feedback to reality

21. Multiple Timescales of Intelligence

22. Critical Assessment

What still looks strong

What was overstated

What matters most today

23. The RCS Core Thesis

Sources