The word system becomes useful when parts are related in ways that produce persistent behavior at the level of a whole.

A system is a bounded set of interdependent elements whose organization and interactions produce behavior over time within an environment.

This definition contains several commitments. A system has elements, but it cannot be understood from an inventory alone. It has a boundary, though that boundary depends partly on the question being asked. It has a state that can change. It interacts with an environment. Its overall behavior depends on relations among parts, not merely on the parts considered separately.

The word and its central idea

English system comes through Latin systema from Greek systēma, an organized whole composed of parts. Its roots carry the idea of things standing together rather than existing as an unrelated assortment.Etymonline: system

The term now covers very different objects:

• the solar system;
• a nervous system;
• an ecosystem;
• a legal system;
• an organization;
• a payment system;
• a software system;
• an AI system.

These examples do not share one material composition or one kind of purpose. What they share is an analytical form: distinguishable elements participate in relations that sustain some pattern of behavior at the level of a whole.

What must a system contain?

The smallest useful account of a system normally identifies:

elements
+ relations
+ a boundary

An account of how the system operates also needs:

state
+ rules or mechanisms of change
+ an environment
+ inputs and outputs
+ time

For an engineered system, further questions become central:

intended purpose
+ constraints
+ performance criteria
+ authority and responsibility

Engineering standards commonly define a system as interacting elements organized to achieve one or more stated purposes.ISO/IEC/IEEE 42020:2019

That purpose-centered definition is appropriate for engineered systems. It should not be projected onto every natural system. A climate system and a river system display organized behavior without needing intentions of their own.

A system is not an inventory

Suppose an online publishing operation contains an author, articles, Markdown files, a Git repository, a static-site generator, a deployment service, a domain, search engines, and readers.

The list tells us what might be present. It does not yet explain the system. The explanation begins with relations:

author ──writes──> article
article ──stored as──> Markdown
Git ──records──> revision
site generator ──transforms──> web page
deployment service ──publishes──> site
reader ──visits──> page
search engine ──indexes──> content

The system exists as an organized pattern of dependencies, transformations, permissions, and flows. If those relations disappear, the same objects may remain, but the publishing capability does not.

This is why a system is not simply the sum of its components. Organization is causally relevant.

How is a system different from a collection?

A collection is defined mainly by membership. A system is defined by interdependence and organization.

One hundred chairs in a warehouse form a collection. Players, coaches, rules, training, communication, and matches may form a team system. A contact list is a collection of records. Communication, roles, authority, and feedback can turn a group of people into an operating organization.

Boundaries make system analysis possible

Everything is connected to something else. If every remote influence must be included, the system expands until it becomes indistinguishable from the world. A useful analysis therefore defines a boundary.

A boundary answers:

Which elements and relations belong to the system under study, and which belong to its environment?

Consider a coffee shop. An analysis of waiting time may include customers, ordering, baristas, equipment, and queue rules. An analysis of profitability may add rent, suppliers, delivery platforms, and pricing. A food-safety analysis may include storage, temperature control, cleaning, and regulation.

The coffee shop has not become three different objects. The analytical boundary changes because the question changes.

INCOSE describes the environment as the part of the outside world that significantly interacts with and affects a system, often as the source of inputs and destination of outputs. Defining the boundary and environment is therefore a basic step in systems thinking.INCOSE: Systems Thinking 101

The boundary is selected, but it is not arbitrary. A model cannot exclude a major causal influence merely because including it would make the diagram untidy.

Open systems exchange with an environment

An environment may provide information, energy, materials, users, prices, legal constraints, threats, and disturbances. A system may return products, decisions, services, waste, risk, and social effects.

environment
    ↓ input
system state and operation
    ↓ output
changed environment
    ↓ new input
continued operation

Most systems studied in biology, society, organizations, and computing are open in this sense. They depend on continued exchange.

A closed system is often an analytical idealization. It means that particular exchanges can be ignored for a particular purpose, not that the object has no relation to anything outside it.

Systems have state and history

A system is not only an architecture diagram. It occupies states, changes state, and carries effects from its history.

A website may be healthy, building, partially deployed, unavailable because of DNS, or updated in a repository while production still serves an earlier release. The same components and nominal connections can therefore produce different outcomes at different moments.

A simple dynamic description is:

current state + input + transition rules
                    ↓
             next state + output

Time delays matter. A page may be live before a search engine indexes it. A product may improve before public expectations change. A policy may appear effective in the short term while accumulating a long-term cost.

Ignoring delay turns a dynamic system into a misleading snapshot.

Feedback changes subsequent behavior

Feedback occurs when an output or consequence returns as an influence on later system behavior.

Negative feedback counteracts a deviation. A thermostat detects a falling temperature, turns on heating, and later turns it off as the room returns to range. Negative feedback often supports stability.

Positive feedback reinforces a change. More engagement may produce more recommendation exposure, which produces more engagement. Positive feedback may create growth, lock-in, polarization, or collapse.

The words positive and negative do not mean good and bad. They describe whether the loop amplifies or counteracts change.

Delayed feedback can cause overshoot and oscillation. Missing feedback can allow a system to optimize a proxy long after the proxy has stopped representing the intended result.

Emergence comes from organization

“The whole is greater than the sum of its parts” is a familiar systems slogan. It is suggestive but imprecise.

What the inventory omits is organization: spatial arrangement, causal interaction, timing, feedback, and constraints. Those relations allow a whole to display properties that isolated parts do not display.

• A single vehicle does not constitute a traffic jam; many mutually constraining vehicles can.
• A single market participant does not determine a market price; structured interaction among many participants may produce one.
• A single neuron does not perform the full cognitive work of a nervous system; organized neural activity supports higher-level capacities.
• A molecule does not have the thermodynamic temperature of a macroscopic body; temperature characterizes a collective state.

Systems biology likewise studies components in the context of their interactions and the constraints imposed by the whole.Stanford Encyclopedia of Philosophy: Philosophy of Systems and Synthetic Biology

Emergence should not be used as a label for mystery. An emergent property still calls for an explanation of arrangement, interaction, scale, constraint, and time.

Does every system have a goal?

No. Purpose, function, and intention must be distinguished.

An engineered payment system is built for stated purposes. An organization may pursue several partly conflicting objectives. A heart performs a biological function that can be explained through physiology and evolution without attributing intention to the organ. A solar system displays lawful behavior without pursuing a goal.

Even in designed systems, four things can diverge:

stated purpose
designer intention
metric actually optimized
outcome actually produced

A recommendation service may claim to improve user satisfaction while optimizing clicks. Higher click-through rates may coexist with lower trust, worse information quality, or compulsive use.

A system should therefore be evaluated by its operation and effects, not only by its declared purpose.

System, structure, mechanism, process, and model

These terms answer different questions.

A publishing system has a structure of repositories, builders, and servers; a mechanism that converts Markdown into HTML; a process of drafting, review, commit, build, and deployment; and a function of making content reliably accessible.

The system is the object under study. A system model is a selective representation of that object. A diagram may omit details usefully, but success in the diagram does not guarantee success in the world.

Systems, entities, and ontology

An entity analysis asks which particular object is being referred to. A system analysis asks how entities and relations are organized into behavior over time. An ontology specifies which kinds of entities and relations a domain recognizes.

entity: an identifiable object
relation: a way objects are connected
system: organized objects and relations operating over time
ontology: an account of the recognized kinds and relations

A system can itself be treated as an entity. A payment system may have an owner, version, status, service boundary, and lifecycle. At a lower level, the same system contains account, order, fraud-control, channel, and settlement subsystems.

Something can therefore be a system at one level and a component of a larger system at another. The relevant level depends on the question.

Related discussions appear in “What Is an Entity? Identity, Reference, and AI Systems” and “What Is Ontology? From What Exists to What AI Can Represent”.

Subsystems and systems of systems

Complex systems are often nested:

component
    ↓
subsystem
    ↓
system
    ↓
larger system

A payment interface may belong to an order-and-payment subsystem, which belongs to an e-commerce platform, which participates in a broader commercial and logistics environment.

A system of systems joins systems that can still operate and be managed with substantial independence. Urban mobility, for example, may involve roads, buses, rail, navigation, ticketing, traffic control, taxis, and ride-hailing platforms. No single component completely determines the whole, yet their interactions shape citywide movement.

This creates coordination problems that cannot be solved by optimizing one subsystem alone.

Are systems discovered or constructed?

Both descriptions capture part of the truth.

Real interactions, dependencies, feedback loops, and constraints are not invented merely by drawing a boundary. Traffic congestion, ecological exchange, institutional authority, and software calls have real effects.

Yet the choice of boundary, scale, state variables, and level of abstraction depends on what the investigator needs to explain. The same person can participate in biological, family, legal, organizational, economic, and information systems.

A useful position is:

Interactions are constrained by reality; system boundaries and levels are selected for inquiry and action.

The model must remain answerable to observed behavior. A convenient boundary that excludes decisive effects is a bad boundary.

System boundaries also carry values and power

Boundary choices determine what becomes visible.

When a system defines who counts as a user, which outcomes count as benefits, which harms count as externalities, which metric receives optimization pressure, and who may change the rules, it embeds practical and political judgments.

A delivery platform that measures only completed orders per hour may improve its internal efficiency while moving safety risk, waiting pressure, and road danger onto workers and the public.

The system did not eliminate the cost. Its measurement boundary excluded the cost.

Systems analysis therefore asks more than whether an operation is efficient:

• Efficient for whom?
• Which outcomes are measured?
• Who absorbs failure and delay?
• Who has authority to change the objective or boundary?
• Which effects appear only in a larger system?

What is an AI system?

An AI system is not synonymous with an AI model.

A deployed language-model application may include:

users and operators
+ interface
+ prompts and context
+ one or more models
+ retrieval and knowledge sources
+ memory and state
+ tools and external services
+ orchestration and control flow
+ identity and permissions
+ runtime infrastructure
+ logging and evaluation
+ human review
+ outcome feedback

The model performs part of the inference. The surrounding system decides what reaches the model, which external state is available, which tools may run, whose authority applies, how outputs are checked, and what changes in the world.

The OECD definition describes an AI system as a machine-based system that infers from inputs how to produce predictions, content, recommendations, or decisions that can influence physical or virtual environments. Its explanatory material describes a model as a core component of such a system.OECD: Updated Definition of an AI System

NIST emphasizes that AI systems are sociotechnical: their benefits and risks emerge from technical components together with operators, users, other systems, and the social context of deployment.NIST AI Risk Management Framework

A model can answer while the system still fails

A language model may generate a correct article draft. Publishing that article reliably requires a larger system to:

• resolve the intended article and language versions;
• read repository rules;
• modify the correct files;
• preserve identity and authorization boundaries;
• validate the build;
• commit and push with the correct repository identity;
• wait for deployment;
• verify the live pages, canonical URLs, language links, and sitemap.

Failure can occur at any boundary. The model may be correct while retrieval supplies stale facts. The plan may be correct while a tool targets the wrong entity. The code may build locally while deployment fails. The page may be live while search metadata is missing.

Therefore:

Model capability does not by itself establish system reliability.

Evaluation must match the system boundary. A model benchmark measures a model under specified conditions. It does not automatically measure the complete product, workflow, organization, or real-world outcome.

A practical method for analyzing a system

1. State the question. Are you trying to explain, design, improve, control, or evaluate the system?
2. Draw the boundary. What belongs inside, what belongs to the environment, and why?
3. Identify elements. Include people, software, physical objects, rules, institutions, data, and resources.
4. Map relations and flows. Track information, material, money, authority, and dependency.
5. Describe states and transitions. What states can occur, and what changes one state into another?
6. Find feedback loops. Which consequences reinforce or counteract earlier behavior?
7. Examine delays. When do outputs and consequences become observable?
8. Separate goals from metrics. What outcome is intended, and what variable actually receives optimization pressure?
9. Locate constraints and authority. Who can change which element, rule, or boundary?
10. Inspect excluded effects. Which people, costs, and risks appear only when the boundary expands?

A final definition

A system can be defined as:

A system is a bounded whole in which interdependent elements operate through some organization and rules, change state over time, interact with an environment, and thereby produce behavior or capacities at the level of the whole.

For designed systems, that account must also include purpose, constraints, metrics, authority, and responsibility.

A system is not merely many things placed together, and it is not merely an architecture diagram. It exists as organized interaction: changes in one part affect others, relations alter outcomes, and the condition of the whole constrains what its parts can do.

Identifying entities tells us what is present. Understanding a system tells us how those entities operate together and why the observed result occurs.

References

• Etymonline: system
• ISO/IEC/IEEE 42020:2019
• INCOSE: Systems Thinking 101
• Stanford Encyclopedia of Philosophy: Philosophy of Systems and Synthetic Biology
• OECD: Updated Definition of an AI System
• NIST AI Risk Management Framework

The word is broad because it does not describe a particular material composition. It describes a role within thought, language, or a model:

An entity is something treated as having a distinguishable identity, so that it can be referred to, described, related to other things, tracked over time, or acted upon.

An entity need not be tangible. It need not exist independently. It need not even exist in the actual world. What it needs, within a particular domain, is enough identity for the system to treat it as the same subject across multiple statements or operations.

Where does the word “entity” come from?

English entity comes through Medieval Latin entitas, formed from ens, a being or something that is, which in turn is related to the Latin verb esse, to be. The word therefore carries an ontological background: it concerns something considered as a being or item of existence.American Heritage Dictionary: entity

That history explains why entity can be used so widely. It may refer to a physical object, a person, an institution, an event, a number, a proposition, or another item admitted by a theory. In philosophy, thing, being, entity, and object may all compete for the role of a maximally general term for whatever a system acknowledges.Stanford Encyclopedia of Philosophy: Object

No single list of entities is philosophically neutral. A physicalist ontology, a mathematical ontology, a legal ontology, and a fictional world may recognize different kinds of things. Calling something an entity is therefore both a semantic move and, in many contexts, an ontological commitment.

An entity is not merely a physical object

A physical object usually has material structure and some spatial boundary. An entity need not.

A corporation has no single body. It depends on law, records, roles, property, contracts, and continued institutional recognition. A meeting is not a durable object, but it can have participants, a time, a location, an agenda, and an outcome. An account exists only within a platform and its rules, yet the platform must still distinguish one account from another.

All three can function as entities because each can be identified and become the subject of further claims.

The English words entity and substance should also be separated. An entity is any item treated as a being or object of reference. A substance, in major philosophical traditions, is more specifically something taken to exist relatively independently or to bear properties. Events, relations, numbers, and properties may count as entities without counting as substances in that stronger sense.Stanford Encyclopedia of Philosophy: Substance

Reference does not establish real existence

Sherlock Holmes can be named, described, compared with other characters, and placed in a network of fictional relations. He is an entity in literary discourse and may be an entity in a knowledge base. None of this makes him a historical person.

At least three questions must therefore remain separate:

An entity can exist at the first two levels without satisfying the third. Plans, hypothetical products, possible events, mistaken identities, and fictional characters all demonstrate why representation and reality must not be collapsed.

Identity is the central problem

Finding a noun is easy. Determining what makes something the same entity is harder.

A person may change names and addresses while remaining the same person. A corporation may replace every employee and continue as the same legal organization. An article may be revised many times while retaining one publication history. A product may keep its commercial name while its capabilities, components, or terms change substantially.

Different types of entities require different identity criteria:

A name is not an identity. Two people can share a name, and one person can use several names. A set of properties is not automatically an identity either. Properties change, records conflict, and two objects may resemble one another closely without being the same object.

A useful entity representation usually needs more than a label:

stable identifier
+ entity type
+ attributes
+ relationships
+ temporal state
+ provenance
+ confidence in the identity match

How does ontology relate to entities?

An entity does not become a useful modeling unit in isolation. A system must decide what kinds of entities it recognizes, which attributes they may have, which relations may connect them, and what counts as persistence or change.

Those decisions form part of an ontology.

An entity is a particular item recognized within a domain; an ontology states which kinds of items the domain recognizes and how they may be organized.

“Shanghai” may be stored as a string in a simple customer table. In a geographical knowledge system, it may be represented as an entity with coordinates, administrative status, districts, and relationships to other places. The difference depends on whether the system needs to identify Shanghai independently and reason about it.

Philosophical ontology asks what exists and how different kinds of beings exist. Computational ontology turns a domain commitment into an explicit model of classes, entities, properties, relations, and constraints. A fuller account appears in “What Is Ontology? From What Exists to Models AI Can Use”. Here ontology matters because it supplies the type system and identity conditions within which entities can be distinguished.

What does “entity” mean in AI?

In AI, an entity is usually an object distinguished from text, images, records, or an environment because the system needs to understand, retrieve, remember, reason about, or act on it.

The term changes meaning across tasks. Named-entity recognition, entity linking, entity resolution, knowledge graphs, computer vision, databases, and AI agents do not operate at exactly the same level.

Named-entity recognition finds mentions, not verified objects

Consider the sentence:

Apple plans to announce a new phone in Shanghai tomorrow.

A named-entity recognition system may label:

• Apple as an organization;
• Shanghai as a location;
• tomorrow as a date.

At this stage, the system has identified mentions: spans of language that appear to refer to named or otherwise categorized entities. In engineering terms, a conventional NER component predicts labeled token spans. spaCy, for example, describes its entity recognizer as identifying non-overlapping labeled spans.spaCy: EntityRecognizer

The output does not yet prove that Apple refers to Apple Inc. rather than a different organization, a title, or an annotation mistake. Nor does it establish that the announced phone is a particular product with a known identity.

The distinction is fundamental:

entity mention ≠ entity identity
entity name ≠ unique referent
entity type ≠ proof of existence

Entity linking connects a mention to a canonical entity

Entity linking attempts to determine which known entity a mention refers to.

“Apple” in a sentence
        ↓ disambiguation
Apple Inc.
        ↓ normalization
knowledge-base ID: company/apple-inc

This requires at least two forms of reasoning:

• Disambiguation: Which entity with this name fits the context?
• Coreference and normalization: Do Apple, Apple Inc., and the Cupertino company refer to the same entity here?

Linking converts a piece of language into an addressable object in a knowledge system. It is still fallible. The selected knowledge-base entry may be wrong, duplicated, incomplete, or out of date.

Entity resolution asks whether records describe the same thing

Entity linking usually begins with language and a knowledge base. Entity resolution often begins with records.

A customer system may contain:

Li Ming, Shanghai, 138****1234
Ming Li, 上海市, 138****1234
李明, Pudong, 138****1234

Do these records describe one person, two people, or three? Shared fields provide evidence, but they do not make the answer automatic. Phone numbers can be reassigned, addresses can be shared, and names can collide.

Entity resolution may involve:

• deduplicating records;
• matching aliases and transliterations;
• detecting that one entity has split or merged in a source system;
• preserving conflicting claims rather than forcing a premature merge;
• recording why two records were considered the same.

This is an identity decision under uncertainty. A useful system keeps the evidence and confidence behind the match instead of treating similarity as certainty.

Knowledge graphs place entities in a network of claims

In a knowledge graph, entities are commonly represented as nodes connected by typed relations:

Apple Inc. ──headquartered in──> Cupertino
Apple Inc. ──released──> iPhone
iPhone ──instance of──> smartphone product

This representation distinguishes:

• entities such as Apple Inc., Cupertino, and an iPhone model;
• classes such as company, city, and product;
• attributes such as dates and names;
• relations such as headquartered in and released.

RDF represents claims as subject–predicate–object triples. Its notion of a resource is deliberately broad: a resource may denote a physical thing, a document, an abstract concept, or another item in the universe of discourse.W3C: RDF 1.1 Concepts and Abstract Syntax

An ontology supplies general rules, while a knowledge graph contains particular claims:

ontology: a company may release a product
claim: Apple Inc. released a particular iPhone model

A graph node is not automatically a verified real-world object. It may represent a class, a fictional entity, a planned object, an uncertain hypothesis, or an erroneous record. Provenance and status remain necessary.

Concepts, classes, entities, identifiers, and records

Several layers are easily confused:

Company may be a concept and a class. Apple Inc. may be an entity. Apple may be a name or mention. company_001 may be an internal identifier. A row in a database may contain claims about the company. None of these layers is identical to the organization itself.

One entity may have many names and records. One record may accidentally combine facts about several entities. A unique database key guarantees uniqueness inside a table; it does not prove that the row corresponds correctly to one real-world thing.

When should something be modeled as an entity?

Not every noun phrase needs its own entity. Six questions help:

1. Does it need a distinct identity? Must the system distinguish this item from similar items?
2. Will it be referred to repeatedly? Will multiple documents, records, or tasks mention it?
3. Does it have its own attributes? Must the system store a status, date, location, or owner?
4. Does it participate in relationships? Must it be connected to other objects?
5. Does it persist through time? Must the system recognize it after some properties change?
6. Can the system act on it? Will it be queried, edited, sent, authorized, purchased, deleted, or monitored?

If most answers are yes, an entity is often appropriate. Otherwise a literal value, label, or temporary span may be enough.

Entity modeling is not free. Every entity type introduces identity rules, lifecycle questions, merge and split behavior, provenance requirements, and access-control consequences.

Can events, states, and intentions become entities?

Grammatical categories do not map directly onto model categories. Nouns do not always denote entities, and verbs do not always remain mere relations.

“Maya signed Contract C36” can be represented as a simple relation:

Maya ──signed──> Contract C36

If the system must record the date, location, version, witnesses, method, and legal status, the signing can become an event entity:

Signing Event E1024
├── signer: Maya
├── object: Contract C36
├── time: 2026-09-24
├── place: Shanghai
└── status: effective

An intention can likewise be modeled as a mental-state entity when the system needs to record whose intention it is, what outcome it concerns, when it was inferred, which evidence supports the inference, and how uncertain the interpretation remains.

Reification makes an event, relation, or state available for further description. It is useful when the system needs that detail, but it should not be mistaken for a discovery that the world naturally divides itself in exactly that form.

How do large language models handle entities?

A standard large language model receives tokens and computes distributed representations shaped by training data and current context. It can learn that Apple often occurs near company, product, iPhone, and Cupertino, and it can frequently disambiguate the word from the fruit.

This competence does not imply that the model contains a single, explicit, canonical Apple Inc. record comparable to a carefully maintained knowledge base. Entity information may be distributed across parameters and reconstructed probabilistically in context.

Consequently, a language model may:

• resolve an entity correctly in one context and confuse it in another;
• conflate a company, its brand, its products, and its website;
• recall an old property without knowing that it has changed;
• invent a plausible person, paper, organization, or product;
• answer without a stable source for the entity claim.

For tasks that require reliable action, model output is usually only one layer:

language model
+ domain ontology
+ entity IDs and resolution
+ current state and provenance
+ time-aware records
+ permissions and action rules

The language model interprets open-ended language. The ontology defines possible types and relations. Entity services determine identity. Data sources establish current state. Authorization determines which operations are permitted.

Why do AI agents need explicit entity identity?

Consider the instruction:

Update yesterday’s article about needs on Moonment.

The request contains several unresolved references:

The agent may understand the general intention while still acting on the wrong file, project, account, language version, or deployment.

A safer path is:

natural-language request
        ↓
mention and reference detection
        ↓
type assignment and coreference
        ↓
entity linking and resolution
        ↓
relation, event, and intent interpretation
        ↓
state, provenance, and authorization checks
        ↓
action on the resolved entity

Intent specifies the change the user appears to seek. Entity resolution determines what the intended action applies to. Authorization determines whether the action may be performed. None can substitute for the others.

Can AI establish that an entity really exists?

Language alone cannot establish existence.

An AI system can estimate that a phrase is probably a person’s name, that context probably indicates a company, or that a mention probably links to a known record. Real-world verification requires additional evidence, such as:

• an authoritative registry or primary source;
• a current database record;
• a file that actually exists in the relevant filesystem;
• a live website or API;
• an authenticated identity and permission system;
• a sensor observation or human confirmation.

The following claims are distinct:

the text mentions something
≠ the model identified it correctly
≠ the system linked the correct record
≠ the record is accurate and current
≠ the corresponding thing exists now
≠ the user is authorized to act on it

Model confidence is not proof of existence. It describes a model’s judgment under particular inputs and assumptions. It does not replace provenance or verification.

A checklist for entity design in AI systems

1. Which entity types does the system recognize, and why?
2. Are mentions, names, classes, entities, identifiers, and records kept separate?
3. What establishes identity for each entity type?
4. How are aliases, namesakes, renaming, and duplicate records handled?
5. Do attributes and relations carry time and provenance?
6. Are events and changing states being flattened into misleading static fields?
7. Are extraction, linking, resolution, and real-world verification separate stages?
8. Can the system distinguish fictional, planned, hypothetical, and actual entities?
9. What happens to historical claims when entities merge, split, or change type?
10. Before acting, how does the system verify identity, current state, and permission?

An entity-rich system can still be unreliable. Without identity criteria, provenance, temporal state, and authorization, it merely attaches confident-looking labels to uncertain referents.

Entities connect language to action

Entities form an interface between language, knowledge, and operations:

objects, events, and states in a domain
                ↓
names and descriptions in language
                ↓
entity, attribute, and relation recognition
                ↓
links to records and external evidence
                ↓
interpretation of needs and intentions
                ↓
authorized action and recorded outcomes

An entity answers, “Which particular thing are we talking about?” An attribute answers, “What is it like?” A relation answers, “How is it connected?” An event answers, “What happened?” An intention answers, “What outcome is an agent trying to bring about?”

The most serious entity error in AI is not failing to label a noun. It is treating an ambiguous expression as though it already named one verified, current, and actionable object.

References

• American Heritage Dictionary: entity
• Stanford Encyclopedia of Philosophy: Object
• Stanford Encyclopedia of Philosophy: Substance
• Stanford Encyclopedia of Philosophy: Logic and Ontology
• spaCy: EntityRecognizer
• spaCy: Linguistic Features—Named Entity Recognition
• W3C: RDF 1.1 Concepts and Abstract Syntax
• W3C: OWL 2 Web Ontology Language Primer

Series: Thinking, Intention, and Action (2/4). Previous: Human Thinking; next: User Intent in AI

What Does It Mean to Say That AI “Thinks”?

The claim that an AI system thinks can refer to three different questions:

1. Can it perform tasks that require reasoning, planning, comparison, and judgment?
2. Does its computation contain internal processes that deserve the functional name thinking?
3. Does it possess consciousness, subjective experience, understanding, or intentions like a person?

The first question has an empirical answer: present systems can perform many tasks that previously required human thought.

The second supports a qualified functional definition:

AI reasoning is the computational transformation of inputs, context, learned parameters, and tool observations into predictions, judgments, plans, and selected outputs.

The third does not follow from performance. Producing a proof, explaining a concept, or planning a project does not establish that a system experiences its activity or understands it in the way a person does.

The relevant distinctions are:

behavioral competence
≠
computational mechanism
≠
subjective experience

This article concerns the first two.

The Base Operation of a Language Model

A language model is trained to predict a token from the tokens that precede it:

P(next token | current context, model parameters)

Training adjusts a large collection of parameters so that the model becomes sensitive to statistical structure across words, syntax, genres, factual statements, arguments, programs, and patterns of explanation.

At generation time, the simplified cycle is:

read the current context
→ score possible next tokens
→ select one token
→ append it to the context
→ repeat

Calling this “next-token prediction” is accurate but incomplete as an explanation of capability. Predicting the continuation of a proof, a program, or a multistep plan can require internal representations that track relations extending far beyond the next word.

The open scientific question is how stable and general those representations are. A model may exhibit a usable concept in one setting, fail after a small reformulation, or rely on a shortcut that worked in the training distribution.

Why Prediction Can Produce Reasoning

Human language contains the products of reasoning and many traces of its process: definitions, proofs, disagreements, plans, diagnoses, corrections, and counterexamples. Learning to predict this material exposes a model to recurring structures such as:

• relevant versus irrelevant evidence;
• premises and conclusions;
• causes and effects;
• goals, constraints, and plans;
• programs and execution traces;
• claims and objections;
• errors and revisions.

Large models can consequently perform deduction, induction, analogy, causal explanation, program simulation, and task decomposition to useful degrees.

These abilities remain uneven. Fluent language can hide an invalid inference. Long dependency chains can fail. A familiar template can produce the right answer without a general method, while a slightly unfamiliar case defeats the same model.

It is therefore unsafe to infer reliable reasoning merely from the presence of reasoning-shaped prose.

What Happens Between Prompt and Output?

Input is divided into tokens and converted into vector representations. A Transformer repeatedly uses attention and nonlinear transformations to construct context-sensitive internal states. The final layers assign scores to possible next tokens.

prompt and context
→ token and position representations
→ attention across relevant positions
→ layered internal transformations
→ distribution over outputs
→ generated continuation

Those internal states are not a transcript written in ordinary language. Researchers can probe activations, attention patterns, and latent representations, but there is no simple one-to-one mapping from a single unit to a complete thought.

A model may also generate a step-by-step explanation. Such text can help decompose a problem and make an answer easier to evaluate. It should not be treated as a complete scan of the computation that caused the answer. Experiments have shown that chains of thought can omit influential cues and rationalize a result after the fact. Turpin et al., Language Models Don’t Always Say What They Think

A verbal rationale is an interface for work and evaluation, not privileged access to every causal step inside the model.

From Prediction to Instruction Following

A base model primarily learns what text is likely to follow other text. An assistant must also learn how a request should guide its behavior.

A common development pipeline includes:

• large-scale pretraining;
• supervised examples of instruction following;
• optimization from human or model feedback;
• runtime system instructions and tool protocols.

GPT-3 demonstrated broad in-context task performance from examples and instructions. InstructGPT showed that scale alone does not guarantee alignment with user requests and that instruction tuning plus human feedback can substantially redirect behavior. GPT-3 · InstructGPT

An assistant’s response is therefore produced by more than the final user sentence. It depends on learned parameters, system rules, conversation history, visible environment, tool results, and decoding choices.

Interpretation, Inference, Decision, and Action

Four stages should be kept distinct:

A model can recommend an action without executing it. A system can execute a tool call after weak reasoning. Separating the stages makes failures diagnosable.

“This file appears redundant” is a judgment. “Deleting it will recover space” is a proposed consequence. “Delete it now” is a decision. “The user authorized deletion” is a fact about permission. None of these substitutes for the others.

How a Model Becomes an Agent

A language model accepts context and emits a continuation. A persistent agent normally requires additional machinery:

• task state to record the objective and current progress;
• planning to decompose work into executable steps;
• tools for search, files, code, browsers, and services;
• memory for results, commitments, and stable conventions;
• observation of tool output and environmental state;
• permissions that determine which actions are allowed;
• feedback and termination rules that define completion or revision.

The operating loop is:

observe
→ interpret the state
→ choose a next action
→ invoke a tool
→ read the result
→ update the plan
→ continue or stop

ReAct formalized a useful version of this pattern by interleaving reasoning traces with actions and environmental observations. ReAct

The acting unit is therefore not the language model in isolation. It is the assembled system of model, tools, state, permissions, and execution environment.

Does an AI Agent Have Goals or Intentions?

Engineered systems can contain objective functions, reward signals, task descriptions, and stopping criteria. These should not be collapsed into human desire or practical intention.

When a model writes, “I will inspect the files first,” the sentence can function as a report of the next operation. Its first-person grammar does not establish a private human-like intention.

This is why an agent can display sustained goal-directed behavior while still requiring external authorization, supervision, and an accountable human or institution.

Action Requires Feedback

A plan produced once cannot guarantee contact with reality. Tools fail, pages change, files disappear, and new evidence defeats earlier assumptions.

Reliable action therefore has a closed loop:

form a hypothesis
→ act
→ observe the result
→ compare actual and expected state
→ revise the interpretation or plan
→ act again

Without observation, action remains a description inside language. With observation, the system can test whether external state actually changed.

The evidence must also be labeled correctly:

• generating a command is not executing it;
• passing a build is not visual acceptance;
• pushing a repository is not proof of deployment;
• silence from a user is not authorization for a consequential action.

Characteristic Failure Modes

Fluency conceals weak evidence

A polished answer and a well-supported answer are different achievements.

Context is partial

The model can use only the files, messages, tool outputs, and environmental state made available to it. An omitted fact can reverse the correct decision.

Long tasks lose state

Extended work needs checkpoints, external records, and explicit completion criteria. Otherwise a system may repeat steps, omit requirements, or report a plan as a result.

Tool output still needs interpretation

Search results, webpages, logs, and documents can be incomplete, stale, mistaken, or adversarial. Retrieval changes the evidence set; it does not guarantee truth.

Objectives conflict

User requests, system rules, physical constraints, and local subgoals may point in different directions. Reliable behavior requires detecting and resolving conflict rather than treating every instruction-like string as authoritative.

Consequences belong to the full system

A model selects a call, an executor changes external state, a platform grants access, and people or institutions assign responsibility. Evaluating only the generated text misses most of the action chain.

How to Verify That an AI Completed a Task

Confidence should come from evidence at each layer:

Did it identify the correct object?
→ Did it obtain sufficient evidence?
→ Does the inference support the conclusion?
→ Was the action authorized?
→ Did the tool actually run?
→ Did external state change as intended?
→ Does the outcome satisfy the original objective?

For consequential or extended tasks, the system should also preserve recoverable intermediate states so that mistakes can be inspected and reversed.

Conclusion

AI reasoning can be described functionally as the transformation of input, context, and observations into judgments, plans, and selected outputs. This creates real functional comparisons with human thinking, but it does not establish identical consciousness or experience.

AI action belongs to a larger architecture:

model
+ context
+ tools
+ state
+ permissions
+ environmental feedback

The central questions are therefore not limited to what answer the model generated. They include what evidence it used, how it checked a plan, who authorized execution, what the tools changed, how the result was verified, and how errors update the next cycle.

AI reasoning computes candidate courses of action. AI agency begins when those computations are connected to authorized tools, observable consequences, and correction through feedback.

Sources

• Brown et al., Language Models are Few-Shot Learners
• Ouyang et al., Training Language Models to Follow Instructions with Human Feedback
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models
• Turpin et al., Language Models Don’t Always Say What They Think

Series: Thinking, Intention, and Action (4/4). Start with How Human Thinking Works

Collaboration Is More Than Prompt and Response

Human–AI interaction is often pictured as:

human writes a prompt
→ AI returns an answer

That picture captures a message exchange. It leaves out why the request arose, how the system selected an interpretation, who authorized an external action, and how the result changes the next decision.

A fuller model is:

human situation, need, and purpose
→ provisional intention
→ linguistic request
→ AI task model
→ calibration of goals and boundaries
→ authorized plan and action
→ observation of consequences
→ human evaluation and system revision
→ next cycle

Effective human–AI collaboration is a continuing process of alignment, action, observation, and correction around a shared task.

Intentions Are Not Fully Formed Before Language

A person does not always begin with a complete objective waiting to be encoded into a perfect prompt. Intentions often become clearer through articulation, comparison, and trial.

A request can contain several levels:

Only part of this structure is usually explicit. The rest may live in earlier turns, the active document, established conventions, institutional rules, or judgments the person has not yet made.

Longer prompts can supply more evidence. They cannot eliminate the underlying problem. Length can also add contradiction, noise, and false precision.

The System Models a Task, Not a Whole Person

An AI system has access to evidence such as:

• the current wording;
• conversation history;
• visible files and interfaces;
• tool observations;
• stored preferences and rules;
• the user’s acceptance or correction of intermediate results.

From these signals it constructs a working task model:

What is the object?
What change is requested?
What constraints apply?
Which facts remain unknown?
Which actions are authorized?
What observable state counts as completion?

That model can become highly accurate without becoming a complete representation of the user’s mind. Restricting claims to available evidence prevents the system from presenting speculation about a person as fact.

Three Kinds of Understanding

Semantic understanding

Can the system resolve the language? For example, what does “use the first one” refer to in the preceding exchange?

Operational understanding

Can it turn the language into a concrete task? Which article, language, format, repository, and action are involved?

Outcome understanding

Does it know what state would satisfy the purpose? Is a generated file enough, or must the work be committed, deployed, and verified at a public URL?

These levels can separate:

understanding the sentence
≠
knowing what operation to perform
≠
knowing what completion looks like

Many failures arise from disagreement about objects, permissions, or completion evidence even when the words were parsed correctly.

Building a Shared Task Model

Human and system gradually establish a shared, revisable representation of the task. It normally includes:

This does not require the person to specify everything at once. A system can use established context and produce reversible work while seeking information only where it changes the result or the permission boundary.

Good collaboration retains decisions already made. It also remains open to revision when new evidence conflicts with an earlier interpretation.

Clarify, Assume, or Act?

Ambiguity does not force a choice between blind guessing and endless questioning. The practical rule depends on:

degree of ambiguity × cost of error × reversibility

Low cost and reversible

Proceed with a stated assumption or a draft. The result itself can help the person clarify the target.

Moderate ambiguity

Preserve alternatives, complete the common work, or present comparable options.

High consequence or difficult to reverse

Confirm the object, scope, and authorization before publication, payment, external messaging, or irreversible deletion.

The purpose of clarification is to control consequences. It should occur where a distinction materially changes the action.

A Prompt Is Evidence, Not a Complete Contract

A prompt directly constrains the current task, but its force still depends on context and conversational commitments.

Closely related sentences license different actions:

• “Can this be published?” requests an assessment.
• “Prepare this for publication” authorizes editing.
• “Publish this on the site” authorizes an external action.
• “I may publish this later” reports a possibility.

A system must distinguish questions, proposals, background information, corrections, and authorization.

Conversation also creates durable commitments. Once the person selects an option, approves publication, or defines a format, those decisions should guide later steps. The operative instruction is distributed across the interaction, not confined to the latest sentence.

Action Tests Understanding

Restating a request does not prove that both sides share the same model. Action exposes hidden disagreement.

A person says, “Update the article on the website.” The system may create local Markdown but fail to commit it; commit without deployment; deploy one language but omit the other; publish both pages but leave discovery files stale.

Verification must therefore proceed through layers:

task interpretation is correct
→ artifact content is correct
→ action actually occurred
→ external state changed
→ the outcome satisfies the purpose

Each layer produces evidence. A failed action also reveals which assumption or completion criterion was missing.

The Joint Action Loop

The process can be described in eight stages.

1. The person encounters a problem

A need, obstacle, opportunity, or unsatisfactory result creates pressure to change the current state.

2. A provisional intention forms

The person selects a direction, although the goal, method, and standard may remain incomplete.

3. The request is externalized

Language carries part of the intention into a prompt, together with available context and constraints.

4. The AI constructs candidate interpretations

The system identifies objects, actions, constraints, missing information, and competing task models.

5. The task is calibrated

History, paraphrase, drafts, options, or a necessary question make the interpretation clear enough for the next action.

6. The AI acts within authorization

The system plans steps, invokes tools, preserves state, and checks permission at consequential boundaries.

7. Consequences are observed

Files, command results, public pages, user reactions, and other evidence show what actually happened.

8. Both sides revise

The person may change the goal, correct a mismatch, or accept the result. The system updates its task model and continues or stops.

The cycle is:

intention
→ expression
→ interpretation
→ calibration
→ action
→ observation
→ evaluation
→ revised intention

A Division of Responsibility

A joint loop does not make human and system responsibility identical.

The person contributes

• the real situation that needs to change;
• value judgments and ultimate purpose;
• private context and unspoken constraints;
• authorization for consequential actions;
• final acceptance of whether the outcome is worth having.

The AI system contributes

• a structured interpretation of available evidence;
• alternative plans and their relevant differences;
• executable steps and tool use;
• explicit uncertainty, permission, and completion states;
• rapid revision when new evidence arrives.

The platform or organization contributes

• identity and access control;
• data and privacy boundaries;
• logs, versioning, and recovery;
• failure handling and assignment of accountability;
• observable status for external actions.

The AI cannot settle every value question for a person. A person cannot explain every failure by pointing only to an isolated model. Outcomes belong to the complete sociotechnical arrangement.

Where the Loop Breaks

A feeling is mistaken for a complete objective

A person knows that an output is wrong but cannot yet specify the desired alternative. The system should help compare concrete possibilities rather than assume a unique hidden answer.

The most likely interpretation becomes “true intent”

High probability means that available evidence favors an interpretation. It does not reveal the person’s complete private purpose.

Content approval becomes action approval

Accepting an article does not automatically authorize its public release.

A plan is reported as completion

“We will update the site” is not deployment evidence. Local files, commits, deployments, and live pages are different states.

Only the output is evaluated

A correct answer can result from an unreliable method. A failed attempt can expose an important unknown. Durable collaboration evaluates results, evidence, and the capacity to correct.

Feedback does not update the next cycle

If corrections are neither retained nor applied, the same mismatch repeats and no learning loop forms.

Improving the Loop

A person can improve collaboration by:

• describing the desired change, not only naming an operation;
• distinguishing exploration, drafting, revision, approval, and publication;
• stating unacceptable outcomes at consequential points;
• locating feedback at the level that was misunderstood;
• allowing the intention itself to change after new evidence.

An AI system can improve collaboration by:

• separating explicit requirements, inferences, and unknowns;
• carrying forward confirmed context;
• using reversible artifacts to advance low-risk work;
• checking object and authorization at high-consequence boundaries;
• reporting observed results instead of plans;
• updating its task model after correction.

Conclusion

A human encounters a situation, develops needs and intentions, and expresses only part of them in language. An AI system uses visible evidence to construct a task model, reason about options, and act through tools. The consequences and the person’s evaluation then revise that model and may revise the original intention.

The central object is therefore not a perfect one-shot prompt. It is a shared loop that remains correctable:

human purpose, values, and authorization
+
AI interpretation, planning, and execution
+
environmental consequences and evidence
+
feedback that changes the next cycle

Reliable collaboration does not require an AI to read an invisible “true mind.” It requires both sides to make the current task, action boundary, and evidence of completion progressively clearer.

Further Reading

• User Intent in AI: Inference Under Uncertainty
• How AI Systems Reason and Act
• How Human Thinking Works: Representation, Reasoning, and Action
• EMNLP 2024: Making Language Models Explicitly Handle Ambiguity