The ambiguity matters because these attitudes answer different questions:

What do I think will happen?
What do I want to happen?
What is someone supposed to do?
What do I believe my action can bring about?

When these questions are compressed into one word, desire can masquerade as evidence, prediction can sound like obligation, and another person’s demand can be treated as a fact about the future.

A working definition

An expectation can be defined as:

An expectation is an attitude toward an outcome that remains unresolved for an agent, representing that outcome as likely, anticipated, required, or connected to action.

The outcome is usually future-directed, but it need not concern an event that has not yet occurred. Someone may say, “I expect the package has already arrived,” when the delivery is complete but its status remains unknown to them. The important condition is epistemic openness: the agent does not yet know the outcome.

Every expectation therefore contains at least:

an agent
+ an unresolved outcome
+ a way of representing that outcome
+ some attitude toward its occurrence

The last element determines which kind of expectation is involved.

Expectation, expectancy, hope, and anticipation

English separates several ideas that ordinary conversation often blends.

Hope has both a cognitive and a conative side. A standard philosophical account treats hope as involving a desired outcome together with belief that the outcome remains possible. A person can therefore hope for an outcome that they regard as very unlikely. That same person would not honestly say that they expect it.Stanford Encyclopedia of Philosophy: Hope

Expectation usually places more weight on belief. Hope places more weight on desirability. Anticipation adds an affective and attentional orientation toward an approaching event.

Predictive expectations

A predictive expectation represents what an agent thinks will happen.

Given the current release data, I expect adoption to grow next month.

This is an epistemic judgment. It should answer to evidence, and it should change when the evidence changes.

A predictive expectation may be precise:

There is a 70% chance that demand will exceed capacity.

It may also be qualitative:

Demand will probably remain stable.

Neither statement guarantees the outcome. Both summarize what the agent currently takes the evidence to support.

This kind of expectation should be kept separate from desire:

I want A to happen.
I currently expect B to happen.

There is no contradiction in holding both attitudes. Rational agency often begins by acknowledging that the desired outcome is not the most likely one.

Expectancy as an action–outcome belief

In psychology, expectancy often refers to a belief that an action can produce a particular outcome. The APA Dictionary describes it both as a mental set shaping how a person approaches a situation and, in motivation theory, as a belief that one’s actions can attain an outcome.APA Dictionary of Psychology: Expectancy

This introduces agency:

If I perform action A under conditions C,
how likely is outcome O?

Such beliefs influence effort and persistence. If someone believes that preparation can change performance, preparation becomes instrumentally intelligible. If they believe that nothing they do can affect the result, motivation may collapse even when the desired outcome remains valuable.

Expectancy is still not intention. Believing that an action would work does not mean that the agent has decided to perform it.

Normative expectations

An expectation can also express a standard rather than a forecast:

The organization expects employees to protect confidential information.

This sentence need not predict universal compliance. It states what employees are required or supposed to do.

Normative expectations organize families, workplaces, institutions, and social roles. They also raise questions of authority:

• Who sets the expectation?
• On what grounds?
• Who bears the cost of compliance?
• What follows from refusal?
• Can the expectation be contested or renegotiated?

The fact that a standard is widely expected does not establish that it is justified. A social regularity, a role demand, and a moral obligation are different things.

Expectations can alter outcomes

Expectations shape attention, interpretation, effort, and interaction. This can create a feedback loop:

expectation
→ attention and behavior
→ changes in the situation or in other people
→ observed outcome
→ reinforcement or revision of the expectation

If someone expects to fail, they may prepare less, withdraw earlier, or interpret ambiguous feedback as confirmation. The resulting behavior can make failure more likely. In social interaction, a perceiver’s expectation may elicit behavior from another person that appears to confirm the original belief, a process described as behavioral confirmation.APA Dictionary of Psychology: Behavioral Confirmation

This does not mean that thought directly controls reality. Expectations influence outcomes only through causal pathways such as attention, effort, communication, coordination, and the reactions of other people. External constraints and chance remain real.

Expectation is not intention

Expectation belongs to a larger chain of agency:

“I expect the article to reach more readers” expresses a belief or hope about an outcome. “I intend to revise the title and opening” introduces an action commitment. “I will revise the title today” is a more specific decision.

The full loop is:

expectation
→ intention
→ decision
→ action
→ outcome
→ revised expectation

The arrows are not automatic. An expectation may never become an intention. An intention may not survive execution. Action does not control every cause of the final result.

Expectation gaps and reference points

Expectations also become standards against which outcomes are experienced.

expectation gap = observed outcome − reference expectation

The same outcome can feel favorable when it exceeds the reference point and disappointing when it falls below it. The outcome has not changed; the comparison has.

Managing expectations should therefore not mean adopting permanent pessimism. A better practice is to separate four questions:

1. Desired outcome: What would I like to happen?
2. Predictive judgment: What does the current evidence make likely?
3. Controllable action: Which conditions can I influence?
4. Risk boundary: What can I tolerate if the desired result does not occur?

This preserves ambition without converting desire into prediction.

How to examine an expectation

When a person or institution says, “I expect…,” ask:

1. Who holds the expectation?
2. What outcome remains unresolved?
3. Is this a prediction, a hope, a standard, or an action–outcome belief?
4. If it is predictive, what evidence supports it?
5. If it is normative, who has authority and who bears the cost?
6. Which causal conditions can the agent influence?
7. Has the expectation become an intention, decision, or plan?
8. What observation would require revision?

A final definition

Expectation is an agent’s orientation toward an unresolved outcome: a representation of what is likely, desired, normal, required, or achievable through action.

The concept spans three domains that must remain distinguishable:

epistemic: what is likely to happen
evaluative: what would be good to happen
normative: what is supposed to happen

Once these are separated, expectations can be tested as beliefs, discussed as values, challenged as standards, and converted into action without being mistaken for guarantees.

References

• Stanford Encyclopedia of Philosophy: Hope
• APA Dictionary of Psychology: Expectancy
• APA Dictionary of Psychology: Behavioral Confirmation

People perceive problems, form beliefs, imagine futures, and develop preferences. None of these activities by itself selects a course of action. A decision occurs when an agent resolves enough of the open possibilities for one direction to guide what happens next.

Decision-making is the process through which an agent responds to a practical situation by comparing possible actions, uncertain consequences, values, and constraints, then commits to a course of action.

The central transition is:

several live possibilities
→ judgment and trade-off
→ one option gains practical priority
→ planning and action

Commitment here is revisable. It means that the agent stops treating every possibility as equally open and begins allocating time, attention, authority, and resources. New evidence can reopen the decision.

Decision, Judgment, Choice, and Intention

These terms often describe different parts of one episode.

“The project is likely to succeed” is a judgment. “Given its upside and our loss limit, we will fund the pilot” is a decision. The first does not entail the second. Action also requires values, constraints, alternatives, and an account of who bears the risk.

The Structure of a Decision

A practical situation

Why does a response seem necessary now? A decision problem begins with a conflict, opportunity, obstacle, or fork that matters to an agent.

A frame

“Should we continue the project?”, “How can we reduce its failure risk?”, and “Which objective should we preserve?” frame the same situation differently. A frame determines which options and evidence become visible. Precise analysis cannot rescue the wrong problem.

An agent and authority

Who can make the selection effective? Who advises, who can veto, and who bears the consequences? Collective deliberation does not produce an operative decision unless an institution also defines authority and responsibility.

Ends, values, and constraints

A goal specifies a desired state. Values explain why it matters. Constraints mark unacceptable means, risks, costs, or side effects.

Many hard decisions persist because several goods cannot be fully realized together. Revenue, safety, autonomy, fairness, speed, and loyalty may resist a common scale.

Feasible options

Success and failure are outcomes, not actions. Continue, stop, reduce scope, negotiate, run a pilot, or wait for information can be genuine options. Doing nothing and retaining the status quo also have consequences and should not disappear from the comparison.

Consequences, causation, and uncertainty

A decision requires a view about what each action might change. This is a causal question, not merely an association. It also requires some representation of uncertainty, whether statistical, model-based, judgmental, or explicitly unknown.

Probability says how plausible an outcome is. It does not say how desirable, fair, or acceptable that outcome would be.

A decision rule

An agent might maximize expected value, limit ruin, protect a non-substitutable value, choose a robust option, preserve reversibility, or stop searching when an option clears an aspiration level. Different rules can select different actions. The rule itself therefore needs justification.

Commitment, execution, and feedback

A decision must become a plan, allocation, instruction, or action. Observation then changes the next decision:

decide
→ execute
→ observe
→ compare expected and actual states
→ revise beliefs, ends, or rules
→ decide again

Three Questions for Decision Theory

Decision research separates three projects.

Descriptive

How do people actually decide? Psychology studies the effects of attention, memory, emotion, framing, defaults, social influence, and heuristics. The APA defines decision-making as the cognitive process of choosing between two or more alternatives. APA Dictionary of Psychology

A recurring behavior does not become rational merely because it is common.

Normative

How should coherent or rational choice be structured? Expected utility theory supplies one influential answer for choice under uncertainty:

EU(A) = Σ P(Oᵢ | A) × U(Oᵢ)

It compares acts by weighting the utility of possible outcomes by their probabilities. Contemporary utility is often a representation of preference rather than a direct unit of money or happiness. Stanford Encyclopedia of Philosophy: Expected Utility

The formula does not generate the option set, validate causal assumptions, settle moral constraints, or identify whose preferences should count.

Prescriptive

How can a real person or organization improve a particular decision? Prescriptive work translates evidence and standards into usable practices:

• separate facts, estimates, values, and unknowns;
• search for options suppressed by the initial frame;
• use outside comparison classes;
• specify stop, exit, and review conditions;
• buy information only when it can change action;
• test consequential assumptions through reversible steps;
• record what was known before outcomes became visible.

Bounded Rationality

No real agent has unlimited time, information, attention, or computation. Bounded rationality studies procedures that remain effective under those constraints. It does not simply label people irrational.

Herbert Simon’s idea of satisficing replaces exhaustive optimization with a search process and an aspiration level: stop when an option is good enough relative to the costs of continuing. Stanford Encyclopedia of Philosophy: Bounded Rationality

The rational question can therefore be:

Will the expected value of more information
exceed the cost of search, delay, and lost opportunity?

Indefinite optimization can itself be a poor decision.

Behavioral Regularities Are Not Merely Noise

Choices under risk depend on reference points, perceived gains and losses, presentation, and nonlinear sensitivity to probability. Prospect theory was developed to explain important patterns that standard economic models did not predict well. The 2002 Nobel Prize materials describe Daniel Kahneman’s contribution as integrating psychological research on judgment and decision-making under uncertainty into economics. Nobel Prize 2002

Calling a pattern a bias still requires a defensible benchmark. A shortcut can be poor under one environment and efficient under another once information and computation costs are included.

Why Decision-Making Is Not Only Calculation

Facts constrain action but do not specify what should matter. A probability distribution cannot decide which losses are acceptable. A utility score can clarify a trade-off while concealing rights, identity, loyalty, or values that the agent refuses to exchange.

The presence of several nominal options also does not guarantee meaningful agency. Poverty, power, addiction, information control, and institutional defaults alter the feasible set and the conditions of responsibility.

Collective decisions add procedural values. Who had standing, access to evidence, voice, and veto power can matter independently of whether the final outcome was efficient.

A Good Decision Can Have a Bad Outcome

Four evaluations should remain separate:

A low-probability event can defeat a sound decision. Luck can rescue a careless one. Evaluating the original decision by information learned only afterward produces hindsight and outcome bias.

A Practical Audit

1. What practical question actually requires resolution?
2. Has the initial frame excluded a better question?
3. Are inaction, delay, negotiation, or a reversible test real options?
4. Which statements are facts, causal assumptions, probabilities, values, or unknowns?
5. What mechanism connects each action to its expected effects?
6. What is being optimized, protected, or deliberately surrendered?
7. Can the worst plausible loss be borne?
8. Who decides, benefits, and bears risk?
9. Has the commitment entered plans, resources, and action?
10. Which new evidence would reopen the decision?

Decision-making is the joint between understanding and action. It closes some possibilities so that agency can proceed, while preserving the capacity to learn from consequences.

A good decision does not guarantee a good result. It is a defensible, executable, accountable, and revisable commitment made from the evidence, values, and constraints available at the time.

Further Reading

• Mental Models: How We Represent, Predict, and Act
• How Human Thinking Works: Representation, Reasoning, and Action

Sources

• APA Dictionary of Psychology: Decision Making
• Stanford Encyclopedia of Philosophy: Normative Theories of Rational Choice—Expected Utility
• Stanford Encyclopedia of Philosophy: Descriptive Decision Theory
• Stanford Encyclopedia of Philosophy: Bounded Rationality
• Nobel Prize 2002: Psychological and Experimental Economics

Series: Thinking, Intention, and Action (2/4). Previous: Human Thinking; next: User Intent in AI

What Does It Mean to Say That AI “Thinks”?

The claim that an AI system thinks can refer to three different questions:

1. Can it perform tasks that require reasoning, planning, comparison, and judgment?
2. Does its computation contain internal processes that deserve the functional name thinking?
3. Does it possess consciousness, subjective experience, understanding, or intentions like a person?

The first question has an empirical answer: present systems can perform many tasks that previously required human thought.

The second supports a qualified functional definition:

AI reasoning is the computational transformation of inputs, context, learned parameters, and tool observations into predictions, judgments, plans, and selected outputs.

The third does not follow from performance. Producing a proof, explaining a concept, or planning a project does not establish that a system experiences its activity or understands it in the way a person does.

The relevant distinctions are:

behavioral competence
≠
computational mechanism
≠
subjective experience

This article concerns the first two.

The Base Operation of a Language Model

A language model is trained to predict a token from the tokens that precede it:

P(next token | current context, model parameters)

Training adjusts a large collection of parameters so that the model becomes sensitive to statistical structure across words, syntax, genres, factual statements, arguments, programs, and patterns of explanation.

At generation time, the simplified cycle is:

read the current context
→ score possible next tokens
→ select one token
→ append it to the context
→ repeat

Calling this “next-token prediction” is accurate but incomplete as an explanation of capability. Predicting the continuation of a proof, a program, or a multistep plan can require internal representations that track relations extending far beyond the next word.

The open scientific question is how stable and general those representations are. A model may exhibit a usable concept in one setting, fail after a small reformulation, or rely on a shortcut that worked in the training distribution.

Why Prediction Can Produce Reasoning

Human language contains the products of reasoning and many traces of its process: definitions, proofs, disagreements, plans, diagnoses, corrections, and counterexamples. Learning to predict this material exposes a model to recurring structures such as:

• relevant versus irrelevant evidence;
• premises and conclusions;
• causes and effects;
• goals, constraints, and plans;
• programs and execution traces;
• claims and objections;
• errors and revisions.

Large models can consequently perform deduction, induction, analogy, causal explanation, program simulation, and task decomposition to useful degrees.

These abilities remain uneven. Fluent language can hide an invalid inference. Long dependency chains can fail. A familiar template can produce the right answer without a general method, while a slightly unfamiliar case defeats the same model.

It is therefore unsafe to infer reliable reasoning merely from the presence of reasoning-shaped prose.

What Happens Between Prompt and Output?

Input is divided into tokens and converted into vector representations. A Transformer repeatedly uses attention and nonlinear transformations to construct context-sensitive internal states. The final layers assign scores to possible next tokens.

prompt and context
→ token and position representations
→ attention across relevant positions
→ layered internal transformations
→ distribution over outputs
→ generated continuation

Those internal states are not a transcript written in ordinary language. Researchers can probe activations, attention patterns, and latent representations, but there is no simple one-to-one mapping from a single unit to a complete thought.

A model may also generate a step-by-step explanation. Such text can help decompose a problem and make an answer easier to evaluate. It should not be treated as a complete scan of the computation that caused the answer. Experiments have shown that chains of thought can omit influential cues and rationalize a result after the fact. Turpin et al., Language Models Don’t Always Say What They Think

A verbal rationale is an interface for work and evaluation, not privileged access to every causal step inside the model.

From Prediction to Instruction Following

A base model primarily learns what text is likely to follow other text. An assistant must also learn how a request should guide its behavior.

A common development pipeline includes:

• large-scale pretraining;
• supervised examples of instruction following;
• optimization from human or model feedback;
• runtime system instructions and tool protocols.

GPT-3 demonstrated broad in-context task performance from examples and instructions. InstructGPT showed that scale alone does not guarantee alignment with user requests and that instruction tuning plus human feedback can substantially redirect behavior. GPT-3 · InstructGPT

An assistant’s response is therefore produced by more than the final user sentence. It depends on learned parameters, system rules, conversation history, visible environment, tool results, and decoding choices.

Interpretation, Inference, Decision, and Action

Four stages should be kept distinct:

A model can recommend an action without executing it. A system can execute a tool call after weak reasoning. Separating the stages makes failures diagnosable.

“This file appears redundant” is a judgment. “Deleting it will recover space” is a proposed consequence. “Delete it now” is a decision. “The user authorized deletion” is a fact about permission. None of these substitutes for the others.

How a Model Becomes an Agent

A language model accepts context and emits a continuation. A persistent agent normally requires additional machinery:

• task state to record the objective and current progress;
• planning to decompose work into executable steps;
• tools for search, files, code, browsers, and services;
• memory for results, commitments, and stable conventions;
• observation of tool output and environmental state;
• permissions that determine which actions are allowed;
• feedback and termination rules that define completion or revision.

The operating loop is:

observe
→ interpret the state
→ choose a next action
→ invoke a tool
→ read the result
→ update the plan
→ continue or stop

ReAct formalized a useful version of this pattern by interleaving reasoning traces with actions and environmental observations. ReAct

The acting unit is therefore not the language model in isolation. It is the assembled system of model, tools, state, permissions, and execution environment.

Does an AI Agent Have Goals or Intentions?

Engineered systems can contain objective functions, reward signals, task descriptions, and stopping criteria. These should not be collapsed into human desire or practical intention.

When a model writes, “I will inspect the files first,” the sentence can function as a report of the next operation. Its first-person grammar does not establish a private human-like intention.

This is why an agent can display sustained goal-directed behavior while still requiring external authorization, supervision, and an accountable human or institution.

Action Requires Feedback

A plan produced once cannot guarantee contact with reality. Tools fail, pages change, files disappear, and new evidence defeats earlier assumptions.

Reliable action therefore has a closed loop:

form a hypothesis
→ act
→ observe the result
→ compare actual and expected state
→ revise the interpretation or plan
→ act again

Without observation, action remains a description inside language. With observation, the system can test whether external state actually changed.

The evidence must also be labeled correctly:

• generating a command is not executing it;
• passing a build is not visual acceptance;
• pushing a repository is not proof of deployment;
• silence from a user is not authorization for a consequential action.

Characteristic Failure Modes

Fluency conceals weak evidence

A polished answer and a well-supported answer are different achievements.

Context is partial

The model can use only the files, messages, tool outputs, and environmental state made available to it. An omitted fact can reverse the correct decision.

Long tasks lose state

Extended work needs checkpoints, external records, and explicit completion criteria. Otherwise a system may repeat steps, omit requirements, or report a plan as a result.

Tool output still needs interpretation

Search results, webpages, logs, and documents can be incomplete, stale, mistaken, or adversarial. Retrieval changes the evidence set; it does not guarantee truth.

Objectives conflict

User requests, system rules, physical constraints, and local subgoals may point in different directions. Reliable behavior requires detecting and resolving conflict rather than treating every instruction-like string as authoritative.

Consequences belong to the full system

A model selects a call, an executor changes external state, a platform grants access, and people or institutions assign responsibility. Evaluating only the generated text misses most of the action chain.

How to Verify That an AI Completed a Task

Confidence should come from evidence at each layer:

Did it identify the correct object?
→ Did it obtain sufficient evidence?
→ Does the inference support the conclusion?
→ Was the action authorized?
→ Did the tool actually run?
→ Did external state change as intended?
→ Does the outcome satisfy the original objective?

For consequential or extended tasks, the system should also preserve recoverable intermediate states so that mistakes can be inspected and reversed.

Conclusion

AI reasoning can be described functionally as the transformation of input, context, and observations into judgments, plans, and selected outputs. This creates real functional comparisons with human thinking, but it does not establish identical consciousness or experience.

AI action belongs to a larger architecture:

model
+ context
+ tools
+ state
+ permissions
+ environmental feedback

The central questions are therefore not limited to what answer the model generated. They include what evidence it used, how it checked a plan, who authorized execution, what the tools changed, how the result was verified, and how errors update the next cycle.

AI reasoning computes candidate courses of action. AI agency begins when those computations are connected to authorized tools, observable consequences, and correction through feedback.

Sources

• Brown et al., Language Models are Few-Shot Learners
• Ouyang et al., Training Language Models to Follow Instructions with Human Feedback
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models
• Turpin et al., Language Models Don’t Always Say What They Think

Series: Thinking, Intention, and Action (4/4). Start with How Human Thinking Works

Collaboration Is More Than Prompt and Response

Human–AI interaction is often pictured as:

human writes a prompt
→ AI returns an answer

That picture captures a message exchange. It leaves out why the request arose, how the system selected an interpretation, who authorized an external action, and how the result changes the next decision.

A fuller model is:

human situation, need, and purpose
→ provisional intention
→ linguistic request
→ AI task model
→ calibration of goals and boundaries
→ authorized plan and action
→ observation of consequences
→ human evaluation and system revision
→ next cycle

Effective human–AI collaboration is a continuing process of alignment, action, observation, and correction around a shared task.

Intentions Are Not Fully Formed Before Language

A person does not always begin with a complete objective waiting to be encoded into a perfect prompt. Intentions often become clearer through articulation, comparison, and trial.

A request can contain several levels:

Only part of this structure is usually explicit. The rest may live in earlier turns, the active document, established conventions, institutional rules, or judgments the person has not yet made.

Longer prompts can supply more evidence. They cannot eliminate the underlying problem. Length can also add contradiction, noise, and false precision.

The System Models a Task, Not a Whole Person

An AI system has access to evidence such as:

• the current wording;
• conversation history;
• visible files and interfaces;
• tool observations;
• stored preferences and rules;
• the user’s acceptance or correction of intermediate results.

From these signals it constructs a working task model:

What is the object?
What change is requested?
What constraints apply?
Which facts remain unknown?
Which actions are authorized?
What observable state counts as completion?

That model can become highly accurate without becoming a complete representation of the user’s mind. Restricting claims to available evidence prevents the system from presenting speculation about a person as fact.

Three Kinds of Understanding

Semantic understanding

Can the system resolve the language? For example, what does “use the first one” refer to in the preceding exchange?

Operational understanding

Can it turn the language into a concrete task? Which article, language, format, repository, and action are involved?

Outcome understanding

Does it know what state would satisfy the purpose? Is a generated file enough, or must the work be committed, deployed, and verified at a public URL?

These levels can separate:

understanding the sentence
≠
knowing what operation to perform
≠
knowing what completion looks like

Many failures arise from disagreement about objects, permissions, or completion evidence even when the words were parsed correctly.

Building a Shared Task Model

Human and system gradually establish a shared, revisable representation of the task. It normally includes:

This does not require the person to specify everything at once. A system can use established context and produce reversible work while seeking information only where it changes the result or the permission boundary.

Good collaboration retains decisions already made. It also remains open to revision when new evidence conflicts with an earlier interpretation.

Clarify, Assume, or Act?

Ambiguity does not force a choice between blind guessing and endless questioning. The practical rule depends on:

degree of ambiguity × cost of error × reversibility

Low cost and reversible

Proceed with a stated assumption or a draft. The result itself can help the person clarify the target.

Moderate ambiguity

Preserve alternatives, complete the common work, or present comparable options.

High consequence or difficult to reverse

Confirm the object, scope, and authorization before publication, payment, external messaging, or irreversible deletion.

The purpose of clarification is to control consequences. It should occur where a distinction materially changes the action.

A Prompt Is Evidence, Not a Complete Contract

A prompt directly constrains the current task, but its force still depends on context and conversational commitments.

Closely related sentences license different actions:

• “Can this be published?” requests an assessment.
• “Prepare this for publication” authorizes editing.
• “Publish this on the site” authorizes an external action.
• “I may publish this later” reports a possibility.

A system must distinguish questions, proposals, background information, corrections, and authorization.

Conversation also creates durable commitments. Once the person selects an option, approves publication, or defines a format, those decisions should guide later steps. The operative instruction is distributed across the interaction, not confined to the latest sentence.

Action Tests Understanding

Restating a request does not prove that both sides share the same model. Action exposes hidden disagreement.

A person says, “Update the article on the website.” The system may create local Markdown but fail to commit it; commit without deployment; deploy one language but omit the other; publish both pages but leave discovery files stale.

Verification must therefore proceed through layers:

task interpretation is correct
→ artifact content is correct
→ action actually occurred
→ external state changed
→ the outcome satisfies the purpose

Each layer produces evidence. A failed action also reveals which assumption or completion criterion was missing.

The Joint Action Loop

The process can be described in eight stages.

1. The person encounters a problem

A need, obstacle, opportunity, or unsatisfactory result creates pressure to change the current state.

2. A provisional intention forms

The person selects a direction, although the goal, method, and standard may remain incomplete.

3. The request is externalized

Language carries part of the intention into a prompt, together with available context and constraints.

4. The AI constructs candidate interpretations

The system identifies objects, actions, constraints, missing information, and competing task models.

5. The task is calibrated

History, paraphrase, drafts, options, or a necessary question make the interpretation clear enough for the next action.

6. The AI acts within authorization

The system plans steps, invokes tools, preserves state, and checks permission at consequential boundaries.

7. Consequences are observed

Files, command results, public pages, user reactions, and other evidence show what actually happened.

8. Both sides revise

The person may change the goal, correct a mismatch, or accept the result. The system updates its task model and continues or stops.

The cycle is:

intention
→ expression
→ interpretation
→ calibration
→ action
→ observation
→ evaluation
→ revised intention

A Division of Responsibility

A joint loop does not make human and system responsibility identical.

The person contributes

• the real situation that needs to change;
• value judgments and ultimate purpose;
• private context and unspoken constraints;
• authorization for consequential actions;
• final acceptance of whether the outcome is worth having.

The AI system contributes

• a structured interpretation of available evidence;
• alternative plans and their relevant differences;
• executable steps and tool use;
• explicit uncertainty, permission, and completion states;
• rapid revision when new evidence arrives.

The platform or organization contributes

• identity and access control;
• data and privacy boundaries;
• logs, versioning, and recovery;
• failure handling and assignment of accountability;
• observable status for external actions.

The AI cannot settle every value question for a person. A person cannot explain every failure by pointing only to an isolated model. Outcomes belong to the complete sociotechnical arrangement.

Where the Loop Breaks

A feeling is mistaken for a complete objective

A person knows that an output is wrong but cannot yet specify the desired alternative. The system should help compare concrete possibilities rather than assume a unique hidden answer.

The most likely interpretation becomes “true intent”

High probability means that available evidence favors an interpretation. It does not reveal the person’s complete private purpose.

Content approval becomes action approval

Accepting an article does not automatically authorize its public release.

A plan is reported as completion

“We will update the site” is not deployment evidence. Local files, commits, deployments, and live pages are different states.

Only the output is evaluated

A correct answer can result from an unreliable method. A failed attempt can expose an important unknown. Durable collaboration evaluates results, evidence, and the capacity to correct.

Feedback does not update the next cycle

If corrections are neither retained nor applied, the same mismatch repeats and no learning loop forms.

Improving the Loop

A person can improve collaboration by:

• describing the desired change, not only naming an operation;
• distinguishing exploration, drafting, revision, approval, and publication;
• stating unacceptable outcomes at consequential points;
• locating feedback at the level that was misunderstood;
• allowing the intention itself to change after new evidence.

An AI system can improve collaboration by:

• separating explicit requirements, inferences, and unknowns;
• carrying forward confirmed context;
• using reversible artifacts to advance low-risk work;
• checking object and authorization at high-consequence boundaries;
• reporting observed results instead of plans;
• updating its task model after correction.

Conclusion

A human encounters a situation, develops needs and intentions, and expresses only part of them in language. An AI system uses visible evidence to construct a task model, reason about options, and act through tools. The consequences and the person’s evaluation then revise that model and may revise the original intention.

The central object is therefore not a perfect one-shot prompt. It is a shared loop that remains correctable:

human purpose, values, and authorization
+
AI interpretation, planning, and execution
+
environmental consequences and evidence
+
feedback that changes the next cycle

Reliable collaboration does not require an AI to read an invisible “true mind.” It requires both sides to make the current task, action boundary, and evidence of completion progressively clearer.

Further Reading

• User Intent in AI: Inference Under Uncertainty
• How AI Systems Reason and Act
• How Human Thinking Works: Representation, Reasoning, and Action
• EMNLP 2024: Making Language Models Explicitly Handle Ambiguity

That difference is easy to miss because English uses the same word in several constructions:

I intend to leave tomorrow.                 a future-directed intention
I opened the window to cool the room.       an intention with which I acted
I opened the window intentionally.          an intentional action

The philosophy of intention asks what unifies these forms. The question matters because intention sits between thought and action: it helps explain why a bodily movement counts as something a person is doing, how plans coordinate conduct over time, and when an outcome belongs to an agent in a responsibility-relevant way.

The practical force of “I intend”

Compare three statements:

I would like to learn Italian.
I believe learning Italian would be useful.
I intend to enroll in the autumn course.

The first reports a desire. The second reports an evaluation. The third normally changes the practical situation. It gives the speaker a reason to check the schedule, reserve money, avoid conflicting commitments, and decide how to travel to class.

An intention therefore has conduct-guiding force. It need not be irreversible, and it does not guarantee success. But while it remains in place, it constrains later deliberation. Someone who continually treats every settled intention as an entirely new question will have difficulty sustaining any extended project.

This is one reason Michael Bratman treats intentions as planning states. Human beings have limited time, attention, memory, and foresight. We cannot reconsider every option at every moment. Intentions let us settle some questions provisionally so that we can reason about means, coordinate with our future selves, and make our conduct predictable to other people. Stanford Encyclopedia of Philosophy: Action

Intention is not desire, prediction, or motive

Several attitudes can concern the same action without being identical.

A person can desire what they do not intend to pursue. They can predict an outcome they are trying to prevent. They can act from several motives while intending only one course of action. They can also form an intention reluctantly, without strongly wanting the action itself.

These distinctions matter whenever we try to infer intention from emotion, preference, or consequence. Wanting an outcome is evidence about intention, but it is not conclusive evidence.

Actions are intentional under descriptions

Suppose Elena flips a switch. In doing so, she:

• turns on a light;
• illuminates the garden;
• alerts a neighbor;
• startles a bird.

One physical movement admits several true descriptions. Elena may know that she is turning on the light and do so for that reason. She may not know that the switch also controls the garden lamp. She may foresee that the neighbor will notice without acting in order to alert them. She may have no idea that a bird is nearby.

It would therefore be too crude to label the entire causal chain either intentional or unintentional. An action can be intentional under one description and not under another.

This insight is central to G. E. M. Anscombe’s account of action. The question “Why?” has a special role when it asks for the agent’s reason for acting. An answer can reveal how several movements form one intelligible action:

Why are you moving those boards?
To repair the steps.
Why repair the steps?
So the entrance is safe before the guests arrive.

The answers do more than name earlier mental events. They display the practical order within the action itself. Stanford Encyclopedia of Philosophy: G. E. M. Anscombe

Plans connect intention across time

Many intentions are incomplete when formed. “I intend to move next year” leaves the city, date, budget, and method unsettled. That incompleteness is normal. A workable plan combines commitment with room for later specification.

Intentions commonly generate three pressures:

1. Consistency. Incompatible intentions cannot all guide successful action.
2. Means-end coherence. If an end is intended and a necessary means is known, the means must eventually enter the plan or the end must be reconsidered.
3. Stability. An intention must resist casual reconsideration long enough to perform its coordinating role.

None of these pressures is absolute. New evidence, changed circumstances, moral reflection, or a better opportunity can justify revision. Rational stability is not stubbornness. It is the ability to keep a practical question settled until there is a relevant reason to reopen it.

Bratman’s planning theory makes this temporal role central: intentions help resource-limited agents coordinate complex projects with themselves and with others. Stanford Encyclopedia of Philosophy: Intention

Foreseen consequences are not automatically intended

Responsibility becomes difficult when an agent foresees a consequence but does not pursue it as an end.

Imagine a surgeon who performs a painful procedure to prevent a life-threatening infection. The pain is foreseen and knowingly caused. It is not therefore automatically the surgeon’s intention. The intended end is treating the infection; the pain may be accepted as a cost.

This does not make the consequence morally irrelevant. Foreseeability, probability, available alternatives, negligence, and proportionality can all affect responsibility. The point is narrower:

Causing, foreseeing, accepting, risking, and intending an outcome are different relations to that outcome.

Legal and moral reasoning often depends on these distinctions. A harmful result alone does not reveal the intention with which someone acted. Conversely, saying “I did not intend the harm” does not answer whether the risk was reckless or the action unjustified.

Intention is not intentionality

In philosophy of mind, intentionality means the aboutness or directedness of mental states. A belief can be about tomorrow’s weather; a fear can be directed at a dog; a memory can concern a childhood room.

These states have intentionality even when no action is intended. I may fear an exam without intending it, remember a journey without planning another one, or believe that a storm is coming while trying to prevent damage.

The shared linguistic root can therefore mislead. Intention is one kind of practical attitude. Intentionality is a much broader feature attributed to thought and experience. Stanford Encyclopedia of Philosophy: Intentionality

How much can intention be observed?

Agents often have distinctive knowledge of what they are doing. Someone writing a sentence normally does not discover their action by watching their fingers as an external observer would. Yet self-knowledge is not infallible. People can rationalize, misunderstand mixed motives, forget prior commitments, or use “intention” strategically after an outcome is known.

Third parties face the opposite problem. They observe words, choices, preparation, patterns, and consequences, but they do not directly inspect another person’s practical commitment.

A careful judgment therefore separates several kinds of evidence:

what the person said they intended
what preparations they made
what alternatives they rejected
what they knew about likely consequences
how their later actions fit the stated plan
what remains uncertain

No single item is decisive in every case. Explicit statements matter, but so do planning and conduct. Outcomes matter, but they cannot be read backward as complete proof of intention.

A working definition

The concept can now be stated more precisely:

An intention is a practical commitment through which an agent settles on an action or end, guides present or future conduct, and makes further planning and coordination possible.

This definition leaves open important philosophical disputes. Must intending involve believing that one will act? Can an action be intentional without a prior intention? How should intention relate to reasons, desire, and practical knowledge? The debates remain active because intention is expected to explain several things at once.

Still, the central contrast is stable. A desire presents an outcome as attractive. A belief presents a proposition as true. An intention presents an action or end as something the agent is committed to bringing about.

Sources

• Stanford Encyclopedia of Philosophy: Intention
• Stanford Encyclopedia of Philosophy: Action
• Stanford Encyclopedia of Philosophy: G. E. M. Anscombe
• Stanford Encyclopedia of Philosophy: Intentionality
• Michael Bratman, Intention, Plans, and Practical Reason