Series: Thinking, Intention, and Action (4/4). Start with How Human Thinking Works

Collaboration Is More Than Prompt and Response

Human–AI interaction is often pictured as:

human writes a prompt
→ AI returns an answer

That picture captures a message exchange. It leaves out why the request arose, how the system selected an interpretation, who authorized an external action, and how the result changes the next decision.

A fuller model is:

human situation, need, and purpose
→ provisional intention
→ linguistic request
→ AI task model
→ calibration of goals and boundaries
→ authorized plan and action
→ observation of consequences
→ human evaluation and system revision
→ next cycle

Effective human–AI collaboration is a continuing process of alignment, action, observation, and correction around a shared task.

Intentions Are Not Fully Formed Before Language

A person does not always begin with a complete objective waiting to be encoded into a perfect prompt. Intentions often become clearer through articulation, comparison, and trial.

A request can contain several levels:

Only part of this structure is usually explicit. The rest may live in earlier turns, the active document, established conventions, institutional rules, or judgments the person has not yet made.

Longer prompts can supply more evidence. They cannot eliminate the underlying problem. Length can also add contradiction, noise, and false precision.

The System Models a Task, Not a Whole Person

An AI system has access to evidence such as:

• the current wording;
• conversation history;
• visible files and interfaces;
• tool observations;
• stored preferences and rules;
• the user’s acceptance or correction of intermediate results.

From these signals it constructs a working task model:

What is the object?
What change is requested?
What constraints apply?
Which facts remain unknown?
Which actions are authorized?
What observable state counts as completion?

That model can become highly accurate without becoming a complete representation of the user’s mind. Restricting claims to available evidence prevents the system from presenting speculation about a person as fact.

Three Kinds of Understanding

Semantic understanding

Can the system resolve the language? For example, what does “use the first one” refer to in the preceding exchange?

Operational understanding

Can it turn the language into a concrete task? Which article, language, format, repository, and action are involved?

Outcome understanding

Does it know what state would satisfy the purpose? Is a generated file enough, or must the work be committed, deployed, and verified at a public URL?

These levels can separate:

understanding the sentence
≠
knowing what operation to perform
≠
knowing what completion looks like

Many failures arise from disagreement about objects, permissions, or completion evidence even when the words were parsed correctly.

Building a Shared Task Model

Human and system gradually establish a shared, revisable representation of the task. It normally includes:

This does not require the person to specify everything at once. A system can use established context and produce reversible work while seeking information only where it changes the result or the permission boundary.

Good collaboration retains decisions already made. It also remains open to revision when new evidence conflicts with an earlier interpretation.

Clarify, Assume, or Act?

Ambiguity does not force a choice between blind guessing and endless questioning. The practical rule depends on:

degree of ambiguity × cost of error × reversibility

Low cost and reversible

Proceed with a stated assumption or a draft. The result itself can help the person clarify the target.

Moderate ambiguity

Preserve alternatives, complete the common work, or present comparable options.

High consequence or difficult to reverse

Confirm the object, scope, and authorization before publication, payment, external messaging, or irreversible deletion.

The purpose of clarification is to control consequences. It should occur where a distinction materially changes the action.

A Prompt Is Evidence, Not a Complete Contract

A prompt directly constrains the current task, but its force still depends on context and conversational commitments.

Closely related sentences license different actions:

• “Can this be published?” requests an assessment.
• “Prepare this for publication” authorizes editing.
• “Publish this on the site” authorizes an external action.
• “I may publish this later” reports a possibility.

A system must distinguish questions, proposals, background information, corrections, and authorization.

Conversation also creates durable commitments. Once the person selects an option, approves publication, or defines a format, those decisions should guide later steps. The operative instruction is distributed across the interaction, not confined to the latest sentence.

Action Tests Understanding

Restating a request does not prove that both sides share the same model. Action exposes hidden disagreement.

A person says, “Update the article on the website.” The system may create local Markdown but fail to commit it; commit without deployment; deploy one language but omit the other; publish both pages but leave discovery files stale.

Verification must therefore proceed through layers:

task interpretation is correct
→ artifact content is correct
→ action actually occurred
→ external state changed
→ the outcome satisfies the purpose

Each layer produces evidence. A failed action also reveals which assumption or completion criterion was missing.

The Joint Action Loop

The process can be described in eight stages.

1. The person encounters a problem

A need, obstacle, opportunity, or unsatisfactory result creates pressure to change the current state.

2. A provisional intention forms

The person selects a direction, although the goal, method, and standard may remain incomplete.

3. The request is externalized

Language carries part of the intention into a prompt, together with available context and constraints.

4. The AI constructs candidate interpretations

The system identifies objects, actions, constraints, missing information, and competing task models.

5. The task is calibrated

History, paraphrase, drafts, options, or a necessary question make the interpretation clear enough for the next action.

6. The AI acts within authorization

The system plans steps, invokes tools, preserves state, and checks permission at consequential boundaries.

7. Consequences are observed

Files, command results, public pages, user reactions, and other evidence show what actually happened.

8. Both sides revise

The person may change the goal, correct a mismatch, or accept the result. The system updates its task model and continues or stops.

The cycle is:

intention
→ expression
→ interpretation
→ calibration
→ action
→ observation
→ evaluation
→ revised intention

A Division of Responsibility

A joint loop does not make human and system responsibility identical.

The person contributes

• the real situation that needs to change;
• value judgments and ultimate purpose;
• private context and unspoken constraints;
• authorization for consequential actions;
• final acceptance of whether the outcome is worth having.

The AI system contributes

• a structured interpretation of available evidence;
• alternative plans and their relevant differences;
• executable steps and tool use;
• explicit uncertainty, permission, and completion states;
• rapid revision when new evidence arrives.

The platform or organization contributes

• identity and access control;
• data and privacy boundaries;
• logs, versioning, and recovery;
• failure handling and assignment of accountability;
• observable status for external actions.

The AI cannot settle every value question for a person. A person cannot explain every failure by pointing only to an isolated model. Outcomes belong to the complete sociotechnical arrangement.

Where the Loop Breaks

A feeling is mistaken for a complete objective

A person knows that an output is wrong but cannot yet specify the desired alternative. The system should help compare concrete possibilities rather than assume a unique hidden answer.

The most likely interpretation becomes “true intent”

High probability means that available evidence favors an interpretation. It does not reveal the person’s complete private purpose.

Content approval becomes action approval

Accepting an article does not automatically authorize its public release.

A plan is reported as completion

“We will update the site” is not deployment evidence. Local files, commits, deployments, and live pages are different states.

Only the output is evaluated

A correct answer can result from an unreliable method. A failed attempt can expose an important unknown. Durable collaboration evaluates results, evidence, and the capacity to correct.

Feedback does not update the next cycle

If corrections are neither retained nor applied, the same mismatch repeats and no learning loop forms.

Improving the Loop

A person can improve collaboration by:

• describing the desired change, not only naming an operation;
• distinguishing exploration, drafting, revision, approval, and publication;
• stating unacceptable outcomes at consequential points;
• locating feedback at the level that was misunderstood;
• allowing the intention itself to change after new evidence.

An AI system can improve collaboration by:

• separating explicit requirements, inferences, and unknowns;
• carrying forward confirmed context;
• using reversible artifacts to advance low-risk work;
• checking object and authorization at high-consequence boundaries;
• reporting observed results instead of plans;
• updating its task model after correction.

Conclusion

A human encounters a situation, develops needs and intentions, and expresses only part of them in language. An AI system uses visible evidence to construct a task model, reason about options, and act through tools. The consequences and the person’s evaluation then revise that model and may revise the original intention.

The central object is therefore not a perfect one-shot prompt. It is a shared loop that remains correctable:

human purpose, values, and authorization
+
AI interpretation, planning, and execution
+
environmental consequences and evidence
+
feedback that changes the next cycle

Reliable collaboration does not require an AI to read an invisible “true mind.” It requires both sides to make the current task, action boundary, and evidence of completion progressively clearer.

Further Reading

• User Intent in AI: Inference Under Uncertainty
• How AI Systems Reason and Act
• How Human Thinking Works: Representation, Reasoning, and Action
• EMNLP 2024: Making Language Models Explicitly Handle Ambiguity

Matching demand and supply is therefore a problem of interpretation, delivery, and learning.

The concepts occupy different positions:

• User intent belongs to the demand side. It concerns what a person is preparing to do or trying to accomplish in a situation.
• Product belongs to the supply side. It organizes capabilities and experiences intended to change that situation.
• Brand connects supply with recognition. It helps people find the source, understand its difference, form expectations, and decide how much provisional trust to place in it.

The firm is the supplier. The product is its vehicle of capability. The brand is the recognition and expectation system around the identifiable source.

Intent gives action a direction. A product offers capability for moving in that direction. A brand helps the user discover, interpret, and evaluate the offer under uncertainty.

Fit is not a single moment in which a company finally reads the user’s mind. It is a revisable relationship built through expression, discovery, use, results, and correction.

Need, Want, Goal, Intent, Demand, and Behavior

Many failures begin by collapsing several layers into one.

Suppose someone says, “I need more notifications.”

That sentence may identify a proposed feature rather than a need. The underlying situation could be fear of missing a deadline. The need may be reliable awareness of urgent changes. The goal may be to respond before a cutoff. The intent may be to check a dashboard every morning. More notifications are only one possible solution, and they may worsen distraction.

A purchase is behavior. It may express an intention, but it can also result from a promotion, inertia, confusion, social pressure, lack of alternatives, or a cancellation barrier. Retention establishes continued use, not the reason for use and not the value of the outcome.

The first discipline of matching is therefore to preserve the layers.

Is There One True Intent?

Sometimes a user has a clear and stable intention: book a train for tomorrow, submit a tax return, or send a file to a colleague. Product work can concentrate on interpreting the conditions and completing the task.

Other situations are less settled:

• the user holds conflicting intentions;
• dissatisfaction is clear but the desired change is not;
• the user names a familiar solution instead of the underlying outcome;
• cost, information, and experience alter the intention;
• immediate impulse conflicts with a longer-term goal;
• the user understands the choice only after trying it.

“True intent” can then mean at least three things:

1. the action the user currently plans to take;
2. the intention the user still endorses after learning the costs and consequences;
3. an action that actually addresses the deeper condition behind the problem.

These can diverge. A person may intend to buy an attention tool, later decide the subscription is not worth its data cost, and discover through use that the real problem was an overloaded work process.

Intent is partly discovered through reflection and partly formed through comparison, commitment, and action. A useful question is not whether a hidden, permanent intention has been uncovered. It is:

Given the current situation, evidence, costs, and values, is this an intention the person understands, endorses, and is prepared to act on?

What a Company Can Know

A company observes signals, not needs directly.

Interviews, search queries, support conversations, purchases, usage, abandonment, workarounds, complaints, and outcomes all provide evidence. Each has limits.

What users say

Language reveals how people understand their situation and which distinctions they can express. Reports can still be distorted by memory, social expectations, limited vocabulary, and question design.

What users do

Behavior proves that an event occurred. It does not by itself identify the cause. A click can indicate interest or confusion. Long time on a page can indicate engagement or difficulty. Renewal can indicate value or high switching cost.

The context of use

Tasks, resources, physical conditions, social rules, skill, time pressure, and organizational environment often explain more than an isolated feature request. Human-centred design treats users, goals, tasks, resources, and environments as parts of the same context.ISO 9241-210:2019

What persists over time

Repeated obstacles, costly workarounds, and stable outcome gaps usually provide stronger evidence than one statement. Longitudinal evidence can also reveal when a product removes one problem but creates another.

A company can improve its model of demand. It cannot turn evidence into direct access to private mental states.

From Evidence to a Product

Insight does not move into a product without loss. At least six transformations occur:

user situation
    ↓ interpretation
need hypothesis
    ↓ selection
target user and value proposition
    ↓ design
product capability and interaction
    ↓ delivery
experience in actual use
    ↓ measurement
outcome, cost, and risk
    ↓ learning
revised need and product model

A failure can enter at every transition.

The company may understand the problem but target the wrong group. A useful capability may be too difficult to discover or learn. A product may create short-term improvement and long-term dependence. The user may benefit while the buyer sees no reason to pay. Average improvement may conceal severe harm to a smaller group.

This is why “the feature works” is a narrower statement than “the product fits the need.”

What Brand Contributes

A product addresses whether a capability can produce a result. A brand affects whether the relevant person can find, understand, and provisionally trust the source.

Discovery

Positioning, category language, distribution, and recognizable cues help a person connect an offer to a situation. A capable product cannot be considered if it is not found at the relevant time.

Interpretation

A clear brand claim tells people whom the product is for, which problem it addresses, what makes it different, and what evidence should be expected. Vague claims make almost any result appear consistent with the promise.

Risk reduction

People cannot test every alternative completely. Source recognition, prior experience, recommendations, and reputation compress information into an expectation. This expectation is probabilistic and must remain open to current product evidence.

Attribution

When a useful experience has no identifiable source, the product may succeed without building memory or trust around the provider. Attribution lets experience affect the next choice.

Accountability

A brand connects many products and actions to the same source. That allows trust to accumulate, and it allows users and institutions to assign responsibility when promises fail.

Brand influence is not neutral. It can help people name a poorly understood need. It can also redescribe ordinary insecurity as a defect that only consumption can repair. The test is whether claims improve understanding and choice or narrow them through manipulation.

Eight Dimensions of Fit

A useful match must survive more than one metric.

1. Problem fit: Does the offer address a problem in the user’s actual situation?
2. Capability fit: Can the product produce the required change?
3. Context fit: Can it work within the user’s time, environment, skill, and workflow?
4. Cognitive fit: Can the user understand what it is, why it matters, and how to use it?
5. Economic fit: Are price, access, learning, switching, and exit costs acceptable?
6. Value fit: Is the method compatible with the user’s identity, principles, and longer-term interests?
7. Trust fit: Is the source credible about performance, risk, data, and support?
8. Outcome fit: Does actual use improve the intended result without imposing unacceptable harm?

No product fits “the user” in the abstract. It fits particular people, situations, times, alternatives, and costs to a greater or lesser degree.

The User Also Needs a Learning Loop

The burden of interpretation does not belong only to the company. A user can clarify intention through a sequence:

notice a problem or attraction
    ↓
describe the current situation
    ↓
separate need, want, goal, and proposed solution
    ↓
form a provisional intention
    ↓
compare brand claims and product capabilities
    ↓
try at a proportionate cost
    ↓
observe results and revise

Useful questions include:

1. What is happening before I name something to buy?
2. Which condition or outcome needs to change?
3. Is this my goal, or a goal offered to me by the seller?
4. Have I confused a desired result with one proposed solution?
5. Would I still address this problem if this brand did not exist?
6. Which money, time, attention, data, and opportunity costs will I accept?
7. What happened after use?
8. After understanding the longer-term effects, do I still endorse the choice?

This process does not uncover a socially untouched inner self. Human desires are shaped by language, culture, comparison, and institutions. Reflection makes those influences more visible and preserves the ability to compare, refuse, and exit.

Two Learning Systems Meet

The supplier’s loop is:

observe situations
→ propose need hypotheses
→ build the smallest useful test
→ observe use and outcomes
→ revise product and brand claims

The user’s loop is:

experience a situation
→ form a provisional intention
→ discover and compare offers
→ use and bear consequences
→ revise self-understanding and choice

They meet at four points:

• Expression: Can the user describe the problem, and can the brand respond in intelligible language?
• Discovery: Can a person with the relevant situation find the offer at the right time?
• Use: Does the product enter the actual context and produce a result?
• Learning: Can both sides learn from success, failure, misuse, and exit?

Product–market fit is often discussed as though the market emits a simple verdict. In practice, the observed result is produced by a distribution channel, price, alternatives, product quality, switching costs, user differences, and measurement choices. Fit must be diagnosed, not merely declared.

Why Matching Fails

A requested solution is treated as the need

The company implements what was requested without examining the situation that produced the request.

Behavior is treated as intention

Clicks, time spent, purchases, and retention are assigned psychological meanings they cannot establish alone.

Average behavior replaces user variation

One metric hides distinct intentions, contexts, and concentrated harm.

The brand promise is too broad

A claim for everyone cannot help anyone determine relevance and cannot be clearly tested.

Immediate impulse displaces endorsed goals

A system optimized for repeated reaction may increase activity while weakening attention, health, or autonomy.

Insight never reaches organizational response

Research remains in a report while product priorities, service, pricing, incentives, and resource allocation continue unchanged. Market orientation requires generating intelligence, sharing it across the organization, and responding to it, rather than merely listening.Kohli and Jaworski, “Market Orientation”

Choice is constrained

Lock-in, opaque information, defaults, and missing alternatives weaken the claim that observed use expresses preference.

Can a Company Create Demand?

A company can reveal an unmet need, create a capability that did not exist, teach people about a new possibility, form a product category, shape a specific desire, or engineer dependence.

Those actions are not ethically equivalent.

The relevant questions are:

• Did the user gain a real capability?
• Are the promise, mechanism, cost, and risk understandable?
• Can alternatives be compared?
• Can the person refuse and leave?
• Do longer-term outcomes remain consistent with interests the user still endorses?
• Are third parties bearing hidden costs?

Influence is unavoidable. Manipulation is not. The difference depends on truthfulness, agency, reversibility, and the distribution of consequences.

The Result

A company can form increasingly reliable hypotheses about demand. It cannot prove that it has read a person’s final and private need.

A product can produce a good fit for specified people and conditions. One successful period does not establish permanent fit.

A user can develop a clearer intention through reflection, comparison, trial, and attention to consequences. There may be no single fixed intention waiting to be found.

The relationship is best understood as reciprocal calibration:

The user moves from an unclear situation toward an actionable intention. The company moves from incomplete evidence toward a need hypothesis. The product exposes that hypothesis to use. The brand helps both sides find one another, set expectations, and remember what happened.

The strongest fit leaves the user better able to understand the situation, the product able to improve a concrete outcome, the brand claim consistent with evidence, and both sides free to revise their judgment.

Sources

• ISO 9241-210:2019: Human-centred design for interactive systems
• ISO 9241-115:2024: Guidance on conceptual design, user-system interaction design, user interface design, and navigation design
• Kohli and Jaworski, 1990: Market Orientation
• Stanford Encyclopedia of Philosophy: Intention
• What Does It Mean to Need Something?
• Intention: How a Commitment Organizes Action
• Brand and Product: Delivery, Expectation, and Trust
• User Intent in AI: Inference Under Uncertainty

Series: Thinking, Intention, and Action (3/4). Previous: How AI Systems Reason and Act; next: How Humans and AI Form a Shared Action Loop

When an AI system says that it has identified a user’s intent, it has not discovered a hidden object inside the user’s mind. It has selected an interpretation that is useful for deciding what to do next.

That distinction is fundamental. In product engineering, user intent is usually an operational variable: search for a flight, cancel an order, summarize a document, edit a file. In psychology and philosophy, intention can mean a practical commitment, a purpose in acting, or the mental organization of action. The engineering label is narrower and more provisional.

An AI system infers an actionable interpretation from available evidence. It does not directly observe the user’s full purpose.

“Intent” is a model of the task

Consider the prompt:

Check this proposal.

Several actions fit the words:

• summarize the proposal;
• verify its claims;
• identify logical gaps;
• edit the prose;
• judge whether it should be approved.

A system needs some representation of the requested task before it can respond. Traditional dialogue systems often assign a message to a predefined intent such as cancel_order, then extract slots such as an order number. A language model can infer and express a much wider range of tasks without a fixed list, but the underlying problem remains: which action does this utterance license in this context?

The inferred intent is therefore a working hypothesis about the task, not a complete theory of the person.

Prompts are often underspecified

Natural language relies on shared context. People omit information because another person can usually recover it from the situation.

“Make it shorter” presupposes a text and a relevant standard of brevity. “Use the first one” presupposes a previously presented set of options. “Publish it” may presuppose a particular site, account, version, audience, and approval state.

Linguistic meaning alone cannot supply all of this. A useful interpretation may depend on:

Research on ambiguity shows why forcing every utterance into one interpretation can be brittle. Ellipsis, polysemy, and missing constraints can leave several readings reasonable at the same time. EMNLP 2024: Making Language Models Explicitly Handle Ambiguity

Inference is better represented as a distribution

A simplified model is:

P(task interpretation | prompt, context, environment, rules)

The system compares candidate interpretations under the evidence it can access. One interpretation may dominate; several may remain close; all may be poor because a crucial fact is missing.

This yields three different conditions:

1. Clear enough to act. One interpretation is strongly supported and the action is reversible.
2. Ambiguous but manageable. The system can state an assumption, produce a draft, or preserve alternatives.
3. Ambiguous and consequential. The system should obtain clarification or confirmation before an irreversible or externally consequential action.

Confidence is decision-relative. The evidence needed to suggest a title is lower than the evidence needed to publish under someone’s name or transfer money.

Semantic similarity is only part of the problem

Language models learn statistical relations among expressions and contexts. That helps them recognize that “clean this up” may request editing, or that code followed by an error message probably requests diagnosis.

But intent inference also involves pragmatics:

• What is the speaker trying to accomplish by saying this now?
• Which earlier object does “it” refer to?
• Is the sentence a request, a question, a correction, or background information?
• Does a polite form conceal a firm requirement?
• Is the user authorizing execution or merely discussing a possibility?

Two prompts can be semantically similar while licensing different actions. “Can this be deleted?” asks about possibility. “Delete this” authorizes an action. “I am thinking about publishing it” does not necessarily authorize publication.

An effective system must therefore interpret language together with conversational commitments and action boundaries.

A deeper goal may remain hidden

Suppose a user asks for a resignation letter. Their immediate task may be clear: draft the letter. Their deeper purpose could be to resign, prepare for a negotiation, explore wording, write fiction, or test the system.

The system can often complete the immediate task without resolving the deeper goal. Confusing the two creates two errors:

• Overreach: claiming knowledge of motives that the evidence does not support.
• Unnecessary friction: demanding a personal explanation when the requested task is already clear and safe to perform.

A good assistant asks only for information that materially changes the work or the permission boundary. It can remain uncertain about a person’s deeper purpose while being precise about the action requested in the current turn.

Confirmation changes status, not metaphysics

If the assistant asks:

Should I identify problems only, or rewrite the proposal as well?

and the user answers:

Rewrite it.

then “rewrite the proposal” becomes an explicit instruction for the interaction. This is stronger evidence than the original ambiguous wording. It still does not prove every motive behind the request.

The most useful state model keeps these distinctions visible:

These labels describe evidence and workflow. They avoid a misleading field such as true_intent = true, which would turn an interpretation into an alleged psychological fact.

Intent and authorization must remain separate

A model may correctly infer what a user wants and still lack authorization to perform the action. It may also have general permission to edit a workspace while misunderstanding which file the user meant.

Reliable systems therefore ask two separate questions:

Interpretation: What action is most likely being requested?
Authorization: Is the system permitted to perform that action now?

This is especially important for publication, messages sent to other people, purchases, deletion, deployment, and access to private data. A confident guess does not create permission.

The inverse matters too. Repeatedly asking for confirmation after permission has already been granted adds friction and can obscure the actual uncertainty. The system should preserve prior authorization while remaining open to new corrections.

Action is a test of interpretation

Because inferred intent is fallible, systems should prefer actions that produce useful feedback at low cost:

• draft before publishing;
• preview before replacing;
• show a diff before merging;
• preserve previous versions;
• state a consequential assumption;
• make uncertain fields explicit rather than filling them with invented values.

The user’s response then supplies new evidence. Acceptance, correction, revision, or rejection updates the working interpretation.

This resembles Bayesian learning in a broad sense: begin with candidate interpretations, observe evidence, update their relative plausibility, and continue revising. But the analogy has limits. A production language model does not necessarily maintain a transparent table of hypotheses or calibrated posterior probabilities, and a plausible interpretation is not thereby the user’s private truth.

What an AI system can responsibly claim

An AI system can sometimes say:

• “The prompt explicitly asks for a rewrite.”
• “Given the previous turn, ‘the first one’ most likely refers to option one.”
• “The user confirmed that publication is authorized.”
• “Two interpretations remain plausible.”

It should be much more cautious about claims such as:

• “This is what the user really wants.”
• “The user’s deeper motive is X.”
• “The person had no intention to do Y.”

The defensible conclusion is precise:

AI can infer, test, and confirm operational interpretations of a request. It cannot turn limited linguistic evidence into certainty about a person’s complete or ‘true’ intention.

Sources

• EMNLP 2024: Making Language Models Explicitly Handle Ambiguity
• OpenAI: Our Approach to the Model Spec
• Stanford Encyclopedia of Philosophy: Intention
• Stanford Encyclopedia of Philosophy: The Pragmatics of How and Why Questions