What do those words tell you? Only three things. Someone is saying something. There’s some object. They want it somewhere it isn’t now.
That’s it.
The words signal intention, but they don’t carry enough meaning by themselves. The language we use rarely represents the full context needed for meaning.
We fill the gaps anyway. We guess there’s a second person, that the object is in view, that “there” is somewhere both people can see. Those guesses come from past situations the words remind us of, and from the words alone they’re brittle. It works the other way too: words can be imprecise, wrong or missing entirely, and meaning still gets across.
If you were in the room, you’d know much more. Maybe they’re directing someone who’s arranging furniture. Maybe they’re talking to themselves while sorting notes. Maybe they’re correcting a kid putting away toys. You’d get that from context, tone and the way they said it, maybe a gesture. You might even know what “that” is, where it is, where it’s going and who’s supposed to move it.
I used to gripe about bag-of-words systems. That complaint doesn’t matter much anymore. LLMs are very good at how we structure language, good enough to reproduce intelligent things for us. That doesn’t mean they’re intelligent.
Think about “Meet me at the coffee shop.” Nobody adds “the one connected to the city water supply, with tables and chairs.” That’s all part of the idea, so nobody says it. What we do say is “I can’t remember what it’s called, but it’s beside the Shake Shack.” The rest comes from where we are, who we are to each other and what we both already know.
When we speak, we say only what’s necessary. We write a little more. The more context we share with someone, the less we need to say. And when we don’t share enough, we ask. People ask each other for more information constantly.
Show people the text of a conversation and they often have no idea what’s going on. Tell them who’s talking and what the situation is, and it makes sense.
We tend to trust that everything these models were trained on adds up to understanding people. It’s the skim of the surface. So much of what we communicate is in how we say it, when, with whom and under what circumstances. And when we speak, we’re built to be brief.
If you’re building an agent that talks with people:
- Give it the context people won’t say out loud: who they are, where they are and what’s already happened.
- Expect short, incomplete, even inaccurate input. People say only what they think is needed.
- Let it ask. A good clarifying question is what a person would do.
- Don’t mistake fluent for intelligent, or confidence for trustworthiness.
- Review more than text. On voice, listen to the calls. Transcripts drop pauses and side speech, which can be why a call failed. In chat, look at what was typed in response to what was present on screen.
What does your agent need to know that nobody has told it?