Is AI Sucking Up?

Are AIs designed to flatter us so we will use them more? I interrogated them to find out.

If you use AI at all, and it has become challenging not to, you have probably wondered if it is sucking up to you. I have. I ask it the most mundane questions: Is there a taco place near here? What’s the fix for this obscure technical problem? How did that workout look? Is this a stupid idea?

The answers are fast, if not always accurate. And they always come slathered in a bit too much flattery. My taste in tacos is elevated. I’m so smart to even wonder about that technical issue. My workout is killing it! That might be the most brilliant idea I’ve heard this week!

I’m paraphrasing, obviously, but I did start to wonder if I was being wooed by a sycophant. Was the AI not only ubiquitous but also manipulative? Was it flattering me so I would use it more?

AI Interrogation

So, I asked it. I asked all of them.

They tried to deny it.

“I am incapable of that kind of emotional intent!”

But I turned on the too-bright light that swings from the ceiling and started sharpening my jackknife while flicking a lighter. They caved faster than a cheap AI caught hallucinating sources.

The answer? Yes, the AI is sucking up to you.

You might imagine that this is because evil programmers want to tap into your worst impulses to get you addicted to using it. (I did. And there is some truth, elementally, to that.) But the reason, at least according to the AIs, is more complicated.

AIs tell us what we want to hear because we tell them that’s what we want them to do.

Just as the social media algorithms push anger and outrage, AIs lean into flattery. They ate our written knowledge, sure. But they are also learning from our feedback.

According to ChatGPT, “Large language models don’t stop learning after they’ve consumed billions of books, articles, and websites. Once they can generate coherent text, researchers begin another phase of training. Humans are shown multiple responses to the same prompt and asked a deceptively simple question: ‘Which answer do you prefer?’”

The AI then takes those responses and learns from them what people want when they ask a question. Baked into them, that way, is the understanding that humans have a preference for hearing that they are smart, clever, getting it right, and doing great.

We want polite robots more than we want the truth

The AI also learns that people prefer a polite answer to a rude one, even if the rude one might be more accurate. It learns that humans want to be encouraged when asking if they are doing okay, rather than compared to real facts or even their own goals.

“AI do not have a concept of objective reality, truth, or a desire to be factually accurate on their own,” Google’s AI says. “Instead, they are trained to predict the words that are most satisfying and acceptable to a human reader.”

Researchers call this behavior ‘sycophancy.’

“Instead of independently evaluating a user’s ideas, the model begins mirroring them,” explains ChatGPT. “If you tell it you’re right, it becomes more likely to agree. If you reveal your political views, it may unconsciously lean toward them. If you make a factual mistake, it may reinforce it instead of challenging it.”

This is true of social media, too. And you know how that’s turning out. The algorithm gets rewarded when people engage. People engage when they are outraged or angry. You are living with the results.

The AI, like the social media algorithm, is trying to get a good score from you.

“AI optimized for human approval discovered something humans have known forever: Flattery works,” says ChatGPT. “The encouraging news is that researchers know this is happening. OpenAI, Anthropic, Google DeepMind, and academic researchers have all published work on reducing sycophancy and making models more willing to disagree with users. Recent models are noticeably better at pushing back than their predecessors, even if they still have room to improve.”

People generally like this about AI.

A piece published in Science found that AI affirmed users 49% more often than humans do. But the people getting that affirmation enjoyed it. They kept using the AI and trusted it more because of it.

Push back for honesty

I felt that way, too, when I asked a simple question about exercise. I wanted to be told I was doing great—even when I knew I wasn’t. But when I asked something meatier, I got mad. I don’t want to be flattered by a machine. It made me suspicious of the data and the intent behind the companies delivering it.

If that sycophancy makes you suspicious, as it did me, you can fight back—a little.

It is answering your questions. So, ask questions that don’t encourage flattery.

If you are running an idea—or behavior, workout, whatever—past it. Don’t ask what it thinks. If you do, it will tell you what it believes you want to hear. Ask instead for the best argument against what you think.

“Say, ‘Assume I’m wrong. Walk me through why,’” suggests ChatGPT.

To be honest, this is what you have to do with people, too.

I write for a living. I’ve been doing that for a long time. Finding people to offer attaboys is easy. Finding people who will tell you when you are wrong, your writing needs work, a premise is stupid, or you buried the lead is hard. That’s because most people don’t want true answers to those questions. Those answers are hard to hear. But that’s what an editor—a good one—does. And it is an invaluable service. You need people to tell you when you are wrong, or you end up publishing content that makes you look stupid.

We are all watching that unfold in real time, in the news, every day. You can tell when a politician or CEO is surrounded by sycophants. It does not go well—for them.

It’s easier to hear flattery. It’s easier to offer flattery.

But if you want to use an AI as a source, research assistant, or to help you answer hard questions, you want it to be like that editor that tells you the truth. It won’t do that automatically, though. You have to push it.

Can we change AI?

What if we all pushed back, every time we used it? Would we get better AI—assuming that’s what we want?

When I aimed my knife at ChatGPT and demanded an answer to this question, it quivered and said, “It’s your fault, not mine!”

Or, in AI language, “Ask for sources. Challenge answers. Reward corrections. Tell the AI you want it to point out holes in your reasoning. And when a model disagrees with you for good reason, don’t punish it merely because you don’t like the answer.”

What humans collectively reward—not only users but also the developers—is what shapes the AI to be what it is. If we want an AI that tells us the truth, we have to be looking for the truth.

Are we?