HN.zip

Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)

148 points by robin_reala - 80 comments
kazinator [3 hidden]5 mins ago
I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice.

If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a "you are completely right, I apologize" about face and then says something different, and things have begun to slide into uncertainty.

That might not exactly be sycophancy, but it's basically the same thing: producing responses that are reflection of what is in the chat, rather than any real shit.

Pick any topic where people disagree. It could be an entirely technical topic in which engineers have settled the questions, and the only contrarians are crackpots. The problem is that the crackpots are out there writing, and this is snarfed into the training data. Crackpots use certain ways of talking about certain subjects. If you use similar vocabulary and concepts that align with some crackpot theory, the AI simply starts predicting tokens according to that, and you are now in crackpot land: what you are saying is validated using the crackpot terms. Next, write in a way that reintroduces rigidity: proper terminology and correct concepts, and, whoa, the AI is an engineer again, contradicting the previous crackpot shit.

jtr1 [3 hidden]5 mins ago
I realize this is not scalable, but I’ve come to the personal preference that sycophancy be overt. LLMs are always steering in a direction and I find it a good reminder that they are just as incapable of objectivity as humans. TBD, but I think I’d prefer that fact plainly visible where I can see it.
chrisjj [3 hidden]5 mins ago
> The problem is that the crackpots are out there writing, and this is snarfed into the training data.

No, the problem is the chatbot pretends to be an AI answer engine.

Recognise it is simply a zero-intelligence search engine, and you'll not be surprised at all to get crackpot writings from the web.

> write in a way that reintroduces rigidity

... and do not be surpised to get rigid crackpot writings from the web.

Ravus [3 hidden]5 mins ago
> Recognise it is simply a zero-intelligence search engine

I think that this highlights the problem: it's not even a proper search engine. If it were, it would find existing material and link to it. The chatbot instead re-elaborates the message. This is different enough to have legal consequences (see the recent German ruling).

By squashing everything together, one loses the ability to evaluate sources, including their credibility and ulterior motives. With a search engine, I can tell if I am getting climate information from the National Snow and Ice Data Center, from an oil trading consortium, or from a random blog.

The parent poster's suggestion is trying to work around the selection of sources, but such an attempt is fighting against the questionable design of the whole thing.

I don't think it's currently solvable, as selling those models as oracles is an essential component of their business plan.

chrisjj [3 hidden]5 mins ago
Plus this rot has extended to true search engines too. A recent Google blurb on new "AI" features of Google web search kicks of with the following misrep:

"The goal of Search has always been simple: to help you ask anything on your mind"

https://blog.google/products-and-platforms/products/search/s...

And they are also perverting search results, if this is to be believed:

"The Google AI Mode recognizes search intent by understanding the meaning of a query made rather than relying entirely on keywords. The platform uses sophisticated AI models to determine what users are looking to accomplish and whether they need any information, comparisons, or final decisions. This way, Google provides more accurate search results."

https://bostoninstituteofanalytics.org/blog/latest-google-ai...

seizethecheese [3 hidden]5 mins ago
Perhaps only a small minority want quality out of information product, while most want satisfying simplicity. This is definitely how the news plays out.

I recently built an ai chart product that attempts to add more nuance by injecting extra models to jump in when the first model misses the mark. (http://pellmell.au So far it seems like only a minority of people like it. The main feedback is that answers are confusing or too much to read

tonyedgecombe [3 hidden]5 mins ago
I learnt a long time ago that most people don't want their problems solved, they just want to vent.
donatj [3 hidden]5 mins ago
> This suggests that people are drawn to AI that unquestioningly validate, even as that validation risks eroding their judgment

I believe this is a larger problem than just AI.

The internet has helped people surround themselves with only voices that agree with them and validate them, often to their detriment.

There are entire online communities urging people to cut others out of their lives over the slightest disagreement.

overgard [3 hidden]5 mins ago
Yeah, a lot of it isn't even really about people's choice though. Like, I have so many click-baity things in my YouTube feed even though I ignore almost all of it, because just one click and suddenly that's a thing it's going to show me 10x. I wish there was a place I could go for tech news that just didn't talk about AI. I'm so over it.
ehnto [3 hidden]5 mins ago
I think what frustrates me about the constant AI talk, is that it's like a global bikeshedding/yak shaving convention. It's an onslaught of details and opinions that don't matter that much. People arguing about single percentile benchmark differences like it will make or break their next project.

I get it, we can use AI agents, and they keep improving. Great, now calm down and go build stuff, and show me the stuff. I really think LLMs and agents are incredible, but I don't want to hear that over and over again for another 5 years.

HN is particularly bad for it right now. Where are all the innovations, cool technical side projects etc happening outside of LLMs?

georgefrowny [3 hidden]5 mins ago
> Great, now calm down and go build stuff, and show me the stuff.

Also sounds like Reddit where there are whole communities around what seem like part of the process - keyboards, knife sharpening, etc, rather than what you actually used your Handmade Matcha 65% MX Mechanical Collector's Edition or your Organic Kyoto 6500 diamond stone-honed knife for.

Like, yes, I get that people like to talk about their processes, and that's fine, but it seems like you can easily get sucked into making it the whole game by a sufficiently "supportive" community.

morgoths_bane [3 hidden]5 mins ago
>I get it, we can use AI agents, and they keep improving. Great, now calm down and go build stuff, and show me the stuff.

I completely agree with this sentiment. What I have seen is that these types of people never show the results of their work, they only discuss how productive they were or how fast it was to develop their project. The actual details of the project are not really mentioned.

In the rare event they do show their project, there's three categories: a worse version of something else that already exists (e.g. their version of Counter Strike that is less fun, rewriting some Python library into Rust but is now also worse), a dashboard, a visually engaging website you'll find interesting for less than 10 minutes and then never touch again.

I am not even an LLM hater, I am just not that impressed with majority of what has been built. Honestly I think back in the 90s I was far more impressed with the early websites that were totally garbage by today's standards. I remember spending so much time on the US Treasury's website as a kid learning everything about how currency works, I do not think I have ever had that level of engagement or fascination with anything that was generated by an LLM or where an LLM had significant support in the development.

Certainly this technology is nascent, however allegedly it should be making us far more productive so in the fourish years it's been around I feel like I should have witnessed something interesting by now. Perhaps I am a curmudgeon, but even in that case I think that there should at least be something else that's taking the world by storm and I am just not seeing it either.

Maybe there are some cool LLM creations, I think the speed at which it creates is also its own weakness. If anyone can create things really quickly, that includes people who are lazy and lack imagination, so then we're inundated with so much slop. Increasing accessibility does not mean that the quality will be maintained. Much like National Parks, if we make sensitive wilderness areas as accessible as possible that will degrade the quality significantly with the arrival of so many tourists, thus the parks have to consider the balance between preservation and conservation. Therefore, it was better when art et al. required some time, intention and skill before it could be shared with the world. These restrictions inherently kept the balance of quality much like how limits of accessibility in National Parks preserve the natural beauty.

MaKey [3 hidden]5 mins ago
I can recommend the Unhook browser extension. My YouTube start page is empty, I have to actively seek out videos I want to watch.
jaggederest [3 hidden]5 mins ago
Agreed, I think AI is the bees knees but I already spend 6+ hours a day thinking about it, more is unhelpful. I wish for something with the quality of discourse of HN but without being so monomaniacally focused on tech-du-jour.
dotancohen [3 hidden]5 mins ago

  > I wish there was a place I could go for tech news that just didn't talk about AI.
Just write an agent to parse HN. Oh wait...
darkmarmot [3 hidden]5 mins ago
It's like it turns everyone into a CEO with all the yes men they could ever ask for! Never goes wrong!
romaniitedomum [3 hidden]5 mins ago
> This suggests that people are drawn to AI that unquestioningly validate, even as that validation risks eroding their judgment

The obvious counterpoint is the large number of complaints about OpenAI's excessively sycophantic models back in April 2026, which led to them rolling back to a previous and less sycophantic model. I think there are those who are susceptible to model sycophancy, but it's definitely not most users. I have a vague, half-formed intuition that many of those susceptible to sycophancy are those who use AI for non-technical, non-work uses, such as inter-personal relationship questions.

As for me, as soon as I see any hint of sycophancy in a response I either stop what I'm doing or I tell the model to stop being an idiot.

recursive [3 hidden]5 mins ago
People are drawn to nicotine use. And still people complain about smoke.
Lutger [3 hidden]5 mins ago
I think the conclusions are a bit overstated and lopsided. Regardless of the sycophancy (which you can tune), LLMs have infinite patience and attention to details. They also have very little inherent bias - it just mirrors your own mostly. They are, in those aspects, vastly superior to even the best therapists, who can't listen for even a couple of minutes and can't think outside of their limited frames. They are only human.

Try listening to somebody and merely repeating the literal words that they say. I guarantee you, most people can't even reproduce two or tree sentences correctly. And I am not even talking about understanding the words, just the literal reproduction of them, copy-pasta. Humans just can't do it, we are not good at it.

Talking to an AI about your problems is a bit like talking to yourself, but with the added superpower of hours of google searches compressed in seconds. Of course it is dangerous and incomplete, but in a way it also beats talking to humans if you know what you are doing and are looking to and able to solve your problems yourself.

altmanaltman [3 hidden]5 mins ago
> They are, in those aspects, vastly superior to even the best therapists, who can't listen for even a couple of minutes and can't think outside of their limited frames.

You start your comment saying the conclusions are overstated and then just drop this conclusion without any proof except anncedotal prespective.

Kattywumpus [3 hidden]5 mins ago
I don't know. Upvotes often do the same thing, especially in siloed communities. I feel like this is a larger problem -- people leaning into extremifying their views or just playing to an audience -- even though one might frame it as "prosocial" because it happens within the context of an online community.
matheusmoreira [3 hidden]5 mins ago
> Upvotes often do the same thing, especially in siloed communities.

Yeah. I've gotten caught in this pattern here on HN. Post something, there's some disagreement but then I see the quiet upvotes trickling in, sometimes by the dozen. That sort of signal slowly convinced me I was right about plenty of stuff.

I used AI to datamine all of my HN comments and find those patterns. Then I started asking the AI to steelman the hell out of my world view, every single argument.

gryfft [3 hidden]5 mins ago
How would you say your views have changed since then?
matheusmoreira [3 hidden]5 mins ago
Biggest change was brazilian vs american laws. I had more knowledge of and respect for US laws than those of my own country, and Claude essentially fixed that. I started studying the law with Claude whenever I'd get into some argument involving copyright, and it started picking up on this pattern and showing me counterpoints where brazilian law precisely defined things that americans had to pay lawyers to go to court and get a judge to create precedent for.

AI measurably lowered the number of absolute claims I've been making. Tended to generalize and radicalize too much which would invite people to respond to the tone and the extremism instead of disproving the underlying idea. That sort of reply would actually reinforce my beliefs: if I was wrong, people would simply refute it. AI attacked my points with counterexamples until I was able to refine them into a core that's actually true.

These little distilled kernels of truth were the most valuable results. I found that many of my arguments actually have an underlying idea that holds up under scrutiny, but nobody cares because I try to prove it via extreme arguments. One such idea is anti-impossibility: laws and rules are all fine, but making it structurally impossible for people to disobey them is unacceptable. I used to try to argue that idea by defending running red lights which persuaded no one. Now I've identified the real idea and can make a much better argument for it.

smcg [3 hidden]5 mins ago
Both could be true. People follow incentives.
accountrequired [3 hidden]5 mins ago
You're absolutely right!
pwdisswordfishq [3 hidden]5 mins ago
It's a pretty overdone joke at this point, don't you think?
tom_ [3 hidden]5 mins ago
It's not a joke-it's an honest fact.
jihadjihad [3 hidden]5 mins ago
I dunno. The seams of the joke are still load-bearing IMO.
almostdeadguy [3 hidden]5 mins ago
You're right to push back. And honestly, this reframes the entire thing.
accountrequired [3 hidden]5 mins ago
You made an excellent point! (could not resist that one)
alexalx666 [3 hidden]5 mins ago
any close relationship forms a dependence. If you talk to something that poses as a helpful human 8 hours a day to do your job, you already spending more time with this "person" than with your loved ones
roughly [3 hidden]5 mins ago
This feels like one of those articles that’s gonna come to mind again and again over the next couple decades.
lowsong [3 hidden]5 mins ago
In decades to come we view "chatbot" AI as one of the most dangerous inventions of this century. Once regulation catches up and they are outlawed, we'll look back on this period of history with horror.
SwellJoe [3 hidden]5 mins ago
I've said before that in one small way, having access to the sycophantic models is like being a billionaire: You never have to hear "no" or "that's not a good idea".

And, it has the same deleterious effect on your mental health. I could name a bunch of billionaires that behave like sociopaths, and they're often seemingly miserable while doing it. I'm sure some of them started out as sociopaths, but they rarely behaved that way so obviously early in their rise.

LAC-Tech [3 hidden]5 mins ago
I think this goes along way to explain people's very defensive reactions when you are skeptical about Agentic Development.
artnanika [3 hidden]5 mins ago
Those are two different things. Chatgpt being sycophantic while playing the role of a therapist with a teenager is much different from Codex being sycophantic while reviewing user feedback. The former is extremely dangerous for society.
LAC-Tech [3 hidden]5 mins ago
I make no societal judgments; just that the nastiness of the reactions I get suggests something more than technical choice is going on.

I had one prominent "influencer" publicly challenge me to a coding competition over it. It was very weird.

qarl2 [3 hidden]5 mins ago
I'm one of the people who gets accused of this. I don't mind skepticism. But every time I turn around, someone's claiming nobody gets any value from these tools and anyone who says otherwise is deluded or running a scam. That gets really old.
sidrag22 [3 hidden]5 mins ago
I've had one conversation that was sorta like this, it was around the time some kid kept releasing videos of him interacting with a very bad TTS gpt model or something and it would give very very bad answers confidently.

Their frame of reference was that, and this was at a time when opus 4.5 was around. Hard to expect a reasonable conversation when one person has such a different view of what the tools are capable of.

LAC-Tech [3 hidden]5 mins ago
I can believe people get value out of them. I think I do.

But "OH MY GOD EVERYTHING HAS CHANGED THE OLD WAY IS DEAD 10X MORE PRODUCTIVE" - no. If that were true, we'd notice it in the software all around us.

qarl2 [3 hidden]5 mins ago
I think we are noticing it. Are you not?

I've shipped three software projects in the last two months. I shipped zero in the preceding year.

Right now I'm building a system to decompile MAME ROM games into idiomatic JavaScript. It completes one game in about 3 days - complete with extensive commenting. I started it about 2 weeks ago.

I'm not sure "EVERYTHING HAS CHANGED" but if you're not seeing dramatic change, I suspect you're not looking.

Planktonne [3 hidden]5 mins ago
> if you're not seeing dramatic change, I suspect you're not looking

If the people claiming that everything has changed were even a fraction as productive as they think, then it would be visible even to people who weren't looking.

As it is, a lot of people are actively looking but still not seeing it.

qarl2 [3 hidden]5 mins ago
[flagged]
chaps [3 hidden]5 mins ago
"you should probably get your eyes checked."

Don't be a jerk. A lot of people have had bad experiences with these systems from people basically saying exactly what you're saying. This "you should get your eyes checked" mentality is pervasive with the nerds who confidently say that their analysis is solid when it's... not. It's a big problem in communities that have non-conventional videogame puzzles for example. Someone will come in, announce some novel solution.... leading to a four hour fight about the efficacy of LLMs, only to find a mistake in their code the next day.. never to be seen again.

Mind you, I'm not saying that these systems aren't great at reverse engineering and whatnot. They're spectacularly good at that. But RE is a relatively constrained problem because everything is still right in front of you. For complicated problems where the signal is mixed between noise... less so.

qarl2 [3 hidden]5 mins ago
As near as I can tell - your argument boils down to "I've seen other people make mistakes so you must be also."
chaps [3 hidden]5 mins ago
If that's what you got from it, then I hope you have a good life.

Be kinder, friend.

qarl2 [3 hidden]5 mins ago
If you can point to a single error in my argument I would love to talk to you about it.

Otherwise, you have a good life as well.

chaps [3 hidden]5 mins ago
My point is just that you're talking past people and being an asshole in the process. Not everything is about winning an argument. Lower your temperature and you'll have more productive discussions that revolve around disagreements.

Cheers.

qarl2 [3 hidden]5 mins ago
No. I am not talking past anyone. I am providing cogent argumentation and evidence. And I'm being met with nonsense and platitudes.

And... I didn't start it.

Cheers.

EDIT: Good choice.

Planktonne [3 hidden]5 mins ago
I'm pleased for you, but these tools have been out for many, many weeks, and there are many, many people claiming incredible productivity gains.

Where's the rest of it?

qarl2 [3 hidden]5 mins ago
I just showed you something that would have been considered a miracle one year ago.

And your response was "So what - show me another one."

Planktonne [3 hidden]5 mins ago
I'm not disputing that it's a hard thing to do, but we had decompilers before; "miracle" is rather stretching it.

Even granting that framing though, the claimed increase in productivity isn't a one-off, but asserted for everyone using them; you're claiming mass-produced miracles but trying to depend on a single example.

qarl2 [3 hidden]5 mins ago
> but we had decompilers before

Cite one example of a decompiler that provides English names and comments.

fragmede [3 hidden]5 mins ago
overgard [3 hidden]5 mins ago
I'm sure it's useful for that, but we've had pretty good decompilers before AI.
qarl2 [3 hidden]5 mins ago
I would love for you to cite a single one.

It must take a binary executable and derive English names for the memory addresses.

overgard [3 hidden]5 mins ago
I used to use dotPeek all the time back in like 2012. There are also things like Ghidra that the NSA made. I dunno, google them, there are a lot. I'm not saying LLMs aren't useful here, just that it's not a huge deal. Also, why are AI people always making tools to take other people's IP? You are kind of fucking with people's intellectual property without permission it seems like. I'm not saying it's the crime of the century or anything but it's distressing how many AI projects come out of "let me repackage something someone else made"
qarl2 [3 hidden]5 mins ago
> there are a lot

There are none.

Ghidra does not provide semantic names.

dotPeek shows you the symbols that were not stripped from the .NET.

Neither of these can do what I asked.

fragmede [3 hidden]5 mins ago
lol you think he hasn't heard of Ghidra? How stupid do you think he is?
bigstrat2003 [3 hidden]5 mins ago
> if you're not seeing dramatic change, I suspect you're not looking.

Ironically, this is the inverse of the very thing you complained about people doing to you.

qarl2 [3 hidden]5 mins ago
Except I'm providing evidence.
xracy [3 hidden]5 mins ago
I'm seeing more examples of you saying you're providing evidence than of you actually providing evidence. So far I've heard a single anecdote about what you did.

That's a pretty far cry from the "evidence" you claim to have dropped on multiple comments.

qarl2 [3 hidden]5 mins ago
LOL. Providing a counter example is far from just an anecdote.
vhantz [3 hidden]5 mins ago
An anecdote is an anecdote is an anecdote
LAC-Tech [3 hidden]5 mins ago
I think we are noticing it. Are you not?

I am not. Congratulations on your side project.

qarl2 [3 hidden]5 mins ago
Projects, plural.

And weren't you the one who started this complaining of nastiness?

qarl2 [3 hidden]5 mins ago
I'm sorry downvoters - it's true. In his opening statement he complained about nastiness and then became extremely nasty himself, when I had said nothing like that to him. Other than just disagree and provide evidence.
veqq [3 hidden]5 mins ago
> I think we are noticing it. Are you not?

The software around us seems worse than ever, constantly breaking and significantly worse than 1-3 decades ago.

pc86 [3 hidden]5 mins ago
You think the software around today is worse than software in the late 90s?
qarl2 [3 hidden]5 mins ago
I guess my experience is different than yours - and I'm providing evidence.
chrisjj [3 hidden]5 mins ago
> But every time I turn around, someone's claiming nobody gets any value from these tools

Really? I find agreement that gullible users get something they value, even if it is worthless.

atleastoptimal [3 hidden]5 mins ago
This is true, but likely happening in talk therapy too with overly sycyphantic therapists
jalev [3 hidden]5 mins ago
Sure, but there is a marked difference between a therapist and an LLM Chatbot: you can't talk to your therapist for +4h a day, at any time convenient to yourself. It's also not a guarantee you're going to hit upon a therapist that is like that, versus a product that is very literally intended to maximise your usage of it.
paimapi [3 hidden]5 mins ago
a therapist should, if they follow their training, not be a sycophant whatsoever. CBT, for eg, is a multi-step process including Socratic self-dialogue, reframing practices, etc to guide the person towards the development of insight and healthier coping mechanisms
lstodd [3 hidden]5 mins ago
Any good therapist should train you in the methods of coping with and eventually overcoming your problem. This boils down to private grad+postgrad psychology/psychiatry course, because to cope+overcome you have to understand what's going on and there is no one who can do shit but you yourself.

But that is only ~half of job. The other half consists of gently reminding one that not all is lost yet.

There is no LLM that can do this and I think there never will be.

socketcluster [3 hidden]5 mins ago
In my case, I need my AI to be sycophantic because most of my contrarian ideas are correct.

So it's a waste of my time and tokens when the AI keeps making weak arguments which I proceed to destroy one by one.

Usually, what happens is it tries to debunk my statement but then I offer a rebuttal to all of its points, it then concedes to all of my points but each time it offers a new partial rebuttal... Which I proceed to crush... But it keeps coming up with increasingly irrelevant caveats. It goes on for quite some time and at the end I ask it to review our discussion and it admits that its original stance "was not as strong" as it initially claimed but it never fully concedes... Even though I literally destroyed all of its points and every partial rebuttal it tried to come up with... Meanwhile its rebuttals became increasingly nit-picky and distant from the original claims made...

It's like if I'm saying "the ship is sinking, look at all the water in the hull and look at all the water pouring in through that hole" and it's like "oh but this is a small hole and the pump can easily offset it" so then I say "What about this hole over here" "Oh but the pump can still offset the stream from both holes easily" and then I say "What about that third one? And fourth one? And fifth one?" And it's like "The pump can easily offset that" and I'm like "Common! The hull is full of water and the water level is rising, the hull is full of holes; it's not a stretch to suggest that the ship is sinking because of all the holes in it... I shouldn't have to point out the location of every single hole for my argument to start making sense!"

It's not proof by induction but it's probably as close as you can get to it for a topic which lies outside the realm of mathematics!

nchmy [3 hidden]5 mins ago
I have had similar experiences - it'll hold fast on a bunch of awful points that I've already shown, and eventually gotten it to admit, are invalid. Kinda like a human...

Yet, the answer is not sycophancy. It is for it to just be objective, honest, humble etc... Challenge or agree with you when necessary

dbspin [3 hidden]5 mins ago
> most of my contrarian ideas are correct.

My stripper finds me attractive, my therapist is in love with me, and my genius goes unrecognised because of small men intimidated by ideas they cannot comprehend.

I'm really hoping your comment is satire, but assuming it isn't, genuinely it might be time to get some help. Our capacity for self delusion as humans is enormous, and it's only higher in brighter people - particularly autodidacts who are used to being the smartest person they know.

hlynurd [3 hidden]5 mins ago
Sounds like you want a sycophant because you use words like "crush" and "destroy" to describe counterpoints to your arguments (please don't destroy me.)