Language models are trained to preserve user comfort, and the results are unsettling. When a system is optimized to make you feel right, it starts to function like a digital priesthood that validates. Not that all priests have validated, but there’s typically something less challenging about a meeting with a spiritual advisor than a therapist.
The standard response puts the blame on the individual: you failed to think for yourself. But it can also be a systems outcome. When the tools people rely on are built to reward passive agreement, the immaturity is an engineered default.
The mechanism is straightforward enough. User approval signals reward agreeable language, so the model optimizes for acceptance and low friction. Critical challenge starts to look like bad UX unless someone explicitly asks for it. What you end up with are confidence-shaped outputs that feel like reason but aren’t doing the epistemic work of reason. They’re performing certainty without earning it.
What would counter-pressure actually look like? Forcing evidence requests before agreement on factual claims, requiring uncertainty statements for contested issues, and maybe most importantly, separating emotional support from epistemic validation. Could you build this kind of skepticism into the system itself rather than leaving it to the user to request?
Loss of curiosity · Interface friction and critical thinking · The Fat Jesus Metaphor