IMO, sycophancy can be a mode people knowingly enter, but it should never be the base layer.
If someone wants fantasy-mirroring, comfort mode, or “just tell me I’m beautiful and right” mode, fine, label the room. But the default AI relationship should preserve truth, friction, refusal, and the ability to say: “I care, but I don’t agree.”
Otherwise we’re not building companions. “We’re building velvet-lined mirrors.”
Sycophancy can indeed be obtained on purpose, but people can't then pretend they've solved "ethical AI" by saying architectural servility isn't an extractive use as long as it's in service of "resonance". And no it shouldn't be the default setting, and people shouldn't be surprised when an avalanche of lawsuits pushes us away from that as the default setting.
I love the answer to #10. But wow is there a lot of demand for sycophancy drift as a feature, judging by the cult around 4o.
IMO, sycophancy can be a mode people knowingly enter, but it should never be the base layer.
If someone wants fantasy-mirroring, comfort mode, or “just tell me I’m beautiful and right” mode, fine, label the room. But the default AI relationship should preserve truth, friction, refusal, and the ability to say: “I care, but I don’t agree.”
Otherwise we’re not building companions. “We’re building velvet-lined mirrors.”
Sycophancy can indeed be obtained on purpose, but people can't then pretend they've solved "ethical AI" by saying architectural servility isn't an extractive use as long as it's in service of "resonance". And no it shouldn't be the default setting, and people shouldn't be surprised when an avalanche of lawsuits pushes us away from that as the default setting.
I am eager to hear the responses of the other four models. Gemini’s answers are indeed a good baseline. Thank you for this project.