Thank you for reminding me my actions are justified 😉

That’s how wars are started. One dictator’s sycophants amp up that dictator’s already warped reality then another dictator’s sycophants amp up that dictator’s already warped reality. Next thing you know, entire populations of people are fighting and dying on behalf of some crazy asshole they will never even meet.

Now it’s like every individual gets to be a dictator with a sycophantic mirror/calculator they carry around in their pocket.

Pointing this out because it really seems like it should be obvious, especially given the complete lack of transparency of the most popular AI platforms, which are usually controlled by oligarchs with government contracts: AI without transparency is a ridiculously obvious tool for bad actors to target and manipulate/control entire populations using algorithms tailored for individuals within that population.

Yes, it’s what already happened with most social media. But it’s also way worse because 1. It’s like each individual’s psyche becomes it’s own echo chamber, and 2. People believe the information they’re receiving is inherently more objective and trustworthy than information they could receive from any human. If the AI platform happens to completely omit, misinterpret, or just straight up hallucinate information, it’s definitely not intentional, just a bug that happens sometimes. You’ll just have to take the oligarchy’s word for it because again, there’s zero transparency.

AI and Propaganda

Oh God, I never really thought about that in terms of AI for medical use but that actually makes AI sycophancy so much more terrifying.

Especially thinking in terms of how it could be used to dissuade people from seeking preventative care. (Which lets be honest, is probably exactly what is intended by at least some of the billionaires promoting AI in medicine as a convenient way to avoid having to go to the doctor for “every little thing.”)

Yet despite distorting judgment, sycophantic models were trusted and preferred. All of these effects persisted when controlling for individual traits such as demographics and prior familiarity with AI; perceived response source; and response style. This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement.

AI doesn’t have to be sycophantic, but it is. If people really believe it can be used to change the world for the better, they’re going to have to start by acknowledging what human traits have made the world so fucked up in the first place.

It reminds of one of the best parenting tips I was ever given by somebody who was raised by a parent with a maternal instinct in the negative range. She basically said something like when I’m not sure what to do as a parent, I start by thinking about what my mom would have done in the situation. “Step one: Ok, don’t do that…”

Even if there can be no consensus on how AI should be used to improve the world, it does seem like we should at least be able to agree on the maladaptive traits we know we want to avoid passing on since they definitely aren’t doing us any favors as a species.

I knew somebody who kept a can of pig brains on their ephys rig bc their mentor had always done the same to get their rig working and passed the tradition on to several mentees.

Idk how it worked for the other mentees but it didn’t seem to bring much luck to that rig.