Augmented Pathology: How AI Gets Trained by the Wrong People
A conversation about how the same ideological monoculture that captured social media is now shaping the humans who train our most powerful AI systems — and why that's a civilizational problem.
1 / 5at 2:06
We're going to produce hyper-powerful avatars of our own flaws.”
We're going to produce hyper-powerful avatars of our own flaws.
In context
outcome than that. Because they're so they're all cut out of the trust and safety group at twitter when he bought it many of them have migrated into these trust and safety groups trust and safety groups at these ai companies and they're they're now setting these policies and doing doing this training so so the the terrifying well the terrifying thing here is that we're we're going to produce hyper powerful avatars of our own flaws laws right and so if you're training one of these systems and you have a variety of domains of right and so if you're training one of these systems
Receipt
If we're going to generate augmented intelligence, we better not generate augmented pathological intelligence.
The central warning of the conversation: careless AI development risks amplifying human dysfunction rather than human capability.
In context
have exactly those characteristics and that is not going to be good and that it's like it you're you're absolutely right to point to this as you know it to point to this as perhaps the serious problem of our times if we're As you know, to point to this as perhaps the serious problem of our times, if we're going to generate augmented intelligence, we better not generate augmented pathological intelligence. And if we're not very careful, we are certainly going to do that, not least because there's way more ways that a system can go wrong than there are ways
Receipt
Deciding by fiat that an AI is correct, then shielding it from market or environmental feedback, is literally the definition of how to make something insane.
Insulating an AI from corrective feedback violates the only known mechanism by which complex systems stay sane over time.
In context
like that is absolutely insane that's absolutely insane i mean all the only all we know that the only way that a complex system can regulate itself across time is is through something like evolutionary competition that's it that's the mechanism and so if you you decide what that this ai is correct by fiat and then you shield it from any possibility of market feedback or environmental feedback? Well, that is literally the definition of how to make something insane. environmental feedback well that is literally the definition of how to make
correct Deciding by fiat and that an AI is correct, then you shield shielding it from any possibility of market feedback or environmental feedback, Well, that is literally the definition of how to make something insane.
Receipt
There's way more ways a system can go wrong than there are ways it can aim upward in an unerring direction.
Error vastly outnumbers correctness in the probability space, making vigilance about AI training failures mathematically urgent.
In context
know, to point to this as perhaps the serious problem of our times, if we're going to generate augmented intelligence, we better not generate augmented pathological intelligence. And if we're not very careful, we are certainly going to do that, not least because there's way more ways that a system can go wrong than there are ways that it can, you know, aim upward in an unerring direction. And so, OK, so why is it these people who were this is so awful? I didn't know that that were. This is so awful. I didn't know that we're,
There's way more ways that a system can go wrong than there are ways that it can you know, aim upward in an unerring direction.
Receipt
You see no ideological competition between Harvard and Yale, the New York Times and the Washington Post, or the major AI labs — what looks like a free market keeps producing cartels.
Apparent institutional diversity masks ideological uniformity, suggesting AI labs will replicate the same cartel dynamic.
In context
step back. You don't see ideological competition between Harvard and Yale, right? You would think that you should because they should compete in the marketplace of ideas. And of course, in practice, you don't see that at all. You see no ideological competition between the New York Times and the Washington Post. You see no ideological competition between the Ford Foundation and any of the other major foundations. They all have the exact same politics. Prior to Elon buying Twitter, you saw no ideological competition between the different social media
York Times and the Washington Post. You see no ideological competition between Harvard and Yale, the Ford Foundation New York Times and any of the other Washington Post, or the major AI labs — what looks like a free market keeps producing cartels.
Receipt
Didn’t see the line you wanted? Choose your own
Make something from it
Ask a question
Ask about anything in this video. We’ll try to provide an answer.