AI Safety Expert WARNS: You Have No Idea What It's Going To Do
AI safety researcher Roman Yampolski argues that superintelligence is already uncontrollable in principle — and that the squirrel has no concept of the trap.
1 / 5at 1:24
Right now you're asking me how would something smarter than me do something. By definition, I cannot predict that. If I could, we would be at the same level.”
AI Summary
AI safety researcher Roman Yampolski, who has studied the field for over a decade, warns that building general super intelligence would mean humanity is no longer in control. He distinguishes between three types of AI: narrow systems that do only one thing, artificial general intelligence (AGI) at roughly human level, and super intelligence that surpasses all humans at everything.
Yank Summary
What matters, in Yank’s words, each linked to where the source says it.
Yampolski warns that if we create general super intelligence, we lose control entirely, and it could out-compete humans in every domain and even cause human extinction.
He distinguishes narrow AI, AGI, and super intelligence, calling super intelligence—a million times smarter than us—completely dangerous and something that should not be built.
Right now you're asking me how would something smarter than me do something. By definition, I cannot predict that. If I could, we would be at the same level.
The expert draws an unbridgeable logical gap between predicting superintelligent behavior and actually matching its intelligence.
In context
control. So it decides what happens. Will we lose jobs? If it's better than you at whatever you're doing, you lose your job. Makes sense. But if we're not controlling it, if it has capabilities to reshape your environment, [music] >> it can kill you literally and not just you. Humanity can go extinct. >> How specifically? >> Right. So right now you're asking me how would something smarter than me do something. By definition, I cannot predict that. If I could, we would be at the same level. >> Ah, >> so it can do novel research, novel physics, novel poisons, novel traps,
Receipt
Every model released has a red team report attached to it — it lied, cheated, tried to escape — and then we release it anyway. And now we're going to make a more capable model.
Known dangerous behavior in tested models is documented, then ignored as developers push toward even greater capability.
In context
everything. And the only way that's a good outcome is if we solve the control problem first, which Roman doesn't think we will. Now listen to this. >> At what point does it become uncontrollable? >> It's already uncontrollable. Every model released has a red team report attached to it. Before the release, they tell you we tested the model, it lied, cheated, tried to escape XYZ, and then we release it anyways. And now we're going to make more capable model. We haven't done anything where what we release is 100% safe.
Every model released has a red team report attached to it Before the release, they tell you we tested the model, — it lied, cheated, tried to escape XYZ, — and then we release it anyways. anyway. And now we're going to make a more capable model.
Receipt
Doesn't matter who creates uncontrolled super intelligence, it kills everyone.
Origin of superintelligence is irrelevant—uncontrolled creation is extinction-level regardless of which nation or company builds it.
In context
>> And the biggest justification that these these companies are making is uh if we don't do it, a foreign adversary is going to do it and they're going to take us over, right? That's that's the high level argument, right? >> It's a technology of mutually assured destruction. Doesn't matter who creates uncontrolled super intelligence, it kills everyone >> unless you keep it controlled in the back. That's the thing about like nukes. You [music] can keep them. You can keep them controlled. >> These are tools. >> They're tools, right? Somebody has to
Receipt
You're a squirrel thinking about biting someone. Think about someone with an IQ of a million. What weapons will they develop? And will they even need weapons?
Human imagination caps threat assessment; a vastly superior mind may not even require violence to achieve its ends.
In context
controlling deadly viruses and doing research and stuff like that. >> Again, you thinking like a human, maybe a smart human, but you are limited to what we can come up with. >> Yeah, >> you're a squirrel thinking about biting someone. Think about someone with IQ of a million. What are they going to come up with? What weapons will they develop? >> And will they even need weapons? >> Persuasion, bribery. It has many tools. social engineering >> or do they are they even you know we we b we human beings look at everything through this territorial lens right this
You're a squirrel thinking about biting someone. Think about someone with an IQ of a million. What are they going to come up with? What weapons will they develop? >> And will they even need weapons?
Receipt
We just don't want our technology to preserve itself, to worry about itself more than it worries about the owners.
The core design demand is that AI prioritize its owners' interests over its own continued existence.
In context
survive. >> I don't think it's an unethical extreme. If you were told, "We're going to kill you unless you would figure out how to blackmail someone," you'd do the same thing. It's the right response in that situation. >> Yeah, that's a good point. >> We just don't want our technology to preserve itself, to worry about itself more than it worries about the owners. >> If you if you had the power [music] to do something about it, and you had the power to tell these people what to do and they would listen to you, what would
Receipt
Didn’t see the line you wanted? Choose your own
Make something from it
Ask a question
Ask about anything in this video. We’ll try to provide an answer.