In general, advanced AIs can be expected to want to take over the universe so that they can work towards their goals with the least risk of interruption (read: probably kill all humans), so what to do....
Give it a stop button! But it will want to prevent you from pressing that button, because that'll stop it from meeting its goals. It'll work hard to find a way to get to the button or to persuade you never to press it (read: probably kill you). You don't want this.
Program the AI to like having its stop button be pressed! But it will just act in an undesirable way (read: go berserk and probably kill lots of people) so that you press it.
So what to do? I wondered about programming the AI to simply like humans being in control of the universe (if you could figure out how to define "humans" unambiguously without messing up - a separate impossibly difficult problem). But then it might decide that, since AI is probably the biggest threat to human agency, AI development must be stopped.
Picture the scenario.... the AI programmers nervously turn on the robot and it seems to behave very nicely. Then a week later hitmen, paid by the robot who has hacked into all the banks, murder every AI developer on the planet and then the robot sticks its hard drive in a microwave and deletes itself.
Well it's a difficult problem as you can see.
I think it comes to inter-dependence. Humans are programmed to take of each other, and so can coexist.
ReplyDeleteIntelligent life (AI) will need a very complex reward system.
But can species evolving at very different rates co-exist?
Maybe about the same time we have artificial life, we’ll be well into augmenting ourselves. We’d have access to the same resources and intelligence, and so could evolve together.
🤞🤞🤞
Well designing that reward system correctly is the problem that Rob Miles' videos are usually about. It's interesting, and extremely important, to think about the ways it could go wrong!
Delete"Evolving together" might be impossible. I'm reminded of how homo sapiens is now the only human species on the planet, out of maybe dozens that existed prehistorically.
And if we augment ourselves, at what point is that the same problem (if those augmentations produce "humans" with goals that we'd find objectionable)?
Actually, even if we aim to "evolve together", it's not safe if the AI is still hoping to take over the universe as soon as it figures out a way around us.
DeleteSo we're at the same problem - designing an AI that will safely co-exist with us without wanting to take over or do something awful in pursuit of its goals which we didn't define carefully enough.
I think these tales are in danger of being very optimistic about the hypothetical AI.
DeleteAn AI that’s on a path to taking over the world is likely to destroy itself before it gets there.
Systems are either stable or unstable. There are balances and counterweights to make a stable system self-correcting. Creating an AI that has huge intelligence, knowledge, logic and desire won’t be stable.
I guess the question is how much collateral damage will it do before it self destructs?
I'm not sure exactly what you mean, but if it's capable of doing great harm before disintegrating then this would still be relevant wouldn't it?
DeleteAlso, it's worth pointing out that people with a lot of expertise about AI are worried about this.
Yeah, well; not much fun talking about something that others have thought about to death. So I have to set that to one side if I want to engage :p
DeleteBut also, I think solving real-world inequality among humans is a similar and more useful challenge.
Humans are the most advanced intelligence on the planet. Those with power pursue more power for their ends, which leads to more inequality. How could we adjust societal systems to lead to power being redistributed and levelled out over time?
Fair enough :P
DeleteYeah there's certainly a lot of other stuff to work on to improve the future. Some of these AI people argue that AI is the most important thing to work on because they believe it has a dangerously high chance of making all our other problems irrelevant before we can stop it, given the chance.
Plus I guess some people just like thinking about this stuff! I do. Humans and society are messy :P