A thought experiment concerning the problems with AI alignment
Thought Experiment
- Suppose you are an 8-year old and your parents left you a trillion dollar company with no adult to serve as a guide to the world
- You must hire an adult to run the company as a CEO
- Candidates include:
- Saints - people who genuinely want to help you and look out for long-term interests
- Sycophants - people who want to make you happy just to satisfy your instructions, regardless of long-term consequences
- Schemers - people with their own agendas who want to abuse power for their own gain
- You have no way to know if you are hiring a sycophant or schemer