Techniques we can use to align AI systems more intelligent than us. We use AI systems to supervise, critique and improve other systems. Methods Iterated Distillation and Amplification Task Decomposition ExIt RRM Constitutional AI AI Debate Reinforcement Learning Weak to Strong Generalization