Musings on the Alignment Problem • 459 implied HN points • 29 Mar 22
- The use of reinforcement learning from human feedback (RLHF) has been successful in aligning models with human intent like following instructions.
- Training AI systems on tasks that are hard for humans to evaluate may not be directly solvable with RLHF due to challenges in generalization and evaluation.
- AI-assisted human feedback, like recursive reward modeling (RRM), can help tackle complex tasks by involving human evaluation in aligning AI systems.