Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safety

Understand

The rapid advancement of artificial intelligence (AI) systems suggests that artificial general intelligence (AGI) systems may soon arrive.

• Many researchers are concerned that AIs and AGIs will harm humans via intentional misuse (AI-misuse) or through accidents (AI-accidents).
• In respect of AI-accidents, there is an increasing effort focused on developing algorithms and paradigms that ensure AI systems are aligned to what humans intend, e.g.
• AI systems that yield actions or recommendations that humans might judge as consistent with their intentions and goals.