Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems

Understand

Autonomous agents acting in the real-world often operate based on models that ignore certain aspects of the environment.

• The incompleteness of any given model -- handcrafted or machine acquired -- is inevitable due to practical limitations of any modeling technique for complex real-world settings.
• Due to the limited fidelity of its model, an agent's actions may have unexpected, undesirable consequences during execution.
• Learning to recognize and avoid such negative side effects of an agent's actions is critical to improve the safety and reliability of autonomous systems.