Ontological Crises in Artificial Agents' Value Systems

Understand

Decision-theoretic agents predict and evaluate the results of their actions using a model, or ontology, of their environment.

• An agent's goal, or utility function, may also be specified in terms of the states of, or entities within, its ontology.
• If the agent may upgrade or replace its ontology, it faces a crisis: the agent's original goal may not be well-defined with respect to its new ontology.
• This crisis must be resolved before the agent can make plans towards achieving its goals.