Knowledge Base ValidationOne of my long term goals is to develop software programs, or agents, that act intelligently--performing tasks faithfully on our behalf. As we ask these agents to operate in increasingly complicated environments, the task of correctly specifying the parameters for correct behavior becomes more and more difficult. As a result, agents that simulate human-level behavior within even modestly complicated domains are time-consuming to develop. One of the most significant bottlenecks in the development process is due to the fact that it is very difficult to ensure that an agent will behave appropriately in its environment without undergoing a lengthy and expensive proving process that involves manually observing the agent's behavior while it performs its task in a wide range of environmental circumstances. My research on KB Validation seeks to reduce the cost of this process by exploring automated methods to identify and report the similarities and differences between an agent's and a human's behavior. Once created, these reports can be assessed by human experts to isolate potential problem areas in the agent's knowledge. In this manner, we seek to identify semi-automated approaches that benefit from the speed of automation and the robustness of human-oversight. Selected Papers Scott A. Wallace. Behavior Bounding: An Efficient Method for High-Level Behavior Comparison. Journal of Artificial Intelligence. 34: 165-208. 2009. Scott A. Wallace.
Identifying Incorrect Behavior: The Impact of Behavior Models on Detectable Error Manifestations. Fourteenth Conference on Behavior Representation in Modeling and Simulation (BRIMS-05). 2005.
Scott A. Wallace and John E. Laird.
Comparing Agents and Humans
Using Behavioral Bounding. International Joint Conference on
Artificial Intelligence (IJCAI-03). pp. 727-732. August 2003.
Scott A. Wallace and John E. Laird. Toward Automatic Knowledge Validation.
In Proceedings of the Eleventh Conference on Computer Generated Forces and Behavioral
Representation. pp. 447-456. May 2002.
Safe AgentsA natural extension to my work in Knowledge Base Validation, is to push validation capabilities into the agent itself to support self-assessment. In contrast to standard agents which assign preferences to actions based solely on their own knowledge, self-assessment agents continuously monitor (or assess) their actions to ensure that external constraints are maintained. In essence, a self-assessment agent attempts to satisfy its own agenda while complying with a mandate given by its designer or some other external source. By performing real-time validation of the agent's behavior as it interacts with the environment, the self-assessment framework provides a natural mechanism for ensuring safe behavior at a relatively high-level of abstraction. However, the possibilities for the self-assessment framework extend far beyond validation and safety assurance. The same mechanisms used for ensuring safety could be used for tasks such as ensuring social compliance or producing variable degrees of autonomy. Selected Papers
Scott A. Wallace. S-Assess: A Library for Self-Assessment. In Proceedings of the Fourth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-05). pp. 256-263. Utrecht, The Netherlands. 2005.
Scott A. Wallace. Abstract Behavior
Representations for Self-Assessment. AAAI Spring Symposium on
Meta-Cognition in Computation (ASSMC 2005). AAAI
Technical Report SS-05-04. pp. 120-125. Palo Alto, CA. 2005.
|