Design approach to a working virtual conversational patient for medical education

Thomas B. Talbot, MD, MS, FAAP  Principal Medical Expert, USC Institute for Creative Technologies  Associate Research Professor of Medical Education, Keck School of Medicine of USC

Presentation Summary

Our team set out to create a conversational virtual patient that would be efficacious and practical.  In order to do this, a number of challenges had to be overcome and technologies created:   emotionally expressive avatars, natural language understanding, conversational AI and semantics, robust conversational interaction, authorability, a universal patient model, performance assessment and constructive feedback.    In the end, we succeeded in the creation of a universal patient model that featured a compelling interaction, high natural language understanding performance, very rapid authorability by non-programmer educators, good usabilty, accurate assessment and a very strong training effect proven in a controlled study.

The path to creating this set of capabilities required varying computing and algorithmic approaches that had to successfully integrate as subsystems without compounding errors in the larger patient model. We discuss how we employed several simple models along with more complex systems to create a functional prototype. We specifically discuss the general design of an modular interactive virtual human, how we modified into a virtual patient and how we integrated a unified medical taxonomy onto this system.     

USC Standard Patient Prototype

Unified Medical Taxonomy

Structure of an Interactive Virtual Human