Natural Interfaces to Virtual Characters and Environments

Abstract

Computers are able to generate amazingly visually and aurally realistic
characters and environments, yet our ability as users to interact with these
virtual creations is limited.  Is this constrained interaction so restricting,
that it reduces the applicability of these potentially revolutionary
technologies?  In this talk, I'll present results of two ongoing research
projects in allowing users to naturally interact with virtual characters and
virtual environments.

First, is a project to train first and second-year medical students on medical
communication skills using immersive virtual humans.  Students naturally
interact with projector-based life-sized virtual characters using gestures and
speech.  Multiple studies have confirmed that the level of immersion
contributed significantly to the experience, and participants reported that the
system is a powerful tool for teaching and training.  This system is being
integrated into coursework in the College of Medicine, and future directions
into non-verbal communication and ethnic diversity research will be explored.

Second, is a project which looks to address some of the difficulties with VR
interaction.  Common VR application domains, such as engineering assembly and
servicing, are hands-on tasks, and the principal drawback of virtual models -
that there is nothing there to feel, nothing to give manual affordances, and
nothing to constrain motions - is a serious one for these applications.

Our approach is to use mixed environments that can rapidly incorporates real
objects.  This allows specific objects to be in the virtual environment in a
matter of hours instead of days or weeks.  This incorporation pipeline includes
laser scanning objects, attaching colored-markers, and employing simple color
tracking using numerous web cameras to quickly generate and render virtual
representations of real objects in the virtual environment.  The user
manipulates real objects, such as tools, parts, and physical correlates to
complex virtual models within a VE.  We have partnered with NASA, and aim to
help develop assembly documents from CAD model designs using these mixed
environments.

Biography

Benjamin Lok is an Assistant Professor in Computer and Information
Sciences and Engineering Department at the University of Florida.
He also holds an Adjunct Assistant Professor position in the Department
of Surgical Oncology at the Medical College of Georgia.  His research
interests include Virtual Environments, Human-Computer Interaction and
Computer Graphics.  He received his B.S. (1997) in computer science from
the University of Tulsa and his M.S. (1999) and Ph.D. (2002) from the
University of North Carolina at Chapel Hill.

 For more information, please see http://www.cise.ufl.edu/~lok.