A number of high-potential future technologies, including multimodal interfaces, rely on the sensing, tracking, and recognition of people, of their actions, and the environment. Computervision techniques based on mainly video cameras have advanced the field but there remain some fundamental limitations and difficulties in:

• Inferring 3D structure based on low-cost and small-sized image sensors
• Integrating information from different kinds of sensors
• Tracking of objects in 3D without the use of additional markers
• Design of systems that can learn by example to recognize particular objects and actions.

ARTTS will address the above issues by:

• Establishing the use of a new kind of sensor technology, based on the time-of-flight
(TOF) principle, for Multimodal Interfaces, object tracking, and action recognition
• Improving the sensor and integrating the TOF information with light intensity
information
• Developing new kinds of tracking techniques based on TOF cameras
• Developing new machine-learning techniques for object and action recognition based
on TOF sensors
• Integrating the results into the development of a multimodal prototype