WP1 - Management (Lead: INB)

The major objectives of this WP are to ensure the sound management of all aspects of the
project.

WP2 - Signal processing and feature extraction (UPB-LAPI)

The first objective is to improve the quality of the TOF signal by advanced signal
processing. By that it will support the objectives of WP 5 on increased depth resolution,
lower power consumption and lower cost.
The second objective is to develop algorithms appropriate for extracting the needed
information from these images with the constraints imposed by the real time condition. This
includes the identification and computation of appropriate features. The new features will: (i)
increase robustness of tracking by reducing the percentage of time when tracking is lost, (ii)
increase accuracy of tracking, (iii) improve classification results as measured by ROC curves
compared to using intensity-image features. Performance will be measured on test sequences
with known ground truth.
The third objective is to fuse the information from TOF and luminance such as to improve
on the first two objectives. Features based on both intensity and TOF signals will lead to
better than state of the art tracking and recognition performance. Further details on the
objectives and the state of the art for fusion are given in Section 0.

WP3 - Object tracking (DTU - IMM)

The objective of this WP is to develop algorithms for 3D object tracking based on the TOF
signal and based on the fusion of TOF and video signals. Since face tracking is of particular
importance, a further objective is to improve face finding and face recognition by
a) usage of new 3D features based on TOF signals
b) extending 2D intensity-based recognition by providing scale and pose from TOF
camera
A further objective will be the evaluation of multiple-view 3D TOF cameras: By moving the
TOF camera relative to the object, additional geometrical information can be acquired, and
accuracy can thus be improved. A further objective is to analyse the combination of the TOF
tracking algorithms with other approaches. For example, a TOF-based pose estimate may be
used to initialize an active-appearance-model algorithm.

WP4 - Action recognition (INB)

The objective is to develop a pattern-recognition toolbox for automatic action recognition.
The users of the ARTTS toolbox should be able to build action-recognition systems by just
designing a database containing labelled examples of each class. Of particular importance
will be the recognition of simple gestures.

WP5 - 3D TOF systems ( CSEM)

The objective of this WP is to deliver an improved miniaturized 3D-TOF camera that
enables low-cost mass manufacturing and outperforms existing 3D-TOF cameras by a
distinct margin.
Compared to existing 3D cameras, the main features targeted by the novel 3D camera are:
• Miniaturized and low-cost
• Low power consumption
• High modulation frequency
• Multi-user capability
• High optical sensitivity

Special attention will be given to the current main disadvantages of the TOF system, namely
power consumption and depth resolution. At the same time, the already proven
miniaturization and low-cost manufacturing potential of the TOF system will be explored.
Other important features to enable the market entrance such as multi-user capability of TOF
cameras, i.e. several cameras operating in the same scene, will be explored and developed.

The main challenges of TOF measurement systems, as described in [14] will remain key
points of the new TOF system.

WP6 - 3D TOF applications (SMI)

The aim of this WP is to demonstrate how the technology developed in WPs 2 to 5 can be
used to build applications with novel features. Following the recommendations of the ESR, a
multimodal prototype (MP) will be built and presented at trade shows. The MP will exploit
interaction based on object tracking and action recognition.

WP7 - Technical evaluation and assessment (INB)

To test, evaluate, and assess the performance of the algorithms and prototype systems. A
central WP has been defined here to monitor the evaluation procedures that are necessary to
assess the technical progress in the project and to provide feedback that will be necessary for
continuous improvements.

WP8 - Dissemination (INB)

The main objective is to provide appropriate dissemination about the goals and results of the
project.