3D Head Rigid Motion Estimation
Aim
Estimate the 3D rigid motion of a moving head from a monocular sequence.
Description of the problem
Let's assume a person that faces a camera and occasionally moves his
head. The real motion of the head can be analysed to rigid and
non-rigid
motion. The rigid motion can be described by 6 parameters: a 3
dimensional translation vector T and a 3x3 rotation matrix R with 3
degrees of freedom:
The problem that was tackled was to estimate the parameters of the 3D
motion for every pair of consecutive frames.
Applications
A automated system that can estimate the 3D motion parameters from a
monocular sequence of a moving head can be used for telecommunication
applications (e.g. video-conference using MPEG4 compression) or
entertainment purposes (controlling virtual 3D characters)
Approach
The 3D rigid motion parameters are estimated using a two second
approach. First, a set of image features are tracked and for each
feature and each pair of consecutive frames, a 2D translation vector is
estimated. Then, for each frame, the 3D motion parameters are
calculated, using all the feature translation vectors. The features are
linked to 3D head model which is assumed adapted at the first frame of
the video sequence. It is also assumed that only rigid motion is
observed.
The features are tracked using the Kanade-Lucas-Tomasi
(KLT) algorithm.
Features are assessed according to their trackability, dissimilarity
and 3D orientation.
An initial estimate of the 3D motion parameters is calculated by an
enhanced version of the 8-point algorithm. A final estimate is provided
by the iterative algorithm Levenberg-Marquardt that minimises the
distances between the tracked features and the projections of the their
associated positions of the 3D model.
The motion parameters are applied to the 3D model and its projection is
drawn onto the image to validate the result. Finally, the facial
features are re-positioned according to the projection of the 3D model.
A summary of the method is given by the following diagram
Publications
N. Sarris, D. Makris, M. G. Strintzis, "Three
Dimensional Model Based Rigid Tracking of a Human Head",
International Workshop on Intelligent Communication Technologies and
Applications with Emphasis on Mobile Communications, May, Neuchatel,
Switzerland, (1999) abstract download
About this work
The above summarises Dimitrios
Makris' final year thesis for the Greek
Diploma. It took place at Aristotle University of Thessaloniki, Greece,
during the academic year 1998-1999.