26-02-2011, 02:17 PM
[attachment=9177]
Robust Audio-Visual Identity Verification
Objective
Identity verification using talking-faces
Providing:
– Robustness to sophisticated forgeries
– Robustness to degraded conditions
Methods to develop:
Lips features extraction, tracking, dynamic modelling and Fusion
Voice conversion and Face animation cloning
Robust audiovisual processing for speaker recognition
Robustness to pose changes for face recognition
Methods developed
– Face animation tracking
• Facial segmentation with IOF-ASM
• Facial Features Tracking
– Lip feature extraction
• Color based lip segmentation
• Motion based mouth opening tracking
– Audio-visual synchrony
• Canonical Correlation and Co-inertia Analysis
• Coupled and Uncoupled Hidden Markov Models
– Objectives
– Investigate imposture techniques
• impostors try to defeat audiovisual identity verification system
– Evaluate/improve robustness to attacks
– Methods proposed
– Face motion detection
• Facial segmentation with IOF-ASM
• Facial Features Tracking
– Audio-video synchrony detection
• Color based lip segmentation
• Features based mouth segmentation
• Motion based mouth segmentation
Facial Segmentation with IOF-ASM
• Tested on a set of 200 Videos of isolated digits
• Initialization
– Viola-Jones based Face Detector
• Segmentation methods
– Active Shape Models (ASM)
– Invariant Optimal Features (IOF) – ASM
• Key points
– Invariant to 2D Rotations
– More accurate than ASM
– Allows a trade-off between accuracy and speed
Facial Features Tracking
• Goal: Track a set of facial landmarks through a video-sequence.
• Technique: Inverse Compositional Image Alignment (ICIA) [1] algorithm.
• Current progress: 104 videos from the controlled sessions of the BANCA database have been tracked. Manual initialization is needed in the first frame