Lip Animation Synthesis: a Unified Framework for Speaking and Laughing Virtual Agent - Télécom Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2015

Lip Animation Synthesis: a Unified Framework for Speaking and Laughing Virtual Agent

Résumé

This paper proposes a unified statistical framework to synthesize speaking and laughing lip animations for virtual agents in real time. Our lip animation synthesis model takes as input the decomposition of a spoken text into phonemes as well as their duration. Our model can be used with synthesized speech. First, Gaussian mixture models (GMMs), called lip shape GMMs, are used to model the relationship between phoneme duration and lip shape from human motion capture data; then an interpolation function is learnt from human motion capture data, which is based on hidden Markov models(HMMs), called HMMs interpolation. In the synthesis step, lipshapeGMMs are used to infer a first lip shape stream from the inputs; then this lip shape stream is smoothed by the learnt HMMs interpolation, to obtain the synthesized lip animation. The effectiveness of the proposed framework is confirmed in the objective evaluation.
Fichier non déposé

Dates et versions

hal-02412183 , version 1 (15-12-2019)

Identifiants

  • HAL Id : hal-02412183 , version 1

Citer

Yu Ding, Catherine Pelachaud. Lip Animation Synthesis: a Unified Framework for Speaking and Laughing Virtual Agent. FAAVSP - The 1st Joint Conference on Facial Analysis, Animation and Auditory-Visual Speech Processing, Sep 2015, Vienna, Austria. pp.78-83. ⟨hal-02412183⟩
41 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More