Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation - Télécom Paris Access content directly
Journal Articles IEEE/ACM Transactions on Audio, Speech and Language Processing Year : 2021

Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation

Abstract

The goal of singing voice separation is to recover the vocals signal from music mixtures. State-of-the-art performance is achieved by deep neural networks trained in a supervised fashion. Since training data are scarce and music signals are extremely diverse, it remains challenging to achieve high separation quality across various recording and mixing conditions as well as music styles. In this paper, we investigate to which extent the separation can be improved when lyrics transcripts are used as additional information. To this end, we propose a joint approach to phoneme level lyrics alignment and text-informed singing voice separation. It is based on DTW-attention, a new monotonic attention mechanism including a differentiable approximation of dynamic time warping. Experimental results show that the method can align phonemes with mixed singing voice with high precision given accurate transcripts. It also achieves competitive results on challenging word level alignment test sets using less training data than state-of-the-art methods. Sequential alignment and informed separation lead to improved separation quality according to objective measures. Text information helps preserving spectral phoneme properties in the separated voice signals.
Fichier principal
Vignette du fichier
2021_Phoneme_level_lyrics_alignment_and_text-informed_singing_voice_separation.pdf (2.18 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03255334 , version 1 (03-08-2021)

Identifiers

Cite

Kilian Schulze-Forster, Clement S J Doire, Gael Richard, Roland Badeau. Phoneme Level Lyrics Alignment and Text-Informed Singing Voice Separation. IEEE/ACM Transactions on Audio, Speech and Language Processing, In press, ⟨10.1109/TASLP.2021.3091817⟩. ⟨hal-03255334⟩
420 View
927 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More