Learning High-Quality and General-Purpose Phrase Representations - Télécom Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2024

Learning High-Quality and General-Purpose Phrase Representations

Résumé

Phrase representations play an important role in data science and natural language processing, benefiting various tasks like Entity Alignment, Record Linkage, Fuzzy Joins, and Paraphrase Classification. The current state-of-theart method involves fine-tuning pre-trained language models for phrasal embeddings using contrastive learning. However, we have identified areas for improvement. First, these pretrained models tend to be unnecessarily complex and require to be pre-trained on a corpus with context sentences. Second, leveraging the phrase type and morphology gives phrase representations that are both more precise and more flexible. We propose an improved framework to learn phrase representations in a context-free fashion. The framework employs phrase type classification as an auxiliary task and incorporates character-level information more effectively into the phrase representation. Furthermore, we design three granularities of data augmentation to increase the diversity of training samples. Our experiments across a wide range of tasks show that our approach generates superior phrase embeddings compared to previous methods while requiring a smaller model size. The code is available at https: //github.com/tigerchen52/PEARL

Domaines

Informatique
Fichier principal
Vignette du fichier
eacl-2024.pdf (701.87 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04465022 , version 1 (19-02-2024)

Identifiants

  • HAL Id : hal-04465022 , version 1

Citer

Lihu Chen, Gaël Varoquaux, Fabian M. Suchanek. Learning High-Quality and General-Purpose Phrase Representations. EACL 2024 - The 18th Conference of the European Chapter of the Association for Computational Linguistics, Mar 2024, La Valette, Malta. ⟨hal-04465022⟩
55 Consultations
13 Téléchargements

Partager

Gmail Facebook X LinkedIn More