Afficher la couverture 'Automatic Text Evaluation through the Lens of Wasserstein Barycenters | Colombo, Pierre' en grand format
/5

0 avis

Automatic Text Evaluation through the Lens of Wasserstein Barycenters

Archive ouverte

Colombo, Pierre | Staerman, Guillaume | Piantanida, Pablo | Clavel, Chloé

Edité par CCSD -

International audience. A new metric BaryScore to evaluate text generation based on deep contextualized embeddings (e.g., BERT, Roberta, ELMo) is introduced. This metric is motivated by a new framework relying on optimal transport tools, i.e., Wasserstein distance and barycenter. By modelling the layer output of deep contextualized embeddings as a probability distribution rather than by a vector embedding; this framework provides a natural way to aggregate the different outputs through the Wasserstein space topology. In addition, it provides theoretical grounds to our metric and offers an alternative to available solutions (e.g., MoverScore and BertScore). Numerical evaluation is performed on four different tasks: machine translation, summarization, data2text generation and image captioning. Our results show that BaryScore outperforms other BERT based metrics and exhibits more consistent behaviour in particular for text summarization.

Suggestions

Du même auteur

Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models

Archive ouverte | Colombo, Pierre | CCSD

International audience. Proprietary and closed APIs are becoming increasingly common to process natural language, and are impacting the practical applications of natural language processing, including fewshot classi...

Heavy-tailed Representations, Text Polarity Classification & Data Augmentation

Archive ouverte | Jalalzai, Hamid | CCSD

The dominant approaches to text representation in natural language rely on learning embeddings on massive corpora which have convenient properties such as compositionality and distance preservation. In this paper, we develop a nov...

Guider l'attention dans les modèles de séquence à séquence pour la prédiction des actes de dialogue

Archive ouverte | Chapuis, Emile | CCSD

International audience. La prédiction d’actes de dialogue (AD) basés sur le dialogue conversationnel est un élément clé dans le développement des agents conversationnels. La prédiction précise des AD nécessite une m...

Chargement des enrichissements...