Cargando...
Fecha
2025-07-01
Derechos de acceso
info:eu-repo/semantics/openAccess
Título de la revista
ISSN de la revista
Título del volumen
Editorial
Association for Computational Linguistics
Citas
Resumen
Various metrics exist for evaluating sequence labeling problems (strict span matching, token oriented metrics, token concurrence in sequences, etc.), each of them focusing on certain aspects of the task. In this paper, we define a comprehensive set of formal properties that captures the strengths and weaknesses of the existing metric families and prove that none of them is able to satisfy all properties simultaneously. We argue that it is necessary to measure how much information (correct or noisy) each token in the sequence contributes depending on different aspects such as sequence length, number of tokens annotated by the system, token specificity, etc. On this basis, we introduce the Sequence Labelling Information Contrast Model (SL-ICM), a novel metric based on information theory for evaluating sequence labeling tasks. Our formal analysis and experimentation show that the proposed metric satisfies all properties simultaneously.
Descripción
Categorías UNESCO
Palabras clave
Citación
Enrique Amigo, Elena Álvarez-Mellado, Julio Gonzalo, and Jorge Carrillo-de-Albornoz. 2025. Evaluating Sequence Labeling on the basis of Information Theory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27849–27860, Vienna, Austria. Association for Computational Linguistics.
Centro
E.T.S. de Ingeniería Informática
Departamento
Lenguajes y Sistemas Informáticos