Recently, the team of Professor Ping Wang, a researcher at the National Archives Administration Key Laboratory of Intelligent Development and Service of Archives and a professor at the School of Information Management, Wuhan University, presented at the Annual Meeting of the Association for Computational Linguistics ACL 2024 a paper titled "Hypergraph based Understanding for Document Semantic Entity Recognition", reporting their latest research on document semantic entity recognition.. The ACL Annual Meeting is the top-ranked international academic conference in the fields of computational linguistics and natural language processing, and is also rated by the China Computer Federation (CCF) as a Class A conference. The first author of the paper is Qiwei Li, a 2023 master's student in Artificial Intelligence at Wuhan University; Wuhan University is the first affiliation of the paper.
With the development of information technology, documents have become the main carriers of information in today's society, containing multiple types of information such as text, tables and images. Manually recognizing these documents often requires a great deal of human effort. OCR tools can only help us recognize text, layout and other simple information in documents. To further understand documents, some researchers have proposed visually rich document understanding (Visually-rich Document Understanding, VRDU), which aims to use visual, textual and other information for a deeper analysis and understanding of documents. Semantic entity recognition (Semantic Entity Recognition , SER) is VRDU an important task in this field. Its goal is to extract and classify text with special semantic information in documents. Existing document understanding models mainly focus on entity categories while neglecting the extraction of entity boundaries.
Figure 1: Traditional semantic entity recognition and hypergraph semantic entity recognition
This paper builds a novel hypergraph attention (Hyper-Graph Attention, HGA) framework for document semantic entity recognition, which uses hypergraph attention to attend to both entity boundaries and entity categories, achieving more accurate entity recognition. Given the text token sequence
and its transformed high-dimensional feature sequence
, the feature corresponding to each token
is regarded as a node in the document. By constructing hyperedges corresponding to different labels among the nodes, semantic entities can be extracted and classified. The method architecture is shown in Figure 2. The hypergraph attention head first performs self-attention computation on the high-dimensional feature sequence to build a node relation matrix, and then constructs hyperedges and recognizes different semantic entities by locating the start and end positions of semantic entities in the matrices corresponding to different labels. During score computation, span position encoding supplements the two-dimensional positional span information in the document to assist semantic entity recognition. We also designed a balanced hyperedge loss to address the imbalance between positive and negative samples caused by a large number of label types.
Figure 2: The hypergraph attention head HGA framework
Based on GraphLayoutLM, we trained our hypergraph attention recognition model HGALayoutLM, and, using the English datasets and the Chinese dataset, conducted relevant experiments., The experimental results verify the effectiveness of the HGA method. Meanwhile, we also conducted experiments on the dataset with more label types CORD (30 label types) to test the effect of the balance factor
in the balanced hyperedge loss. The experimental results show that when is 0.4, the model achieves the best results. As the value of deviates further from 0.4, the model performance gradually declines and even falls below the original results. This demonstrates the effectiveness of the balanced hyperedge loss.
This research was supported by the National Natural Science Foundation of China, the Natural Science Foundation of Hubei Province, the Fundamental Research Funds for the Central Universities and other projects.
