Machine Reading Comprehension and Lexicon-Enhanced BERT Based Named Entity Recognition Ancient Chinese Science and Technology Texts

Abstract

Abstract:

Recognizing named entities in the texts of ancient Chinese science and technology haspresented a unique challenge in recent years. In response we introduce a novel named entity recognitionNER method DLEBERT-MRC which is grounded in a machine reading comprehension MRCframework and utilizes a domain lexicon-enhanced BERT model to extract contextual information fromboth questions and target texts. The introduction of domain-specific lexicons through a bilinear attentionmechanism between transformer layers significantly enriches contextual information. SoftMax is employedin the decoding layer to accurately predict the start and end positions of entities in the input text.Furthermore we constructed an ancient Chinese science and technology NER dataset employing theBIOES scheme. The dataset derived from Baidu Encyclopedia is designed to align with thespecifications of MRC tasks. Experimental evaluation verifies the effectiveness of the proposed methodwhile ablation experiments demonstrate the importance of each component of the model.

Key words:

Named entity recognition, Machine reading comprehension, Domain lexicon enhanced BERT, Ancient Chinese science and technology, Digital humanities

Pan Jun, Xiao Mingcheng, Tao Xiangxing（School of Science Zhejiang University of Science and Technology）. Machine Reading Comprehension and Lexicon-Enhanced BERT Based Named Entity Recognition Ancient Chinese Science and Technology Texts[J]. Libraly Journal, 2025, 44(415): 64-74.

References

[ 1 ] 李明杰. 古代科技文献整理的目标,路径与当前亟需解决的问题[J]. 图书情报研究, 2021(3): 14 21.

[ 2 ] 潘俊,李萌配,王贤明. 应用深度学习的中文命名实体识别研究综述[J]. 数字图书馆论坛,2023,19(5):1 9.

[ 3 ] 孙茂松,黄昌宁,高海燕,等. 中文姓名的自动辨识[J]. 中文信息学报,1995(2):16 27.

[ 4 ] 朱锁玲,包平. 方志类古籍地名识别及系统构建[J]. 中国图书馆学报, 2011, 37 ( 3 ): 118124.

[ 5 ] 皇甫晶,王凌云. 基于规则的纪传体古代汉语文献姓名识别[J]. 图书情报工作,2013, 57(3): 120 124.

[ 6 ] 李娜. 面向方志类古籍的多类型命名实体联合自动识别模型构建[J]. 图书馆论坛,2021, 41(12): 113 123.

[ 7 ] 黄水清,王东波,何琳. 基于先秦语料库的古汉语地名自动识别模型构建研究[J]. 图书情报工作,2015, 59(12):135 140.

[ 8 ] 袁悦,王东波,黄水清,等. 不同词性标记集在典籍实体抽取上的差异性探究[J]. 数据分析与知识发现,2019, 3(3):57 65.

[ 9 ] 严承希,唐雪梅,杨浩,等. HanNER:一个面向汉语古籍语料命名实体自动抽取的通用框架[J].情报学报,2023, 42(2):203 216.

[10] 崔丹丹,刘秀磊,陈若愚,等. 基于LatticeLSTM的古汉语命名实体识别[ J]. 计算机科学, 2020, 47(增刊2):18 22.

[11] 徐晨飞,叶海影,包平. 基于深度学习的方志物产资料实体自动识别模型构建研究[J]. 数据分析与知识发现,2020, 4(8):86 97.

[12] 刘懋霖,赵萌,王昊. 面向古诗词的物象库构建方法及其分布规律研究[J]. 图书馆杂志,2024,43(1):96 108.

[13] Kenton J D M W C Toutanova L K. BERT Pretrainingof deep bidirectional transformers forlanguage understanding C / / Proceedings of the17th Conference of the North American Chapter ofthe Association for Computational LinguisticsHuman Language Technologies 2019 41714186.

[14] Cui Y Che W Liu T et al. Pre-training withwhole word masking for Chinese BERT J . IEEE /ACM Transactions on Audio Speech and LanguageProcessing 2021 29 3504 3514.

[15 ] 王东波, 刘畅, 朱子赫, 等. SikuBERT 与SikuRoBERTa:面向数字人文的《四库全书》预训练模型构建及应用研究[J]. 图书馆论坛,2022,42(6):31 43.

[16] 林立涛,王东波,刘江峰,等. 数字人文视域下典籍动物命名实体识别研究:以SikuBERT 预训练模型为例[ J]. 图书馆论坛,2022, 42 (10 ): 42 50.
[17] 刘江峰,冯钰童,王东波,等. 数字人文视域下SikuBERT 增强的史籍实体识别研究[J]. 图书馆论坛,2022, 42(10):61 72.

[18] Li X Feng J Meng Y et al. A unified MRCframework for named entity recognition C / /Proceedings of the 58th Annual Meeting of theAssociation for Computational Linguistics 20205849 5859.

[19] 范涛,王昊,张卫,等. 基于机器阅读理解的非遗文本实体抽取研究[J]. 数据分析与知识发现, 2022, 6(12):70 79.

[20] 喻雪寒,何琳,王献琪. 基于机器阅读理解的古文事件抽取研究[J]. 情报学报,2023, 42(3):316326.
[21] Liu W Fu X Zhang Y et al. Lexicon enhancedChinese sequence labeling using BERT adapterC / / Proceedings of the 59th Annual Meeting ofthe Association for Computational Linguistics andthe 11th International Joint Conference on NaturalLanguage Processing 2021 5847 5858.

[22] Huang Z Wei X Kai Y. Bidirectional LSTMCRFmodels for sequence tagging EB / OL .2024-03-20 . https / / arxiv. org / pdf / 1508.01991. pdf.

[23] Zhang Y Yang J. Chinese NER using lattice LSTM C / /Proceedings of the 56th Annual Meeting of theAssociation for Computational Linguistic 20181554 1564.

[24] Huang L Yu W Ma W et al. A survey onhallucination in large language models principlestaxonomy challenges and open questions EB /OL . 2024-06-20 . https / / arxiv. org / abs /2311. 05232.