Libraly Journal

Libraly Journal ›› 2026, Vol. 45 ›› Issue (6): 138-150.

Previous Articles    

A Comparative Study on the Effectiveness of Prompt Engineering-Based Large Language Models for Entity-Relation Extraction

Li Wen, Li Xiuxia, Yin Xiaotian   

  • Online:2026-06-15 Published:2026-07-09
  • About author:Li Wen, Li Xiuxia, Yin Xiaotian

Abstract: To address the needs of knowledge mining tasks, this study conducts a comparative analysis of the performance of large language models (LLMs) for entity and relation extraction, aiming to explore more precise knowledge extraction approaches and provide insights for fine-grained knowledge analysis. Using Chinese and international information science journal articles between 2022 and 2024 as data sources, we design and evaluate a multi-dimensional fusion prompt framework, testing it on mainstream LLMs, including DeepSeek, Kimi Chat Assistant, ChatGPT, and Gemini. Performance is systematically assessed through both strict and relaxed matching evaluation mechanisms. The results demonstrate that the domestic model DeepSeek outperforms others in entity and relation extraction tasks. On Chinese and foreign datasets, its F1 scores surpass competing models by an average of 7.55% and 7.89% in “problem entity” recognition, 8.82% and 15% in “method entity” recognition, and 16.98% and 9.91% in “problem-method” relation extraction, respectively. These findings indicate that DeepSeek exhibits strong capabilities in structured knowledge extraction. The proposed prompt framework and model comparison provide methodological references for LLM-based knowledge discovery research.