高级检索

大模型文本简化的篇章结构变化分析与评测

Discourse Structure Variation for Large Language Model Evaluation in Text Simplification

  • 摘要: 该文基于修辞结构理论,从篇章层面对大语言模型(GPT-4o)在中文新闻文本简化任务中的表现进行了系统评估。研究以篇章树与语义关系描述篇章结构,在新闻文本上构建了篇章结构标注数据集,探索性地提出了九项用以表征文本简化性质的篇章结构量化指标,对比、分析了大模型与专家简化文本在篇章结构上的差异。研究表明:大模型简化文本的篇章树相对扁平。进一步分析显示,大模型在删除、合并操作中易导致篇章结构层面的简化过度,出现核心信息遗漏与照应失衡的现象。该文提出了引入篇章结构信息的评测与优化思路,并探讨了其在模型结构约束与自我校正中的应用潜力。

     

    Abstract: Based on Rhetorical Structure Theory(RST), this study evaluate the discourse-level performance of the large language model GPT-4o in Chinese news text simplification. Discourse structures are represented through rhetorical trees and semantic relations. A discourse-annotated dataset of news texts is constructed, and nine quantitative indicators are proposed to describe the properties of text simplification at the discourse level. The comparison between expert simplifications and model outputs shows that GPT-4o produced relatively flat discourse trees. Further analysis revealed that deletion and merging operations by the model often caused excessive simplification, leading to information loss and imbalance in referential coherence. The study proposes the incorporation of discourse structural information into text simplification evaluation and optimization, and highlights its potential for model constraint and self-correction mechanisms.

     

/

返回文章
返回