近年,人工智能的语言生成技术突飞猛进,基于自然语言生成技术的聊天机器人ChatGPT能够自如地与人对话、回答问题。为了探究机器生成语言与人类语言的差异,该文分别收集了人类和ChatGPT在中文开放域上3 293个问题的回答作为语料,对两种语料分别提取并计算描述性特征、字词常用度、字词多样性、句法复杂性、语篇凝聚力五个维度上的161项语言特征,利用分类算法验证用这些特征区别两种语言的有效性,并考察、对比这些特征来阐释人类、机器生成两种语言的异同。研究结果发现,两种文本在描述性特征、字词常用度、字词多样性三个维度的77项语言特征上存在显著差异,相较于机器回答语言,人类回答语言表现出易读性高、论元重叠度低、口语色彩明显、用词丰富多样、互动性强等特点。
Abstract
Recent advancements in artificial intelligence have led to significant strides in language generation technologies, with chatbots like ChatGPT demonstrating proficiency in conversation and question answering. This paper investigates the differences between machine-generated language and human language by analyzing responses to 3 293 open-domain Chinese questions from humans and ChatGPT. The analysis examines 161 linguistic features in five dimensions: descriptive characteristics, word frequency, lexical diversity, syntactic complexity, and discourse cohesion. Classification algorithms are employed to assess the efficacy of these features in distinguishing between the two types of language. The results reveal significant differences in 77 linguistic features across descriptive characteristics, word frequency, and lexical diversity. Human language tends to exhibit higher readability, lower argument overlap, a more colloquial style, a richer vocabulary, and greater interactivity compared to machine-generated language.
关键词
ChatGPT /
人类语言 /
语言特征 /
机器学习
{{custom_keyword}} /
Key words
ChatGPT /
human language /
linguistic features /
machine learning
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] OUYANG L, WU J, JIANG X, et al. Training language models to follow instructions with human feedback[J]. Advances in Neural Information Processing Systems, 2022, 35: 27730-27744.
[2] PU J SH, HUANG Z Y, XI Y D, CHEN G D, et al. Unraveling the mystery of artifacts in machine generated text[C]//Proceedings of the 13th Language Resources and Evaluation Conference, 2022, 6889-6898.
[3] MA Y, LIU J, YI F, et al. AI vs. human-differentiation analysis of scientific content generation[J]. arXiv preprint arXiv: 2023.02155,2022.
[4] LOUWERSE M M, MCCARTHY P M, MCNAMARA D S, et al. Variation in language and cohesion across written and spoken registers[C]//Proceedings of the Annual Meeting of the Cognitive Science Society, 2004: 843-848.
[5] DEMPSEY K B, MCCARTHY P M, MCNAMARA D S. Using phrasal verbs as an index to distinguish text genres[C]//Proceedings of the FLAIRS Conference, 2007:217-222.
[6] CROSSLEY S, MCNAMARA D. Interlanguage talk: What can breadth of knowledge features tell us about input and output differences?[C]//Proceedings of the 32rd International FLAIRS Conference, 2010: 229-234.
[7] NASSERI M, THOMPSON P. Lexical density and diversity in dissertation abstracts: Revisiting English L1 vs. L2 text differences[J]. Assessing Writing, 2021, 47: 100511.
[8] GUO B, ZHANG X, WANG Z, et al. How close ischatgpt to human experts?: Comparison corpus, evaluation, and detection[J]. arXiv preprint arXiv:2301.07597, 2023.
[9] CUI Y, ZHU J, YANG L, et al. CTAP for Chinese: A linguistic complexity feature automatic calculation platform[C]//Proceedings of the 13th Language Resources and Evaluation Conference, 2022: 5525-5538.
[10] 熊兵.基于语料库的旅游文本英译文词汇特征及翻译研究[J]. 华中师范大学学报(人文社会科学版), 2016, 55(05): 94-103.
[11] 李绍山. 易读性研究概述[J]. 解放军外国语学院学报,2000, 23(4): 1-5.
[12] 邢诗吟. 基于语料库的初中英语记叙文写作语言特征研究[D].厦门: 集美大学硕士学位论文, 2022.
[13] 张倩倩. 基于小学语文教材的文本易读性公式研究[D]. 江苏: 江南大学博士学位论文, 2022.
[14] BALOTA D A, CHUMBLEY J I. Are lexical decisions a good measure of lexical access?: The role of word frequency in the neglected decision stage[J]. Journal of Experimental Psychology: Human Perception and Performance, 1984, 10(3): 340.
[15] HABERLANDT K F, GRAESSER A C. Component processes in text comprehension and some of their interactions[J]. Journal of Experimental Psychology: General, 1985, 114(3): 357.
[16] 蔡建永. 汉语二语文本可读性研究[D].北京: 北京语言大学博士学位论文,2020.
[17] JOHANSSON V. Lexical diversity and lexical density in speech and writing: A developmental perspective[J]. Working papers/Lund University, Department of Linguistics and Phonetics, 2008, 53: 61-79.
[18] 张必隐.阅读心理学[M]. 北京: 北京师范大学出版社, 1992.
[19] 黄伯荣,廖序东. 现代汉语[M]. 北京: 高等教育出版社, 2017.
[20] 杨彬. 篇章动态视角下副词性成分的叙事价值分析[J]. 当代修辞学, 2023,01: 42-50.
[21] 蒋跃,董贺. 计量特征在人机译文语言风格对比中的应用[J]. 语言教育, 2015, 3(03): 69-74.
[22] STAMATATOS E. A survey of modern authorship attribution methods[J]. Journal of the American Society for Information Science and Technology, 2009, 60(3): 538-556.
[23] 陈绍新. 元功能理论视角下的英语商务合同汉译研究[J]. 湖南第一师范学院学报, 2017, 17(6): 92-97.
[24] MCNAMARA D S, GRAESSER A C, MCCARTHY P M, et al. Automated evaluation of text and discourse with Coh-Metrix[M]. Cambridge University Press, 2014: 100-106.
[25] 吴思远. 基于多层面语言特征的汉语文本可读性自动评估研究[D]. 北京: 北京语言大学硕士学位论文, 2020.
[26] HUDSON R, Word meaning[M]. London: Routledge, 2003.
[27] 陆前,刘海涛.依存距离分布有规律吗? [J]浙江大学学报(人文社会科学版),2016,46(4): 63-76.
[28] 洪秋月,熊智伟. 语言经济原则下热搜词条的语篇衔接研究[J]. 今古文创, 2023(5): 129-132.
[29] 彭宣维. 代词的语篇语法属性、范围及其语义功能分类[J]. 语言教学与研究, 2005(1): 56-65.
[30] SEIDEL G. Ambiguity in political discourse[M]//MAURICE B,Political language and oratory in traditional society, London: Acadmic Press, 1975: 205-228.
[31] 贾宇丹.中国外应专业研究生学术语篇非正式语体特征研究[J]. 名家名作,2022(21): 85-87.
[32] CAIN K, NASH H M. The influence of connectives on young readers’ processing and comprehension of text[J]. Journal of Educational Psychology, 2011, 103(2): 429.
[33] HALLIDAY M A K, HASAN R. Cohesion in English[M]. London: Routledge, 1976.
[34] 何清强, 王文斌, 吕煜芳. 汉语叙述体篇内句的特点及其二语习得研究: 基于汉英篇章结构的对比分析[J]. 语言教学与研究,2019, 06:1-11.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
教育部人文社科青年基金(23YJCZH264);国家语委重大科研项目(ZDA145-17)
{{custom_fund}}
PDF(4031 KB)
