A Unified Model for Solving the OOV Problem of Chinese Word Segmentation

doi:10.1145/2699940

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 自然语言处理团队

	A Unified Model for Solving the OOV Problem of Chinese Word Segmentation
	Li, Xiaoqing1 ; Zong, Chengqing1 ; Su, Keh-Yih 2
刊名	ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING
	2015-06-01
卷号	14 期号:3 页码:29
关键词	Algorithms Languages Experimentation Performance Chinese Word Segmentation Out-of-vocabulary Words Model Integration Domain Adaptation
DOI	10.1145/2699940
文献子类	Article
英文摘要	This article proposes a unified, character-based, generative model to incorporate additional resources for solving the out-of-vocabulary (OOV) problem of Chinese word segmentation, within which different types of additional information can be utilized independently in corresponding submodels. This article mainly addresses the following three types of OOV: unseen dictionary words, named entities, and suffix-derived words, none of which are handled well by current approaches. The results show that our approach can effectively improve the performance of the first two types with positive interaction in F-score. Additionally, we also analyze reason that suffix information is not helpful. After integrating the proposed generative model with the corresponding discriminative approach, our evaluation on various corpora-including SIGHAN-2005, CIPS-SIGHAN-2010, and the Chinese Treebank (CTB)-shows that our integrated approach achieves the best performance reported in the literature on all testing sets when additional information and resources are allowed.
WOS研究方向	Computer Science
语种	英语
WOS记录号	WOS:000370686400003
资助机构	Natural Science Foundation of China(61333018) ; International Science & Technology Cooperation Program of China(2014DFA11350) ; Hi-Tech Research and Development Program ("863" Program) of China(2012AA011102)
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/11346]
专题	自动化研究所_模式识别国家重点实验室_自然语言处理团队
通讯作者	Li, Xiaoqing
作者单位	1.Chinese Acad Sci, Inst Automat, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 2.Acad Sinica, Taipei, Taiwan
推荐引用方式 GB/T 7714	Li, Xiaoqing,Zong, Chengqing,Su, Keh-Yih. A Unified Model for Solving the OOV Problem of Chinese Word Segmentation[J]. ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING,2015,14(3):29.
APA	Li, Xiaoqing,Zong, Chengqing,&Su, Keh-Yih.(2015).A Unified Model for Solving the OOV Problem of Chinese Word Segmentation.ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING,14(3),29.
MLA	Li, Xiaoqing,et al."A Unified Model for Solving the OOV Problem of Chinese Word Segmentation".ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING 14.3(2015):29.