Information-Theoretic Measures for Objective Evaluation of Classifications

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	Information-Theoretic Measures for Objective Evaluation of Classifications
	Baogang Hu; Ran He(赫然); Xiaotong Yuan
刊名	Chinese Acta Automatica Sinica
	2012
卷号	38 期号:7 页码:1169-1182
关键词	Abstaining Classifications Error Types Reject Types Entropy Similarity Objectivity
英文摘要	This work presents a systematic study of objective evaluations of abstaining classifications using Information-Theoretic Measures (ITMs). First, we define objective measures for which they do not depend on any free parameter. This defi- nition provides technical simplicity for examining “objectivity” or “subjectivity” directly to classification evaluations. Second, we propose twenty four normalized ITMs, derived from either mutual information, divergence, or crossentropy, for investigation. Contrary to conventional performance measures that apply empirical formulas based on users’ intuitions or preferences, the ITMs are theoretically more sound for realizing objective evaluations of classifications. We apply them to distinguish “error types” and “reject types” in binary classifications without the need for input data of cost terms. Third, to better understand and select the ITMs, we suggest three desirable features for classification assessment measures, which appear more crucial and appealing from the viewpoint of classification applications. Using these features as “meta-measures”, we can reveal the advantages and limitations of ITMs from a higher level of evaluation knowledge. Numerical examples are given to corroborate our claims and compare the differences among the proposed measures. The best measure is selected in terms of the meta-measures, and its specific properties regarding error types and reject types are analytically derived.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/21079]
专题	自动化研究所_智能感知与计算研究中心
推荐引用方式 GB/T 7714	Baogang Hu,Ran He,Xiaotong Yuan. Information-Theoretic Measures for Objective Evaluation of Classifications[J]. Chinese Acta Automatica Sinica,2012,38(7):1169-1182.
APA	Baogang Hu,Ran He,&Xiaotong Yuan.(2012).Information-Theoretic Measures for Objective Evaluation of Classifications.Chinese Acta Automatica Sinica,38(7),1169-1182.
MLA	Baogang Hu,et al."Information-Theoretic Measures for Objective Evaluation of Classifications".Chinese Acta Automatica Sinica 38.7(2012):1169-1182.