Cyclic Differentiable Architecture Search | |
Yu HY(俞宏远)1,4; Peng HW(彭厚文)2; Huang Y(黄岩)1,4; Fu JL(傅建龙)2; Du, Hao2; Wang L(王亮)1,4; Lin, Haibin3 | |
刊名 | IEEE Transactions on Pattern Analysis and Machine Intelligence |
2022 | |
卷号 | 2022期号:TPAMI.2022.3153065页码:DOI 10.1109 |
文献子类 | 计算机视觉 |
英文摘要 | Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search. It tries to find the optimal architecture in a shallow search network and then measure its performance in a deep evaluation network. The independent optimization of the search and evaluation networks, however, leaves a room for potential improvement by allowing interaction between the two networks. To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CyDAS. Considering the structure difference, CyDAS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation. First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized. Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation. Repeating the above cycle results in a joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network. The experiments and analysis on CIFAR, ImageNet and NAS-Bench-201 demonstrate the effectiveness of the proposed approach over the state-of-the-art ones. Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet. In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0. Our code and models are publicly available at https://github.com/researchmm/CyDAS.git. |
语种 | 英语 |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/48518] |
专题 | 自动化研究所_智能感知与计算研究中心 |
通讯作者 | Wang L(王亮) |
作者单位 | 1.中国科学院自动化研究所 2.Microsoft Research Asia 3.the Department of Computer Science, Stony Brook University 4.中国科学院大学 |
推荐引用方式 GB/T 7714 | Yu HY,Peng HW,Huang Y,et al. Cyclic Differentiable Architecture Search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,2022(TPAMI.2022.3153065):DOI 10.1109. |
APA | Yu HY.,Peng HW.,Huang Y.,Fu JL.,Du, Hao.,...&Lin, Haibin.(2022).Cyclic Differentiable Architecture Search.IEEE Transactions on Pattern Analysis and Machine Intelligence,2022(TPAMI.2022.3153065),DOI 10.1109. |
MLA | Yu HY,et al."Cyclic Differentiable Architecture Search".IEEE Transactions on Pattern Analysis and Machine Intelligence 2022.TPAMI.2022.3153065(2022):DOI 10.1109. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论