U-DARTS: Uniform-space differentiable architecture search

Lan Huang, Shiqi Sun, Jia Zeng, Wencong Wang, Wei Pang, Kangping Wang

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)
29 Downloads (Pure)

Abstract

Differentiable architecture search (DARTS) is an effective neural architecture search algorithm based on gradient descent. However, there are two limitations in DARTS. First, a small proxy search space is exploited due to memory and computational resource constraints. Second, too many simple operations are preferred, which leads to the network deterioration. In this paper, we propose a uniform-space differentiable architecture search, named U-DARTS, to address the above problems. In one hand, the search space is redesigned to enable the search and evaluation of the architectures in the same space, and the new search space couples with a sampling and parameter sharing strategy to reduce resource overheads. This means that various cell structures are explored directly rather than cells with same structure are stacked to compose the network. In another hand, a regularization method, which takes the depth and the complexity of the operations into account, is proposed to prevent network deterioration. Our experiments show that U-DARTS is able to find excellent architectures. Specifically, we achieve an error rate of 2.59% with 3.3M parameters on CIFAR-10. The code is released in https://github.com/Sun-Shiqi/U-DARTS.
Original languageEnglish
Pages (from-to)339-349
Number of pages11
JournalInformation Sciences
Volume628
Early online date1 Feb 2023
DOIs
Publication statusPublished - May 2023

Keywords

  • Automatic machine learning
  • Deep learning
  • Image recognition
  • Neural network architecture search

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Software
  • Control and Systems Engineering
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'U-DARTS: Uniform-space differentiable architecture search'. Together they form a unique fingerprint.

Cite this