Learning Architectures from an Extended Search Space for Language Modeling

Understand

Neural architecture search (NAS) has advanced significantly in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell.

• In this paper, we extend the search space of NAS.
• In particular, we present a general approach to learn both intra-cell and inter-cell architectures (call it ESS).
• For a better search result, we design a joint learning method to perform intra-cell and inter-cell NAS simultaneously.