Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning

Understand

We use Bayesian optimization to learn curricula for word representation learning, optimizing performance on downstream tasks that depend on the learned representations as features.

• The curricula are modeled by a linear ranking function which is the scalar product of a learned weight vector and an engineered feature vector that characterizes the different aspects of the complexity of each instance in the training corpus.
• We show that learning the curriculum improves performance on a variety of downstream tasks over random orders and in comparison to the natural corpus order.