Second International Workshop on Symbolic-Neural Learning (SNL-2018)

July 5-6, 2018
Nagoya Congress Center (Nagoya, Japan)

On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data

Nan Lu (The University of Tokyo), Gang Niu (Riken), Aditya Krishna Menon (The Australian National University), and Masashi Sugiyama (RIKEN/The University of Tokyo)

Abstract:

Empirical risk minimization (ERM) is a natural criterion in supervised classification for training arbitrary models such as deep neural networks. In this paper, we focus on training arbitrary models from only unlabeled (U) data following ERM without additional assumptions. We raise a question---what the minimal supervision is for training any binary classifier, and answer it in two steps: we prove this task is impossible given a single set of U data, but it becomes possible given two sets of U data with different class priors. Our proof is constructive and results in unbiased risk estimators and therefore the consistency of learning is theoretically guaranteed. Experiments demonstrate that by minimizing our risk estimators we can even successfully train AllConvNet and ResNet from two sets of U data.