IPAB Workshop - 28/04/2022

Title: Data and label efficient learning with gradient matching

Abstract: Large-scale datasets, comprising millions of samples, are becoming the norm to obtain state-of-the-art machine learning models in multiple fields including computer vision, natural language processing and speech recognition. At such scales, even storing and preprocessing the data becomes burdensome, and training machine learning models on them demands for specialized equipment and infrastructure in addition to expensive manual labeling cost. In the first part of the talk, I will introduce a training set synthesis technique for data-efficient learning, called Dataset Condensation, that condenses a large dataset into a small set of informative synthetic samples that can be used for training deep neural networks from scratch in a fraction of standard training time. In the second part, I will discuss an automatic annotation method that can produce large-scale annotated images by using a generative neural network.