ANC Workshop - Iain Murray and Cian Eastwood

Speaker: Iain Murray

Title: Some Low Dimensional and Shallow Computations

Abstract: Sometimes definite integrals that we need to compute have no neat solution, but aren't really nasty enough to warrant sledge-hammers like MCMC either. This week I've been doing some remedial playing with numerical representations of functions, and computations with FFTs, and Chebfun. I aim to show a couple of examples.

Speaker: Cian Eastwood

Title: Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration

Abstract: Source-free domain adaptation (SFDA) aims to adapt a model trained on labelled data in a source domain to unlabelled data in a target domain without access to the source-domain data during adaptation (due to e.g. privacy regulations or storage/bandwidth constraints). Existing methods for SFDA leverage entropy-minimization techniques which rely on the source model achieving a good level of feature-space class-separation in the target domain. In this paper, we show that even the most innocuous of shifts are capable of destroying this class-separation and thus the performance of these techniques. We address this problem for a particularly pervasive type of domain shift called measurement shift, characterised by a change in measurement system (e.g. a change in sensor, lighting or weather conditions). In the source domain, we store a lightweight and flexible approximation of the feature distribution under the source data. In the target domain, we retrain the feature-extractor such that the approximate feature distribution under the target data realigns with that saved on the source data. We call this method Feature Restoration (FR) as it seeks to extract features with the same semantics from the target domain as were previously extracted from the source domain. We additionally propose Bottom-Up Feature Restoration (BUFR)---a bottom-up training scheme for FR which boosts performance by preserving learnt structure in the later layers of a network. Through experiments we demonstrate that BUFR successfully restores feature-space class-separation in the target domain, and as a result, often outperforms existing SFDA methods in terms of accuracy, calibration error, and data efficiency.