Gradient-based Learning of Complex Latent Structures
Imposing structural constraints on the latent representations learned by deep neural models
Led by Pasquale Minervini, Antonio Vergari, and Edoardo Ponti with Emile van Krieken (Postdoctoral Researcher)
Imposing structural constraints on the latent representations learned by deep neural models has several applications, which can improve their explainability, generalisation, and robustness properties. For example, we can learn more explainable models by making them selectively decide which parts of the input to consider; or we can improve their generalisation properties by learning representations suitable for reasoning tasks, such as deductive reasoning and planning, and comply with any desired constraints. The main reason why learning complex discrete – or mixed continuous-discrete – representations is not widely popular is that back-propagating discrete decision steps used to be problematic, but several practical solutions have been proposed recently (e.g., see Niepert et al., 2021; Minervini et al., 2022). In this project, we investigate how we can derive better methods for back-propagating through mixed continuous-discrete complex latent structures and how we can leverage them for learning more explainable, data-efficient, and robust deep neural models.
- Emile van Krieken, Thiviyan Thanapalasingam, Jakub M. Tomczak, Frank van Harmelen, Annette ten Teije: A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference, NeurIPS 2023
- Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van den Broeck, Antonio Vergari: Semantic Probabilistic Layers for Neuro-Symbolic Learning. NeurIPS 2022
- Pasquale Minervini, Luca Franceschi, Mathias Niepert: Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models. CoRR abs/2209.04862 (2022)
- Mathias Niepert, Pasquale Minervini, Luca Franceschi: Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions. NeurIPS 2021
- Edoardo M. Ponti, Alessandro Sordoni, Yoshua Bengio, and Siva Reddy: Combining Modular Skills in Multitask Learning. arXiv e-prints (2022): arXiv-2202.