AIAI Seminar - 19 June 2023 - Talks by Miguel Mendez Lucero, Jonathan Feldstein and Jiawei Zheng

Speaker:  Miguel Mendez Lucero

Title:         Semantic Objective Functions: A Neurosymbolic Framework for Constraint Learning and Knowledge Distillation

Abstract:

Issues of safety, explainability, and efficiency are of increasing concern in learning systems deployed with hard and soft constraints. Symbolic-based Constrained Learning and Knowledge Distillation techniques have shown promising results in this area, by embedding and extracting knowledge, as well as setting logical constraints into models in an explainable and safety-critical manner. Through an integration of mathematical logic and information geometry, we provide a construction and theoretical framework for these tasks that generalizes symbolic-based approaches. We propose a loss-based method that embeds knowledge—enforces logical constraints—into a machine learning model that outputs distributions. This is done by constructing a  distribution from the external knowledge/logic formula, and constructing a loss function as a convex combination of the original loss function with the Fisher-Rao distance or Kullback-Leibler divergence to the constraint distribution. This construction includes logical constraints in the form of propositional formulas (Boolean variables) and  any statistical model that was pretrained with semantic information. We evaluate our method on a variety of learning tasks, including image classification with logic constraints, transferring knowledge from logic formulas, and knowledge distillation from general distributions. Our method provides a more general framework whilst still improving the accuracy in the aforementioned tasks

Speaker:  Jonathan Feldstein

Title:         Parallel Neurosymbolic Integration with Concordia

Abstract: 

Parallel neurosymbolic architectures have been applied effectively in NLP by distilling knowledge from a logic theory into a deep model. However, prior art faces several limitations including supporting restricted forms of logic theories and relying on the assumption of independence between the logic and the deep network. We present Concordia, a framework overcoming the limitations of prior art. Concordia is agnostic both to the deep network and the logic theory offering support for a wide range of probabilistic theories. Our framework can support supervised training of both components and unsupervised training of the neural component. Concordia has been successfully applied to tasks beyond NLP and data classification, improving the accuracy of state-of-the-art  on collective activity detection, entity linking and recommendation tasks.

Speaker:  Jiawei Zheng

Title:         Alignment-based conformance checking over probabilistic events

Abstract:

Conformance checking techniques allow us to evaluate how well some exhibited behaviour, represented by a trace of monitored events, conforms to a specified process model. Modern monitoring and activity recognition technologies, such as those relying on sensors, the IoT, statistics and AI, can produce a wealth of relevant event data. However, this data is typically characterised by noise and uncertainty, in contrast to the assumption of a deterministic event log required by conformance checking algorithms. In this work, we extend alignment-based conformance checking to function under a probabilistic event log. We introduce a weighted trace model and weighted alignment cost function, and a custom threshold parameter that controls the level of confidence on the event data vs. the process model. The resulting algorithm considers activities of lower but sufficiently high probability that better align with the process model. We explain the algorithm and its motivation both from formal and intuitive perspectives, and demonstrate its functionality in comparison with deterministic alignment using real-life datasets.