Joint work
preprintAdequate Losses via Quantitative Linear Logic
As neural components are increasingly embedded in existing symbolic software -- including
safety-critical systems -- the question arises of how to specify and enforce the safety
of the newly introduced neural parts. Unlike traditional logical specifications, these
must be amenable not only to the standard Boolean interpretation, but also to training
and optimisation. The latter calls for a quantitative interpretation of the logical
syntax, subject to further requirements such as smoothness and differentiability.
Moreover, the qualitative and quantitative sides of the logic must share a unifying
proof-theoretic and categorical semantics. Finally, the new logic should link cleanly
to the substructural and program logics that underpin the verification of existing
symbolic programs.
In this paper, we present a logic that ticks all of these boxes.
We introduce a family of calculi, \(p\)QLL, indexed by a hardness degree \(p\), prove a cut-elimination theorem for them, and establish completeness with respect
to enriched residuated `soft' lattices. At \(p = \infty \), \(p\)QLL reduces to multiplicative additive linear logic (MALL), and provability in \(p\)QLL converges to provability in MALL as \(p \to +\infty \).
We express optimisation objectives in the syntax of this logic and prove the quantitative
adequacy of neuro-symbolic loss functions -- a result that has eluded the neuro-symbolic
machine learning community for nearly a decade.
preprintQuantitative Linear Logic for Neuro-Symbolic Learning and Verification
Differentiable Logics are deployed in neuro-symbolic learning tasks as a way of embedding logical constraints in the training objective of neural networks. A differentiable logic consists of a syntax to write logical properties and a semantics to interpret them as real-valued functions to be folded in the loss function. A defining trade-off of the field is that between logical properties of the connectives, and analytic concerns for the semantics, with both aspects being relevant in applications. At one extreme we find fuzzy logics, that have well-established algebraic and proof-theoretic foundations, and at the other ad-hoc differentiable logics like Fischer's DL2, conceived for deep learning applications. However, no satisfactory foundation has emerged yet. We propose a resolution to this long-standing tension via a novel logic, QLL, with foundational ambitions. Our design is driven by naturality---the idea that, since logical constraints are translated to losses, the semantics of the connectives should be pertinent operations used in ML practice (that is, sum and log-sum-exp) on additive quantities (like logits). We then judge the result on two aspects: logical adequacy---that they satisfy most of the standard logical laws of Linear Logic; and empirical effectiveness---test-time performance (as measured by adversarial attacks) is well-correlated to the actual verification of the logical constraints (as measured by off-the-shelf neural network verifier), which makes QLL stand out among SoTA techniques.