Tutorial: Probabilistic circuits: how structure can boost your deep generative models

Abstract

In this tutorial, I will introduce the framework of probabilistic circuits (PCs), tractable computational graphs in which imposing structure guarantees exact probabilistic reasoning in polynomial (often linear) time. After introducing their syntax and showing some algorithmic and theoretical results, I will discuss how PCs have achieved impressive results in probabilistic modeling, sometimes outperforming intractable models such as variational autoencoders and normalizing flows. I will also discuss how PCs are special cases of neural networks, and build a calculus in which restricting a network to sport certain structural properties enables tractability for compositionally many reasoning tasks. This opens up to a unified view of probabilistic ML models and to a range of ways to learn PCs from data. I will discuss how we can learn both the structure and parameters of PCs and discuss modern approaches to scale them on GPU. Lastly, I will showcase several successful application scenarios where PCs have been employed as an alternative to or in conjunction with intractable models, including robust image classification, lossless compression, predictions in the presence of missing values, fairness certification, and knowledge graph completion. 

Speaker

Antonio Vergari is a Lecturer (Assistant Professor) in Machine Learning at the University of Edinburgh. His research focuses on efficient and reliable machine learning in the wild; tractable probabilistic modeling and combining learning with complex reasoning. He recently was awarded an ERC Starting Grant on automating probabilistic reasoning for trustworthy ML. Previously he was postdoc in the StarAI Lab lead by Guy Van den Broeck at UCLA. Before that he did a postdoc at the Max Planck Institute for Intelligent Systems in Tuebingen in the Empirical Inference Department of Bernhard Schoelkopf. He obtained a PhD in Computer Science and Mathematics at the University of Bari, Italy. He likes to tease and challenge the probabilistic ML community at large on how we desperately need reliable ML an AI models nowadays. To this extent, he organized a series of tutorials, workshops, seminars and events at top ML and AI venues such as UAI, ICML, AAAI, IJCAI and NeurIPS and last year a Dagstuhl Seminar.

Event details