Researchers from the Institute for AI are tackling key challenges in AI, ranging from diffusion models to language models. Their work introduces new methods to accelerate image generation, improve scientific computing, and uncover hidden structures in language models.
From Tokens to Lattices: Emergent Lattice Structures in Language Models: Bo Xiong and Steffen Staab investigate how language models can learn conceptual knowledge. They show that language models capture conceptual knowledge and organize it implicitly in lattice structures [1].
Active Learning for Neural PDE Solvers: Daniel Musekamp et al. introduce AL4PDE, a benchmark to evaluate Active Learning for neural PDE solvers. The benchmark shows that Active Learning can make training neural PDE solvers more data-efficient and reliable [2].
Discrete Copula Diffusion: Anji Liu and co-authors identify a fundamental limitation in diffusion models—they fail to capture dependencies between output variables at each denoising step. To address this issue, they provide a formal explanation and introduce a general approach to supplement the missing dependency information by incorporating another deep generative model [3].
Learning to Discretize Denoising Diffusion ODEs: Vinh Tong et al. introduce a lightweight sampling technique for efficiently generating high-quality images from diffusion models. Their technique offers an efficient method for optimal time discretization when sampling from pre-trained diffusion models. Their paper has been accepted for an oral presentation at ICLR 2025 [4].
References:
[1] Bo Xiong, Steffen Staab. From Tokens to Lattices: Emergent Lattice Structures in Language Models. The Thirteenth International Conference on Learning Representations (ICLR 2025) [OpenReview]
[2] Daniel Musekamp, Marimuthu Kalimuthu, David Holzmüller, Makoto Takamoto, Mathias Niepert. Active Learning for Neural PDE Solvers. The Thirteenth International Conference on Learning Representations (ICLR 2025) [arXiv] [OpenReview]
[3] Anji Liu, Oliver Broadrick, Mathias Niepert, Guy Van den Broeck. Discrete Copula Diffusion. The Thirteenth International Conference on Learning Representations (ICLR 2025) [arXiv] [OpenReview]
[4] Vinh Tong, Dung Trung Hoang, Anji Liu, Guy Van den Broeck, Mathias Niepert. Learning to Discretize Denoising Diffusion ODEs. The Thirteenth International Conference on Learning Representations (ICLR 2025) [arXiv] [OpenReview]