Researchers from the Institute for AI received two workshop awards at ICML 2026 in Seoul, South Korea. FutureSim [1] won the Best Paper Award at the Workshop on AI Forecasting, and GraphBench [2] received the Runner-Up Award at the Graph Foundation Models workshop. Both papers address a shared challenge: as AI systems grow more general, the community needs rigorous, realistic benchmarks to measure progress.
FutureSim: Replaying World Events to Evaluate Adaptive Agents
AI agents are increasingly deployed in dynamic, open-ended environments where new information arrives continuously. Yet most benchmarks evaluate agents on static snapshots, telling us little about how well they adapt over time. FutureSim (https://openforecaster.github.io/futuresim/), built in collaboration with the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, addresses this by building a grounded simulation that replays real-world events in the order they occurred. Agents are tasked with forecasting world events beyond their training knowledge cutoff while interacting with a chronological replay of the world: real news articles arrive day by day, and forecasting questions resolve over the simulated period.
Evaluating frontier agents in their native (and custom) harnesses over a simulated three-month period (January to March 2026), FutureSim reveals a clear separation in capabilities: the best agent reaches only 25% accuracy, and several models achieve a worse Brier skill score than making no prediction at all. The authors also demonstrate how the environment can serve as a realistic testbed for emerging research directions such as long-horizon test-time adaptation, search, memory, and reasoning under uncertainty.
The paper was presented as an oral at the workshop and selected for the Best Paper Award by both the organizers and a public prediction market.
GraphBench: Next-Generation Graph Learning Benchmarking
Machine learning on graphs has made substantial progress across domains such as molecular property prediction and chip design. However, benchmarking practices remain fragmented, relying on narrow, task-specific datasets and inconsistent evaluation protocols.
GraphBench, a large collaborative effort with partners including RWTH Aachen, Oxford, Google Research, and NVIDIA, introduces a comprehensive benchmarking suite spanning diverse real-world domains: from social networks and hardware design to combinatorial optimization and earth systems, and covering node-level, edge-level, graph-level, and generative tasks. It provides standardized evaluation protocols with consistent dataset splits, metrics that account for out-of-distribution generalization, and a unified hyperparameter tuning framework. The suite also establishes principled reference baselines using state-of-the-art message-passing neural networks and graph transformers, all accessible through an easy-to-use Python package at graphbench.io.
By unifying heterogeneous tasks within a single framework, GraphBench enables reproducible and robust graph learning research and paves the way for the fair evaluation of graph foundation models, a contribution recognized with the Runner-Up Award at the workshop.
References
[1] Shashwat Goel*, Nikhil Chandak*, Arvindh Arun*, Ameya Prabhu, Steffen Staab, Moritz Hardt, Maksym Andriushchenko, and Jonas Geiping. 2026. FutureSim: Replaying World Events to Evaluate Adaptive Agents. Workshop on AI Forecasting at ICML 2026. https://arxiv.org/abs/2605.15188
[2] Timo Stoll, Chendi Qian, Ben Finkelshtein, Ali Parviz, Darius Weber, Fabrizio Frasca, Hadar Shavit, Antoine Siraudin, Arman Mielke, Marie Anastacio, Erik Müller, Maya Bechler-Speicher, Michael Bronstein, Mikhail Galkin, Holger Hoos, Mathias Niepert, Bryan Perozzi, Jan Tönshoff, and Christopher Morris. 2026. GraphBench: Next-Generation Graph Learning Benchmarking. Graph Foundation Models workshop at ICML 2026. https://arxiv.org/abs/2512.04475
