AI4OPT Seminar Series

Date: Thursday, February 15, 2024

Time: Noon – 1:00 pm

Location: Coda Building (756 W Peachtree St NW, Atlanta, GA 30308)

Speaker: Wolfram Wiesemann

Large Scale and Data-Driven Markov Decision Processes

Abstract: Markov decision processes (MDPs) constitute one of the predominant modeling and solution paradigms for dynamic decision problems affected by uncertainty. MDPs model the dynamics of a system through a random state evolution that generates rewards over time. The decision maker aims to select actions that influence this state evolution so as to maximize rewards.In this talk, we review recent advances in MDPs along two directions: (i) the construction of data-driven policies that combine the (traditionally separated) tasks of estimating the system’s behavior and selecting actions that maximize rewards in the estimated system, and (ii) the exploitation of structure to solve large-scale problems. In view of (i), we will show how the consideration of data-driven policies naturally leads to the study of robust MDPs, where the decision maker combats overfitting by hedging against the worst system dynamics that are plausible under some given training data. We will also discuss how alternative models of robustness offer different trade-offs between the competing goals of out-of-sample performance and complexity of the involved policies and computations. As for (ii), we will review weakly coupled MDPs, which allow us to alleviate the well-known curse of dimensionality when combining a potentially large number of MDPs via a small number of linking constraints.

Bio: Wolfram Wiesemann is Professor of Analytics & Operations as well as the head of the Analytics, Marketing & Operations department at Imperial College Business School. His research interests evolve around decision-making under uncertainty, with applications to supply chain management, healthcare and energy. Wolfram is an elected member of the boards of the Mathematical Optimization Society and the Stochastic Programming Society, and he currently serves as editor in-chief of Operations Research Letters as well as a department co-editor for Management Science.

To receive AI4OPT seminar announcements, please sign up to our mailing list at: https://lists.isye.gatech.edu/mailman/listinfo/ai4opt-seminars

Past seminars can be found at: https://www.ai4opt.org/seminars/past-seminars