Workshop

Machine learning for predictive control and real-time optimization

by Panagiotis Petsagkourakis (University College London, UK),
Antonio del Rio Chanona (Imperial College London, UK),
Benoit Chachuat (Imperial College London, UK)

SCOPE:
The first part of this workshop will be concerned with real-time optimization (RTO), with a view to enhancing RTO using supervised learning techniques. The supervised learning method of choice, Gaussian process (GP) regression [1], will be reviewed in the first talk alongside Bayesian optimization. The second talk will present a brief summary of different RTO paradigms, with particular emphasis on modifier adaptation [2]. A recent methodology whereby GP regression is integrated within modifier adaptation in combination with trust-region and Bayesian optimization approaches [3] will be presented in the third talk. The last talk in this part of the workshop will highlight a promising development that exploits multi-fidelity GP regression to further improve the convergence rate and practical implementation of GP-assisted RTO.
The second part of the workshop will be dedicated to reinforcement learning (RL) for optimization and control. The emerging field of RL has led to remarkable empirical results in rich data domains like robotics and strategy games. However, so far, no adoption has been made into process engineering. This workshop aims to introduce and showcase the use of RL in process optimization and control. An introduction to RL will initially be given in the first talk, where different methods will be conceptually explained and preliminary mathematical formulations will be explored [4]. The second talk will highlight the recent developments on how models can efficiently be used for safe and fast learning of RL agents in practical implementations [5,6].

SCHEDULE & FILES:

  • Introduction to Gaussian processes and Bayesian optimization (40 min)
  • Introduction to real-time optimization (20 min)
    • Introduction to Real-Time Optimization slides
  • Integrating supervised learning within real-time optimization (30 min)
    • Modifier Adaptation Meets Bayesian Optimization and Derivative-Free Optimization slides
  • Multi-fidelity Gaussian processes for real-time optimization (20 min)
    • Real-time optimization using multi-fidelity Gaussian process slides
    • Modifier Adaptation and Gaussian Processes python code
  • Introduction to reinforcement learning (40 min)
    • Reinforcement learning crash course slides
  • Reinforcement learning for process optimization and control (50 min)
    • Reinforcement learning for process optimization and control slides

REFERENCES:

[1] C. E. Rasmussen, & C. K. I. Williams, Gaussian Processes for Machine Learning.
[2] A. Marchetti, B. Chachuat, and D. Bonvin, Modifier-Adaptation Methodology for Real-Time Optimization, Industrial & Engineering Chemistry Research, 2009, 48 (13), 6022-6033
[3] E. A. del Rio Chanona, P. Petsagkourakis, E. Bradford, J. E. Alves Graciano, B. Chachuat, Real-time optimization meets Bayesian optimization and derivative-free optimization: A tale of modifier adaptation, Computers & Chemical Engineering, 2021, 107249 (147)
[4] A. Barto and R. S. Sutton, Reinforcement Learning: An Introduction
[5] P. Petsagkourakis, I.O. Sandoval, E. Bradford, D. Zhang, E.A. del Rio-Chanona, Reinforcement learning for batch bioprocess optimization, Computers & Chemical Engineering, 2020, 106649 (133)
[6] M. Mowbray, P. Petsagkourakis, E. Antonio del Río Chanona, R. Smith, D. Zhang, S. Chance Constrained Reinforcement Learning for Batch Process Control, 2021, https://arxiv.org/abs/2104.11706