A comparative study between model predictive control and reinforcement learning for robust reference tracking in chemical processes

Abstract

Artificial intelligence (AI) has shown notable capabilities in various domains, including process control. Reinforcement Learning (RL), as a method within AI, is known for its strong ability to handle complex dynamic systems, which are frequently encountered in this domain. In this research, we conducted a comparison of three off-policy RL methods Twin Delayed Deep Deterministic Policy Gradient (TD3), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC) - against Model Predictive Control (MPC) for the purpose of reference tracking. This evaluation was carried out in the use case of a polymerization reactor in which the jacket temperature and the input flow rate of a reagent are adjusted to control both the molar equivalent of the reagent and the reactor temperature. The comparison was performed in two scenarios: one without disturbances and the other with varying levels of disturbance. The findings in both scenarios indicate that TD3 outperforms the other RL methods and MPC in controlling the molar equivalent of the reagent, particularly in scenarios with high levels of disturbance. Conversely, MPC is more effective and demonstrates greater robustness in controlling the reactor temperature. Despite these distinctions, all RL methods and MPC exhibit comparable performance, highlighting the potential for RL to function as a reference tracking controller.