Real-World Implementation of Offline Model-Free Reinforcement Learning for Thermostat Control

Abstract

This study introduces a model-free, offline Reinforcement Learning (RL) approach for optimizing the thermostat control in heating systems. Specifically, historical data from a real-world building was used to train the RL agent that is based on the Dueling Double Deep Q-Network (DQN) algorithm. The RL agent's objective is to balance thermal comfort with energy savings, by controlling indoor temperature set points. Traditional rule-based thermostat control methods often depend on fixed schedules and lack adaptability to varying conditions, resulting in energy inefficiencies. Training the RL model offline eliminates the requirement for live interaction, thereby preventing potential disruptions to occupant comfort. During evaluation, the RL model achieved an 18.66% reduction in energy use compared to a rule-based controller, while maintaining recommended indoor temperature levels.