Project titleReinforcement Learning combined with Transfer Learning for HVAC Control in Residential and Commercial Buildings

Maintaining optimal HVAC (Heating, Ventilation, and Air Conditioning) conditions inside buildings remains an essential yet challenging component of modern-day architecture. HVAC automation shows promising benefits in energy consumption savings, ensuring consumer comfort, as well as maintaining optimal conditions for the well-being of stored goods. However, many challenges arise with HVAC automation, due to the huge variability of environmental conditions and materials. Early work has been using model-based control strategies, while more recently, people have been using Machine Learning (ML) that avoids expliciting a model. Nevertheless, using supervised ML is challenging since it requires capturing a potentially large continuous state space into a predefined set of labeled training examples, which is a complex, long, and expensive process. Additionally, the computational requirements for training such supervised models into an HVAC control policy are very demanding. Reinforcement Learning (RL) and, more recently, Transfer Learning (TL) have been used as alternative ML solutions, showing promising results.

However, current works do not take full advantage of the benefits of the two learning methods due to the use of a rather limited, discrete set of room state descriptors and minimally-descriptive binary reward systems. In this work, we attempted to expand the use of RL for online HVAC control by enlarging the input state space and by defining a richer reward system for training the RL controller. We also considered TL for transferring control policies from low-dimensional state descriptors to high-dimensional state descriptors, and from residential to commercial buildings. All work was performed in simulation using EnergyPlus, and the standard Q-Learning algorithm was deployed.

Primary contact nameNada Soliman
Primary contact emailEmail hidden; Javascript is required.
Primary contact mobile phone(+974) 5561 5143
Other student team members
Student name Email Phone
Abdalla Mohamed +1 (608) 707-2520
Students/participant(s) programs
  • Computer Science
Faculty advisor(s)
Advisor name Email Affiliation
Gianni Di Caro CMU-Q
Eduardo Feo Flushing CMU-Q
For CMU-Q advisor(s), please select their program(s)
  • Computer Science