The residential sector consumes a significant portion of the electricity sold in the United States. Above 60% of the energy used in the sector is used to operate heating, ventilation, and air conditioning (HVAC) systems and water heating (WH) systems. With the increase of intelligence in the grid and the new decision and control options enabled by the Internet of Things; control of these devices can be used to support the grid. This article presents a scalable multiagent system for optimizing HVAC and WH systems while maintaining comfort. It allows a utility to orchestrate the shifting of energy from critical periods without direct control, but instead by using a price signal. The architecture, optimization formulation, implementation strategy and results from an implementation project are discussed.