Henry and Lucy's youngest daughter Caroline was only four years old when her family came to Montpelier, and she called the mansion home for the rest of her life. When her mother died in 1824, Caroline inherited the house, and her two husbands followed her there.

Over the years she became less and less able financially to maintain the nineteen-room mansion, and eventually she closed off the chamber floor to economize. Her bedroom on the salon level opened onto a "parlor" which was her main living space, as well as her dining room.