Le Refuge is an alcove that is believed to have been created using the reclaimed stonework from the original entrance of Alveton Lodge.
Behind a door in the alcove is a summerhouse within the terrace which was also likely to have been used as a changing room for the Roman Bath, when it was used as a cold bathing pond. On either side of the door are two vents, one of which was used as an air vent, whilst the other was a chimney for a heating system.
The statue in the centre of the Roman Bath, represents triton blowing into a conch shell.
Le Refuge is considered one of the oldest features in the Gardens and is thought to date from around 1813 and would therefore pre-date the Roman Bath located in front of it, which is listed as being added in 1819.