You have to take into consideration, just how society moved in those dark times. If the earl throws a dinner party at Alton Towers to his very rich and successful peers, conversation would have naturally swung at some, point to cost of staff and upkeep. The earl would have been open to slave owners needing to sell, "why pay for 10 Englishmen. when you can have 100 slaves!" Then we have another dark path. High society would have been judged by the amount if slaves owned. The higher the count, the more influence and sway.
Britain is dominated by the slave trade. It jaw dropping seeing people from Manchester comment on how racist Liverpool as a city was because of amount of money made from the slave trade. This Manchester, a city built on cotton. That's not exactly the cleanest industry in regrades to the slavery