Bromley College was founded in 1666 to provide houses for clergy widows and Sheppard's College in 1840 to provide houses for unmarried daughters of clergy widows, who had lived with their mothers at Bromley College. Houses in both colleges have been converted into flats and widows/widowers of clergy, retired clergymen and their spouses, divorced and separated spouses of clergy may now be admitted. Unmarried daughters or stepdaughters of a deceased former resident may also apply.

Bromley College was founded in 1666 by the Will of John Warner, Bishop of Rochester 1638-1666, to provide housing for "twenty poore widowes of orthodoxe and loyall clergiemen". 

The original building consisted of twenty houses built around a quadrangle with residences for a Chaplain and a Treasurer.

This quadrangle, built in classical style, is often called the "Wren Quadrangle" as at one time it was thought that Sir Christopher Wren was the architect. It is now acknowledged that Captain Richard Ryder, one of Wren's Surveyors, was responsible for the design and construction.

The second quadrangle of twenty houses was built in the late 18th Century from bequests of a Mrs Helen Bettenson and William Pearce, brother of Zachary Pearce, Bishop of Rochester 1756-1774.

Sheppard's College was founded in 1840, following an appeal by the Chaplain, Thomas Scott, to Mrs Sophia Sheppard, the wealthy widow of Dr Sheppard, Fellow of Magdalen College, Oxford.

The Chaplain saw the problem of an unmarried clergy daughter becoming homeless when her mother died and Mrs Sheppard provided the means to accommodate them by building a terrace of five houses.