In 1086 the Domesday Book recorded a manor on this site at the northern edge of Coleby and, no doubt, a manor house of some kind was in existence at that time. Over the centuries the house was probably replaced several times. In 1361 the Coleby Estate was inherited by Hugh Swynford, husband of Katherine Swynford, who made history through her liaison with John of Gaunt. The present building was started in 1628 for Sir William Lister and is constructed in coursed rubble and ashlar with red tiled ridge roof and stone stacks. The Lister family owned the Hall until 1734 when it passed to a relative, Thomas Scrope and during his tenure the building was altered and extended. The grounds were landscaped and two temples were built, one of these, dating from 1762, still stands. In the early nineteenth century Coleby hall was inherited by the Tempest family of Broughton Hall in Yorkshire, remaining in their ownership until 1928. The hall is now divided into three private houses with the stables and other outbuildings converted into a further five homes.
Coleby Hall is private property and not open to the public.