Originally opened in 1846, referred to White Hall. Ten years later the county purchased more property on the north side of the St. Joseph River. By the turn of the century it was decided the old structures need replacement to better serve the county. In 1905 the county board of charities purchased a farm northwest of South Bend. In 1907 the Saint Joseph County Infirmary opened, the name change to reflect its change in mission. The facility contained the main house, a pole barn, and a pump house. There was also a poor farm cemetery 1/2 mile east of the home.
By the 1930s, the facility became so overcrowded that patients determined as "feeble-minded" had to be housed with the insane. Although by the early 1950s, those diagnosed with severe mental illness were moved to Norman Beatty Mental Hospital. The lock-up cells however, were used through the 1970s for those that had escaped, were a danger to others or frequently broke rules. In the late 1970s the facility changed it's name to Portage Manor and farming operations ceased. Today the main focus of the facility is residential long-term care.