New Haven was founded in 1836 as a riverboat stop on the Missouri River called "Miller's Landing." Founder Phillip Miller operated a wood yard on the river to fuel the steamboat trade. The arrival of the "Iron Horse," the Union Pacific railroad, in the 1850s, brought more commerce and activity to the area. In 1856, town fathers changed the name from Miller's Landing to "New Haven," reflecting the growth of the town from its origins as a steamboat stop.

German immigrants helped settle the area through the 19th century, many of them coming from the Borgholzhausen, Germany area. Today, New Haven maintains an active Sister-City partnership with Borgholzhause.

New Haven continued to grow through the 20th century. The town is home to several beautiful churches built in the 1800s and early 1900s, and much of the original downtown district, dating from the late 1800s, remains. Both the downtown area, as well as a residential neighborhood near downtown are listed on the National Register of Historic Places.

New Haven's history is promoted and preserved by two museums: The John Colter Museum downtown and the Springgate Museum, hosted by the New Haven Preservation Society. Books about New Haven and John Colter are available for purchase.