Located at the intersection of Main Street and Market Street in the City of Poughkeepsie
Known as the Nelson House Annex, the building was renamed as Pomfret House by Walter Averill I, when he bought the Nelson House in 1918, after his hometown of Pomfret, Connecticut.
When the City of Poughkeepsie rerouted Market Street in the 1920s, a passway was built through the first floor of the Pomfret House to allow traffic to flow North to South until the new Market Street was completed.
Pomfret and the remainder of the Nelson House (part was sold to Dutchess County in 1963) continued to operate as the Nelson House Motor Inn until 1969 when both buildings were purchased by the County for offices.
The building served as County offices until 1996 and then was abandoned. In 2012, it was condemned and demolished. New office and retail buildings have been built on the site.