The first building occupied by the Academy on Audubon Terrace was the administration building at 633 West 155 Street. The building was designed by William Mitchell Kendall of McKim Mead & White and was completed in 1923. It currently houses the administrative offices, members’ room, portrait gallery, library, and several exhibition galleries. The Academy's second building, located at 632 West 156 Street, was designed by Cass Gilbert and was completed in 1930. It adjoins the administration building and contains a large terrace-level exhibition space and a 730-seat auditorium. Upon completion of the second building, two sets of bronze doors, which were the work of Academy member Herbert Adams, and that represent Arts, Letters, Poetry, Music, Painting, Sculpture, Inspiration, and Drama, were installed at the Academy's two terrace-level entrances. An additional set of bronze doors were designed by Academy member Adolph A. Weinman and installed at the administration building's 155-Street entrance in 1938. These are dedicated to the novelist Mary E. Wilkins Freeman, one of the Academy's first female members (elected in 1926), and to the women writers of America. In 2005 the Academy purchased the headquarters building of its former neighbor, the American Numismatic Society. The east wing of the building, completed in 1907, was one of the first structures built on Audubon Terrace. It was designed by Charles Pratt Huntington, whose other works include the master plan of Audubon Terrace and the buildings of the Hispanic Society, Geographical Society, and Museum of the American Indian.

The American Numismatic Society is a national institution advancing the study and appreciation of coins, medals, and related objects of all cultures as historical and artistic documents. Founded in 1858, it remains the second oldest museum in Manhattan. The ANS houses the world's finest numismatic library and America's most comprehensive collection of numismatic objects. The ANS will be moving to its new building at 140 William Street where it will establish the first Museum of Money. Until the museum opens you can come and see what compelling stories coins and bills can tell, such as the recently opened exhibition Drachmas, Doubloons and Dollasr: The History of Money. This is the first tiem that 900 of the Society's most valuable coins, medals and paper currency have been brought together for public view. Presented chronologically, the exhibition highlights the significance of money as propaganda, artwork, and a reflection of social climate and economy. Items on display range in date from a 7th century BC Lydian electrum coin to recent paper money, such as Romanian 2000 lei note commemorating the total solar eclipse on Aug 11, 1999.