Detail of the main tower. In 1849 James Renwick, Jr. designed "The Castle," nick-named for its red brick, parapets, and high-flying flags. Construction ended in 1855. The Smithsonian is constructed of red sandstone from Seneca Creek, Maryland, in the Norman style, a 12th-century combination of late Romanesque and early Gothic elements.
http://www.150.si.edu/siarch/guide/start.htmRenwick's other works include, among others, St. Patrick's Cathedral, New York City, and the Smithsonian's Renwick Gallery.
http://c250.columbia.edu/c250_celebrates/remarkable_columbians/james_renwick.htmlCommandered for the duration of the Civil War, "the building was returned to Corcoran in 1869. He hosted a lavish ball February 20, 1871, to celebrate the building and to raise money for the long-stalled Washington Monument. The event, called 'the most magnificent reception ever given in the United States,' was attended by President Ulysses S. Grant and included a special balcony for musicians who were accompanied by canaries singing from cages suspended from the ceiling.
"After extensive renovations, the building finally opened as his art gallery in 1873."
http://americanart.si.edu/visit/about/architecture/renwick/