The White House Historical Association says, "The earliest known drawing of the White House is Hoban’s plan, which he drew in 1792... Hoban’s original design had two stories and a raised basement, but some thought the house was too large.
"There was also a question of whether enough quality sandstone could be collected for building such a large house. Stone was also needed to build the Capitol, where Congress would work. George Washington agreed that the President’s House could be reduced to two stories by eliminating the raised basement. He knew that the design would enable future presidents to make additions if they needed more space. As Washington said, the President’s House and the other government buildings would need to change according to needs 'beyond the present day.'
"Hoban was hired to oversee construction. Stonemasons from Scotland, along with free laborers and hired slaves, worked on the building from spring through fall each year until November 1800, when John and Abigail Adams moved in. Adams came to a house that was still unfinished."