About the Capitol
Another good starting point to learn about the building's history (and to learn about almost anything else about Georgia) is the New Georgia Encyclopedia:
One of the most recognizable features of the Atlanta skyline is Georgia's gold-domed state capitol, located several blocks southeast of the center of downtown. Although dwarfed by nearby skyscrapers, the dome glistens from the same hill that supported the first state capitol building in Atlanta.
Early Capitol Buildings
In all, three different buildings in Atlanta have served as statehouse; the first two have been torn down. This story begins in April 1868, when Georgia held a statewide referendum to ratify a new constitution. Article 10 of the 1868 Constitution designated Atlanta as the new state capitol. Shortly afterward the contents of the old Milledgeville statehouse were shipped by train to Atlanta, where there was not yet a building to house the new state government. City and county officials came to the rescue by volunteering the building then used jointly as the Atlanta City Hall and Fulton County Courthouse.
Excerpt from Georgia State Capitol from the New Georgia Encyclopedia, written by Edwin L. Jackson, University of Georgia