Aarhus GF owes its name to roots that have little to do with football: Aarhus Gymnastikforening, or the "Aarhus Gymnastics Association," was founded on 26 September 1880, with gymnastics and fencing as its main sports. The football department was created in 1902, and its first match, played that November against Aarhus Idrætsklub Olympia, ended in a 5–2 defeat. Supporters call the club "De Hvide" (The Whites) and "Byens Hold" (The City's Team). The side won the Jutland championship in 1908 and joined the newly created nationwide league competition in 1927.
After a single season in the second tier (1952/53), AGF returned to the top flight and over the next twelve seasons won four Danish championships (1955, 1956, 1957, 1960) and five Danish Cups. The period also brought the club's first European experiences. In the inaugural European Cup (1955/56), AGF lost its first-round tie to Stade de Reims 4–2 on aggregate, with Léon Glovacki scoring three of the French side's goals. In 1960/61, as national champions, the Danes reached the quarter-finals, where they were eliminated by the eventual winners, Benfica. That generation included Henning Enoksen, Henry From and Hans Christian Nielsen, silver medallists at the 1960 Olympic Games, as well as John Amdisen, who won four titles and five cups with the club between 1955 and 1965.
The second period of success came in the 1980s. After returning to the top division in 1976, the club spent thirty consecutive years there. In 1982 it came close: a 2–2 draw with B.93 on the final day handed the title to Odense Boldklub, and in 1984 AGF lost out to Vejle Boldklub by a single point. In 1986 the fifth championship finally arrived, thanks to players such as Flemming Povlsen, Jan Bartram and John Stampe, the last of whom made 444 appearances for the club between 1977 and 1991. Danish Cup wins followed in 1987, 1988 and 1992. In 1988/89 AGF reached the quarter-finals of the Cup Winners' Cup, losing 1–0 on aggregate to the eventual winners, FC Barcelona.
The most memorable season without a title was 1995/96. The team, featuring Stig Tøfting and Håvard Flo, led the league for long spells, but in round 30, with the score at 3–3 against Brøndby IF, the visitors' goalkeeper Mogens Krogh headed in an equaliser, still remembered as the decisive moment of that race. Brøndby took the title and AGF finished second, though it did beat the same opponent 2–0 in the Danish Cup final. The rivalry with Brøndby is now regarded as the club's most important, while the fixture against Aalborg BK, known as the "Battle of Jutland," pits two of the oldest clubs in the region against each other.
AGF's historic home was the stadium in Aarhus, opened on 5 June 1920 in the presence of King Christian X and designed by Axel Høeg-Hansen. Its attendance record of 23,990 was set on 23 October 1962 in a 4–0 home defeat to Esbjerg fB. The ground was rebuilt between 1999 and 2001, and demolition began in early 2024. Since February 2025 AGF has played at the temporary Vejlby Stadium in the north of the city. The new ground, designed by Zaha Hadid Architects under the working name "Skovens Arena" ("Forest Arena"), is meant to hold around 24,000 spectators, with the opening planned for 2027.
After 2000 the club went through serious financial and sporting problems. It was relegated from the top flight in 2006, 2010 and 2014, returning after a single season each time. Since 2014 the club has been led by Jacob Nielsen, and bronze medals in 2019/20 and 2022/23 pointed to improvement. In 2025/26, the first season under head coach Jakob Poulsen, a former player, AGF finished first with 67 points (19 wins, 10 draws, 3 defeats). The title was sealed by a 2–0 away win at Brøndby on 10 May 2026. It was the club's sixth championship and its first in forty years, and it earned a place in the Champions League qualifying rounds.