FK Sarajevo was founded on 24 October 1946 through the merger of two local clubs, Udarnik and Sloboda, and adopted its current name three years later. Among the club's founders was Safet Džinović, its first chairman, while in 1948 Yugoslav footballer Miroslav Brozović joined as player-manager and led the side to promotion to Yugoslavia's top division.
In the 1966–67 season, FK Sarajevo became the first club from Bosnia and Herzegovina to win the Yugoslav First League title, earning a place in the European Cup. In 1967–68 the club eliminated Cypriot side Olympiakos Nicosia before facing Manchester United in the next round; after a goalless draw at home, Sarajevo lost 2–1 at Old Trafford to the eventual champions. The club claimed a second Yugoslav title in the 1984–85 season.
Following Bosnia and Herzegovina's independence, FK Sarajevo remained one of the country's leading clubs, winning five Bosnian Premier League titles and eight national cups, making it the third most decorated club in the country's top flight, behind HŠK Zrinjski and FK Željezničar. In the 2018–19 season, the club completed a league-and-cup double. Malaysian businessman Vincent Tan owned the club from 2013, and since March 2019 it has been run by the Vietnamese investment group PVF, headed by Nguyễn Hoài Nam.
The team plays its home matches at the Asim Ferhatović stadium, named after one of the club's most celebrated players, wearing the traditional maroon-and-white colours that give rise to its popular nickname, the "Maroon-Whites".