Frei started his youth football by FC Frauenfeld and FC Winterthur, but he came through the ranks at FC Basel. With Basel he was Swiss Champion at both U-16 and U-18 levels. During the 2006-2007 season Frei played in the U-21 team, he played so well that he was given a professional contract.
Frei played his first team debut on 22 July 2007 in the 1:0 home win against FC Zürich. He played a major part in Basel's title and cup win in the 2007-08 season. He scored his first goal for the club on 4 December 2008 during Basel's 3–1 win over FC Aarau at St. Jakob-Park.
In July 2009, due to the fact that Basel strengthened their squad for the begining of the 2009–10 Swiss Super League season, Basels new trainer Thorsten Fink loaned Fabian Frei to FC St. Gallen. During his two year loan in St. Gallen he played 64 league games in which he scored 13 goals.