Larsson was born in Eskilstuna and initially played football for his home-town team, IFK Eskilstuna. He joined Arsenal at the age of 16 courtesy of an Arsenal scout who watched him play a match for his country. Having been on the substitutes bench for a match against Panathinaikos, he made his first-team debut against Manchester City in the League Cup on 27 October 2004, where he played out of position at left back.
Larsson joined Birmingham City on a season-long loan from Arsenal in August 2006, with an option to make the move permanent. He made an immediate impact, scoring late winners in his first few games against Crystal Palace and Shrewsbury Town. He also scored two goals against Newcastle United in the FA Cup. At the end of January 2007, Larsson signed permanently for Birmingham City on a four-year deal for a £1 million fee. He scored Birmingham's goal of the season, against Sheffield Wednesday in April 2007, running half the length of the field with the ball and finishing from inside the goal area.