Walton Casuals Football Club was formed in 1948 by a group of ex-servicemen and first played in the Surrey Intermediate (Central) League. The team joined the Surrey Senior League in 1969, the same year they moved to their current ground at Franklyn Road.
In 1971 the team switched to the Suburban League and spent the remainder of the decade ground-sharing with neighbours Hampton & Richmond Borough and Addlestone. In 1992 the club joined the Surrey County Premier League followed in 1995 by a step up to the Combined Counties League. In the 2004-05 season the Stags were Combined Counties League champions and were promoted to the Isthmian League. They played one season in Division One before league re-organisation saw them placed in Division One South for the 2006-07 season.