Clayton played college football at LSU where he was part of the NCAA champion LSU Tigers. During his three years with LSU, he caught 182 passes for 2,582 yards and 21 TDs for the Tigers, and ended his LSU career with the record for career TD receptions with 21. The record was later broken by Dwayne Bowe in a game against Tennessee on November 4, 2006. He finished second in school history in receptions with 182, just one shy of tying Wendell Davis's record of 183. He is ranked fourth in career receiving yards with 2,582 and seventh in career 100-yard games, and is the only player in LSU history to have at least 700 yards receiving in three straight seasons.