Nelson attended Chester High School in Chester, Pennsylvania and was a student and a letterman in basketball. In basketball, he helped lead his team to two PIAA state championships.
College career
Nelson began his play at Saint Joseph's University in the 2000–2001 NCAA season. He had a breakout freshman season in which he was named unanimous National Freshman of the Year. During his junior season in 2002–2003, he averaged 19.7 points per game, 5.1 rebounds per game, and 4.7 assists per game. He declared for the NBA Draft in 2003, but remained for his senior season because he wanted to improve his draft status.