UConn had argued that issues at Tennessee, which fired coach Holly Warwick in March, met the NCAA's recently revised standards for a waiver.
The school will appeal the NCAA's decision.
Coach Geno Auriemma and the Huskies were hoping Westbrook, a 6-foot point guard from Salem, Oregon, would not have to sit out a year after transferring. That is generally required by the NCAA in the sports of football, men's and women's basketball, baseball and men's hockey, unless players already have completed their undergraduate degrees.
Schools are allowed to apply for waivers to allow players to compete immediately based on specific circumstances that are out of their control. It's the NCAA's discretion whether to grant them.
UConn athletic director David Benedict said he was surprised by the decision "because the NCAA talks to us about serving the best interest of the student.''
Westbrook averaged 14.9 points and 5.3 assists last season for Tennessee. The junior has two seasons of eligibility remaining.
No. 5 UConn opens its season on Nov. 10 against Cal.
Information from The Associated Press was used in this report.