Evansville's Sam Cunliffe, a transfer who had previously played at Arizona State and Kansas, hit a pair of free throws with 6.8 seconds remaining to seal the win. Wildcats guard Tyrese Maxey missed a 3-pointer at the buzzer.
The Purple Aces had been picked to finish eighth in the Missouri Valley Conference's preseason poll following last season's 11-21 record and 5-13 finish in conference play in coach Walter McCarty's first season.
"They are a great team, the toughest team to play us," said McCarty, who starred as a player for the Kentucky team that won a national title in 1996. "But I believe in my guys. I told them, 'We're gonna go in there and get them. We're gonna surprise some people today.' I told them, 'This is a great day to be a Purple Ace. Let's go do it.'"
Asked where Tuesday night's win ranked among his biggest moments in Rupp Arena, McCarty said, "This is at the top."
"To be able to come back home and play against the No. 1 team in the country and be able to perform the way that we did, I don't know if anything matches this other than winning a national championship," he said. "It's awesome, man. To be able to come here and play on this type of stage. My guys, I got a good group of guys, I really do. They love each other. They're very connected.
"To bring this group in here and to be able to do that is just awesome; it doesn't get any better than that."
Cunliffe finished with 17 points, and K.J. Riley had 18 points for Evansville, which led much of the game despite shooting just 38.3% from the field.
Immanuel Quickley had a team-high 16 points (5-for-12) for the Wildcats, who were held to 37% shooting from the field and became the first AP No. 1 to lose at home to an unranked nonconference opponent.
"Coach told us we could win, this is a winnable game, and we believed it from the jump," Riley said. "We didn't say they have a lot of guys on the draft board. We have faith in ourselves as a team and the work we put in and the preparation."
Last week, Kentucky defeated then-No. 1 Michigan State in the Champions Classic. On Tuesday night, the Wildcats suffered a home loss to a Missouri Valley Conference team that wasn't picked to finish in the top half of its league.
It's the first time in the AP poll era that two No. 1 teams have lost in November.
"I've done this for 37 years, and things like this happen," Kentucky coach John Calipari said. "It's what you do from here. We may say three weeks later that this is the best thing to happen to us. I've said it from Day 1: The whole thing with this team is toughness."
Kentucky had won 52 consecutive games against unranked opponents at home. ESPN's Basketball Power Index (BPI) had given Evansville a 4% chance to win.
Calipari credited the Purples Aces for being "the tougher team" on Tuesday night.
"If we would've somehow pulled it out, it would have been unfair, because they were the tougher team," Calipari said. "They made shots, and that's tough to do in this building. Give credit to Walter. He had his team better prepared than I had my team. They deserved to win."
The Purple Aces were 25-point underdogs at Caesars Sportsbook, matching the third-largest betting upset in the past 15 seasons in a college basketball game involving two Division I teams.
Kentucky (2-1) was the first No. 1 team in the Associated Press poll to lose to a team that had 11 or fewer wins in the previous season since Stanford during the 2003-04 season, according to ESPN Stats & Information research.
Evansville (2-0) also became the fourth team to secure its first road win against an AP-ranked squad that was the top team in America in the poll.
"It's amazing. Come to a school like this and you talk about doing things like this, making history, playing in front of these bright lights," Cunliffe said. "And, you know, we prepared the right way and we had our mindset in the right spot, and to actually do it is surreal.
"I mean, I couldn't have imagined this ever in my life. So, you know, for our team to see the preparation that we put in and to see something great come from, it is just unbelievable."