Rhule got a seven-year deal at Baylor, sources told ESPN's Brett McMurphy.
Rhule, 41, replaces former Bears coach Art Briles, who was fired in late May in wake of the school's sexual assault scandal. Rhule, a native of State College, Pennsylvania, has a 28-23 record in four seasons at Temple.
"We could not be more excited," Baylor athletic director Mack Rhoades said in a statement. "... When we set out on our search for a new leader of our football program, we wanted a coach who shared our values, who had demonstrated success, who showed a true commitment to the overall student-athlete and who we believed could lead Baylor to a national championship.
"We found all of that and more in Matt and I know he will be a perfect fit with the Baylor Family."
Jim Grobe, who guided the Bears to a 6-6 record as acting coach this season, will step down after Baylor plays Boise State in the Motel 6 Cactus Bowl on Dec. 27.
Rhule takes over a Baylor program that has been plagued by off-the-field problems for much of the past year. In late May, the Baylor Board of Regents -- after hearing the Pepper Hamilton findings on how the university handled sexual assault complaints -- fired Briles, sanctioned athletic director Ian McCaw and removed Ken Starr from his role as president. McCaw and Starr would later leave the university altogether.
"I am truly honored and humbled to join the Baylor family," Rhule said in a statement . "Baylor is a tremendous institution with a history of football success and I know the passion that so many have for the Bears will help bring the community together to reach even greater heights. I am excited to get started."
In Rhule's first season at Temple in 2013, after inheriting one of the worst programs in the FBS, the Owls went 2-10. Temple went 6-6 in his second season before breaking through with a 10-4 campaign last season.
This season, Temple finished 10-3 and defeated No. 19 Navy 34-10 in Saturday's AAC championship game. The Owls will play Wake Forest in the Military Bowl Presented by Northrop Grumman on Dec. 27, and sources told ESPN that assistant head coach/tight ends coach Ed Foley will serve as the interim head coach.
While much of Rhule's coaching experience happened in the Northeast, he seems a good choice to take over the football program at the world's largest Baptist university. His father, Dennis, attended the Nazarene Theological Seminary in Kansas City, Missouri, and was a longtime pastor at a church near Times Square in New York.
Rhoades also considered Arkansas State's Blake Anderson, SMU's Chad Morris and California's Sonny Dykes before deciding to hire Rhule.
On Monday, Baylor set up a Tuesday meeting with Oklahoma State's Mike Gundy, but that meeting between Gundy and Baylor officials was canceled Tuesday morning, a source said.