After a 4-3 loss to champions Liverpool on the opening day, Leeds, back in the top flight after a 16-year absence, produced more vibrant attacking football along with defensive jitters.
Patrick Bamford's clinical finish in the 50th minute gave Leeds breathing space and Costa's sumptuous second goal of the game six minutes later had Leeds running riot.
Fulham, promoted via the playoffs, were far from finished as goals by substitute Bobby Decordova-Reid and Mitrovic in quick succession kept the result in the balance.
With the game being played at a punishing high tempo Leeds lived dangerously at times but hung on for their first top-flight victory since April 2004.
Like Leeds, Fulham's style was easy on the eye but Parker's side now have no points from two games, having conceded seven.
"It was a little too close in the end," Bamford, who also scored at Liverpool, told BT Sport.
"I think there will be a few long meetings this week to see what we can do better. The third goal made us panic a bit."