Mohamed Salah scored a penalty to make it 1-1 but Jurgen Klopp's side are now 10 points behind City having played a game more.
Gundogan had an eventful afternoon after missing a penalty in the first half. After the match, Guardiola said that goalkeeper Ederson could take the next spot kick.
Both teams started the match slowly but it was Liverpool who had the first chance after 25 minutes when Sadio Mane headed over from close range.
Roberto Firmino also came close but his volley from outside of the area was tipped over the bar by Alisson.
With 10 minutes to go before the break, City were awarded a penalty when Fabinho brought down Sterling in the box but Gundogan blazed it over the bar.
Gundogan made up for his miss by putting the visitors ahead after 49 minutes when he reacted quickest to Alisson's save from Foden's strike.
City went ahead 10 minutes later when Foden capitalized on Alisson's mistake to set up Gundogan for his second.
The Liverpool goalkeeper made another massive error as his stray pass went straight to Bernardo Silva who set up Sterling before Foden scored a fourth moments later.
"It's brilliant, every player wants to play in big games like this and we showed that courage from minute one," Foden said.
"We definitely forced the errors, it's something we're working at in training."
Liverpool boss Jurgen Klopp said that the team side are now targeting Champions League qualification by finishing in the top four.