Raphinha struck the only goal of the game to restore Barca's nine-point lead at the top of LaLiga as Athletic fans protested following allegations the Catalan club bought favourable treatment from referees.
The home crowd threw bank notes marked with "mafia $" and Barca's badge onto the pitch during in the first half and chanted for Xavi's side to be relegated "to the second division."
"I respect the fans at San Mames," Xavi said in his postgame news conference. "They have always treated me well, but I was surprised at the hostile atmosphere towards Barca.
"It was sad to see. [Barca] are being judged before time, which I don't think is good for society. Everyone is free to [air their opinion], and I respect all opinions, whether I agree with them or not, but it made me sad."
Pressed on if he expected the same atmosphere at other upcoming games and if it had an effect on his players, Xavi said: "I don't have anything else to add, [just that] it makes me sad."
"We have not spoken about it in the dressing room with the players," he said. "We are here to win and are focused on the football. We are all professionals."
Prosecutors on Friday filed corruption charges against Barcelona following the revelation of payments totaling €7 million to a company owned by the former vice president of the refereeing committee between 2001 and 2018.
Former presidents Sandro Rosell and Josep Maria Bartomeu, along with ex-referee Jose Maria Enriquez Negreira, also are listed among the defendants.
Barca vehemently deny any wrongdoing and say they have never bought referees or influence and that they were paying Negreira for "technical reports on refereeing."
On Sunday, Real Madrid said they would join the complaint filed by prosecutors against Barca, prompting Blaugrana president Joan Laporta to once again tweet the club's "innocence," claiming they are "the victims of a campaign" against them.
Later on Sunday, Barca won 1-0 in LaLiga for the ninth time in the league this season to strengthen their grip on top spot ahead of next weekend's Clasico against Madrid at Camp Nou.
Athletic, who hit the woodwork twice, thought they had scored a late equaliser through Inaki Williams, only to have their joy cut short when referee Gil Manzano ruled out the goal due to an earlier handball by Iker Muniain.
"I have seen the images, and I don't know it it's more shoulder or arm," Athletic coach Ernesto Valverde said. "But it's not about that; it's a question of what is football and what is not.
"I think VAR is there to correct blatant mistakes. That's why it came in. Big errors. Balls that have gone in or not, penalties. To call over the referee to see if it touches [a hand] or not five passes earlier. I don't know if that belongs in the world of football."
Xavi said, after reviewing the images, it was a "clear handball," but he conceded his team were not at their best against Athletic despite recording a 19th clean sheet in 25 league games.
"It is true we have to play better, but this is a difficult place to come," the Barca boss said. "It's normal to suffer at San Mames, but it's a vital win.
"We knew it would be a difficult away game. We have still got the nine-point lead, and we go into the Clasico with confidence. But we have to improve, above all in attack."