The Gunners set up a semi-final meeting against either Reading or Bradford thanks to a 2-1 win over 10-man United at Old Trafford.
Former United striker Danny Welbeck returned to haunt Van Gaal by scoring the winner after Wayne Rooney had equalised Nacho Monreal's opener.
The main talking point from the match was the latest episode in what has been a largely disappointing start to Di Maria's career at Old Trafford.
After being cautioned for simulation, the former Real Madrid midfielder remonstrated with referee Oliver and then grabbed the back of his shirt and yelled at the official when he had turned his back on him.
Replays showed the first booking on Di Maria was harsh and Aaron Ramsey had pulled the player back, but the Â£59.7million man's actions thereafter deserved sanction, according to Van Gaal.
"I think he's touched the referee and that's forbidden in every country, so he has no excuses," the United manager said.
"In Spain he knows that he doesn't touch the referee, but that is also in his emotion.
"I've already spoken with him, he knows my opinion but also I have to see on the video."
Di Maria actually started the game reasonably well, setting Rooney up for his goal with a wonderful curling cross.
But his dismissal clearly angered his manager and the player himself, who refused requests for an interview post-match.
Van Gaal tried extremely hard to look for positives in a tense post-match press conference.
The United boss, who has come in for criticism despite losing only three of his last 23 matches, said Valencia was United's best player before he handed Arsenal a gift-wrapped ticket to Wembley when he sold David de Gea short with a comical backpass, and added that the team had shown "fantastic fighting spirit".
He also bristled at claims he made a big error by selling Welbeck, who has scored eight goals for Arsenal since his Â£16million move to north London.
The Dutchman claimed last summer that he would be better off without the boyhood United fan because he had not scored enough goals for the club.
But the academy product, who grew up in Longsight, just four miles from Old Trafford, made the 63-year-old eat his words as he capitalised on the terrible mistake by Valencia to tap in the winner.
When asked whether he stood by his decision to ditch Welbeck and stick with Rooney, Robin van Persie and Radamel Falcao, the reply was emphatic.
"Yes," Van Gaal said bluntly.
"A lot of times against the club who a player was playing for (he) shall be very motivated but I think we gave that goal (away)."
When it was put to him that Falcao, who did not make it off the bench, had flopped this season, Van Gaal fired another retort back.
"He has scored four goals and (has made) three or four assists," he said.
"And he stimulates also the other players so that is an aspect of a profession but it is now easy for you to say that.
"You are very happy to say that
I see that in your face."
Van Gaal's opposite number Arsene Wenger was happy to see Welbeck score the winner that secured Arsenal's first victory at Old Trafford in nine years.
The Frenchman said he expected Welbeck to be fired up for the game, given that he was facing his old employers.
"We want all to do well when we play against our former club to show we are great players," the Arsenal manager said.
"It is not easy to deal with the mental aspect of it
He focused just on his game and played football the way he wanted to
He has showed mental strength.
"He is intelligent
He is a great talent and has done extremely well for the national team as well."
Wenger was delighted to move one step closer to retaining the FA Cup, but cautioned against expecting an easy win in the next round.
"Last year we played a tricky game at Wembley in the semi-final and you expect that again," said Wenger, whose team progressed to the final last year after a penalty shootout triumph over Wigan.
''I feel we played a good game from the start and we deserved to win the game.''
Wenger hopes the win at Old Trafford will boost his team's chances of qualifying for the Champions League.
"Every win makes you confident," he said.
"We have won at Manchester City and that helped us to win here
When you have won big games, it is more mental
The belief is important."
Source : PA