David de Gea produced two brilliant last-gasp saves and Ashley Young also cleared off the line in injury time as United held on to a 2-1 victory at Old Trafford.
Marouane Fellaini gave United a first-half lead and Juan Mata restored their advantage after a Steven Nzonzi equaliser - although Marcos Rojo also claimed to have got the final touch to the decisive strike.
But it was De Gea and Young who ensured United held on to record a fourth successive Barclays Premier League win, with Mame Biram Diouf twice denied and Marko Arnautovic also frustrated.
Van Gaal said: "I am not happy with the performance
I was not pleased with the first half
We were not as aggressive as in the last match and we want to play in the same style.
"I think we didn't keep our positions on the pitch so we didn't create so many chances in the first half.
"Okay the goal was a very nice goal, but the opponents didn't do anything, only defending.
"In the second half we played much better, kept the positions, were more aggressive - but then you have to finish it
(James) Wilson could have finished it, Fellaini could have finished it.
"In England when it is like that you know the coach of the other team shall do something and, with Peter Crouch, we knew in advance it would be difficult to defend long balls and (it took) a great save from De Gea and a little bit of luck from Ashley Young.
"But you have to see the whole match and say there is one team that deserved it and that is Manchester United."
Although Van Gaal felt the performance was unconvincing, a momentum is starting to build behind United.
After an indifferent start to the season, the Red Devils are fourth in the table and just a point behind third-placed Southampton despite a lengthy injury list.
Van Gaal said: "Maybe we forced the luck more and that is coming also because of the fighting spirit
When you see the team (like this) that is why the fans are also excited.
"We have fought until the end and also (despite) the long balls to Mr Crouch, we have survived
It is good for the players, good for the team, good for me and it is also good for the party we have tomorrow."
Stoke manager Mark Hughes felt his side, who lost to an 85th-minute Liverpool goal last weekend, deserved more from the game.
Hughes felt Rojo was offside as he rose in attempting to get his head to the Mata free-kick that proved the winning goal and bemoaned his side's luck in injury time.
Hughes said: "On the balance of play we were very much in the game, especially in the first half.
"It was a really good performance and we picked ourselves up from the disappointment of conceding early on.
"We scored a really good equalising goal, a good strike from Steven Nzonzi.
"We were encouraged at half-time and were looking forward to the second half.
"We are disappointed with the award of the second goal
We clearly think it is offside
It doesn't matter if he (Rojo) touches it, he is clearly offside and interfering with play
He was ahead of the ball in an offside position
The referee and assistant should have cancelled that out.
"We didn't allow that to affect us but the episodes at the end - it was unbelievable, demonstrating the sort of luck we are having at the moment."
Source: PA