An out-of-sorts United side, which included captain Wayne Rooney, David De Gea and Radamel Falcao, toiled for over an hour against their struggling hosts until a piece of individual brilliance set them on their way to a 2-0 win.
Herrera marked his FA Cup debut with a thumping 64th-minute volley that left Yeovil goalkeeper Jed Steer clutching at thin air.
Louis Van Gaal's first experience of this competition was certainly a sobering one and the Dutchman was forced to make a double half-time switch and send on World Cup finalist Angel Di Maria as his side toiled in Somerset.
And it was the former Real Madrid winger who eventually saw off Gary Johnson's plucky outfit in the dying seconds when he clipped the ball over Steer as United countered at pace.
There would have been eyebrows raised in the changing room of League One's bottom club when they saw their opponents' team sheet, but star names Falcao and Rooney were proving to be more useful in their own box than Yeovil's in the first half
The pair were heading away a string of deliveries from the Glovers, who were forcing corners and keeping De Gea occupied.
Yeovil did not look like a side bereft of confidence, having won just one of their last six league games, as they hassled and harried Van Gaal's men.
Captain Joe Edwards was at the heart of their pressing game along with the returning Kevin Dawson, and the midfield duo helped restrict United to long-range efforts from Paddy McNair and Rooney in the opening exchanges.
Keeper Steer, fresh from conceding three goals in Monday night's home defeat to Leyton Orient, probably expected another long afternoon but it took 32 minutes before his gloves were warmed by the visitors.
Even then, James Wilson's low drive was routine for the shotstopper, who then watched on as Herrera fired well wide of his left-hand post just before the break.
Van Gaal's response to a tepid 45 minutes was to withdraw both wing-backs, switch to a back four and send on Juan Mata and Jonny Evans.
Nine minutes into the second half Yeovil carved out a fantastic chance for second-round hero Kieffer Moore but the targetman failed to get any real purchase after being found by Dawson at the near post.
And with an hour gone, and only Wilson's wayward 47th-minute header to reflect on since the interval, Di Maria was sent into the fray for his first appearance since November after injury.
The Argentinian's introduction meant, for the second successive match, Falcao was surprisingly subbed early on.
But United fans had barely had time to digest the change when they enjoyed a perfect view of Herrera sparking this cup tie into life with 64 minutes on the clock.
The Spaniard let a through ball bounce just outside the box before spinning and hooking a right-footed rocket past Steer.
Di Maria was sent on to unlock the stubborn Yeovil rearguard but it was his defensive awareness that denied Yeovil a certain leveller
Sam Foley's corner was met by Edwards inside the box and there was Di Maria to clear off the line.
Yeovil rang the changes in a bid to secure a lucrative Old Trafford replay but they were hit by a sucker punch as the game ticked into the 90th minute, Di Maria dashing clear of the Town backline after Rooney's pass before clipping in the second.
Source : PA