Kenny Dalglish's side took the lead when Daniel Agger punished poor goalkeeping by David de Gea to head home in the first half.
South Korean midfielder Park Ji-Sung equalised for United before half-time, but Dutch international Kuyt had the last word in the first meeting between the two sides since Liverpool striker Luis Suarez was hit with an eight-game ban for racially abusing Patrice Evra in a league game on October 15.
Kuyt's 88th minute goal came after sloppy defending from Evra and allowed the Reds -- who reached the League Cup final in midweek -- to avenge last season's FA Cup third round exit at Old Trafford.
With tensions between both sets of fans running higher than normal after the last meeting three months ago, both Dalglish and Mancheter United boss Sir Alex Ferguson pleaded for calm ahead of the fourth round tie.
Evra, who captained 11-times FA Cup winners United, was booed every time he touched the ball but both teams behaved on the pitch much to the relief of both clubs and the Football Association.
France full-back Evra earned a ticking off from referee Mark Halsey for his lunge on Maxi Rodriguez as the home side settled quickly in a frantic start.
Rodriguez forced De Gea, recalled to United's side after being dropped for the last three matches, into a fourth minute save after letting fly from distance.
United were missing Wayne Rooney, Nani and Phil Jones to ankle injuries, but they responded quickly with veteran Ryan Giggs' half-volley easily gathered by Pepe Reina before Antonio Valencia was denied by the woodwork after a terrific 18th minute attempt.
The visitors were beginning to take control but three minutes later Liverpool broke the deadlock through an unlikely source.
Agger has not scored in a Liverpool shirt since April 2010 but the former Brondby player leapt high above the United defence to nod Steven Gerrard's corner past De Gea, who completely failed to deal with the delivery.
Anfield erupted and Suarez, watching the tie with his young family from the main stand, joined in the celebrations before Park took advantage of slack defending to haul United back on level terms six minutes before the interval.
Instead of building on their lead, Liverpool were content to sit back and were punished when Jose Enrique was hustled off the ball by Rafael da Silva who set up Park to fire past Reina.
It was the least United deserved and came at the end of some real dominance in possession.
They went close to taking the lead for the first time when Giggs, playing in his 70th FA Cup game, produced a cross which Valencia only narrowly failed to steer home at the back post.
Liverpool's appeals for a penalty after Chris Smalling handled fell on deaf ears before Gerrard forced De Gea into a full-stretch save from a 65th minute penalty.
At the other end, Liverpool defender Martin Skrtel made a goal-saving clearance after Danny Welbeck looked certain to score after darting past Reina.
With a replay looming, Dalglish sent on Craig Bellamy, who scored the decisive goal against Manchester City in midweek which booked Liverpool a place in the League Cup final.
But this time it was Kuyt who was Liverpool's hero as the Netherlands striker beat De Gea at his near post after Evra was caught out of position by Andy Carroll's flick in the 88th minute.