Glenn Murray headed an extra-time winner as the Championship outfit claimed a memorable 2-1 quarter-final win at Old Trafford.
There had been ten changes made to the side by Fergie and although he acknowledged that Palace played well, he expected better from his side.
He told MUTV: "I don't know where to start to be honest.
"My apologies are to the fans because that was not a Manchester United performance.
"I don't want to take anything away from Crystal Palace. I wish them every success and hope they get to the final because it was a fantastic, mammoth effort from them tonight. Every one of their players worked their socks off to try to get to the semi-final.
"I am just very disappointed."