The Manchester United manager hit out at Roberts for going against the campaign, with the Reading striker believing the anti-discrimination body have failed to tackle racism over the last year in the wake of the John Terry and Anton Ferdinand case.
Roberts said on Thursday: "I find it hard to wear a T-shirt after what has happened in the last year. I won't wear one.
"I'm totally committed to kicking racism out of football but when there's a movement I feel represents the issue in the way that speaks for me and my colleagues, then I will happily support it.
"I think people feel let down by what used to be called 'Let's Kick Racism Out of Football'. People don't feel like they have been strong enough."
But Ferguson believes every player in the country should back the plan to ensure the right message is given and insisted all his players - including Rio Ferdinand whose brother was involved in the John Terry case - will wear the shirts before the game at Old Trafford.
"I have to disagree with Jason Roberts, he is making the wrong point," Ferguson said. "Everyone should be united, all the players in the country wearing the top, the warm-up tops.
"I do not know what point he is trying to make or trying to put himself on a different pedestal to everyone else.
"He really should be supporting all the rest of the players who are doing something. If you are doing something then everyone who believes in it should do it together, we should not have sheep walking off. He is making the wrong message.
"Yes, all my players will wear it. All the players will be wearing it, except for Roberts who seems to be different."
Ferguson has revealed Ashley Young will return to action against Stoke after a lay-off with a bruised cartilage and is hoping the former Aston Villa winger can add more attacking threat to his forward-laden United side.
"I don't think he is forgotten, it's always the same when a player is injured. Very few players cross the ball as well as he can, his goal-scoring ratio is very good.
"I don't think you can forget that or underestimate it. I think any player with the ability to come to our club from Aston Villa will want to do well."
The return of Young and the recent comeback of Darren Fletcher added to the good form of Tom Cleverley means Ferguson is more than happy with his midfield options despite claims United are light in the area.
He added: "Young coming back gives us another option. The strength in our squad is in the attacking options and we are getting better in the midfield with Darren Fletcher coming back.
"Cleverley is also doing well and Anderson. All the fears about midfield positions are evaporating."