I don't believe that to be true, on the whole. I think fans, managers etc. make the ref the centre of attention by questioning his every decision, drawing focus to him. If everyone accepted the refs' decisions and got on with the game, they'd stop noticing him so much.
I also find this complaint a bit... I'm not sure hypocritical is the right word, but something like that... how we complain about ref's making themselves the cetre of attention, then complain that they won't come out to answer questions after games.
Blaming refs is usually an easy refuge for fans who either don't know the rules or continue to view games as if they're kids...so 'we' never foul but 'they' always do etc. There are a few near us at Withdean who abuse the linesman every game for every decision (really they do every time)! They are either letting off steam after a crap week at home and/or work or else they are cretins.
But there is a problem with the FA fast-tracking younger refs. Neither Steve Cook (home v Stockport) or Gavin Ward (home v wycombe) or Oli Langford (Yeovil away) commanded respect from the players. And most players do know when a ref is broadly having a good game. All of those three refs are probably still in their 20s-and it's just too soon for them I think.
If as we hear, some players will 'take the piss' out of managers and coaches and need boundaries and discipline, then imagine how they will approach referees. You've got to ref for many years at park level before you are let near an FL ground in my opinion.