A nudge with the upper half of your body is just a normal part of the game. If slight nudges like that were always penalised, then there would be dozens of fouls every game. He did foul him slightly before, trying to pull him back - but that was totally unconnected.