If defender makes contact with the ball and it doesn't dramatically change the trajectory he is offside. (you are still offside from a deflection)
I don't think that's right. The defender was trying to play the ball; a deliberate action which means it's not a deflect, so it would be the same as Duffy making contact: not offside.