I'm not religious by any stretch but I think that is wrong, without religion there would be no conformity, without conformity rules are impossible to enforce so it would be survival of the fittest. And that is the beginning of the end of civilisation as we know it.
Ever been to Japan? A model of conformity in a land not exactly beset by religion.