In my experience the only people who don't like wearing speedo's are those who are self conscious of the size of their penis and those who actually have a small penis.
All depends in which boat you sit really.
It's not the size of my penis that causes me worries - it's the mountain that stops me seeing said penis !