Lived in Brighton (or Hove) my whole life - haven't been on the pier for YEARS. Also, never been inside the Royal Pavilion.
You're missing out on an absolute treat, it's superb. Fortunately, I wasn't dragged round it when I was at school which I imagine diminishes anyone's interest of a site as fascinating as the Pavilion. It's a wonder, really it is, go as soon as you can, you won't regret it.
Also, anyone that has never been to the Booth Museum needs to get there now. Superb, a real gem if you have kids.