How long do you have the phone?
I had the same problem and Apple replaced the phone but mine was in warranty. It seems to be a fault with earlier versions of the iPhone 4. I purchased mine via Vodafone but I walked in off the street to an Apple Store and it was literally replaced immediately as the minute I mentioned the problem it was recognised as a fault.