Hi Foxy,
I think you need to make an appointment with your solicitor and discuss where you stand in detail. No-one can give you good advice without seeing your contracts and building agreement. Tell your solicitor that you want to pull out, and that you want to get your deposit back. Ask him/her what can be done to achieve this, and what if any risks are attached to that option.
I'm not putting you off, it's just that sometimes you need to just take the time to sit down with your solicitor, ask all your questions, and make it very clear what you want to achieve.
Kate.