I'd imagine they'd want you to bill through a limited company to ensure you can't be considered an employee. I don't think the costs of your own company makes sense if you're making roughly what you're making now, maybe an umbrella co. (ie: you get paid more than you currently do, buy said you wanted to work less, so guessing your new total will be about similar).
You'll lose all benefits except a daily/hourly rate, so holidays, bank/public holidays, redundancy etc etc etc. with a huge one being non renewal as already mentioned. Not all companies value contractors, you may be treated differently, and may not like the new dynamic.
There will be a big cost to do this, just make sure it's worth it.