Previous owner left rubbish in house

We rang our Estate agent and requested a pre inspection visit to ensure that the attic and shed were empty. She rang the vendors who advised that this had been done.

Rather than take anyone's word, I requested our solicitor wrote outlining the Clause in our contracts i.e House in good order, etc. and requested that attic / shed etc where cleared out. Just to show that we'd be following up with them if they didn't.

I am finding myself in a similar situation. About to move in to a new home but the previous owner has left behind a few items (including a matress).

The vendor was contacted and agreed to move all items. They went to the property to leave a set of keys but did nothing about the remaining items. So now they have no keys to the property and the items remain.

I'm astonished at the cheek of some people - I'm no saint myself but I would never do something like this. Don't want to get too 'hippy' on it but it saddens me to think that there are people that would deliberately try to get one over on their fellow man.

What is my best course of action at this stage, as I don't intend to let this lie. Is the Small Claims Court an option (for the costs I will incur in removing the unwanted items).