Besides the technical legalities of the situation one has to look at the whole picture before maikinga decision or, from us, giving advice ?
Is he the sole tenant ?
How long has he been there ?
Besides the heater complaint has he been O.K.? (paid on time and place in good state)
His he paying a rent that is comensurate with property/area?
Is it easy to fill the place if he moves out?
If he's been there for several months and everything is fine (besides heater thing) then I'd say just get a new heater for him, especially if it may not be easy to immediately get a substitute good tenant.
If he's a slow paying pain and you could get more out of another tenant then let him leave. As regards paying for a new heater ...I'm not sure that you could get away with making him pay (out of the deposit I assume). You say "he broke the switch". Did he ? Not easy to prove.
You say the heater is old -or, rather, "obselete". Just because it was working beforehand doesn't mean he broke it ,rather than it broke through natural age, wear and tear.
As a LL it's not a battle I'd fight too strongly.