well, if you cannot prove the bank called the builder then you have no case against them on that score. There are several ways you can try to get proof... but if you don't have a good relationship with either the bank or the builder then it may be impossible. So you'll have to let that slide.
You should make written notes now if you haven't done so already. Write down times and dates of calls, to whom, and what was said. Ring the bank, tell them that such and such an individual rang the buiilder on such and such a date, and informed him of your circumstances... you have to get them to confirm that... otherwise all is lost.
Also ring the builder, get him to confirm that he was contacted by the bank...
I would record all calls... this may be considered underhand but if people will say things that they won't put into writing then it is your only choice... This may not be acceptable in a court of law but it's hard to deny a recording... so it may help with an out of court settlement.
The builder hasn't broken a contract apparently, he has simply refused to provide more work.