A barrister withdrawing from a case is a very, very serious matter, particularly in circumstances such as you've outlined. There are ethical guidelines and that would seem to be a clear breach of them. They're set out in the [broken link removed] link in the page I've linked to (Word document). there's also a complaints procedure.
First thing in the morning find out if the version you've presented is what's actually happening - there's a fair chance that what is happening is that the barrister is caught in an ongoing High Court action and can't get away from that to deal with your case. This is common enough as most barristers do not just work for one client at a time - the barrister has ethical responsibilities to that client too, and sometimes court dates will conflict, even with the best of intentions.
In such a situation, the barrister may be able to get away from the High Court action for a while to deal with yours, or may be able to get the hearing held over to "second calling" (basically being put back to the queue until later in the day) and deal with it later, or more likely the solicitor will explain that the barrister is unavoidably detained in another court for an ongoing trial, and seek an adjournment. The judge will agree to this unless to adjourn would make the motion pointless, for example, if it was an injunction to stop someone from doing something they are about to do. If the judge were to refuse, the solicitor does have a right of hearing and can run the case if necessary.
Don't worry too much - contrary to popular belief, judges generally want to see justice done, and will not prejudice a party to a case if they can avoid it. The motion will most likely be adjourned to a later date.