If the rates have risen since you fixed your mortgage, then there will likely be no fees due. This may be why you have been able to pay extra with no fee.
I'm not sure which rate BOI use, market rate, cost of funding etc. but the break fee is basically to cover any loses of you leaving one...