What I think you should do,is have some sense of what that women went through.sympatise and show you care. You got your house built but they had to go through the pain. I would at the very least tell her it must have been a right pain for her and as a gesture treat her to a weekend away,which maybe you should have done during the build rather than moving in,and ignoring the fact that people lived through it and it would have been more sincere. You have no idea of the stress of the noise that a build has on people. Have a bit of concern and consideration for those who lived through it and maybe then you wouldnt be seen as not giving a damm about them,apart from giving them your mobile number. You say the builders went over their time on a few occaisions,which may be very little to you,but when you have to live through it 5 and a half days a week belive me every extra minute is very very difficult.Thats my view from both sides and hopefully you will get to see what she went through someday and will understand.hope its of some help that you will see how upsetting and annoying it is for the other person and I understand when you're not there it is difficult to really know.
Best of luck.