Depnds - can be anything from nothing-at-all, to knock-it-down-you-chancers. If they're the kind of design changes that can be required by site conditions or upping the specification, then so long as it complies with building regs it should be ok (but if this is ongoing rather than historic, consult the planners - they'll usually give a fairly prompt and clear yea or nay); if changes are significant or non-compliant but the planners don't see it may be fine until the house is sold, at which point you could end up with serious problems/inability to certify clean title, and expensive rectification or a white elephant.