I just had lunch with a friend of mine, who works in the planning office, so I asked him the question.
He said that there would be no issue from their side, and if there was a complaint, then they would just request a written undertaking that the wall would be reinstated. He also said that they would look favourably on it as the alternative is a skip on the road, which would take up a parking space.
His only caveat is if the wall was a protected structure, there would be an issue, otherwise you're good to go.