I am not sure if you are trying to find out if its your problem or your neighbours, but if you are: If the fence is a partition between 2 houses then the matter is complicated. It will depend on the way the fence is constructed. As far as I know for a block wall, the wall is the property of the person with the pillars on their site.