If you're puting down ply, best thing would be to remove the old floorboards, otherwise you'll be raising the level of the floor quite a bit.
Alternatively, if you're not concerned about raising the floor, remove most of the adhesive, and use plenty of flexible gap-filling adhesive between the ply and floor boards. This should provde plenty of support under the ply and allow for any old stuff left behind. Also screw the ply down into place to eliminate any movement.
Leo