Avoid solvents, use clean, cotton buds dampened with clean warm water to pick up dust and dirt from the grooves. Works a treat and won't cause any damage. When you have removed the stubborn dirt your records can be kept clean very easily using a "velvet sponge" type cleaner available from record stores.
Or you could play them and try not to listen to them first time round, I have recently acquired a Rega P3, and some LP's were old, i am not too fussy about the stylus quality as my LP's are only a small % of my overall collection. The stylus might get a bit more wear with this method though