A lot of friends of mine recommend staying on the island.
I stayed in a budget hotel in Mongkok and it was pretty grubby at night but kind of interesting all the same. We found it very hard to get somewhere cheap because we left it rather late to book, unlike when we went on to Thailand the following week where we had no problem at all getting somewhere nice on a budget.
The fake handbags by the way are just that - fakes. Probably the same thing as what you'd get in Pennys really.