Hi there,
That bar that's located on the cliffs/rocks is called the buza bar. There
are 2 of them and they are both well worth a visit. The drink is a bit more
expensive in there but the experience is well worth it.
We found that the Arsenal restaurant did great food and was not overly
expensive, especially give the location. It's right on the harbour and has the back section cafe/bar facing the Stradun. Nice coffee and cake in there too. Great place to people watch.
I'd definitely recommend walking the city walls and I wouldn't bother with
the headphones. Maybe share one between you! Also, contrary to what we were told, there are plenty of places to get a late drink both inside the city walls and there are a couple of places just outside. One nightclub on the left when you come out the main city gates and one bar across the road on the right.
We also went Kayaking and it was a great day out, but make sure you are
in good form, it's a good workout. There's also a great pizza place just off
the Stradun where you can get a large pizza slice for 2 euro, and it's open
very late.