Ask around for other spots. I don't really eat anywhere fancy when I am down there.
Places to go:
Snorkeling:
(1) dry tortugas - this is a cool fort, with lots of great snorkeling around it.
http://www.nps.gov/drto/index.htm
(2) hit the back country - take a boat tour, or a snorkeling tour out here. Allegedly, it is the best place to go on the water
(3) you can swim with dolphins on one of the trips
Nightlife:
(1) Irish Kevin's on Duval is my favorite place. There is always a guy playing an acoustic guitar there. It is normally a really fun time.
(2) Fat Tuesday's - maybe a block away from Irish Kevin's. Lots of frozen drinks. It's a cool place to hang when the weather is nice (it's always nice).
(3) Rick's - a dance club. Shake your moneymaker.
(4) Bull and Whistle bar - head upstairs to the Whistle Bar. It is a rite of passage for everyone that goes to Key West.
(5) Sloppy Joe's - I'm not a fan of Sloppy's, but you should check it out anyway.
Pools:
(1) Dante's - if you want to hang by a pool, this is the place: http://danteskeywest.com/
Other stuff:
(1) Hemingway House - the place is huge. It is cheap to get into, and it is a nice way to kill a bit of time.
(2) Southern Most Point - a photo at the SMP is a must