If you are intent on staying close by, go to the City Market. It's right downtown. Hell, go there no matter what, it's easy.
You could also catch a harbor tour in walking distance from the historic district and go to the USS Yorktown and/or Fort Sumter.
For me, the Yorktown was #1. It's not every day you can tour a WWII aircraft carrier (I've also been on her sister ship, the Lexington, in Corpus Christi). It's the second Yorktown as the first was sunk at Midway.