With the kids, I'd try to avoid walking down Bourbon Street, as it's not just a lot of drunks, but the strip clubs seem to be multiplying.
If you are going to Bourbon Street, I wouldn't recommend going down Bourbon from Canal because the first few blocks down Bourbon from Canal are pretty sketchy.
