1. You have to strip the varnish. This can be done with a varnish remover and painter's scrapers.
2. The stain you want (mahogany) is lighter than dark walnut. If you just apply mahogany stain over the current stain, you won't get the result you want. But if you sand the stain, you risk wearing away some of the wood details/edges. You might trying bleaching the wood to lighten the stain (using vinegar to stop the chemical process when you get the color you want). Remember, you don't need to bleach/sand the wood back to white, just to a light enough shade to get the mahogany stain to look right. The key here is trial and error.
3. I would consider taking the trim apart and doors off the hinges. It's considerably easier and less frustrating to do all of this work on a bench in the garage than having to squat or work around stairs and nooks.
4. Is it worth the effort? Take a look at how much similar doors and trim would cost new. A project like this might take a long time and in the end, it might not be worth the savings you envision now. A successful DIYer knows when to pick his battles.