When Saban took over the MSU team in 1995, they hadn't finished with a winning record since 1990 and they were starting a self imposed scholarship restrictions to avoid penalties from the NCAA from their prior coach. They had to forfeit all games in 1994 due to rules violations.
He then took them to 4 bowls in 5 years and never finished below .500, not to mention he won a lot of big games his last couple seasons, defeating Ohio St twice, ND and PSU who all had far superior talent. His teams had a habit of losing some games they weren't supposed to though (I think he over comes this problem with sheer talent at Bama)... but excelled in big games which got him noticed.
In year 4, his team went 6-6 but beat the #1 team in the country and lost 3 games in the last minute.
In year 5 he took MSU to 9-2 which punched his ticket to Baton Rouge.
What you saw with Saban at MSU was continual process and improvement each year, even though they lost a number of games they shouldn't have (likely due to some game day coaching flaws).
Continuous improvement over 5 years is a sign of a good coach, which is why he was a hot commodity.
I also have no idea what you are talking about with a "weak conference", the Big 10 was much better back then and the SEC was pretty good as well.