But would his agents stop searching for the ring because the White Council went to Dol Guldur?
Maybe I just need to reread the RotK appendix, it's been a while. The treatment of Saruman has never sat well with me, but maybe my memory from the books was off.