If I haven't talked to you in more than 2 years and you annoy me, I'm either defriending you or removing you from my news feed. You have to be vicious about it as well.
I think I have something like 130 or so friends on there and it works well for me. There's probably 30 or so people that are off my facebook news feed as well since they mostly just posted annoying shit, but weren't assholes or anything like that.