(warning: I own the top-voted answer on that question - don't think it matters, but just in case.)
In my opinion, it's the sites' users responsibility to downvote answers that:
- do not answer the question directly,
- do not provide any added value (so as to push them away and reduce noise, and encourage their deletion by their owner),
- are of poor quality.
Though if possible we should try to edit answers that could be valuable so they can be rescued and made more releant, and it's best to notify their author first that they're doing it wrong, because:
- they might not know better (yet),
- they might be typing addendums to their answers (I usually sketch mine out gradually, starting with a very basic answer and then fleshing them out as I go in the next 10mins).
So, what I usually do is:
- directly downvote utter crap (let's call a cat a cat),
- downvote interesting but incorrect answer, leaving a comment to their author that I will gladly cancel or even invert my -1 if they address some issue.
Now, when it comes to mods, I think they should feel happy to be proactive and to delete any answer that is a lost cause and just noise, without much warning. There are some questions that have been around a while, and some minor answers on them just often add unnecessasry noise. If there's not one bit of useful information in these, I'd suggest to delete them. If there's some useful information in these, either leave them to stand on their own or merge them into the accepted answer, if possible.
Note that I'm not saying that as a strong problem. I often see old questions getting marked as "protected" if they had good answers and someone jumps in a long time after (say, a 2010 question with already highly-voted answer and where new information is not likely to be added over time as the context may not matter).
Just my 2 cents.
Update
Quick update on what I was saying above about adding a quick -1 to useless answers to favor their removal by their author, or to request an improvement that would warrant removing the -1 and switching it to a +1...
I couldn't remember on which questions I had used this technique, but I just did on this SO question, with good effect (you can see that if you had high privileges or mod rights): a good answer was posted, and a few seconds later a nearly identical one without added value. By upvoting the good one and downvoting the bad one, it created a shift between a +1-rater answer and a -1-rater answer. As you can imagine, the +1 quickly became +4 (even though the other one also got upvoted back to 0). I explained my logic to the one I downvoted, and he kindly removed his answer. This, to me, seems the best approach. However, this is only easy to do right at the time where questions are posted and first answers appear, and the window of opportunity is very, very small.
Afterwards, it is a bit harsh to request deletion if there's a loss of points involved or an emotional attachment to an answer (which sometimes makes me wonder if voting shouldn't be disallowed - or maybe allowed, but with hidden results - for the 10 minutes following the posting, so that early answers can be fleshed out and improved and that it's not a game of "I was first, so the timeshift is in my favor" but of "my answer is actually better, even though I posted it shortly after".
Update 2:
Though apparently, the above approach on that question I listed may have pissed off the person whose answer I downvoted (but that looks to me like it's what downvotes are for and would seem an egoistic reaction) AND may have pissed off others who feel like I was an ass for being so. Not sure, though.