Personally I find unexplained down-votes very frustrating, also IMHO, this isn't constructive criticism. On the contrary it promotes bad behavior..
I suggest enforcing adding a comment when casting a down vote, rather than just giving a hint. It can be anonymous, at least for the person being voted on
UPDATE: If it would make any difference, please, note that this is for both the answer & question down-votes