People get plenty of feedback on downvote reasons, just not usually from the voter. People frequently leave comments without downvoting, and if someone later downvotes for the same reason I've seen them actually apologize that it wasn't them. Same goes if the downvote came before the comment: "I'm not the one who downvoted you, but…"
Heck, authors often preemptively put downvote reasons into their own posts: "This will probably be closed as not constructive, but…" Then they're shocked when someone downvotes them without providing a reason. I downvoted an answer just yesterday with two disclaimers that he knew absolutely nothing about the topic (he was right).
If they're still mystified about downvotes without a reason, they ask in a comment and get answers 9 times out of 10. If they don't get an answer from their comment, they can ask in meta, where I've yet to see such a question go unanswered.
As far as tactical downvoting, I think the -1 does a pretty good job 99.9% of the time, and when it doesn't, it's usually corrected quickly with sympathy upvotes. You need to judge from the end result, not from a temporary sub-optimal state.