Can we improve our style of self-moderation for not-too-bad questions?

Question

Over the last months and maybe years, I observed some tendency on this site of what I would call “hostile downvoting”. I see often questions which are not-too-badly written (sometimes even well-written), belong clearly to the realm of software engineering, but still get downvotes without any explanation. Often, they get not even a close vote (which could serve as a hint of what the downvoter thought what was wrong with the question).

To make this clear – I see no problem in downvotes without explanation for really crappy questions. Chances are high such questions are closed soon and so the OP get an information about why this happened. I talk about questions which are on-topic and can often be answered sensibly.

Another variant of this hostility are questions which are mostly fine, but in the last sentence the OP ask not only for an approach to solve their issue, but does a minor “rule violation”, for example by asking for external pointers. Such questions immediately get a downvote and close vote for being a “3rd party resource request”, with no further explanation. Often, they could be saved by a small edit, or by simply ignoring the fact there is one minor issue in there which does not make the whole thing a crap question.

If these were just singular, rare cases, I would not have put so much thought into it. However, given the frequency I think I have seen this behaviour, I guess there is a broader problem here, something which might be perceived as a "hostile voting culture".

To give you some examples:

• Are automated UI tests necessary? What should be tested? Well written, definitely on-topic, maybe a little bit broad. But why a downvote with no close vote and no comment on what is wrong here?

• I accidentally overhauled someone's entire project. Any acceptable way to pull request? Well written, definitely not too broad, perfectly answerable, but why two anonymous downvotes and close votes (as too broad).

• How to authenticate devices across organisation boundaries without a third party CA. See my comment below the question – 2 close votes for a well-written question where the OP simply asked “which technologies exist” at the end – which could be fixed by a minor edit. So why those downvotes with no hint to the OP? After I fixed that question, downvotes stopped, but the downvotes stay there – since they were anomymous, we have no chance to inform the casters they could retract them now.

• Consequences of not doing a REST API the “right” way? Again, an anonymous downvote, no close vote, no idea what the downvoter did not like.

• Should a Product Owner be responsible for an initial prototype? – I would understand the two downvotes if someone had cast a close vote of the “primarily opionion based” type, or given an explanation in this direction. But again, no explanation, no close vote, but enough answers to show the question was not too bad.

• How to specify that a number is a random number, or a rounded number, etc - 3 downvotes, no comments, no close votes (ok, 4 upvotes either) for a question which is definitely on-topic, definitely answerable. Ok, the question seems to express a misunderstanding of the OP, but hey, that's exactly the reason why he asked us here, that is IMHO not reason justifying any downvote.

Of course, you may find flaws in all of these examples, none of them is a “perfect question”, of course, but I hope they help to illustrate my point (and I am pretty sure I could quickly add more to this list, all of them from the last three weeks).

I guess some of these downvotes or close votes are

• given in good faith, to keep the site free of too much crap

• cast because the downvoters mistake the “downvote button” for a “I disagree” button

• cast because others are following the bad example of earlier voters

Maybe the "no comment for downvotes culture" has also something to do with the new "Code of Conduct". An honest comment for a question one does not like is not easy to write for everyone in a manner which is perceived friendly enough to not violate the CoC. It is so much easier just to downvote without writing anything - that's always in line with the CoC, at least literally - but is it really? Or is it a quite subtile violation of the spirit of CoC, one which is not easy to prove?

So here is my point:

IMHO by trying to keep the site clean this way, we produce too much collateral damage. Shouldn't we try to change something here?

I think many of those not-too-bad questions could be saved by investing a little bit more time. Some ideas:

• let us read questions more carefully

• count to ten and sit on our fingers before hitting the downvote button

• if the question is no blatantly off-topic, let us take more time to tell the OP what they could improve (and give them enough time to fix)

• try to avoid to over-interpret the site’s rules and the precanned close reasons - and consider not to cast a close vote for every minor rule violation

• don’t vote for closing everything immediately that looks a bit broad as “too broad” or “too opinionated” – some broad questions turn out to be answerable on a second thought (maybe in a broad manner, but still comprehensive).

• after casting a downvote or close vote, take the time and come back later and see if the OP has fixed the issue, and if yes, consider to retract the downvote or close vote.

As mentioned by Amon in the latest moderator election questionnaire, the site shows a decreasing activity across multiple years, and Robert Harvey wrote “we may have gerrymandered the scope of this site a bit too much”. To me, these observations look like symptoms of the same disease – there is a part of our community that is way too fixated on searching for excuses for downvoting and closing a new question instead of searching ways to fix a saveable question or communicate problems.

The site renaming helped us to make the focus of the site more clear to sort out the really crappy questions quicker and to drive away the plz fix my code askers. I think now it is time we should stop to drive away people who understood the site’s scope well and just need a little bit help to get their question asked in a better way.

So finally, here is my question:

• am I the only one who observes this and has a problem with this issue?

• do you agree with my suggestions to improve the situation, or do you have additional or better ideas?

(and if you have a different opinion, I would be happy if you give an answer or leave a comment instead of just abusing the downvote button for saying "I disagree").

Answer 1

I agree. Obviously you can't stop people down-voting questions they feel are bad, but given the low volume of votes questions get in general and how quickly they seem to get downvoted, its got to be some of our prolific users down-voting questions as they appear on the new questions page

I would suggest a different solution though. Upvote more questions.

Answer 2

Indiscriminate moderation actions are bad, more nuance is good. But this doesn't mean that we have a downvote epidemic at hand or that any kind of corrective action is currently required.

In this post, I explain my understanding of when which vote is applicable, and respond to some points raised on this question. It is intended to be suitable as a reference or introduction to sensible voting.

To put this answer and value system into perspective, it is my goal for this site to ensure a consistent stream of interesting, useful, and relevant Q&As about software engineering concepts and software design problems. It is not my goal to be helpful at all costs.

Does Software Engineering have a negativity problem?

Fig 1: Weekly upvotes (top, green) and downvotes (bottom, dark red) since start of 2015. Y-axis does not start at zero. Taken from site analytics.

No. Posts on Software Engineering generally receive multiple times more upvotes than downvotes. Downvotes saw a peak in the Winter 15/16 and have since seen a declining trend, with a slight resurgence in recent months.

When we look at recent questions, most questions have non-negative scores.

However, the above graph lumps all posts together. It is likely that questions see far less upvotes than answers, in particular taking into account Hot Network Questions which mostly benefit answers. The number of total votes also doesn't say anything about the number of votes per post. During the shown time period, the rate of questions has differed dramatically, which could mean that the same number of downvotes is now concentrated on fewer posts. At a glance, downvotes seem to be well correlated with the number of questions, though this would need a SEDE query for a proper analysis.

It is also indisputable that many questions receive baffling downvotes, without these directly leading to negative question scores. The effect is that the site does appear more reserved than enthusiastic.

My interpretation is that while this site generally has a mostly healthy approach to voting, that there does also exist a pattern of unwarranted downvotes. However, these are not terribly relevant in the large scheme of things.

Why moderation is necessary.

A free for all does not work, and would lead to an unattractive mess where valuable content is far and rare. A certain amount of curation and moderation is absolutely necessary.

This means we must curate questions to filter out those that do not meet the topicality or quality requirements of this site. The community is then free to spend its effort on the many non-bad questions we receive as well.

But this also means that we must not gerrymander our criteria until they are so narrow that no question would pass: a consistent stream of quality questions does not just require quality, it also requires questions in the first place.

Moderation actions are not just an inward-facing curation mechanism, but also outward-facing education: we are communicating community norms to askers and onlookers. Ideally, their future questions will be more in line with our norms. But this requires that their feedback is clear, friendly, and actionable.

Unfortunately, no action is also communication: that we don't care. In this sense, any feedback might be better than ignoring a question. Indeed, Stack Overflow data from 2014 “What happens to new users?” shows that both downvotes and upvotes have a similarly large positive effect on user retention, whereas no reaction is even worse than deleting their post.

On the proper use of votes.

Fig 2: Amon's vote calibration scale.

With our votes, we have a number of options available for moderation and curation.

At the most basic level, upvotes reward quality content. Upvotes send a signal that some post is on-topic, correct, well-researched, shows effort.

In contrast, downvotes signal low-quality content, for example when a post shows little effort or grossly misses the scope of the site.

When votes are wielded on principle, that indicates they might be wrong.

For example, minor faults in a question do not warrant downvotes. As illustrated in Fig 2, downvotes are best applied to the really bad stuff, not posts that aren't perfect. There's a large area on the quality axis where no action is needed (but beware of the “no action is also communication” problem discussed above). Instead, edits or comments might be more helpful.

Just like indiscriminate downvotes, indiscriminate upvotes are bad as well. The worst of these are pity upvotes to counter an existing downvote. Such votes sacrifice curation and moderation actions for a false sense of friendliness. Unless the post has downvotes that are clearly unwarranted, please do not vote in reaction to the current score. Vote from your own belief of whether the quality of the post's content warrants a particular vote. Note also that indiscriminate votes deny feedback to the OP.

Close votes/flags are the primary way of enforcing topicality. Not downvotes. When a post is unanswerable within the scope of this site – whether it is too broad or off topic – then closing the post should be the primary moderation action.

Downvotes are orthogonal to close votes. Whereas up- and downvotes are a statement about general quality, closing a question is a statement about topicality and answerability. It is perfectly normal that a good and upvote-worthy question is a bad fit for this site and should be closed, or that a question fits the scope and other formal requirements but shows really little effort and should be downvoted.

There is one unfortunate exception for downvotes: the site mechanics link question score with appearance of the question on the front page, and with eligibility for fast-delete.

• In my perception, overall question quality has improved so that it is no longer necessary to proactively downvote bad questions to push them off the front page. We have so many questions that are fine, we don't need to spend our effort on hiding the few bad eggs.

• The use case of downvoting really bad questions in order to fast-delete them is still valid. However, deletion is only appropriate for irredeemable questions, i.e. questions that are off topic and cannot be edited to become on topic. This likely applies to less than 15% of questions we receive.

Non-solutions.

Shaming. In the end, how someone wields their votes is down to their personal, subjective decision, and that is fine. The votes of other community members will balance things out eventually.

It is OK to discuss how votes are intended or can be used most effectively, but it is not OK to accuse individuals of voting the wrong way.

It's also generally impossible to attribute votes to the correct person. Occasionally, I've tried to help a poster understand what the problems with their question are, only to be accused of downvoting anyway. No! I'm trying to help fix the problems.

Requiring comments for downvotes. This is a recurring request on MSE and MSO, see Why isn't providing feedback mandatory on downvotes, and why are ideas suggesting such shot down? for a canonical discussion. While the system does encourage constructive comments after a downvote, a hard requirement would be damaging to the community. In particular, both comments and votes are legitimate moderation actions on their own.

• Having to jump through extra hoops to place a downvote would discourage their use, thus deprive the community from a curation mechanism and the OP from a feedback mechanism.
• Furthermore, non-anonymous comments would be a target for harassment (see “shaming” above), whereas anonymous downvotes would be a channel to harass the poster. Neither approach seems desirable.
• There is no guarantee a comment would contain actionable advice rather than placeholder text to allow the downvote.

Indiscriminate voting. As discussed above, indiscriminate voting in either direction is problematic and does not serve the community.

• Downvoting nearly everything doesn't provide actionable feedback to posters and distorts how others perceive the community norms. Downvoting due to site mechanic reasons only applies to a small fraction of posts, and cannot be used as a general justification. Please look at Fig 2 to recalibrate the voting scale.

• Upvoting nearly everything makes legitimate curation and moderation activities more difficult, deprives posters of accurate feedback, and provides a false sense of friendliness. Please look at Fig 2 to recalibrate the voting scale.

Conclusion.

While I don't think we have a huge voting problem, we do regularly see votes that are not aligned with community norms. Hopefully, this post helps to promote a healthy approach to voting.

I do agree with Ewan in that there are too few votes in general (this ties in to my moderator questionnaire discussion that slightly too few people are engaging in active community moderation). Perhaps we can use this discussion as motivation to vote more, with any kind of appropriate vote.

Answer 3

I do agree that some questions are unnecessarily down voted, or at least unnecessarily down voted early.

The tooltip on the down vote reads:

This question does not show any research effort; it is unclear or not useful

There are questions that fall into this category. But the questions like the ones that you point aren't don't seem to. They are well written, clear, and perhaps even useful. They just don't meet our community criteria for a good question.

The fast down voting is useful for getting the unsalvagable questions off the homepage. Coupled with votes to close and flags, can ensure that they are closed and/or deleted. But salvagable questions don't need that fast response. In fact, I'd argue that getting salvagable, but broad or opinionated questions that may be salvagable off the homepage hides them from people who can help get the questions into good shape.

I'd say that these questions should be closed, but I'd also recommend holding off down votes and leaving comments instead. If you don't have time to leave a comment, that's OK. But just leaving a down vote seems like an off-putting measure that hurts the asker and perhaps the community by ultimately hiding good questions.

Answer 4

Sorry, Doc, too many active voters see this site primarily as a curated list of beautiful questions, rather than primarily a community where people can ask for help. I don't see the voting patterns changing without some sort of change to disincentivize overzealous moderation.

For example, I think users' moderation privileges (downvoting, close voting, and commenting) should be suspended for a few days if a question they downvoted or voted to close is reopened. That way people would have to stop and think if the vote they are about to take is really aligned with the community as a whole.

Answer 5

What about making votes a currency, rather than a privilege?

It is also possible to penalize casting votes badly, either way. Downvoting a question which doesn't end up deleted is a bad downvote. Upvoting a question which does end up deleted is a bad upvote.

With both proposed solutions, my intention is to make the casting of votes an action which carries a personal investment from the voter.

I perceive that there is a small minority of very highly rated users taking up the imaginary responsibility of "tidying up" this website. This is an extremely common phenomenon in virtually any forum-based online community. And thus these "gardeners" use moderation options liberally without much concern for their impact on the other users. People come to the site for information, not for a "well moderated" community.

The natural energy of these mega-users is an asset which is mostly put to good use on this platform. Yet, it is possible to improve the system further in order to tame those who are excessively zealous.