Answers quality in hot questions

Question

For few recent months, I've got a habit of downvoting answers which quality doesn't look OK to me.

These probably can be generally described as low effort and/or these lacking relevance to question asked.

• ^{Opinionated slogans, claims that are not backed up by appropriate references or by compelling presentation of personal experience, posts that appear to ignore prior answers covering same grounds, stuff like that...}

As far as I can tell, many (probably most) of my downvotes go to answers in "hot questions".

While I downvote maybe one of the answers to 5-10 "regular" questions, I noticed that almost every question with views over 2K brings answers that look bad to me.

"Hot garbage waves" in the answers appear to happen once or twice a week on average, frequent enough to feel the connection between these and respective questions making their way into SE collider list.

Is that something to worry about?

---

My particular concern is the poisonous effect these mis-answers have on questions, making interesting and well presented problems look the same as non-constructive popularity contests.

• _{"Oh look: 'my guess is fairly prosaic. Git is brilliant...' - with upvoted answer like that it can't be a real question".}

What is especially depressing is that regular ways to deal with this kind of issues just don't work. It's typically not difficult to edit the question to repel garbage answers, I can easily name a handful of active regulars who can and do just that.

Thing is though, it takes some time to figure how to clean up ambiguous wording while preserving the essence of original. In regular questions this works like a charm, but when editing a hot one, I often find out that when I'm done with edit, someone already posted an answer that invalidates my edit. And answer that exploits another minor ambiguity. And yet another, and so on, until my brain explodes!

• ^{It feels like all one gets is just like 60 seconds to figure protective edit to cover every word and letter in the question that could possibly be misinterpreted by some random passer-by and exploited for their senseless cheap shots. That's just... impossible. And more, it feels quite unfair to over-police text of such questions: per my observations "hotness algorithm" have been smart enough to pick questions that have reasonably good wording as-is.}

The way how things work now helps to attract new contributors but I would appreciate if achieving this important goal ^{1, 2, 3} would somehow be less damaging for good, highly visible questions.

protect your own users from scale ...human interaction, many to many interaction, doesn't blow up like a balloon... (A Group Is Its Own Worst Enemy )

---

Here are some additional observations based on discussions that followed posting this question.

• Low quality answers in hot questions likely contribute to the fact that currently, it is hard for users to understand site standards, which results in posting many "bad" questions.

  ^{Taking into account that these are highly visible posts, with thousands of views, it is pretty possible for them to be a steady source of misguided contributors... There are... broken windows... "why can't I post X when Y exists".}

• Until recently, low quality answers also threated to penalize contributors who post good ones because when piled on in hot questions, garbage answers trigger CW which consequently turns all the upvotes given to good answers into zero reputation for their authors.

  more than 1700 answers scored 25+ turned community wiki

For the sake of precision, note that number 1700 above is only indicative since search does not exclude answers turned CW for other reasons. Also note some of the answers could be high-voted "unfairly": per analysis 10+ voted answers in sampled questions (below), this could happen, although rather infrequently.

Background research

For a little data to back up what was written above, I quickly went through few sampled questions with more than 10K views asked for last half year.

Please bear in mind that below list only partially represents the issue: it would be hard to do similar walkthrough for questions with over 2K views since these appear about 10x more frequently.

• How does learning assembly aid in programming?
  20 answers total, I'd downvote about 7-8
  _{Of 5 answers scored 10 or above, 3-4 look OK to me.}

• Why can't the IT industry deliver large, faultless projects quickly as in other industries?
  protected, 27 answers, I'd downvote about 12-13
  _{Of 7 answers scored 10 or above, all look OK to me.}

• What is the Mars Curiosity Rover's software built in?
  protected, 2 answers, all look OK
  _{Both answers are scored above 10 (way above:).}

• Do I need to use an interface when only one class will ever implement it?
  13 answers, I'd downvote about 6-8
  _{Of 4 answers scored 10 or above, 3-4 look OK to me.}

• Is it a good idea to design an architecture thinking that the User Interface classes can be replaced by a command line interface?
  17 answers, I'd downvote about 8-9
  _{Of 5 answers scored 10 or above, 3-4 look OK to me.}

• Why is Clean Code suggesting avoiding protected variables?
  12 answers, I'd downvote about 6-7
  _{Of 3 answers scored 10 or above, all look OK to me.}

• Torvalds' quote about good programmer
  17 answers, I'd downvote about 7-8
  _{Of 5 answers scored 10 or above, 3-4 look OK to me.}

• How can I really master a programming language?
  protected, 14 answers, I'd downvote about 8-9
  _{Of 3 answers scored 10 or above, 2-3 look OK to me.}

• What functionality does dynamic typing allow?
  protected by mod notice, 16 answers, I'd downvote about 8-9
  _{Of 4 answers scored 10 or above, all look OK to me.}

• Is the use of "utf8=✓" preferable to "utf8=true"?
  protected, 3 answers, I'd downvote 2 (especially the one where author didn't even bother to format their code, <meta charset="URF-8">)
  _{The only answer scored above 10 is one that looks OK to me.}

• What triggered the popularity of lambda functions in modern mainstream programming languages?
  12 answers, I'd downvote 2-4
  _{Of 5 answers scored 10 or above, all look OK to me.}

• Does having a higher paid technical job mean you do not get to code any more?
  protected, 4 answers, I'd downvote 1-2
  _{There's only one answer scored 10 or above; it looks OK to me.}

• Is there an excuse for short variable names?
  protected, 16 answers, I'd downvote about 8-9
  _{Of 4 answers scored 10 or above, all look OK to me.}

• Why is the sudden increase in number of Git submitters on Debian popcon graph in 2010-01?
  protected, 11 answers, I'd downvote about 6-7
  _{Of 4 answers scored 10 or above, 2 look OK to me.}

• Why aren't user-defined operators more common?
  12 answers, I'd downvote about 4-6
  _{Of 5 answers scored 10 or above, 4 look OK to me.}

For the sake of completeness, note that besides 199 answers reviewed above, there are also 19 answers deleted in 16 questions listed above. These answers are visible only to moderators and to users with sufficient privileges. Of these 19, 9 are deleted by moderators, the rest has been deleted by owners. Of 10 answers deleted by owners, I would downvote 7-8.

URL used to get above questions is:
  https://softwareengineering.stackexchange.com/search?tab=newest&q=views%3a10000%20closed%3a0

_{Note I wrote "I'd downvote" above since I did not really do that to all the answers I checked because of voting limits.}

Summing up, I would downvote about 101-118 of 218 answers I reviewed per above research.

update

Some data related to recently added observations on community wiki status.

• Questions with 2K+ views - about 25% are CW, 307 of total 1247

• Questions with 10K+ views - about 60% are CW, 62 of total 101

Quoting self, wow. Just... wow.

Answer 1

Most of the "hot" questions with a large number of views have been advertised on social sites outside of SE (reddit, hackernews, etc), so the majority of the views come from Non-Programmers.SE or Non-StackExchange users.

Often these users don't know about SE rules or guidelines for answers, and think they can add something to the "discussion" by posting their own thoughts on the matter.

I looked at a few of your example questions, and a lot of low-quality answers are by low-rep users who had accounts created at roughly the time they posted their answer.

The best you can do in these cases is to downvote the low-quality ones and upvote the high quality ones so the best answers float to the top, and leave a friendly comment explaining why you downvoted so the user knows what's wrong with their answer, and doesn't make the same mistake again.

Answer 2

Most likely reason for the issues with answers quality is a particular bug in a hotness formula ("fake popularity"). This bug causes some of the questions with many answers to be placed unreasonably close to the top of the collider.

Issues caused by this are explained in details in a post about particularly troublesome ("sticky") kind of questions affected by the bug in formula:

One problem with hot questions seems to be that there is no way for "hotness algorithm" to differentiate genuine popularity from fake one, that is from popularity introduced by the algorithm itself... the algorithm simply is incapable to detect cases when popularity is its own byproduct...

For questions that are not too hot... quality is mostly maintained by site / tag regulars (business as usual), collider brings moderate amount of interested newcomers from other sites with their views, votes and fresh perspective, everything is nice and fair...

But when question gets closer to the top of collider, things change... it attracts a lot of visitors whose desire to answer is not balanced by local community norms. At this point "intended life-cycle of a hot question" breaks.

Hotness algorithm assumes the question gets bumped mostly by good answers and that infrequent deviations are corrected by local community through voting and comments. Thing is though, this does not work on top of the collider...

• Post further explains why community moderation ("quality control" through voting, comments and editing by site regulars) tends to break on questions impacted by unnatural hotness score:

  ...There are just too many new visitors to keep things under local community control, and there are just too many new voters and commenters to get things going as designed... So comes next and next and next round of garbage answers, bumping the question, bringing mindless upvotes, over and over and over again.

  Hotness algorithm calculates this as genuine popularity, keeps it accordingly high at collider, bringing more visitors that are breaking things further and so on and so on. Note it's not limited to newcomers only, everyone is invited to the party. Community regulars can clearly see that usual quality norms are broken... and that they are free to follow the New Order just like anyone else.

  That's how fake popularity makes shit stick to the ceiling and helps to keep it there.

• It also explains why there are so much community wiki (triggered by too-many-answers) among questions affected by hotness formula bug:

  Regular open questions ("appropriate" ones) tend to have much less than 30 answers indeed.

  But when we look at "sticky" questions, things seem to be different. These questions aren't "inappropriate", otherwise these would be closed long time ago. Still, 10 examples of sticky questions have more than 270 answers total, about 27 average per question...

  ...at least 2/3 answers in sticky questions are just a garbage that somehow leaked in (probably because "fake hotness" gave question an unnaturally high exposure in collider).

---

Here are feature requests to address this:

• Trial run of modified "hotness formula" for Programmers questions
• In hotness formula, discard answers when voting evidence indicates that these are not good data points

  Please stop counting proven low score answers in hotness formula. Please roll the dice fairly, let user voting and time decay contribute to hotness score as intended. Please promote to collider audience less brain-damaging content to learn from.