Some thoughts about noise and moderation

Question

I've noticed that both stack overflow and now programmers are following a similar path. First the sites get going, there's lots of interesting questions and discussions. As the sites become more and more popular there seems to be some issues that make them less enjoyable

1. Moderation abuse - More people get more points and more moderation power, resulting in more discussions regarding which questions should be closed. While I understand the need for this it seems that a lot of what in the beginning where interesting and popular topics become less and less common.

2. Irrelevant questions, more closed questions and general lower quality overall. Popularity has it's price and as something gets more popular you'll always lose the "community feeling" and have quality issues. Another problem is that people stop visiting the sites if all they se is negative, closed and stupid questions.

3. Duplicates, after a while the sheer volume of content makes it hard to ask any questions because it's already been covered. The is partly a problem of spamming the new questions with "noise" and also that you don't feel like contributing to something where "everything"'s already been covered. This problem is aggreviated by stringent moderation that too aggressively shuts down questions that are "kinda" similar

These are hard problems to solve for sure, or you might argue that there is no problem and everything works fine. I do however have a couple of suggestions for possible improvements, feel free to shoot them down :)

• When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

• Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

Anyways, just a couple of things to think about. I really appreciate this resource so I'd hate if the level of discussion was lowered due to it's popularity and overall noise.

Perhaps the bias should be a bit more lenient against more discussions type questions if they add real value.

Answer 1

When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

This doesn't seem plausible, since Stack Overflow is one THOUSAND times larger than this site, and it has no such question vetting mechanism (other than getting much stricter about the quality filters new users must pass). Granted Stack Overflow is the "concrete, sitting in front of a compiler" yin to Programmers "abstract, standing in front of a whiteboard" yang, but .. based on the data, no evidence this is really required. Super User is also very very large these days and doesn't do that.

Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

This runs counter to everything we have learned about community; the larger the community, the more highly engaged mini-moderators you need to help run the site. I have only seen damage (and lots of it) from sites that were UNDER moderated, never ever ever from sites that had this mythical "too much" moderation.

Perhaps the bias should be a bit more lenient against more discussions type questions if they add real value.

Can you provide examples of this? Remember the gold standard here, in a nutshell, is

1) Can your peers learn anything from this that will advance their professional career?

and

2) Does this solve a real problem your peers are facing?

... and absolutely not "is this fun?" or "is this entertaining?" ; we spend a lot of time suppressing entertainment in favor of education. It's certainly human nature to favor mindless whee discussiony fun over learning, so it's something we have to actively guard against. Constantly ask yourself:

Am I learning anything here?

Answer 2

It's only abuse if questions are getting closed incorrectly. :) Do you have evidence of moderation abuse by moderators and 3k+ users? See also the relatively recent Frequently closing popular questions post.

Now on to the suggestions. :)

When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

Assuming for a moment that we want to place questions in a limbo they may never come out of, let's define some terms here. Do you mean "moderator" as in "diamond community moderator" or as in "a user with access to a subset moderator tools"?

If it's the former, I don't think your idea would be workable. There are only 4 of us, we have day jobs, and we rely a lot on the community to do the bulk of its own moderation or to at least signal to us via flags what needs attention. Remembering/finding all the duplicates would be a challenge, if nothing else.

If it's the latter... I think there would still be problems:

Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

You run the risk of not having enough people around to moderate or placing the power, such as it is, in the same hands all the time if the same people end up active on the site. However, if someone is active on the site doesn't mean they're interested in picking up the janitorial moderation duties. With your proposed system, you could end up with the same users consistently in the top X% and not interested in moderating at all. That would be bad, especially if combined with your first suggestion of getting every question approved.

Answer 3

I was thinking about posting a separate question titled "P.SE Considered Harmful?", but I think this question raises the issues I wanted to talk about in a non-snarky way. I come to very different conclusions then @Konrad though.

I hit P.SE when I have a few minutes to kill during compiles and test runs. I'm looking for a few minutes of titillation and amusement and usually find it. I sometimes contribute an answer, but usually only when I have some personal experience, and a narcissistic compulsion to share my anecdote with the world. Can anyone get value (other than entertainment and confirmation bias) from random anecdotes responding to one or two paragraph subjective questions? If my behavior is typical of many users, then I fear that the wisest, most intelligent moderation in the world won't be able to keep P.SE from devolving into noise.

This contrasts with SO, which is a better reference tool than ever, but now much less entertaining. I participate on P.SE because it's easy, but value SO far more highly because it's useful.

Note that I have nothing but appreciation for the job the moderators and the SE staff have taken on, and there may be many users like @Konrad who are approaching P.SE in a mature way. I'm just concerned that having opened up a site to tackle subjective questions, they will inevitably be overrun by barbarians like me, no matter how the moderation is done.