4

I've noticed that both stack overflow and now programmers are following a similar path. First the sites get going, there's lots of interesting questions and discussions. As the sites become more and more popular there seems to be some issues that make them less enjoyable

  1. Moderation abuse - More people get more points and more moderation power, resulting in more discussions regarding which questions should be closed. While I understand the need for this it seems that a lot of what in the beginning where interesting and popular topics become less and less common.

  2. Irrelevant questions, more closed questions and general lower quality overall. Popularity has it's price and as something gets more popular you'll always lose the "community feeling" and have quality issues. Another problem is that people stop visiting the sites if all they se is negative, closed and stupid questions.

  3. Duplicates, after a while the sheer volume of content makes it hard to ask any questions because it's already been covered. The is partly a problem of spamming the new questions with "noise" and also that you don't feel like contributing to something where "everything"'s already been covered. This problem is aggreviated by stringent moderation that too aggressively shuts down questions that are "kinda" similar

These are hard problems to solve for sure, or you might argue that there is no problem and everything works fine. I do however have a couple of suggestions for possible improvements, feel free to shoot them down :)

  • When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

  • Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

Anyways, just a couple of things to think about. I really appreciate this resource so I'd hate if the level of discussion was lowered due to it's popularity and overall noise.

Perhaps the bias should be a bit more lenient against more discussions type questions if they add real value.

  • 5
    "make them less enjoyable" I think the premise is incorrect. The goal is not ENJOYMENT, but LEARNING. To the extent that we can provide both learning and entertainment, that's great -- but these two concepts are often not only at odds but actively working against each other. – Jeff Atwood Jul 7 '11 at 11:33
  • 4
    Jeff, I have to disagree, enjoyment is a one of the most fundamental factors for learning any skill and learning without it is one of the most difficult things there are. The reason people become proficient at piano, basketball, programming or whatever is not that they have talent as much as they enjoy the activity and therefore spend more time doing it. I think seeing enjoyment as a secondary objective is a mistake. Btw, if you're learning something that's not enjoyable you're probably better off learning something else, that is ;) – Homde Jul 7 '11 at 11:48
  • 3
    please read this very, very closely. scobleizer.com/2009/11/02/… particularly the more conversations I got involved in the less I found I was learning. – Jeff Atwood Jul 7 '11 at 12:01
  • If I'm understanding you correctly your point is that you at every cost want to avoid the stackexchange sites becoming "chat rooms" and keeping them strictly q&a chatrooms will always devolve over time. That's certainly a valid point and a policy that might be good. I suspect there might an other side to the coin though. That a Q&A sites devolve over time as well, both due to experts being crowded out and also because questions are more finite than discussions. As the sites become more encyclopedic the activity of your users will diminish. Just a thought. Anyways, keep up the good work :) – Homde Jul 7 '11 at 12:16
12

When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

This doesn't seem plausible, since Stack Overflow is one THOUSAND times larger than this site, and it has no such question vetting mechanism (other than getting much stricter about the quality filters new users must pass). Granted Stack Overflow is the "concrete, sitting in front of a compiler" yin to Programmers "abstract, standing in front of a whiteboard" yang, but .. based on the data, no evidence this is really required. Super User is also very very large these days and doesn't do that.

Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

This runs counter to everything we have learned about community; the larger the community, the more highly engaged mini-moderators you need to help run the site. I have only seen damage (and lots of it) from sites that were UNDER moderated, never ever ever from sites that had this mythical "too much" moderation.

Perhaps the bias should be a bit more lenient against more discussions type questions if they add real value.

Can you provide examples of this? Remember the gold standard here, in a nutshell, is

1) Can your peers learn anything from this that will advance their professional career?

and

2) Does this solve a real problem your peers are facing?

... and absolutely not "is this fun?" or "is this entertaining?" ; we spend a lot of time suppressing entertainment in favor of education. It's certainly human nature to favor mindless whee discussiony fun over learning, so it's something we have to actively guard against. Constantly ask yourself:

Am I learning anything here?

  • The amount of moderators needed can certainly be discussed and under moderation is even worse. My main thought is that moderation should perhaps be part of a separate interface dedicated to that task. Now it seems we get the moderation but we're still left seeing the "ugly left overs" of that moderation. You might argue that moderation is much more simpler and more likely to occur if it's part of the normal interface, and you might be right. But you could easily switch on and off "moderator" view that displays hides more "noise" (bad questions). – Homde Jul 7 '11 at 11:57
  • I never meant that content that is just fun and entertaining should be promoted or perhaps even accepted. But when thoughtful interesting questions that are borderline of a more discussion variety are systematically killed you might be throwing out the baby with the bathwater. But it's certainly up to you to decide what kind of site and community you want. Just don't fall into the google trap of demanding evidence and statistics for everything, building and maintaining communities is an art, not science ;) – Homde Jul 7 '11 at 12:01
  • I think having moderation powers related to recently earned points rather than total points could be a beneficial change. – Jeremy Jul 7 '11 at 15:21
  • @Jeremy That has potential to become a vicious circle - the more time you spend on moderation, the less time you have to ask and answer question and earn points. And vice versa. – Adam Lear Jul 7 '11 at 15:47
  • @Anna Lear No its self-regulating. If you actually do nothing but moderation then eventually you'll lose moderation powers and be forced to start participating in the site again. It seems to me the whole point is that we want the involved community to moderate; people who are not involved should not be moderating they aren't doing anyone any favors. – Jeremy Jul 7 '11 at 16:27
  • @Jeremy Not asking or answering doesn't mean not reading, not commenting, or otherwise not being present/involved. The existing system self-regulates already: far as high rep users with access to mod tools/close votes go, no single person has the power to fully moderate. All actions require agreement from other users and/or diamond moderator involvement. Those who aren't involved do not generally tend to make the right calls or flag things correctly and therefore don't receive that support. – Adam Lear Jul 7 '11 at 16:29
3

It's only abuse if questions are getting closed incorrectly. :) Do you have evidence of moderation abuse by moderators and 3k+ users? See also the relatively recent Frequently closing popular questions post.

Now on to the suggestions. :)

When sites such as programmers become big enough, switch to "moderation mode" where questions have to be "accepted" by a moderator to show up. This will hopefully reduce noise a lot since moderators will pre-emptively be able to keep off duplicates, irrelevant questions etc etc. I will also reduce or remove the distracting comments about questions validity.

Assuming for a moment that we want to place questions in a limbo they may never come out of, let's define some terms here. Do you mean "moderator" as in "diamond community moderator" or as in "a user with access to a subset moderator tools"?

If it's the former, I don't think your idea would be workable. There are only 4 of us, we have day jobs, and we rely a lot on the community to do the bulk of its own moderation or to at least signal to us via flags what needs attention. Remembering/finding all the duplicates would be a challenge, if nothing else.

If it's the latter... I think there would still be problems:

Make moderation powers available only to a select few. You need enough moderators that content goes through quickly but not so many that the quality of the moderation goes down. Moderation powers should probably be based on being amongst the top x% of users for a period of time (like a quarter). If moderation powers are based only on points than anyone can start moderating by scraping together points over time which might result in some dubious decisions.

You run the risk of not having enough people around to moderate or placing the power, such as it is, in the same hands all the time if the same people end up active on the site. However, if someone is active on the site doesn't mean they're interested in picking up the janitorial moderation duties. With your proposed system, you could end up with the same users consistently in the top X% and not interested in moderating at all. That would be bad, especially if combined with your first suggestion of getting every question approved.

  • I can't point to any specific "abuse", I think it's more a question of editorial "style". Perhaps my preferences differs from others, that's ok, just throwing it out there :) With moderators I mean users who get access to moderator "powers". You probably would have to tinker around with the exact amount and mechanism. It could be something like the top percentage getting a notification asking them if they want to moderate, perhaps for a month at a time so people can take shifts. – Homde Jul 7 '11 at 12:57
  • @mko I'm not convinced that the current system needs changing for something more complicated, but I'd like to thank you for bringing up concrete suggestions and doing so rather politely. :) – Adam Lear Jul 7 '11 at 13:01
  • If you wanted to get really complex you could have something like a (invisible) meta-moderation where a persons proficiency as a moderator where judged based on how well he conformed to the decisions of other moderators of high moderator rating. More proficient moderators would then be asked to moderate before and more often then less proficient moderators. The point system would only act as a seeding round into the moderator system. Hmm, might be too complex, still interesting though :) – Homde Jul 7 '11 at 13:02
  • No problem, glad to be of any, if little, help :) – Homde Jul 7 '11 at 13:03
2

I was thinking about posting a separate question titled "P.SE Considered Harmful?", but I think this question raises the issues I wanted to talk about in a non-snarky way. I come to very different conclusions then @Konrad though.

I hit P.SE when I have a few minutes to kill during compiles and test runs. I'm looking for a few minutes of titillation and amusement and usually find it. I sometimes contribute an answer, but usually only when I have some personal experience, and a narcissistic compulsion to share my anecdote with the world. Can anyone get value (other than entertainment and confirmation bias) from random anecdotes responding to one or two paragraph subjective questions? If my behavior is typical of many users, then I fear that the wisest, most intelligent moderation in the world won't be able to keep P.SE from devolving into noise.

This contrasts with SO, which is a better reference tool than ever, but now much less entertaining. I participate on P.SE because it's easy, but value SO far more highly because it's useful.

Note that I have nothing but appreciation for the job the moderators and the SE staff have taken on, and there may be many users like @Konrad who are approaching P.SE in a mature way. I'm just concerned that having opened up a site to tackle subjective questions, they will inevitably be overrun by barbarians like me, no matter how the moderation is done.

  • Hadn't really thought about it 'til you posted this, but... It's become much more common for me to encounter educational posts on SO since P.SE launched, to the point where I don't dread visiting the top-voted lists anymore. Perhaps this site is serving a useful purpose after all... – Shog9 Jul 10 '11 at 1:29
  • 1
    @Mr. CRT, I've occasionally thought that P.SE is the "sacraficial anode" for SO. (en.wikipedia.org/wiki/Galvanic_anode) – Charles E. Grant Jul 10 '11 at 1:34
  • 2
    @Mr.CRT Perhaps this site is serving a useful purpose after all... I appreciate the humour there, but honestly the flippant attitude towards PSE is starting to get tiring. The issues this site runs into come largely from people not taking it seriously and it's rather frustrating for those of us who are trying to improve it. – Adam Lear Jul 10 '11 at 2:49
  • @Charles That's a good point. I think some (a lot?) of the questions on PSE can benefit from individual experiences, so that's not necessarily a bad thing. There's a difference between sharing personal anecdotes backed by experience/references that relate to a question and posting easy answers. – Adam Lear Jul 10 '11 at 2:51
  • @Anna: I do apologize - and let me take this opportunity to express my appreciation for the monumental and often under-appreciated job you and the other moderators do here, every day and as part of the recent cleanup effort in particular. Finally, while I do tend to down-play their general usefulness, it is clearly helpful that P.SE provides a home for some of the topics that so often proved a distraction on SO - and I remain hopeful that they'll become something more here than they ever were there. – Shog9 Jul 10 '11 at 4:21
  • @Mr.CRT Thanks. :) – Adam Lear Jul 10 '11 at 4:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .