The problem

Tons of extremely low quality questions from unregistered accounts. A few recent examples:

All of them are now deleted, apologies to those with less than 10K, but we can't keep crap like that around for long. If you'd like me to show you screenshots to get an idea of how painfully horrible these were, feel free to ping me in chat.

The pattern

All these questions seem to me like Stack Overflow rejects, extremely low quality questions about implementation issues / programming tools. Stack Overflow requires users to register before they asked, and I think that some of these questions appeared on Programmers after they were closed & deleted on Stack Overflow and / or the askers' accounts were automatically question banned or suspended. It might just be that noobs are naturally more interested in implementation issues, and Stack Overflow has no involvement in this, but my spider sense is tingling.

We have a few confirmed cases of people posting off topic questions on Programmers to sidestep their question ban / suspension on Stack Overflow, but this isn't easy to confirm for unregistered accounts. Some smells:

  • A handfull of cases where the unregistered accounts posting low quality Stack Overflow type questions shared IPs with quite a few other unregistered accounts that also did the same (it might be an accident, or it might be the same guy/gal taking advantage of the system),
  • A couple of cases where we confirmed the unregistered account was question banned on Stack Overflow, either by them admitting it in comments, or by asking someone with access to the database to confirm it for us.

Moderation issues

Moderating unregistered accounts is extremely inefficient:

  • Suspensions don't work, the only thing the suspendee has to do is clear their cookies and they can post again.
  • Automatic question bans are enforced at a deeper level, but they seem to also fail for unregistered accounts.
  • Deleting / destroying an account for posting very low quality questions is overkill, we typically only delete spam accounts or accounts that posted extremely offensive content.
  • Even for good but off topic questions, migrating to Stack Overflow is discouraged, the unregistered accounts will most likely lose ownership of the question if they don't register there.

Suggestions

  1. Require users to register before asking, same as with Stack Overflow.

    I personally don't like it, but it's the obvious solution. We could do a trial run for a couple of months and evaluate the results.

  2. Tweak the question ban algorithm to be a bit stricter for unregistered accounts.

    This is completely up to Stack Exchange, my thoughts would be:

    • Given the overlap with Stack Overflow, have the algorithm check question banned IPs on Stack Overflow, and not let the account post more than one question, if their first one was downvoted and closed as Off Topic or Not A Real Question.
    • Be more aggressive with unregistered accounts in general. Couple of Off Topic questions, ban.
  3. ???

Hard data

Closure (closure and/or deletion) rates:

  • 35% for unregistered users, 17% for registered (all time),
  • 72% for unregistered users, and 40% for registered (last 6 months, retrieved Aug 10),
  • 70% for unregistered users, and 38% for registered (last 3 months, retrieved Aug 10).

As for people banned on SO:

  • 79 questions asked this month, 39 were closed.

    It seems that science doesn't point at Stack Overflow as the (main) cause of our troubles.


Thoughts?

  • 17
    The answer seems obvious; require registration. Although I know that SE doesn't want to do it on every site, I think the history and nature of this site probably justifies the change. Anecdotally, the question quality on SO improved significantly when they decided to require registration for asking questions. – Robert Harvey Aug 28 '12 at 22:46
  • If someone does run the numbers, can they also provide us with the percent of questions from newly registered accounts that are closed/downvoted too? I'd be interested in seeing how it compares with unregistered accounts. It may be that requiring registration won't do anything to stop the bad questions, although you would be able to take more moderator actions against the account if they're registered. – Rachel Aug 29 '12 at 2:10
  • @Rachel The percentage of closures for registered accounts is a little bit over half the percentage for unregistered accounts. Getting data for newly registered accounts might be problematic, we need to define what we mean by newly registered first, for example does someone who registered today, but has 3K rep on SO, count? – yannis Aug 29 '12 at 2:17
  • @YannisRizos Hrrrmm what about about only counting a user's first question on this site? Or perhaps only count questions asked in the first week of the account being created? – Rachel Aug 29 '12 at 2:43
  • @Rachel You mean the first week the account was created in general? As in any of the network's sites? – yannis Aug 29 '12 at 2:44
  • @YannisRizos No, just the first week the account was created here, regardless of what other SE accounts they have – Rachel Aug 29 '12 at 2:49
  • 1
    @Rachel Given SO's size and the fact that it's our second largest referrer, a SO user discovering ProgSE after they have earned a decent amount of rep on SO is not an uncommon scenario. Rep is partly a metric of familiarity with the SE platform, and if someone is new here but not new on the network, chances are that they will at least read the FAQ before posting. Also, as I've already mentioned, the percentage of closures for registered accounts (in general) is half of the one for unregistered ones, registered accounts aren't really a problem. – yannis Aug 29 '12 at 2:52
  • @YannisRizos It was just something I was curious about - closure/downvote ratio of newly registered users vs unregistered ones. I think it skews the numbers to include all questions by registered users because obviously a long-term registered user who asks many questions is likely to understand the site better,and ask better questions. – Rachel Aug 29 '12 at 3:23
  • @Rachel Hey, the more data we get the better ;) – yannis Aug 29 '12 at 3:24
  • 1
    related - MSO discussion regarding similar change implementation at Stack Overflow: Encouraging users to create an account (and keep it) – gnat Aug 29 '12 at 11:19
  • This site has too high percentage of low quality questions. Try the registration requirement. Try something. – joshp Sep 2 '12 at 23:19
  • @YannisRizos Where did you get the data from? And would it be possible to get the closure rate for questions from newly created accounts, such as accounts created on the same day the question was asked? – Rachel Sep 7 '12 at 17:32
  • @Rachel I got the data from SE, and the data you are asking for is available on SEDE (for registered accounts). SEDE hasn't been updated in a while, but the next time it does it will follow a near real time schedule (weekly updates). Or so they promised ;) – yannis Sep 8 '12 at 12:59
  • @YannisRizos Data.SE is inaccurate because of deleted questions. I know we've been averaging around 50 deletions per day, and possibly more since I can only see the last 50 deleted posts. I was hoping you'd be able to get more accurate numbers. The % closed/deleted questions from unregistered vs newly registered users would tell us if requiring registration would make a difference or not – Rachel Sep 8 '12 at 15:54
  • @Rachel Well then, you'll have to ask SE for the data you want (by doing what I did here, ask a Meta question ;). 50 deletions per day is an extreme figure, where did you get that from? The numbers I'm seeing are way lower... – yannis Sep 8 '12 at 16:05
up vote 14 down vote accepted

In the past 60 days, 76% of all questions from unregistered users have been either down-voted below 0, closed, deleted, or some combination of the three.

That's a total of 182 questions, compared to 877 similarly bad questions from registered users. It's a relatively small number, but as a percentage of the total it's high enough that implementing #1 holds little risk - at worst, you'll lose maybe a question a day from someone too lazy to register.

So, I've turn on the registration requirement in order to ask questions here. Let's check back in a month or so and see if it's made any difference...

  • 877 "bad" questions from registered users are somehow balanced by about 2500 "good" ones from them - right? (here, I am using 38% closure rate reported for registered users) – gnat Sep 8 '12 at 10:27
  • 1
    Yup, the total number of bad is much higher, but so is the total number of worthwhile posts. – Shog9 Sep 8 '12 at 14:58

Due to our perception as a sort of second SO (deserved or otherwise) I think doing #1 would be a good idea, because we constantly have to deal with both our expected growing pains, and fallout from SO as they grow. I think this puts our site in a special category where requiring registration would be justified.

Our site definition seems difficult enough for registered users (SO and our own) of to grasp, hoping non registered users will magically be better at asking on topic questions seems like a pipe dream.

Closure rate 70% for unregistered users

is an awful lot of trouble. In comparison, stats for registered users make one feel that simple act of registration has a magic effect of cutting half of it (to 38%).

...We could do a trial run for a couple of months and evaluate the results.

Assuming that requiring registration for questions can be cancelled if trial run results turn out negative 1 above looks like a low risk test.

Per my reading, such a trial would fit the spirit of guidance provided in Optimizing For Pearls, Not Sand:

...we’re determined to keep question quality high, even at the cost of refusing a little sand. It’s true that you can’t have Q&A without questions, but having the wrong sorts of questions is far more dangerous. The fastest way to kill any Q&A site is to flood it with low-quality questions. I think Mark Trapp summed it up best in this meta answer:

To put it another way, when I go to a Stack Exchange home page, I see a list of questions. If most of those are terrible questions with little to no indication that I’d be wasting my time by reading them, the value proposition of visiting and participating is diminished: I have better things to do.

Compare that to answers on a specific question: I’ve made a conscious choice to look into what I think is an interesting question. I already made the decision that the question is worth my time. If I find the answers to be useless, I have a few different options, as an interested party, to register my displeasure, including writing my own answer. Being able to write your own answer is key: if your answer is good enough, it’ll rise above the junk answers and everyone will be better off for it.

There is no such action for question lists. I can’t say “these questions suck, show me this question I just thought up instead”: that’d be silly. So, it’s imperative the question list have a high signal-to-noise ratio, and removing the penalty for those users who do take the time to read a question and later find it to be useless so they can down-vote is conducive to that.

Fundamentally, answers can be filtered in ways that questions cannot. While there is a tension between having “enough” questions and a bunch of amazing, highly skilled answerers twiddling their thumbs waiting around for something to do, in the long run we’d much rather err on the side of having interesting and on-topic questions for these folks to sink their teeth into.

We feel that the world is awash in questions, but not answers. Answers are the real unit of work in any Q&A system. Therefore, the only logical thing to do is to maximize the happiness and enjoyment of answerers. If this means aggressively downvoting or closing unworthy and uninteresting questions, so be it. Without a community of people willing to answer questions, it really doesn’t matter if there are questions at all, does it?


1 if trial run results turn out negative - (not incidentally,) above guidance also provides a criteria on how to judge trial run results: maximize the happiness and enjoyment of answerers.

PS. As an answerer like one of mentioned above (5+ K rep on answers), I am not particularly happy about current questions quality, which I think is clearly indicated by 5+ K downvotes I cast. One thing I find particularly frustrating is that sometimes, after downvoting bad questions, I find myself out of votes that I would use to upvote good ones - this happens frequently enough to make me feel that better prevention of low quality questions is needed.


update

Registration requirement is turned on:

http://i.stack.imgur.com/LJAQh.png

  • Note that the % tells only part of the story - in raw numbers, registered users post nearly 5 times the crap that unregistered users do (see my answer for details). However, they also post significantly more decent stuff, so I guess we can tolerate keeping them around for a while yet. – Shog9 Sep 8 '12 at 4:14
  • @Shog9 yes I figured there's another part of story and was unsure about it as you point. But the fact that we can run the change as trial and have an option to roll it back makes me believe this data is sufficient to try registration – gnat Sep 8 '12 at 7:39

Due to the very nature of P.SE, I strongly advocate for implementing #1.

It seems rather unlikely to me that a software developer has a quick question that fits P.SE, but is unwilling to register here. Stack Overflow is a much more likely place for one-off questions (because any developer can run in to the help, how to fix Error -733 in YAPEF 2.7? situation, but it takes a certain attitude to discuss the architectural issues suitable for P.SE), and even they do not allow unregistered users to post questions.

Proposal to vote on: If Stack Exchange wants to invest in #2 try #2 before escalating to #1 if needed. If they don't then go straight to #1, if SE allows it.

  • 3
    The merit of #2 depends heavily on whether those people would have been banned by any changes. I'd want to know roughly what % of closed questions would have been prevented – Ben Brocka Aug 29 '12 at 0:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .