27

First of all, I realize that this topic has already been discussed ad nauseam, so I apologize in advance if nothing fruitful comes of this. I strongly feel that there is a large disconnect between the community's understanding of the site's purposes and the collective understanding of the moderators. I'm bringing this up again because I feel that previous answers on this topic have been a bit dismissive, and I was hoping that a more data-driven approach might elicit a more meaningful discussion.

I did a few queries on data.stackexchange.com to get some statistics about the number of questions being closed on each site. Unfortunately, saving the queries is not working properly for me, or I'd link directly to them. Instead, I've tabulated the results in this spreadsheet. I found the following:

Site           | % Closed with Score <= 0 | % Closed with Score >= 5
---------------+--------------------------+--------------------------
Stack Overflow | 66.35%                   | 5.08%
Programmers    | 38.5%                    | 23.3%
English        | 47.47%                   | 6.27%
Gaming         | 47.55%                   | 9.18%

The percent columns represent the percentages of all closed questions with scores less than or equal to zero, and greater than or equal to five, respectively. Overall, a relatively high percentage of questions are closed on Programmers compared to other Stack Exchange sites, but I'm not going to give specific numbers on that because to do so would probably not be constructive.

Please note some caveats about these numbers: the community on each site is different and there is no accepted or target ratio for any of these values. The numbers represent all non-deleted questions from the beginning of each site's existence. It's difficult to make proper comparisons because of differences in the number of users and questions on each site, as well as how mature each site is. I am using these numbers merely to illustrate why I think there is room for improvement.

Without wishing to sound whiny, from a personal perspective I'm rather uncertain about asking any questions on Programmers. Even after reading the FAQ and the Good Subjective, Bad Subjective post, I don't feel like I really have a good understanding of what kinds of questions are acceptable. I think the guidelines do a decent job of explaining what not to ask, but are not nearly specific enough on what should be asked. (For instance, there are four examples of bad questions, but not a single example of a good question.)

Now, I am not suggesting that the mods are wrong, or that we should accept questions which do not fall within the guidelines. What I am saying is that these data, to me, make it very clear that the community and the mods are not in complete agreement about the purpose of the site or about how to interpret the question guidelines.

What I am asking for are suggestions as to how we can:

  • Bring the community's idea of the site in line with official Stack Exchange policies and guidelines.
  • Reduce the number and percentage of closed questions (and resultant disgruntled users).
  • Maintain a friendly and mutually respectful atmosphere between moderators and users.

Thanks!

  • 3
    Thanks, Mark - those questions are good reading on this topic. I do want to state, additionally, that I appreciate the work you guys do and I don't want to make your jobs any harder - especially given that you're volunteers. I just feel that we as a community can do better than we are doing now, probably through better educating users about the exact purpose of the site. I realize that a lot of work has already gone into that and that it's not an easy task, but I strongly believe that there's always room for improvement. :) – Mitch Lindgren Aug 17 '11 at 1:35
  • 3
    There's room for improvement, but rehashing the same issue over and over and over again doesn't really seem constructive. There are some issues with this analysis, which we talked about here. – user8 Aug 17 '11 at 1:37
  • 3
    @Mark I think going through that in an answer here would be a good idea. – Adam Lear Aug 17 '11 at 2:23
  • 3
    +1 for posting data. I have come across many questions that have been closed that (in my opinion, anyway) do not violate the guidelines from the FAQ. – Jay Elston Aug 21 '11 at 22:22
  • 1
    Are you sure that this isn't just a reaction to having your question closed as a duplicate? Does the whole system need to change as a result? programmers.stackexchange.com/questions/101543/… – sylvanaar Aug 22 '11 at 15:59
  • 3
    @sylvanaar, I did not ask that question. As I mentioned, I have not asked any questions on Programmers. I did (and do) feel that that question was closed unnecessarily, and that did prompt me to start investigating the number and scope of question closures on the site, but I did not post this as an angry reaction to that particular closure. My interest in this is strictly out of a desire to improve this community, and the data I gathered are obviously completely impersonal. For what it's worth, I am clearly not the only community member who feels that this is worth investigating. – Mitch Lindgren Aug 22 '11 at 21:01
  • @Mitch Lindgren you are right, you answered it not asked it - sorry for the mistake. – sylvanaar Aug 22 '11 at 21:11
  • 6
    I asked it, however, and the difference between Mitch and me is that I'm prepared to walk away from it and Mitch is prepared to try and discuss changing things so that moderator behaviour and community preference line up better. In my view, that level of interest perhaps deserves more recognition rather than a simple list of possible duplicates. He did go to the trouble of providing data to underline his concerns. – temptar Aug 26 '11 at 9:24
  • 2
    The problem with your stats is that it doesn't take in consideration those who will never post (because they observed the destruction behavior) or those that (tried) to post a few and that will never try again, ever. Nobody knows how much it is, unfortunately. – user2567 Sep 21 '11 at 14:21
  • This is a really great question. Maybe it should be moved to general meta – Revious Dec 16 '14 at 10:39
3

I'll start the discussion by stating that I'd like to see examples of good questions in the FAQ, along with explanations of why those questions are good. I don't mean any disrespect to the hard work that has already gone into the FAQ, but it's clear that a large percentage of community members are finding it an insufficient resource to determine what kinds of questions they should be asking. Examples of existing questions which are on-topic and strictly adherent to Stack Exchange subjectivity guidelines would be very helpful.

  • 1
    You got it the wrong way around. It's not " that a large percentage of community members are finding it an insufficient resource to determine what kinds of questions they should be asking" but the other way around. In the FAQ should stand the common denumerator of questions that are asked (if this is really a community site then the community majority should mean something as well). – Rook Aug 24 '11 at 17:34
-2

Right off the top of my head, go to the top questions by month:

https://softwareengineering.stackexchange.com/?tab=month

Rule out any [closed] questions, obviously -- and possibly anything that was contentious enough to get closed and reopened based on the revision history of the question -- and there's your list of good, on-topic questions.

I'm sure the mods could hand-pick favorite "best of" here, but is that really necessary?

  • 2
    There will be quite a few [closed] questions to be 'ruled out' in this stack exchange. 3 out of the top 6 questions this month are marked as closed. – Jay Elston Aug 22 '11 at 6:30
  • 7
    So the kinds of questions we should strive for are "Why do people use programming books?" and "How do I deal with a slow and undedicated colleague in the team?"....seriously? – user29776 Aug 23 '11 at 15:13
13

I see several types of closed questions on the Programmers' stack exchange that concern me.

  1. Non-duplicate-duplicates: Questions closed as duplicates that are at least some degree different from the question that is cited as a duplicate. (The concern here is that a good question, when asked in a different light, can elicit different, possibly better answers).

  2. Poor questions: Questions closed as not fulfilling enough of the 6 guidelines for a good question that are "on the border" of fulfilling enough of the guidelines that, to the casual observer, appear to be good questions. (The concern here is that closing good questions makes the exchange less valuable as a medium for exchanging information about, in this case, the programming profession).

  3. Mis-labeled closed questions: Questions closed as not fulfilling guidelines that are actually good questions, but clearly off-topic, such as this one on what is an iteration, or this one from a 14-year old beginner. (The concern here is that newbies can become confused between what it means to stay on topic and how to ask good questions. Another concern is that moderation can take the appearance of being incompetent when it lists the wrong reason for closing a question.)

  4. Marginally off-topic questions: Questions that are closed as being off topic that, while not being exactly on topic, have the casual appearance of being on topic, such as this one about interruptions. (The concern here is that closing good questions makes the exchange less valuable as a medium for exchanging information about, in this case, the programming profession. Another concern is that moderation can take the appearance of being in the control of mindless bureaucrats rather than reasoned, pragmatic elders.)

One way to handle this is to ask the moderators to take some extra time to explain close decisions.

For instance, noting that Programmers' stack exchange is for expert programmers would be sufficient to close this one and this one also.

For ***poor questions***such as this one on GIT or this one on regex/regexp, ticking off the guidelines that are not present would really help casual users understand the elements of a good question better. For instance, the regex/regexp question could be closed because it is merely mindless fun, it cannot be answered through facts or references, it does not invite sharing of experiences, it is not asked in an impartial tone, it only elicits short answers, and it will not inspire good answers. (Note -- I am curious about why the regex/regexp question is not a good question...

The entire community can help out with mis-labeled closed questions by flagging then for re-labeling.

When closing, marginally off-topic questions -- either close them with the caveat that they be reworded to apply to expert programmers, or explain why the question is considered off-topic. For instance, the question about interruptions is a real problem for programmers as interruptions impact the productivity of programmers differently than they impact most other professions, so either it can be reworded to apply to programmers, or it should be explained that this is off-topic because other professions (such as mathematicians) also have the same problem with interruptions, so this question will always be considered off-topic in this stack exchange.

I do not know the answer to the non-duplicate-duplicate issue, this one is heavily discussed in meta.stackoverflow as well. It is just one of the minor annoyances of using stack exchanges, I guess.

The moderators for this stack exchange have been very busy lately. I do not know if it is fair to ask them to shoulder the additional thought and care that the above recommendations would entail.

  • 2
    Correct. Sometimes It feels to me that downvoting and closing questions is more a competition than a way to filter the site and instruct others on how to elaborate good questions... – AJC Oct 1 '11 at 17:36
3

Mitch, that's an interesting set of figures and the differences between behaviour on StackOverflow and Programmers.Stackexchange are particularly disconcerting as an indication of the lack of line up between the community behaviour and moderation on Programmers.SE. It says to me that StackOverflow is getting its buyin right in a way that Programmers.SE is not.

I'm obviously not privy to what the site owner wants for the site, but when you build a community based site, you can only exert a certain amount of control over the behaviour of that community before you change the community utterly. In some cases, by getting rid of some poor user behaviour, you can cause the site to grow; in other cases you just kill the site. Or you may not wind up with the community you want. I'm reminded by one comment here above that this is a site for expert programmers. The questions, however, are not expert level questions, and a lot of the more nuanced questions seem to be getting closed because nuance is being missed. This is a key criticism I have about one of my questions getting closed, incidentally. Duplicates are not always duplicates and the list of duplicates which Mark Trapp provided for this question are absolutely not duplicates. They are related, yes, but not duplicates.

Mitch - I am not sure what can be done in response. When there is that level of disconnect between what the site providers what to provide and what the community would like to gain, it's hard to tell any audience "we're not giving you this, so have that instead" because communities just move on. I think there is an issue in that for the most part, elements of the community here see this site as having a hefty element of discussion - but that the questions/answers format is enforced in a rigid manner which tends to kill off discussion for questions to which there are non-binary answers. The response seems to be that those questions which have no straight answers and which thus, generate a large number of answers, are not necessarily good questions per the preferences of the site. While I can see the site's rationale for that, I don't agree that it's realistic. Unfortunately.

Jeff - in response to your answer - given that there are some question marks over mod decisions in terms of closing questions, I would not necessarily be in favour of them being responsible for deciding what the best questions are. There's a voting protocol in place for questions and I'd be inclined to wonder why that would need to be over-ridden - again it boils down to the community not being in tune with the site owners, right?

  • 2
    As I pointed out to Mitch (link to the chat in the comments below the links I provided to other questions), the problem isn't as dire as the vocal minority would like to believe, and saying there's a fundamental disconnect based on a few questions here on meta or the number of closed questions you happen to see is a case of selection bias. The traffic for the site has been going up, not down, while the % closed to asked has been going down, not up. – user8 Aug 26 '11 at 17:22
  • 2
    There's always room to improve, but making the starting point "this site is broken because the powers that be don't want to listen to the community" makes any attempt to improve far less productive, and indeed the one concrete suggestion to improve guidance and isn't just a rant about the powers that be is the currently the lowest voted answer. – user8 Aug 26 '11 at 17:23
  • 1
    Except that's not what I said Mark. Firstly Mitch's figures would suggest that things aren't that great either. I think his figures do suggest there is a disconnect particularly if you compare the Stack Overflow figures which are almost inverse. I'd also add that at no point did I suggest that the site was broken - I did refer to what the owners of the site might want not quite matching what the community hopes to gain from a site and how that impacts community behaviour. Implying that I had a starting point of the site being broken is incorrect. – temptar Aug 26 '11 at 19:20
  • Mitch's figures, if you'll read the chat transcript linked, are inaccurate and misleading, as they only capture one facet of the activity of the site while ignoring others and comparing the results to a control that isn't qualitatively the same as the subject. – user8 Aug 26 '11 at 19:33
  • You said: "When there is that level of disconnect between what the site providers what to provide and what the community would like to gain, it's hard to tell any audience 'we're not giving you this, so have that instead' because communities just move on....The response seems to be that those questions which have no straight answers and which thus, generate a large number of answers, are not necessarily good questions per the preferences of the site. While I can see the site's rationale for that, I don't agree that it's realistic. Unfortunately." – user8 Aug 26 '11 at 19:35
  • 1
    I'm not sure how one can read that as something other than you thinking there's a fundamental disconnect with the site that's causing it material harm. But it's simply not true. While you may disagree with the way the site's run, people aren't "moving on". That's a theme that ties all of these questions together: that something's happening that's causing or going to cause a mass exodus from the site. It's just not happening: the site's growing. – user8 Aug 26 '11 at 19:36
  • I have read the chat concerned now and will accept that a direct comparison may not be possible based on the changes you made Oct 1 last year; it might be more instructive to get hold of figures for the last 11 months. I don't agree that gaming is a suitable comparison, probably because I have zero interest in the gaming site but do have an interest in stackoverflow, I doubt that's unique. However, it may be difficult to quantify the cross over between the two sites. – temptar Aug 26 '11 at 19:45
  • There is a lot of nuance in what I posted - if I meant the site was broken, those are the words I would have used. I did, however, point out that the site would ebb and flow based on how the community perceived it. It's also hard to be certain what the community thinks at any given point in time - hindsight is the best view for that. The site itself may grow but a lot depends on the churn of regular users. – temptar Aug 26 '11 at 19:49
12

Update - October 20th, 2011

Current statistics:

  • Total Questions: 13,440
  • Total Closed but not deleted: 1,321
  • Closed Percentage: 9.38% (▼ 7.56% from September 20th, ▼ 10.81% since this post was written)
  • SE 2.0 Closed Percentage (no Trilogy): 5.36% (▼ 0.72%)
  • Network Closed Percentage (w/Trilogy): 2.56% (▼ 0.04%)

At this point, we have a lower close percentage than Web Applications, Android, English, and Webmasters.

There was some concern that we're bleeding users: not sure how long they've been tracking it, but the Stack Exchange site tracks new 200+ users over a rolling two weeks period: Programmers has consistently had the second highest number of 200+ users (only Stack Overflow itself has beaten us), and the number has gone up since I've been tracking it.

We've also updated the FAQ to be much more straightforward, which will hopefully alleviate some confusion about what is and isn't on-topic here.


While I don't think it's particularly constructive to approach a problem this way, I think it's important, if we do, to work with accurate numbers.

The numbers

As of right now, there are 2,816 closed—but not deleted—questions on Programmers. There are 13,925 questions total on Programmers. This makes the closed, but not deleted ratio of questions 20.2%. In the 10 days since you measured this, it's gone down 3%.

Focusing on the same metrics you did your post, here are number closed with at least 5 score and number closed with less than or equal to 0 score:

  • = 5 score: 774 (27.5%)

  • <= 0 score: 265 (9.4%)

This puts the ratio of 5+ score questions closed higher than what you had and the ratio of <= 0 score questions much lower than what you had.

To understand why, we need to include data you didn't have access to when you did this survey: the number of deleted questions.

The number of deleted questions on Programmers.SE is 2,975: more than the number of closed questions you see. If we also include these numbers into the mix, we get a different story:

  • = 5 score: 891. % of closed or deleted: 15.3%

  • <= 0 score: 2,223. % of closed or deleted: 38.3%

This paints a bit of a different picture than measuring just the closed questions and the results presented in your survey. Why?

It's because we routinely delete low-scoring questions as they are uncontroversially bad for the site, and the scores on those questions agree.

The reason we don't delete nearly as many high-scoring answers is because "closed" doesn't mean "bad". It means the question doesn't meet the site's guidelines in some way, but there's some value, and it could even be improved and reopened.

That is, deletion—not closure—is where bad questions go to spend eternity in Hell. Closure, on the other hand, is the purgatory of questions. They're not on-topic, but they're providing some value: enough to stick around while we figure out what to do with them.

A Comparison

But now that we're working with accurate numbers, how do they compare to other sites, particularly Stack Overflow?

From a purely quantitative standpoint, Stack Overflow's numbers are in a different league:

  • Publicly viewable questions: 1,997,856
  • Closed, but not deleted: ~39,048 (2%)
  • Deleted: 194,008 (8.9%)
  • = 5 score:

    • Closed (not deleted): ~1,964 (5%)
    • Closed + deleted: ~4,142 (1.8%)
  • <= 0 score:
    • Closed (not deleted): ~26,123 (67%)
    • Closed + deleted: ~199,326 (85.5%)

Like Programmers, Stack Overflow's ratios skew a lot higher when deleted low-scoring questions are added, and lower when it deleted high-voted questions are added. This isn't coincidence: the same moderation style is used on both sites.

Some context

But it's important, from a qualitative standpoint, to put these numbers into context. Stack Overflow is not qualitatively similar to Programmers.SE, and comparing their closed/deleted percentage to ours is about as meaningless as comparing our percentages to Quora or Yahoo! Answers. We just simply don't have the same issues or types of questions they have. If we did, there wouldn't be a separate site: we'd just be part of Stack Overflow.

It's also important to put Stack Overflow and Programmers.SE into historical context:

Stack Overflow was the first, and for a long period of time, the only Stack Exchange site. It did not, for much of its history, have the benefit of knowledge gleaned from the year of trial and error with the Stack Exchange 2.0 project. And notably, it did not have the benefit of 40+ other sites with 40+ other sets of problems to crystalize what makes and doesn't make a good question.

This difference in background and two year head start is a constant cause of problems: 27% of all questions migrated here from Stack Overflow are closed and/or deleted. Thousands upon thousands of questions that should be off-topic there and on-topic here never get closed or migrated.

Because of its history as the flagship Stack Exchange site, it also has enormous scaling issues. They have ~10 volunteer moderators for 2 million questions: one for every 200,000 asked. We have 4 for 16,000: one for every 4,000. We are just better equipped to close bad questions.

The history of Programmers.SE

But going into our history a little bit: Stack Overflow had a serious problem with popular questions that didn't really belong anywhere: "What's your favorite programmer cartoon?", "What do you like about programming?", etc. When Stack Exchange 2.0 and the Area 51 engine launched, one of the first proposals to gain traction was Not Programming Related, intended to house all of these types of questions. It launched on September 1st, 2010 and for a period of 3 weeks, surged in popularity as all the questions repressed on Stack Overflow were asked on this new site.

Unfortunately, the quality was all over the place: the site had some of the worst questions and some of the best questions. After a few weeks of this, new guidelines were put into place to try to raise the level of questions.

For the rest of the beta period, these guidelines were enforced sporadically as the community tried to come to terms with its original charter and what it would take to make a useful site.

On December 16, 2010, Programmers.SE was launched under its new mission, having been the test-bed for a number of guidelines about questions that later generations of Stack Exchange sites benefitted from.

This left 3.5 months of questions asked during a particularly formative point in the Stack Exchange network: more than a quarter of our site's lifespan. Compare this to Stack Overflow, which has had the benefit of 2.5 years of maturity and wasn't originally created to be the toilet bowl of a different site.

While we're slowly resolving those initial growing pains, it's going to take some more time. The ratios will normalize, but they aren't going to normalize in the next month: it going to take several months to a couple of years to stabilize into a site as mature as the currently-three-year-old Stack Overflow.

Conclusion

The situation isn't as dire as it might seem from one reading of the numbers. It was dire at one time, but that was last year. Now, we're moving forward: our traffic is increasing and our closed ratio is decreasing. More and more, people are taking ownership of Programmers.SE as something to be proud of, and the quality of questions and answers is noticeably rising.

I think there's a lot we can improve, but it's going to be far more constructive and productive to isolate specific failings and correct those, not to make generalizations about the site as a whole. If we stick to generalizations, we might as well just shut this site down and start over.

  • 10
    Your (updated) stats have nothing interesting to say except what you want them to say. Many users will never come back after how they have been managed. – user2567 Sep 21 '11 at 14:20
  • 2
    @Pierre303 That's moving the goal posts. The question specifically questions the site's policies because of closure rates: now that they're falling, it's a completely different set of problems that's also conveniently difficult or impossible to measure? I get that you don't like how the site turned out, but all the data we do have shows that Programmers.SE is doing fine or improving. – user8 Sep 21 '11 at 18:17
  • 8
    you see it improve because what you did is a sort of natural selection (not to use another word...) in order to keep those who don't care or agree with your vision, and throw out all the others. The objective is therefore not to make the internet better, but to make it you way. Which is perfectly fine, SE is owned by a company is not the humanity. I just wanted to point out your stats are wrong and doesn't reflect the reality. – user2567 Sep 22 '11 at 7:30
  • 3
    @Pierre303 Prove it. Demonstrate how the stats are wrong by providing the real stats. You keep complaining about how Programmers.SE has been ruined: provide some evidence that Programmers.SE is not on the right track. – user8 Sep 22 '11 at 7:42
  • 8
    that's the point: it's impossible. We can't ask every user independantly (and those who never login). I base my point of view empirical observations that include general user behavior on the site and personnal feedback I receive in private (taking such clear positions on the policies helped gather that kind of feedback). – user2567 Sep 22 '11 at 9:41
6

I think it's pretty simple and nearly inevitable. Most of the sites have a single, simple rule : "No subjective questions." P.SE replaces that with a half dozen guidelines about what constitutes a reasonable subjective question.

For a few people who use the site extremely regularly, that's not (much of) a problem, but for most others, it's harder to be sure they're following the guidelines -- especially since most of the guidelines are themselves at least somewhat subjective.

Don't get me wrong: I think the situation is probably open to at least some improvement (what isn't, after all?) but I think it's normal and probably expected that a higher percentage of questions here will be closed, migrated, need editing, etc., than on most of the sister sites.

  • 2
    Exactly. I see a fair number of questions closed that are worthy questions for discussion or answering, and often it's closed only by the Moderator. – Wayne Molina Oct 3 '11 at 17:37
  • 1
    @WayneM I'll grant you that some questions may be worthy of answering, but discussion isn't what we do here. Questions that are discussion-oriented are best served in chat or on other sites like Reddit or Quora. – Adam Lear Oct 4 '11 at 3:21
  • 2
    What "we do here" is defined by "what we do" not by some guidelines on paper somewhere. Overly heavy handed attempts to "engineer" a community are only damaging to said community, IMO. The moderators need to back off and let the community be what it is. – mindcrime Oct 22 '11 at 21:56
11

I am very unhappy with the amount of closing and the types of questions being closed.

I feel very strongly that many of the questions being closed are very appropriate for this site, and the # of upvotes and interest these questions often retain would signify that others agree.

I am using this site considerably less these days due to the narrow range of topics permissible. It is adding significantly less value than it used to.

  • 1
    Popularity isn't always a measure of how well a question is on-topic for a site. If you find a question you think was inappropriately closed, please flag it or make a meta post explaining how you think it fits on the site. We're always open to reviewing closed questions. – Adam Lear Oct 5 '11 at 13:21
  • 9
    @Anna: If a site is community-run, then popularity is almost certainly the only way to determine if something is acceptable. The only question is if the formal norms represent the actual norms. – Paul Nathan Oct 5 '11 at 15:07
  • @Paul - ultimately the "formal norms" are absolutely bleeding irrelevant. Communities are what they are and trying to "social engineer" them into your own vision is usually unproductive in the long run. – mindcrime Oct 22 '11 at 21:58
  • 3
    To add to this: Now it appears that when a question is deleted, any rep you gained from posting the question or an answer is removed as well. This leaves people to ask why they should bother participating in the site when their work may be summarily disappeared at any moment. (It's also interesting to note that of the top 50 questions by votes, 46 are community wiki. Why even have rep on this site?) – Kyralessa Mar 2 '12 at 1:29
-1

Perhaps part of the disconnect is that the site is called Programmers.StackExchange, so people assume the site is about Programmers.

In reality, the FAQ says the site is about "conceptual questions about software development", and questions that aren't related to that get closed.

So we have a community that thinks the site is about Programmers, and moderators that think the site is about conceptual questions on software development.

It probably wouldn't be a popular solution, but if you want a site that's about software development I feel it should be called something like softwaredevelopment.stackexchange.com, not programmers.stackexchange.com. Programmers.stackexchange should be a site about programmers.

And personally, I want to participate in a site about programmers, not one about software development :)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .