Update - October 20th, 2011
- Total Questions: 13,440
- Total Closed but not deleted: 1,321
- Closed Percentage: 9.38% (▼ 7.56% from September 20th, ▼ 10.81% since this post was written)
- SE 2.0 Closed Percentage (no Trilogy): 5.36% (▼ 0.72%)
- Network Closed Percentage (w/Trilogy): 2.56% (▼ 0.04%)
At this point, we have a lower close percentage than Web Applications, Android, English, and Webmasters.
There was some concern that we're bleeding users: not sure how long they've been tracking it, but the Stack Exchange site tracks new 200+ users over a rolling two weeks period: Programmers has consistently had the second highest number of 200+ users (only Stack Overflow itself has beaten us), and the number has gone up since I've been tracking it.
We've also updated the FAQ to be much more straightforward, which will hopefully alleviate some confusion about what is and isn't on-topic here.
While I don't think it's particularly constructive to approach a problem this way, I think it's important, if we do, to work with accurate numbers.
As of right now, there are 2,816 closed—but not deleted—questions on Programmers. There are 13,925 questions total on Programmers. This makes the closed, but not deleted ratio of questions 20.2%. In the 10 days since you measured this, it's gone down 3%.
Focusing on the same metrics you did your post, here are number closed with at least 5 score and number closed with less than or equal to 0 score:
= 5 score: 774 (27.5%)
- <= 0 score: 265 (9.4%)
This puts the ratio of 5+ score questions closed higher than what you had and the ratio of <= 0 score questions much lower than what you had.
To understand why, we need to include data you didn't have access to when you did this survey: the number of deleted questions.
The number of deleted questions on Programmers.SE is 2,975: more than the number of closed questions you see. If we also include these numbers into the mix, we get a different story:
= 5 score: 891. % of closed or deleted: 15.3%
- <= 0 score: 2,223. % of closed or deleted: 38.3%
This paints a bit of a different picture than measuring just the closed questions and the results presented in your survey. Why?
It's because we routinely delete low-scoring questions as they are uncontroversially bad for the site, and the scores on those questions agree.
The reason we don't delete nearly as many high-scoring answers is because "closed" doesn't mean "bad". It means the question doesn't meet the site's guidelines in some way, but there's some value, and it could even be improved and reopened.
That is, deletion—not closure—is where bad questions go to spend eternity in Hell. Closure, on the other hand, is the purgatory of questions. They're not on-topic, but they're providing some value: enough to stick around while we figure out what to do with them.
But now that we're working with accurate numbers, how do they compare to other sites, particularly Stack Overflow?
From a purely quantitative standpoint, Stack Overflow's numbers are in a different league:
- Publicly viewable questions: 1,997,856
- Closed, but not deleted: ~39,048 (2%)
- Deleted: 194,008 (8.9%)
= 5 score:
- Closed (not deleted): ~1,964 (5%)
- Closed + deleted: ~4,142 (1.8%)
- <= 0 score:
- Closed (not deleted): ~26,123 (67%)
- Closed + deleted: ~199,326 (85.5%)
Like Programmers, Stack Overflow's ratios skew a lot higher when deleted low-scoring questions are added, and lower when it deleted high-voted questions are added. This isn't coincidence: the same moderation style is used on both sites.
But it's important, from a qualitative standpoint, to put these numbers into context. Stack Overflow is not qualitatively similar to Programmers.SE, and comparing their closed/deleted percentage to ours is about as meaningless as comparing our percentages to Quora or Yahoo! Answers. We just simply don't have the same issues or types of questions they have. If we did, there wouldn't be a separate site: we'd just be part of Stack Overflow.
It's also important to put Stack Overflow and Programmers.SE into historical context:
Stack Overflow was the first, and for a long period of time, the only Stack Exchange site. It did not, for much of its history, have the benefit of knowledge gleaned from the year of trial and error with the Stack Exchange 2.0 project. And notably, it did not have the benefit of 40+ other sites with 40+ other sets of problems to crystalize what makes and doesn't make a good question.
This difference in background and two year head start is a constant cause of problems: 27% of all questions migrated here from Stack Overflow are closed and/or deleted. Thousands upon thousands of questions that should be off-topic there and on-topic here never get closed or migrated.
Because of its history as the flagship Stack Exchange site, it also has enormous scaling issues. They have ~10 volunteer moderators for 2 million questions: one for every 200,000 asked. We have 4 for 16,000: one for every 4,000. We are just better equipped to close bad questions.
The history of Programmers.SE
But going into our history a little bit: Stack Overflow had a serious problem with popular questions that didn't really belong anywhere: "What's your favorite programmer cartoon?", "What do you like about programming?", etc. When Stack Exchange 2.0 and the Area 51 engine launched, one of the first proposals to gain traction was Not Programming Related, intended to house all of these types of questions. It launched on September 1st, 2010 and for a period of 3 weeks, surged in popularity as all the questions repressed on Stack Overflow were asked on this new site.
Unfortunately, the quality was all over the place: the site had some of the worst questions and some of the best questions. After a few weeks of this, new guidelines were put into place to try to raise the level of questions.
For the rest of the beta period, these guidelines were enforced sporadically as the community tried to come to terms with its original charter and what it would take to make a useful site.
On December 16, 2010, Programmers.SE was launched under its new mission, having been the test-bed for a number of guidelines about questions that later generations of Stack Exchange sites benefitted from.
This left 3.5 months of questions asked during a particularly formative point in the Stack Exchange network: more than a quarter of our site's lifespan. Compare this to Stack Overflow, which has had the benefit of 2.5 years of maturity and wasn't originally created to be the toilet bowl of a different site.
While we're slowly resolving those initial growing pains, it's going to take some more time. The ratios will normalize, but they aren't going to normalize in the next month: it going to take several months to a couple of years to stabilize into a site as mature as the currently-three-year-old Stack Overflow.
The situation isn't as dire as it might seem from one reading of the numbers. It was dire at one time, but that was last year. Now, we're moving forward: our traffic is increasing and our closed ratio is decreasing. More and more, people are taking ownership of Programmers.SE as something to be proud of, and the quality of questions and answers is noticeably rising.
I think there's a lot we can improve, but it's going to be far more constructive and productive to isolate specific failings and correct those, not to make generalizations about the site as a whole. If we stick to generalizations, we might as well just shut this site down and start over.