This experiment is over. I will be adding an answer to show the results and suggest a plan for going forward.
For the next 30 days, closing a question will require just 3 votes. Also 3 votes will be sufficient to reopen a closed question. In essence, each close and reopen vote has 5/3rds the normal value. After 30 days I will revert the site setting back to 5 and evaluate the results.
I have tested this change on a development instance and questions that currently have 3 or 4 close votes will not be closed automatically. Rather they will be closed when the next close vote comes in.
I think it's fair to say there's a lot of dissatisfaction with the question closing system on Programmers. One proposed solution is to increase the number of close votes available to 10k users. Some of us on the community team have considered various schemes to scale close votes with reputation under the theory that people most invested in the site (as measured by reputation) will have the most incentive to curate questions. That might still happen, but we'd like to be pretty sure that it will solve the various problems with question closing.
It's pretty clear that organic closing (via the close votes queue or people voting to close questions they find naturally) isn't working very well. To demonstrate, here's how close votes on questions asked in the last 30 days are distributed on Super User:
Most questions have no close votes (not shown). 461 questions have just one (the yellow circle on the left) and the number slopes down to 142 with 4 close votes. 5 votes is a natural attractor because people will keep voting to close off-topic and otherwise closable questions until they reach that limit. There's a long tail beyond 5 close votes (also not shown) due to votes expiring, close wars and so on. In essence, questions with at least 1 close vote, but not 5, are in a grey area in terms of closablity. This general pattern turns out to be extremely common across a variety of sites.
In contrast, here's the same graph for Programmers—Stack Exchange:
Remember this is only showing questions that have been asked in the last 30 days; we aren't talking about Not Programming Related questions. Instead of a fairly smooth gradient between questions that should clearly be open to questions that clearly should be closed, this graph shows something closer to a binary state. My hypothesis is that since the people contributing the most close votes feel the need to conserve votes, they don't vote to close borderline questions. There's too great a risk those votes will expire impotently.
I've also included the counts of questions that have been edited or deleted in each close vote bin. Relative to other sites, questions on Programmers with close votes are more likely to be deleted and less likely to be edited. It's entirely possible that a greater proportion of questions are unsalvageable on this site. But I suspect the underlying problem is that the questions that could most benefit from an improvement are not getting close votes in the first place. That might not be a problem, but close votes can provide incentives for askers to fix problems even before their questions are closed. Close votes also put questions in front of reviewers who might be inspired to improve them.
Another problem that we discovered very dramaticly is that closed questions are rarely reopened. Most closed questions can't (and shouldn't) be reopened, but statistically speaking, we expect more than 1-2% will be. My hypothesis is that it takes so much effort (and coordination) to close questions that people are not interested in undoing that work even if there's been an edit to fix a question's problems.
To be clear, the breakdown in the close voting system is not limited to Programmers. It is, however, rare across the network for the division between open and closed questions to be so stark. We hope this test will help us understand how the close vote system breaks in order to (hopefully) fix it.
Why 3 votes?
Initially users with enough reputation could unilaterally close and reopen questions on Stack Overflow. In December 31, 2008, the close vote system was created and required 3 votes. Between then and the next April, the limit was increased to 5. (Shog9 remembers 3 lasting a few weeks.) The documentation of the site setting reads:
KEEP THIS AN ODD NUMBER
So we need an odd number greater than 1 and less than 5.
How will this change be evaluated?
There's a nontrivial chance something will break requiring me to revert the change. If we continue for the full 30 days, we will compare before and after the change:
Obviously, it's now easier to close questions. Insofar as bad questions are put on a path toward either corrective edits or deletion, that's a good thing. But this site closes well over 50% of new questions, which is the highest rate on the network. (Considering total questions closed to questions asked in a time period, the rate is more like 70%.) Paradoxically, I'd like to see the close rate remain steady or decrease.
Reopening questions is one way to mitigate against a high close rate. Since the bar for reopening is now lower, we'd expect more questions will be reopened. Currently the reopen rate is about 2% of closed questions.
Edit rate on closed questions
In general, the only path to reopening a question is editing. It's also the most effective way to prevent questions from being closed in the first place. Since it's easier to reopen questions, there's even more reason for askers to fix their own closed questions. In the last 30 days 9% of closed questions were subsequently edited.
About 65% of questions that get at least one close vote on Programmers are deleted. We aren't exactly looking at the cream of the crop here, but that's noticeably more than other sites. (Again, this only counts questions asked and deleted within the last 30 days.) We can't force people to ask questions that don't deserve deletion, but hopefully more questions can be salvaged with edits.
Finally, I'm curious if this change helps people enjoy the site more overall. At the end of the experiment, I'll post my results and ask for any followup thoughts you as a community have.
How you can help
Relatively few people are active in closing, reopening and deleting questions. Even fewer are active in editing. In order for this experiment to improve the quality of content on this site, more people need to be involved in identifying questions with potential to be unlocked with a judicious edit. Fixing tags and removing signatures are good first steps. But title and (occasionally heroic) body edits might be needed to pull some questions out of mediocrity. Remember: once the question is posted, it belongs to the community as much as it does the original author.
In addition, please use the answers below to let me know how the experiment is going.