Should surveys be allowed?

Question

To clarify, I'm referring to statistical surveys and questionnaires that are well designed, ask open-ended questions, are relevant to programmers as per the diagram in the FAQ, aren't rants, and generally add value.

Specifically, questions more like this one (although that's a mediocre example, at best) or questions like what a professional or academic study would entail (if anyone has read The Dark Side of Software Engineering, they included a sample of questions they asked to professional software developers and their responses).

I think that, overall, well-written survey questions could add value to people looking to tap into the total knowledge and experience here on Programmers, as well as add legitimacy to the site by bringing in academics and software engineering researchers to access practitioners and perhaps provide answers to questions based on the latest academic research, where applicable. However, it would have to be monitored and controlled to ensure only high-quality surveys are allowed - ones that don't add value or demonstrate relevance are closed.

These questions should also be made community-wiki. By nature, they are designed to bring in a large number of answers, typically relating to personal experiences and knowledge gained from them. There's no right or wrong answers, and as such, reputation should not be assigned to the question or any answers provided for it.

This is similar to another discussion here on Meta, but provides a much more narrow definition of what is allowed and what isn't.

Answer 1

I'd suggest reading Real Questions Have Answers and particularly the answer on Meta Stack Overflow created by our very own Aaronaught, which goes into more finer detail about why just such a bad fit for Stack Exchange, but in short, the purpose of Stack Exchange is to get an answer to an actual problem a person is facing.

One answer that actually solves the problem is what we're looking for: not a host of suggestions. A future visitor, who has the same issue, should be able to look at the set of answers and be able to determine two things:

• What the community has decided is the correct answer (highest number of votes)
• What the question asker has decided is the correct answer (accept flag)

When you use Stack Exchange for polling, these two concepts break down. When every answer is equally valid (it's a survey, after all), voting doesn't mean anything: what do people vote for? Their favorite answer? There's no value in that in the Stack Exchange network: it adds another, hidden dimension to voting that a future visitor, not aware of the exception to the rule being made for the specific question, isn't going to pick up on. "Use voting as a means to vet the answers...except on this question! Ignore the voting on this question!"

Secondly, I'd consider this to be an abuse of the user base. Stack Exchange users are not "participants", sitting around waiting to take surveys for whatever purpose someone coming to the site needs. They're experts, who have generously devoted their time and knowledge to help people solve problems. Every time we forget that there are actual people here with specific issues and specific knowledge by reducing them to one point on a graph, we diminish the value of their contributions. Answering a few questions for a survey is not the same thing as providing a long, thoughtful answer about a specific problem.

Finally, Community Wiki is not the answer. Community Wiki—for questions—was made a moderator-only function specifically to combat its abuse to keep questions that were off-topic or bad fits for Stack Exchange (but were ostensibly interesting) open. Recently on the Stack Exchange blog, Grace Note went into a good amount of detail about the point of Community Wiki:

With suggested edits now in place, you could argue that the removal of reputation from voting is now the only function of community wiki. Unfortunately, this means it is often seen as a magic switch to allow questionable content.

One of the first feature requests I saw on Meta Stack Overflow was Moderator Filtering of Highest Voted Questions, which was deemed necessary because questions like Coolest Server Names show the wrong side of the site. The actual problem-solving nature of sites is too easily buried under the weight of all these “fun” community wiki questions. At one point, “Our top voted post is an actual question!” was a point of pride. That’s … not a positive sign for a Q&A network.

Instead, as Grace Note continues, we should be looking for quality content that stands without Community Wiki. We might be able to do that by rewriting survey questions into a question about a problem the asker is actually facing: this would bring back the voting and acceptance functions into play, and make a question that meets our general guidelines without having to abuse Community Wiki.