The guidance for off-topic questions has this to say about the use of statistics:

Asking for a link to something or statistics

This is a subtler meaning of the 'favorite off-site resource' part of the wording of the close reason. Asking a question that doesn't draw upon the community's expert knowledge as programmers but rather asking it to be a crowdsourced search engine falls into this area.

Links to things suffer from link rot. Statistics become out of date over time. Neither of these contribute significantly to the collective knowledge of problem solving that the Q&A format provides.

I don't understand what the issue is with questions that invite answers that involve statistics. To me that guidance is bad for the quality of answers on this site. Allow me to illustrate.

Take for example this question about code reviews: How to deal with no code reviews in my new place when I come from that practice?

In my view, having to argue in favor of the best practice of code reviews is something programmers are likely to face (hence on-topic) and the best possible answer to this question is to provide provable statistics that demonstrate the value of the practice of code reviews (and possibly statistics that compare the different ways of performing code reviews). This way the asker has better than anecdotal arguments in favor of code reviews as a best practice. In my opinion statistics are a higher quality answer when it comes to best practices answers, because they remove the bias inherent in an answer that is a sample of 1.

I can understand forbidding link-based answers, and requiring that the statistics are repeated in the body of the answer (citing the off-site resource where they were obtained), but why is it a good thing for P.SE to automatically close questions about best practices which can best be answered through statistics?

  • 2
    One possible argument I might envision is "just google it if you need statistics". But on the one hand you can apply that to anything that might be asked here, and on the other hand the best statistics are typically not easily found in google because they are in books and papers. E.g. the work of Capers Jones is very useful to understand best practices, but the statistics are not easily found online. Nov 6 '14 at 10:09
  • 3
    having to argue in favor... -- this sort of questions is addressed in another guidance, How do I explain ${something} to ${someone}?
    – gnat
    Nov 6 '14 at 10:27
  • 6
    1. statistics change over time, 2. statistics are easily misrepresented in nice graphs. Nov 6 '14 at 10:47
  • @ratchet freak: a lot of things in our industry change over time. If we can't give answers which will change over time, what's left to answer? As for the argument that statistics can be misrepresented, this is true, but that is because they actually represent something beyond personal anecdote. At least with statistics you have a graph, you have facts to discuss the factuality of. Answers based on statistics can actually be factually verified by other people. That makes them better answers than anecdotal experience in my book. Nov 7 '14 at 8:34
  • Do statistics ever constitute proof? It's certainly comforting to see where one stands in relation to the big picture, but that seems like a different mission than answering "conceptual questions about software development."
    – sea-rob
    Jan 5 '15 at 22:53
  • I'm curious: do you reject statistics in general, or just in relation to software development? As I see it, much of modern science would not be possible without statistical research. Jan 6 '15 at 8:11
  • Well, it warrants a more sophisticated view of how statistics can benefit this forum. Without a coherent, rigorous methodology, simply gathering and posting numbers will end up falling far short of "statistical research", let alone "proof".
    – sea-rob
    Jan 6 '15 at 8:47
  • Personal experience can be a valid answer or it cannot, and it is definitely welcome here, without any sort of rigorous methodology to verify that the experience is as portrayed. I don't understand why you would require answers involving statistics to commit to a higher standard than other answers, because they are easier to verify than anecdotal answers (given proper sources). There exists a voting and comment mechanism to weed out bad answers. Bad statistics could be voted down. The only reason I can see to reject statistics outright is if you believe they are always bad answers. Jan 6 '15 at 10:09

You must log in to answer this question.

Browse other questions tagged .