What can we do to help users understand our site better?

Question

As of right now, the close/migrated question rate for the past month or so is around 43% (see notes below)

The majority of the closures are valid based on our site standards, however the fact users are asking so many invalid questions makes me think that we could be doing something better with the way we are communicating with our users about what the site is for. Most people don't willingly post unwelcome questions on a Q&A site

The majority of questions are closed because they are:

• Code questions - These should be asked on SO, and get migrated there if they are suitable for SO. If the question quality is not suitable for SO, they simply get closed.

• Requests for broad recommendations, like book suggestions, what to learn next, or general advice in X situation. Any kind of broad question like this is usually closed on SE sites as unanswerable since there is no one right answer to the question, and they usually turn into a poll with everyone chiming in with their own recommendation or opinion

• Questions about or for programmers, that are unrelated to conceptual software development. The site is meant for Q&A on conceptual software development, so questions asked should be related to that.

Things that contribute to our high close/migration rate include:

• Users not reading the FAQ, and not understanding what this site is for

• Users migrating bad questions from Stack Overflow because they don't understand our site's scope. Per Yannis, 41% of migrations from SO were rejected in the past 90 days (about 70-80 questions), but that's for a 90-day period, not the last 30 days. The average for a 30-day period would be somewhere around 25

• Users who have been suspended on SO, or are banned from asking questions, so are asking code questions here instead. Yannis said they get one of those every few days or so, so its not a lot but it does contribute to the problem (around 10-15 questions over the past month)

(If I missed something in either list list, let me know in a comment and I'll add it)

So my question is, what can we do to improve the way we present the site to users, and prevent so many bad questions from being asked?

---

For those who want to know how the close/migration percentage was determined:

46% is the number is obtained by going to the question list, sorting by Newest, and counting the number of questions since 4/4, then filtering for closed:1 and doing the same thing. There are 1029 questions total, while 441 are closed, so 441 / 1029 = 42.8%

closed:1 includes migrated:1, so a separate filter for migrated:1 shows 77 migrated questions in the past month, so (441 - 77) / 1029 = 35%

And no, I didn't count those one at a time. :) I sorted by date, found the date I wanted to go back to, and multiplied the page number - 1 by the number of questions per page, and only counted the # of questions on the final page.

Answer 1

Nice timing. I came across this question while trying to determine for myself whether I should post my question on P.SE or on SO. For me, (and yes, this is anecdotal and unsupported by researching existing questions -- I'm just speaking from the heart as a P.SE newbie) there are two problems:

1) I find P.SE's mandate to be vague and confusing. I've read the P.SE and SO faqs repeatedly just now, but am still uncertain about the difference between the two sites. I think the difference is that:

• P.SE is for Q&A on the Software Development Process (methodology, concepts)
• SO if for Q&A on specific programming problems/tools

If I'm right, then I'd like to propose placing the words "Q&A on the Software Development Process" prominently, e.g. in the site masthead. (Incidentally, I don't see the "Welcome!" post-it note (and its tagline) that Rachel screen-captured. I do not have the tagline anywhere on the P.SE homepage -- finding the word "interested" turns up nothing.) Now, I'm sure that the authors of the current tagline put their heart and soul into it and find it meaningful, but for me, an outsider, it does not successfully introduce the site. Why emphasize "Professional Programmers"? If the site is on process/concepts (and explicitly not on career advice), why differentiate between professional and amateur/student programmers? And what is added by including the words "interested in conceptual questions about"? The only words in the tagline that truly matter are "software development [concepts]", or as I've rendered it, "Software Development Process". Bottom line, consider rewriting the tagline and showing it more prominently.

If I'm wrong about the difference between P.SE and SO, then perhaps we could rewrite the opening FAQ to clarify the distinction. In fact, either way it should be clarified, because clearly us nubes ain't getting it.

The mandate is also unclear in that (independent of SO) it's unclear what topics are in/out. e.g. Q's on "freelancing and business concerns" are allowed, but "career advice" is not. The former is out of place here, as it is unrelated to process/methodology/concepts, and yet seems closely related to the latter.

2) The stated goal of P.SE seems self-contradictory. The directive "You should only ask practical, answerable questions based on actual problems that you face" is incompatible with such broad, abstract concepts as software architecture/engineering.

Thanks for your work putting together P.SE. I'm sure it serves a necessary niche in the SE network. It just needs a little bit more work to clarify the parameters of that niche and to present itself well within the network.

Answer 2

Users not reading the FAQ, and not understanding what this site is for

I am pretty new here, so I might be completely off, but I think that's the main problem. When I first found the site, I read the FAQ, it was pretty easy to find and quite informative (but very long!). I'm not an expert on what is on topic for Programmers, but I still haven't stumbled upon a closed question that would have been asked if the OP had read the FAQ in the first place. I don't know if this is possible but one statistic I would be very interested to see would be the exactly that, how many people actually visited the FAQ before they asked their questions, and how many of the questions asked by people not visiting the FAQ were closed.

My experience with the site is very limited, and so is my data set, but I don't think it's unreasonable to expect people to familiarize themselves with the guidelines before asking. I haven't asked a question here yet, and do not know if I will soon (most of my questions are already asked and answered) but when I do I will give the FAQ another look to see if my question is ok. I think it's the sensible thing to do, it will only cost me a few seconds and it will help me get better answers, as from what I've seen so far the questions that are disallowed are those that don't generate useful answers.

This site, Meta, is also very useful, there are several past discussions that are applicable to this one and I'm surprised no one mentioned them:

• Can the migration path from SO be reconsidered?
• How can we help people who ask “What language should I learn next” ask a better question?
• How to prevent “Why is my question being closed?” and User Frustration?
• What can be done about bad questions on Programmers?
• Questions barely or not related directly to programming. We need handling them.

It also seems to me that people keep answering obviously off topic questions, although that has already been identified as something that we shouldn't do:

• Should we seek for alternatives to propose to OP when closing their off-topic question?
• Please don't post answers on obviously off topic / bad questions!

That is not really helpful, although I understand that most people will think to answer first and clean up later, it's very misguiding to the person asking especially when answers are from senior members. It's like saying your question is off topic, but no problem, you got your answers. I've already lost 25 reputation points from answering a very bad question by this guy, and I'm really glad the community and the moderators re-acted and other people didn't waste their time answering his questions.

I'm getting a bit off topic, so I will stop here, with this amazing find:

Programmers is not supposed to be a beginners site, and a lot of closed questions come from beginners. I don't think there is anything we can do here it is only natural that the site will have more closed questions than others. I like the advanced aspect of the site, and the work that is being done in the disciplined aspect of it, questions like this one should be the norm not the exception.

Answer 3

I went through all closed questions of the past month and deleted:

1. Cross posts

   Questions that were identified as having asked on other sites before being asked on Programmers. None had answers, all were downvoted (which is good), and they were quite a few of them (didn't count, but aproximately 1/10).

2. Questions posted here to avoid bans or suspensions on Stack Overflow

   Argh.

3. Rejected migrations

   Questions that were migrated here, mostly from Stack Overflow, and we closed them. Those stayed around locked, and there were quite a few of them.

4. Succesful migrations

   Questions that were migrated away and had found a better home at the target site, which means that they were got a few up votes (>3), and up voted answers (and in most cases accepted answers). Yes I checked each and every one of them.

5. Downvoted questions with no answers or no upvoted answers

   All hopelessly off topic.

6. Heavily downvoted questions with no highlish upvoted answers

   Heavily downvoted <= -3, highlish upvoted <= +2, all hopelessly off topic.

Questions that satisfied any of the above criteria were not deleted if they had at least one re-open vote, and of course duplicates were left untouched. I asked The Workplace mods if they wanted a couple of questions, one was deleted by the OP before we get a chance to migrate (which is funny as the mod I talked with liked the question) and the other was rejected (but not deleted, as with some rewording it might fit another site, waiting confirmation). I've also re-opened a question that was closed as not constructive, as it had two re-open votes and a good(ish) answer.

I was honestly curious to see what was going on with the closed questions, I'm happy with all closures, they are all justified, regardless if they were delete worthy or not. However deleting the questions was only a by product of the process, I only did it because it made sense to clean up while going through the closed questions, my primary motivation was to get a better feel of our closures and our overal closing attitude. My conclusions and thoughts are (in no particular order):

1. Snarky comments

   I saw a lot of comments I didn't really like. The majority of commenters were being nice and helping OPs understand why their question was off topic, but there were a lot of comments that were... umph. Not outright rude, but not particularly helpful either. I do enjoy sarcasm as much as everyone, and perhaps a little more, but I think we should be a bit kinder to newer users. I understand that several of us are getting increasingly tired of having to deal with the same crap questions over and over, but I would like to ask everyone to refrain from commenting on newer users questions if they are not feeling productive.

   I would like to suggest a very easy workflow: If you happen upon a snarky comment, and you feel you can provide better guidance to the OP, please do, even if it means re-iterating the earlier comment. And after you post your comment, flag the earlier one so we can remove it. Simple as that, and please avoid responding to the snarky comment with an even snarkier. A few of our higher rep users already do that, and I think they are setting a perfect example for the rest of us.

2. Stack Overflow sucks

   Not the real Stack Overflow, that's a wonderful place, but the part of it that finds it's way here. The questions I deleted were of extreme low quality to begin with, but those that came from Stack Overflow were the worst of the worst. That includes direct migrations, questions asked here because of Stack Overflow bans or suspensions, and questions asked here after being closed on Stack Overflow because some "friendly" commenter pointed OP to Programmers.

   Most Programmers regulars are probably aware of how troublesome our migration relationship with Stack Overflow is, however it's worth noting that 59% of migrations from there the past 90 days were good migrations, and Stack Overflow is our top referring site (excluding search engines). So it's not all bad.

3. We should be deleting some questions more quickly

   Cross posts, questions that were asked here to circumvent bans or suspensions on other sites, and failed migrations don't really serve any purpose staying around for long. All they do is contribute in our questions lists looking like a battlefield, without having any chance of being salvageable. 10K+ users please contribute delete votes when you see a comment identifying a question in one of the first two categories, lower rep users please flag if you happen upon those questions a week after they were asked.

4. Not constructive

   We have some not constructive questions that I think could be salvaged. I didn't have the energy to look deeper into them, but I invite everyone to go through highly upvoted closed questions and see if you can help bring them up to shape. Don't go at it alone, if you find a question you think could be re-opened, post it in chat and ask others to evaluate and help. Don't go blindly casting re-open votes, always keep in mind that re-open votes are more powerfull in twos (and even more in threes).

The numbers

Of course deleting the questions changed the numbers Rachel references, the current numbers are (for the last 30 days, and at the time of writing this, new questions may have popped up):

Open questions         = 599
Closed questions (all) = 283
Total questions        = 882
----------------------------
Migrated questions     = 37
Closed (excl. migrat.) = 246
----------------------------
Closed % (incl. migr.) = 32%
Closed % (excl. migr.) = 27,8%

27,8% is our current close rate, having deleted an abudance of crap. Again this wasn't about hiding things under the carpet, this was about getting a clear idea of how many questions we close excluding:

1. Questions we have little control over (first three categories of the deletions),
2. Migrations,
3. Hopelessly off topic questions that aren't salvageable (last two categories of the deletions).

I am excluding migrations (from us to other sites), because:

• The questions are labeled [migrated] and not [closed], it's reasonable to assume that they don't contribute to scaring newer users off (which is the basis of this question)
• Migrations are cool, we are only trying to find a better place for the question, and most OPs really appreciate that.

And don't forget that our closure rate includes duplicates, something that's not really cool, I strongly believe that most OPs appreciate finding quick answers or at least don't scare off because of duplicates. However duplicates are still labeled [closed] and they might contribute in giving the wrong idea for the site.

Which leaves us with the hopelessly off topic questions, the heart of the matter, and of course the rest of the closed questions that I didn't delete. Yes, there is room for improvement. Yes, our expectations are a bit higher than most other Stack Exchange sites. Yes, we all want more people to love the site as much as we do, and yes we all want less closed questions. But is it a problem? I think not, at least not a major one and certainly not one we can't deal with efficiently.

Let's start by being a bit more helpful when guiding newer users, and a bit more aggresive when protecting borderline not constructive questions. We (the mods) have said countless times that bad answers may lead to a borderline question getting closed, however the community hasn't really been responsive in defending those magnificently interesting questions by vehemently cleaning up the crap answers.

Use your down votes people, and please for the love of the Flying Spaghetti Monster, resist the urge to add yet another opinion heavy answer to the pile. Great answers with references and relevant experiences is what we are looking for, treat everything else like it should be treated. The best way of lowering our close rate is by re-opening borderline questions.

The upcoming contest will probably bring us a lot new users, the coming month is a perfect opportunity to help the site grow. Be nice(r) to the newcomers, spend a few more seconds explaining why their questions might not fit or better yet, edit them to fit. Please avoid completely un-productive comments & discussion about stuff only a handful of people may find interesting, long comment discussions, etc.

Answer 4

A big problem is that the majority of Stack Overflow users neither know nor care what our site standards are now. They did (or thought they did) once and are still using those to judge whether to vote to migrate questions here.

You need to educate Stack Overflow users on the following:

1. What our site scope and standards are now.
2. What constitutes a good question - i.e. one that's constructive, answerable, etc. etc.

Basically we need to teach them that you:

Don't migrate crap

Answer 5

I was curious as to whether the closure rate has been getting worse over time, so I wrote this Data Explorer query. There has definitely been a surge during the past year. (Note that this includes deleted questions; also, the explorer dump stops at March 15 right now.)

We had to deal with a similar surge on Quant.SE, where I am a moderator. We dealt with it by revamping our FAQ to be extremely clear about what we did not want on our site. We've also been vigilant about getting the wrong people off the site with a combination of persuasion and suspensions. It was pretty harsh, but it worked. We have halved our closure rate in the past few months while maintaining a steady visitors growth rate.

One thing I notice on Programmers.SE is that there is still disagreement about what's on-topic. Many of the what-language-should-I-use and career advice questions get a few answers before being shut down! It might be instructive for the poor souls who mistakenly answered an off-topic question to be told exactly why they should not have bothered.

Answer 6

I really don't know why you need to close questions at all. If a question is related to programming, even if quite tenuously, then it should be left to the users of the site to arbiter whether it is relevant or not. The mechanism for that is quite simple and already exists - people will vote for questions they like, downvote questions they don't and, most importantly, provide answers to questions they feel are interesting.

I find it totally ridiculous that you can have a question that has lots of answers and is provoking discussion to be suddenly closed because some unelected moderator decides to exert their power. No wonder we are at the ludicrous situation where so many questions are being closed. Personally I hardly even bother coming here any more (and I know for a fact other people feel the same) as half the questions I'm interested in are suddenly closed (without often the mundane and dull questions left). Why should people bother wasting time supplying an answer to a question when they know that it could (seemingly) arbitrarily be closed at any moment?

I subscribe to the StackExchange twitter feed and it's amazing how many of the questions selected to appear in that feed (presumably automatically based on popularity) end up being closed. If people are interested in it, contributing and provoking discussion then what gives someone else the right to stamp down on that? I remember when this site was set up it was to act as an "overflow" for the questions that weren't quite suitable for StackOverflow yet were programming related (with an example cited as the (in)famous "What is your favourite programming cartoon?"). The irony is that questions like that would never be allowed here any more.

I'm sure moderators think they are doing their best by following some "charter", but who wrote that? Who was consulted? Again I come back to letting the people decide what should be here via the mechanisms that already exist with moderators only needing to close the most egregious violations.

Answer 7

I think that part of the problem may be the implicit focus from the site name. The site name is Programmers, which is different from every other successful SE site. Generally, the other SE sites talk about an action, where our title is talking about a group of people. I think that this leads to some of the confusing -- calling it "Programming" might be a better choice.

Part of this might affect Programmers more than other sites because of the site history as random-crap-as-a-programmer.

Answer 8

the fact users are asking so many invalid questions makes me think that we could be doing something better with the way we are communicating with our users about what the site is for

Above issue looks real, as indicated by 54.32% "bad" questions asked.

I think this is likely related to issues with answers quality in hot questions (discussed here in more details):

• "Hot garbage waves" in the answers once or twice a week
• poisonous effect these mis-answers have on questions, making interesting and well presented problems look the same as non-constructive popularity contests
• regular ways to deal with this kind of issues just don't work

  ^{It feels like all one gets is just like 60 seconds to figure protective edit to cover every word and letter in the question that could possibly be misinterpreted by some random passer-by and exploited for their senseless cheap shots. That's just... impossible. And more, it feels quite unfair to over-police text of such questions...}

---

An "outsider" reading low quality answers in hot questions, could naturally think,

hey here I can chit-chat about how git is fantastic and get my portion of cheap upvotes, c00l

Taking into account that these are highly visible posts, with thousands of views, it is pretty possible for them to be a steady source of misguided contributors.

_{Above makes me doubt that widespread misunderstanding of site purpose is really caused by either of name-FAQ mis-match, or misleading site name / tag line, or scope change. There are much larger broken windows than that: "why can't I post X when Y exists".}

If above reasoning is correct, then ensuring good quality of the answers in hot questions will help users reading these better understand the site and eventually lead to less of a bad posts from misguided users.

---

Update: recent feature request intended to address above issues is Trial run of modified "hotness formula" for Programmers questions

Connection between mentioned request and the way how users can be confused about what Programmers are about is probably best explained in a comment made by Ben Brocka:

Often hot questions have lots of bad or meh answers that are bad enough to drag down quality of the content and general discourse (encouraging similarly bad new answers)...

Answer 9

So... Where should people ask questions like "Recommend an editor that doesn't wrap lines."

I always had the impression that SO was for questions about programming — "What is the pythonic way to use exceptions?", and that P/SE was for questions about the process of programming — "How can I learn to write better code?"

I came here yesterday to answer some questions, and someone had asked about a text editor that didn't wrap lines. I thought "I know the answer to that! Use Vim, and the command ":set nowrap". However, by the time I got to the question it had been closed with no explanation, and I was left feeling confused and disillusioned with the whole site. The person wasn't asking for a general editor recommendation. They were asking for an editor with a very specific feature. It seemed like a decent question to me, but obviously not acceptable on SO, and apparently not acceptable here either. (Although reading your FAQ, it appears that you think it should be acceptable on SO, so why was it closed rather than migrated?)

I think part of the problem is the fact that nobody knows where to ask subjective questions. They see a title like '\*Programmers*\', and think "Programmers between two comment tags? That must be a place to comment about programming! I'll ask my subjective question there, instead of on SO where it will be closed." I think that your subject area (as specified in your FAQ) is much narrower than what is suggested by your title. Perhaps if you change the title from "\*Programmers*\" to "Programming Concepts", you would get better questions.

It would also help if people explained why they downvoted answers or questions. A single comment would suffice. (Downvoting without commenting is a bit counterproductive; either the person will keep making the same mistake because they don't know any better, or they will give up on the site completely.)

Answer 10

These are just my related thoughts in a spirit of sharing them...

• Sometimes you just look at a site like this as a resource that may or may not yield a desired result. In this light, the FAQ is irrelevant. It sounds disrespectful but I don't mean it to be - it's just pragmatism of someone trying to solve a problem with available means.

• The closing of a question has an impact on the person who authored it. It's pretty harsh to have your peers reject you, even if it doesn't really mean anything outside of a sandbox, it makes a person less and less likely to want to interact.

Answer 11

Change the Programmers.SE tagline

The tagline is usually the first thing users see when they learn about our site. It currently is

professional programmers interested in conceptual questions about software development

It is used in our site description on Google

in the welcome message displayed to new users

and in the Migration path for the site from SO (according to Yannis, Stack Overflow is currently our largest referring site, excluding search engines)

This makes it sound like we are a SE site for Professional Programmers [interested in conceptual questions about software development], and not a Q&A site for conceptual questions on software development only.

To further the confusion, the site is named and branded with Programmers in mind (whiteboard-look, triple-monitor logo, coffee mugs, comment marks around site name, etc), so by all first impressions, we are a site for programmers to ask questions and get answers from other programmers.

The idea that it is misleading can also be backed up just by looking at our closed questions, where the majority of closed questions seem to come from new users that are looking for an answer from a programmer. I've started leaving comments on closed questions clarifying the site scope, and have received many replies from users not aware of the site scope, including from some long-term users

Most people won't willingly post or migrate bad questions to a site, so I think changing the tag line to make it clear that this is a site for Q&A on conceptual software development will go a long ways towards reducing the number of bad questions that get asked.

Some examples I can think of would be:

This is a question and answer site for ...

... conceptual questions about professional software development

... software professionals to get answers on conceptual software development

... conceptual software development questions by professional programmers

(These are just examples. If enough people agree that the tag line is misleading and should be modified, we'll discuss what it should be then)

Answer 12

Write 10,000 pages user manual for using SE sites, consisting of random collection of different posts, all filled with specious reasoning trying to squeeze objective meaning out of language designating inherently ambiguous and incomplete ideas, subject to numerous interpretations, many of them as good as next set of interpretations, and pretend that selected interpretation gets Q&A closer to some platonic ideal of objectivity. Blame the users when they don't meet that ideal you can't attain but still desire.

You can detect what is bad stuff (e.g. spam, or awfully phrased question), but can't define what is good stuff. Close to nobody will bother reading your FAQs, manuals, and policies. Fortunately, for complex reasons.