7

Some recent posts make me feel like these could be automatically rejected by quality filter, thus saving community efforts for better quality questions and answers.

Above are full literal quotes of the posts, I did not change anything.

Recently it has been announced that quality filter is tightened at Trilogy sites (for your convenience, announcement is quoted below).

  • As far as I understand, SE team is open to consider similar change for Programmers:

    the sad truth is that on Programmers, even questions with a maximum quality score fare worse on average than the ones passing the threshold I just put in place on SO. We can try tweaking the threshold there too, but it's probably worth a separate discussion on meta.programmers.

Could we please somehow adjust quality filter at Programmers to prevent posting low quality stuff like in above examples?

For the reference, announcement of filter adjustment at Trilogy sites is as follows:

I've bumped up the threshold below which questions will be blocked. The majority of recently-asked questions that fall below the new threshold do not fare well on Stack Overflow (i.e. they are closed, deleted, and/or down-voted).

The down-side is that short questions will be harder to post (this is more than just a length check, but short + poor spelling / caps / punctuation / formatting will damn a post more readily). At this point, I think that's a fair trade-off on Stack Overflow.

I've also increased the threshold on Super User and Server Fault; although the volume is lower there, they field even fewer reasonable questions in this range. A quick check of other high-traffic sites does not appear to justify raising this anywhere else at this time.

FWIW: Stack Overflow already rejects something like twelve hundred questions a day based on this check - that'd amount to about 13% of questions if a lot of them weren't just the same folks retrying more or less the same text over and over.

Update for the reference, at the "quality score" list provided in this answer: first of above examples scores 70, second - 83, third - 81.

5
  • 1
    Not sure if it's because I'm more online on Programmers recently, do more reviews or we get more trash. But lately I use up my close votes every other day. So I totally agree with tightening the filters. Especially since here on Programmers I think very short questions are even less likely to have any value. There can always be found a few exceptions. Mystical had a few good examples like "What's the difference between git pull and git fetch?". But I think everybody really interested in good answers should be able to add enough info.
    – thorsten müller
    Apr 17, 2013 at 9:36
  • 1
    OMG the first question... i can't even...
    – maple_shaft Mod
    Apr 17, 2013 at 11:04
  • 1
    Corollary - I wonder if it would help to decrease the number of questions new users are allowed and / or to further restrict the rate at which the questions can be asked.
    – user53019
    Apr 17, 2013 at 14:26
  • 1
    @GlenH7 I'm not exactly sure what counts as a "New User" by SE's standards, however if I've written this Data.SE query correctly, over 50% of P.SE questions are from users that created an account less than 30 days before posting.
    – Rachel
    Apr 17, 2013 at 14:49
  • 1
    Third one's in the list - search for "oppurtunity".
    – Shog9
    Apr 18, 2013 at 21:22

4 Answers 4

Reset to default
4

After analyzing the data, I'm not convinced raising the sensitivity of the current filters will do much good on Programmers:

  • The "quality" checks were designed for sites like Stack Overflow which attract a lot of short, very poorly written questions. Of the examples you gave, only 1 would currently be blocked on Stack Overflow - and that's with the recently-raised quality-score threshold.

  • You obviously get some questions that the algorithm would consider low-quality (and some are already being blocked) but very, very few. More on that in a bit.

  • The majority of questions asked here fare poorly. By which I mean, they're either closed, deleted, or down-voted. Even questions that max out the automated quality score fare poorly over 50% of the time. Coupled with the relatively low volume of low q-score questions, I doubt you'd even notice if I raised the threshold here.

That said, here's the data - see what you think: all questions asked on Programmers in the last year with a quality score less than 100

There are 1238 of them - about 9% of all questions asked on the site in that time. For reference, the threshold on Stack Overflow right now is 75, which is about as high as I could put it without a terrible false-positive rate.

This is now raised to 73 on Programmers. MichaelT makes a good point: while this won't do much in terms of absolute numbers, it may help with the perception a bit by blocking some of the more egregious questions.

19
  • Thanks for running the research!
    – user53019
    Apr 18, 2013 at 16:43
  • 2
    FWIW: as you go through that list and look at what those scores correspond to, think about just how bad the stuff that is being blocked is.
    – Shog9
    Apr 18, 2013 at 16:50
  • from the list you provided I'd say SO's score 75 might do here, "without a terrible false-positive rate". All open questions listed under 75 would better be more informative. But since we've got lower traffic and less contention of community moderation allowing us to do more 'manual filtering", I'd rather prefer to give smaller value a try, like 70-73. Would that make sense?
    – gnat
    Apr 18, 2013 at 16:57
  • 1
    @gnat: this would be a threshold of 73 - 88 questions for the entire past year.
    – Shog9
    Apr 18, 2013 at 17:10
  • 2
    that's about 1-2 questions a week, correct? Matches my subjective perception of a frequency of shockingly bad questions. I would be interested in giving this a try for 3-4 months. What about answers, are these supposed to run through the same filter as questions?
    – gnat
    Apr 18, 2013 at 17:19
  • 3
    Based on that list, I think setting a threshold of 73 would be worth it. How long does that take, maybe a minute or two?
    – Robert Harvey
    Apr 18, 2013 at 18:04
  • 3
    I can only look at the open and closed questions in that list (not a lot of them compared to the deleted)... consider the question of "will this lead to a better experience for both the people asking the question and the people reading and maintaining the site?" The first group realizing that their question is poorly done, writing a better one and help reduce the "everything on P.SE gets closed" impression when shockingly bad questions get asked.
    – user40980
    Apr 18, 2013 at 18:05
  • 1
    So... does this count as [status-completed]?
    – yannis
    Apr 18, 2013 at 21:38
  • Looks like you missed a good "oppurtunity" to practice decline culture skills. :) Thanks!
    – gnat
    Apr 18, 2013 at 21:41
  • @Shog9 does the filter apply to answers? Wonder what would be score for "try to rember the logic behind every code. dont just try to memorize it. if u try to memorize ur surely going to forget it within few days."
    – gnat
    Apr 19, 2013 at 19:32
  • @gnat: there is, and that fell waaay south of it - but unlike questions, answers aren't blocked; they just have to click through programmers.stackexchange.com/questions/how-to-answer before it'll post.
    – Shog9
    Apr 19, 2013 at 21:37
  • 1
    @Shog9 I see, thanks. Is there a way to at least submit low-score answers to Low Quality queue for review?
    – gnat
    Apr 20, 2013 at 6:43
  • 1
    @gnat: it probably would have been - but the moderator deleted it first (there's an automatically-generated "low quality" flag for these).
    – Shog9
    Apr 20, 2013 at 15:35
  • @Shog9 tangential thoughts to the quality filter - if something scores a (made up number) 70 which is good enough for P.SE, but not enough for site SO, and it was migrated from P.SE to SI, are there any notices or such that are (or should be) part of the migration process?
    – user40980
    Apr 22, 2013 at 18:06
  • 1
    Simple - the quality filter sucks, @gnat. We have a better one in testing currently.
    – Shog9
    Aug 28, 2014 at 23:21
2

Since all three questions you cited as examples were posted by the same person, and were all heavily downvoted and deleted, it's likely that this user is question-banned, and the problem has already solved itself.

The quality filters are not a panacea. All they do is look for textual patterns that are highly predictive of question failure. In that sense, they are completely statistical in nature; only those things that accurately red-flag questions that are likely to fail are included in the filter.

The resulting effect is that some things which you might expect to be caught by the filter (such as not capitalizing i for example) are not, because they are not sufficiently predictive of question failure.

What remains is the actual meaning of the question, and since we have yet to figure out how to get computers to divine meaning, we still need human beings to moderate such questions.

7
  • "problem has already solved itself" - how's that? Let's see, 7+7+6=20 votes down, 2+1+3=6 votes to close, 3 mod-deletes, does this qualify as solved itself?
    – gnat
    Apr 17, 2013 at 19:54
  • @gnat: This person won't be posting questions again. Problem solved. Don't abdicate your moderation responsibilities to a machine.
    – Robert Harvey
    Apr 17, 2013 at 19:59
  • well I am really confused with word "itself". Other than that, I agree that problem is solved (by 20 DVs, 6 VtC and 3 mod-deletes). As for abdicating, we already do, don't we? I mean, quality filter is already in action, we only discuss whether to tweak its settings
    – gnat
    Apr 17, 2013 at 20:05
  • I am reminded of the folks at Meta.SO who are always asking for certain tags or words to be blacklisted. It's a bottomless pit; there are always side-effects, and users always find ways to work around the pr0blem [sic].
    – Robert Harvey
    Apr 17, 2013 at 20:16
  • don't get me wrong, I wouldn't mind voting and editing, if (note: if) it turns out that tweaking the filter doesn't make sense
    – gnat
    Apr 17, 2013 at 21:21
  • 2
    FWIW, lack of proper capitalization does factor in, but it won't by itself block a question. If your question is borderline already and you forget to capitalize your "i", then that might just block it. txtspk also hurts.
    – Shog9
    Apr 18, 2013 at 3:57
  • 4
    @Shog9 - txtspk should really hurt.
    – user53019
    Apr 18, 2013 at 16:47
2

Yes, it's worth the effort to tighten whatever quality mechanisms are in place.

Period.

Yes, we'll probably filter out some otherwise decent questions that way. That's a price I'm willing to pay to filter out crap questions like "how do I build my own beginner's linux distro?" Questions like that make me want to open another meta question to identify the strongest but just-this-side-of-OK-for-Stack-Exchange word to describe it with.

"Removing" my answer as the research doesn't indicate it would provide any benefit.

5
  • 1
    You know, there's a delete link under every post that can be very useful when you want to remove them.
    – yannis
    Apr 18, 2013 at 17:55
  • 1
    You may wish to reevaluate your revaluation based on the new new research
    – user40980
    Apr 18, 2013 at 18:15
  • 1
    @YannisRizos - for real? A delete link that even lil'ol me can use and pretend to have mod like powers? :-P </lighthearted snark> I wanted the strikeout text to indicate my opinion had changed in light of the (first round of) evidence. And I know 10k+ users can still see deleted answers, but not everyone participating in this Q is 10k+ as evidenced by MichaelT's comment.
    – user53019
    Apr 18, 2013 at 18:18
  • 2
    @MichaelT - noticed that.... it appeared that everything below 73 was crap with one or two exceptions. Easier to handle one or two exceptions in Meta than 88 exceptions.
    – user53019
    Apr 18, 2013 at 18:20
  • 2
    @GlenH7 too bad we don't have double strikethrough so you can double strike your original, and strike the modified if/when you change your mind again. ;-)
    – user40980
    Apr 18, 2013 at 18:23
1

We should tighten up the quality filter if there is evidence it would help. I'd like some clarification of what Shog9 said, but it sounded to me like the algorithm they have isn't very predictive for this site. As soon as gnat finishes tuning his bayesian filter and submits the patch to SO we should do it.

(Just kidding about the filter. As far as I know).

4
  • "We should if there is" - what should we? :) wait? see? test?
    – gnat
    Apr 17, 2013 at 19:13
  • 1
    @gnat - we should tighten up the quality filter, as the question asks.
    – psr
    Apr 17, 2013 at 19:40
  • I see! thanks. "We should, if there is... etc" - correct?
    – gnat
    Apr 17, 2013 at 19:45
  • 1
    @gnat - Yes. If there is any predictive algorithm we should use it, but I'm not sure there is one in our case.
    – psr
    Apr 17, 2013 at 20:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .