8

The Stack Exchange platform has automatic triggers in place based on number of answers, number of answers per unit of time, answers from new users per day, answers from new users that get deleted, number of answers, and number of answers per time period. Some actions deliver moderator flags while others trigger automatic protection.

I've noticed that a lot of the moderator flags that I've been handling that are system-triggered don't result in me doing anything. Since these values are configurable, I wanted to throw out some configuration options that are available to us.

See Improvements to handling questions with too many answers for more details.

Do note that I haven't poured over the Data Explorer, but I looked through lists of questions using the site search tools. If anyone wants to write some SEDE queries and improve these numbers - feel free to and I'll edit appropriately.

Warning for New Answers

Right now, a question page will display the answer box for all questions with less than 15 answers. After the 15th answer has been posted, the answer box will be replaced with a button. If a user clicks on the button, they will be presented with a warning dialog that indicates the question has a large number of answers and these should be reviewed to make sure that any new answer adds value.

This limit should be lowered to 5. Most questions tend to have less than 5 answers. It looks like there are a few very old legacy questions with more that have a historical lock on them. Although there are a good chunk of questions that are somewhat opinionated, but still good questions for this community, it seems like most of the best solutions tend to be written in the first 5 answers. Of course, the warning will not prevent a 6th or 7th or 8th answer from being written, it will hopefully help reduce some of the other flags.

Number of answers from new users per day before auto protection

Right now, this is set to 3. That means if 3 users post new answers in a single day, the question is auto-protected and more new users cannot post answers. No change to this value, since it seems like it properly mitigates the effect of low-quality drive by answers to extremely popular questions.

Number of answers from new users deleted before question is auto protected

Right now, this is set to 3. I think this should be increased to 5. This will allow new users to post high quality answers, but if a question is attracting a lot of bad answers, it will get handled. The idea is to be more open with people joining the community. If new users stumble on a question and can contribute something new, they won't be blocked from doing so unless the question is attracting a lot of sudden attention or if it appears to be a magnet for low quality answers.

Number of answers before the question is flagged

Right now, this is set to 30. We don't see this flag very often, so I'm thinking that it is too high. Most of the questions with over 30 answers are legacy questions. I propose lowering this to 15, which is the current trigger for warning users about new answers and is 3x higher than most questions will get. This will bring any question with 15 answers to the immediate attention of a moderator where we can review the answers and handle low quality answers - edits, down votes, or deletions as appropriate.

Number of answers per time period before the question is flagged.

Right now, this is set to 10 answers in 7 days. We get a good number of these flags that result in little action. Although some answers are worse than others, they regularly don't meet a definition of low quality that results in deletion. It's also well before users get a warning about posting new answers.

I think this flag is likely to be OK, for now, with the change made to warning users about new answers. I would be more likely to be harsher on questions if users have appropriate warnings about quality first.

  • I'm thinking about this more and I'm thinking that number of new users per day before auto protection should be increased to 5 and the number of answers from new users deleted before auto protection should remain at 3. I think that will have the effect of letting new users contribute to popular questions, but locking down the ones that draw low qualty answers. – Thomas Owens Jan 15 '18 at 13:52
  • 1
    Thomas, auto-protection almost never kicks in over here which suggests that 3 is safe (or even can be decreased to 2). This is because quick upvotes to new users answers (eg from grateful asker) invalidate protection criteria (as explained eg here: "two of the people answering got fairly quick upvotes on their answers. Enough to go over the 10 rep requirements. At that point, when a new answer comes in, those are excluded...") – gnat Jan 16 '18 at 8:09
  • 1
    @gnat That's a good point. The definition of "new user" should probably be changed, but that appears to be beyond scope of the configuration options currently available. – Thomas Owens Jan 16 '18 at 14:24
  • FWIW a while Shog provided stats related to this matter; if my reading is correct 21 question escaped auto-protection due to quick upvotes on new users answers in a year between Mar '15 and Feb '16. That's about 2 questions a month and I haven't yet made my mind whether this would justify requesting system changes – gnat Jan 16 '18 at 14:37
1

Your arguments look sane; let's just try it. As of right now...

  • ...Authors will be warned when adding an answer if there are 5 answers already on the question. (also, comments scoring < 1 will be collapsed by default)

  • ...It'll take 5 answers from new users in a day to auto-protect a question (unless at least 3 are deleted).

  • ...A moderator flag will be raised when a question gets 15 answers.

  • I don't like the collapsed comments, though. Hate it on Workplace, too. :( Approve of the rest. – Thomas Owens Jan 17 '18 at 21:12
  • 5 answers to trigger auto-protection don't feel right because as discussed in comments above 3 seem to work well enough ("quick upvotes to new users answers" etc). What's your take on this @ThomasOwens? – gnat Jan 17 '18 at 21:13
  • @gnat I honestly don't think it will matter either way, if up votes are enough to bump a user out of "new user" status. We aren't triggering it at 3, so without a system-level change. I think the most important things are the earlier warning to answerers and earlier mod flag. – Thomas Owens Jan 17 '18 at 21:19
  • I see, thanks @ThomasOwens. Probably Shog can run a simulation on older questions (before auto-protection was introduced in Apr '14) to check how many of these would trigger it at 5 answers vs 3 answers – gnat Jan 17 '18 at 21:25
  • 1
    It's pretty tedious to simulate this accurately, @gnat... So I ran a "mostly-accurate" check: over the past 365 days, at least 24 questions would've qualified for auto-protect based on answer author's score at the time of answer creation. 11 of these were protected automatically, 6 of those after 3 answers. Of the rest, 1 was protected by a moderator and the remaining 12 were protected by you; in some cases, this manual protection was done for the reason given above (authors no longer qualified as "new users"); in other cases, it was done preemptively (< 3 new-user answers had been posted) – Shog9 Jan 18 '18 at 22:52
  • so the worst case - assuming that change from 3 to 5 will totally break auto-protection (it probably will) - we're talking about something like 2 questions a month that could possibly qualify. That's not much - probably @ThomasOwens is right and we needn't bother, at least not for now – gnat Jan 19 '18 at 21:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .