27

In the Close Votes review queue, I got Graphics Card Adapter Is Facing Backwards as an audit question.

The question had a colorful history: closed by a moderator as unclear, subsequently edited a number of times, reopening was rejected by the community, it was modified some more, reopened by a moderator, then it received a decent answer. Reopening the question happened back on July 18. (For reference, the timeline is here.)

So for 2 1/2 weeks, the question has been in acceptable form, as assessed by a moderator. The question is now open, with no visible sign of any issue (other than some downvotes from when it was first posted). It looks OK to me, also. I verified that the question was determined to be acceptable, then voted to leave it open. That triggered the "STOP! Look and Listen" audit failure notice.

This situation looks oddly similar to a question here from 4 1/2 years ago: Yet Another Bad Audit – It was wrong to agree with a moderator.

Does this sequence of events imply that the audit bot collects questions based on their status at that time, then uses them for audits over a long period of time, and never rechecks them to verify that the status hasn't changed?

If so, that seems like an unreliable basis. For this question, assuming the bot looks for questions that have failed reopen review, the question was in that state for a total of 3 minutes. But if it is uncommon for moderator intervention to make an audit question invalid, perhaps it's just another shortcoming contributing to "close enough" that allows a totally automated system?

4
  • 8
    Sorry for making you fail an audit....
    – Mokubai Mod
    Aug 4, 2019 at 17:56
  • 1
    FWIW, it seems that question has only been used for one audit, i.e. the one that you "failed", and that was long after it had been reopened. I really have no idea what happened here.
    – Ilmari Karonen
    Aug 7, 2019 at 14:33
  • Good observation and question... I think whatever is controlling this logic or algorithm is worthy enough to at least be considered for review to make adjustments so these sort of trick audit reviews base a condition on the current status of the question regardless of it's historical property values. Shall we call upon and invite the super mods such @shog, or others at this level to advise to get an actual even somewhat satisfactory answer to this question? Perhaps it's been answered elsewhere or the community could get a peak at the correlated source code to help.
    – Vomit IT - Chunky Mess Style
    Aug 8, 2019 at 3:18
  • 2
    @PimpJuiceIT, my impression is that the folks responsible for the audit system view it differently from the average person doing the reviews. It's just part of gamifying the SE sites, and a certain percentage of bad audit questions is a tradeoff for the simplified maintenance of an automated system. As long as there aren't too many bad audit questions, the bad ones have the same effect as the good ones. But reviewers are willing to tolerate only so many "undeserved failures", then we lose reviewers. I suspect they won't want to complicate the system for something that happens only rarely.
    – fixer1234
    Aug 8, 2019 at 4:38

1 Answer 1

Reset to default
3

As outlined in the FAQ the criteria for "bad" Close/Reopen audits are questions which are quickly closed with no answers. AFAIK downvotes also play a role. This particular question was downvoted and closed within minutes, and then the bot had almost two days to pick it up, before it got reopened.

Edit: I felt this was something worth a discussion on meta.SE.

1
  • 1
    Your FAQ link from Tim Post includes advice to post on the site Meta and include the [disputed-review-audits] tag. That implies that when that FAQ was created, there was a provision for culling bad audit questions. The tag no longer exists, so it looks like a decision was made to not invest human resources in keeping the audit system clean; close enough is good enough.
    – fixer1234
    Aug 14, 2019 at 18:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .