9

I was doing some reviews, and I ended up at this answer.

It clearly is no SPAM, it clearly deals with the subject at hand and indeed gives some answers in a structured way, so I thought "That looks like a good answer".

However, that was not the case: apparently the answer was generated by some A.I. chatbox.
Ok, that can happen.

Reaction from the Superuser website:
NO, THAT CAN NOT HAPPEN! SOME NEXT TIME YOU WILL BE FORBIDDEN TO PARTICIPATE TO REVIEWS!

Look: I understand that A.I. generated answers are to be removed from the site, but you need to understand that they might be very difficult to detect. Hence, using such answers as audit tests goes too far.

By the way: I see there's a "chatgpt" tag but no general "A.I." or "artificial-intelligence" one???

15
  • From what I'm seeing it was delete voted by 5 other users - and audits are automated and picks up from there
    – Journeyman Geek Mod
    Commented Jul 13, 2023 at 6:34
  • @JourneymanGeekOnStrike: this is exactly my point: people who know the subject very well might recognise that the answer is A.I. generated, most other people don't. So either A.I. generated questions are avoided being used as audit tests, or the general messages is spread that only specialists are allowed to perform reviews (although I fear that this might heavily slow down the reviews process).
    – Dominique
    Commented Jul 13, 2023 at 7:30
  • 3
    Or if it seems fishy, you could skip it. But there's nothing we can really do about what turns up on audit
    – Journeyman Geek Mod
    Commented Jul 13, 2023 at 7:32
  • @JourneymanGeekOnStrike: that's exactly what I'm telling you: A.I. generated questions do not seem fishy anymore, and that will get worse and worse in future.
    – Dominique
    Commented Jul 13, 2023 at 7:33
  • 4
    I mean, its fishy to me, but I've seen faaaaar too many of these things
    – Journeyman Geek Mod
    Commented Jul 13, 2023 at 7:39
  • I just noticed my 'educate reviewers' got a downvote. Maybe one of the reviewers I'm talking about is certain it doesn't apply to them ;) @JourneymanGeekOnStrike I know you're on strike right now, but we do have one reviewer who really needs a review ban. they're not learning anything at all.
    – Tetsujin
    Commented Jul 13, 2023 at 9:49
  • Even if I wasn't I can't see who voted - and there's not very much I can or should do about it. Lets blame our dear Mr Posts keys shall we? :D
    – Journeyman Geek Mod
    Commented Jul 13, 2023 at 9:51
  • @Tetsujin: I can tell you that I've also just downvote your "educate reviewers" question, for the simple reason that you mention a "Golden Stupidity badge", that's not very respectful. On the other hand, I've upvoted your answer here because it gives useful insights on how to tackle bad posts during the review process.
    – Dominique
    Commented Jul 13, 2023 at 10:02
  • @JourneymanGeekOnStrike - I was just joking about the previous downvoter. The abysmal reviewers, however, are easy to track from the review history [which seems to have become my second home this past couple of weeks :\
    – Tetsujin
    Commented Jul 13, 2023 at 10:08
  • 2
    Dominique - I'm far too jaded by the abysmal quality of reviews at the moment that 'respectful' went right out of the window long before I posted that question. There is a desperate need for reviewers to learn. If you hadn't asked this question about audits, you wouldn't have learned either & still be failing them.
    – Tetsujin
    Commented Jul 13, 2023 at 10:10
  • 1
    The only correct action on that low quality, garbage answer, was to indicate it was unacceptable. It purpose of the audit was to suggest it was spam, and determine, if you could properly identify that the contribution wasn’t spam but still unacceptable. You apparently failed the audit. As a reviewer you must take it upon yourself to properly identify acceptable and unacceptable contributions. If you can’t do that use the skip option
    – Ramhound
    Commented Jul 13, 2023 at 12:01
  • 1
    “A.I. generated questions do not seem fishy anymore, and that will get worse and worse in future.” - No, This absolutely is not true, the content being generated by ChatGPT and other LLM all follow the same format including the deleted garbage that you reviewed. Unfortunately, There isn’t a nicer word, to describe that contribution you reviewed.
    – Ramhound
    Commented Jul 17, 2023 at 1:09
  • That party question received two extremely low quality incorrect inaccurate ChatGPT generated answers, and every user except one, properly identified and selected the correct review action.
    – Ramhound
    Commented Jul 17, 2023 at 1:21
  • 1
    Any screenshot of the answer will help those with low rep see what you mean
    – Vylix
    Commented Jul 21, 2023 at 5:21
  • Related on Meta SO: Review Suspension: ChatGPT/LLM Audits. This is more of a problem with the overall audit system that may automatically choose questionable posts for audits.
    – Andrew T.
    Commented Jul 22, 2023 at 8:45

1 Answer 1

Reset to default
10

The question states "Please be more careful with audit tests" but the question body appears to be accusing the audits of being "too hard".

Sorry, but the onus is on you to correctly review, not on the audit to only present easy ones.
Review isn't just a numbers game, or a race to the end to win some points. It's supposed to ensure overall quality of submissions. If you find you're just dashing through to the end, think of something else to do instead.

When you're reviewing answers, especially late answers, don't try to work from just the answer itself. Look at the whole question & weigh the value & intent of the new answer against the others.

In this particular case you would have seen that the answer had already been deleted, which would have enabled you to choose a more appropriate action, & shown the audit you were paying attention.

The audit is not all that 'smart' so any action towards deletion or downvoting would probably be sufficient to 'pass'.

If at any time you are still unsure, then skip & leave it to someone more certain, or with specific knowledge in the subject covered by the QA.
You don't have to do every review. The skip button is as important as the others. You lose nothing by skipping if you don't know.

Also see How do we better educate new Reviewers in the Review queues? for what happens when reviewers don't skip, or don't read things properly.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .