49

Open letter here: https://openletter.mousetail.nl

I haven't decided what I think about this yet. It seemed fairly apparent months ago that technical support was a prime target for ChatGPT et al., and this is a technical support WWW site.

I posted this as a public service announcement on another Stack Exchange meta site, and given that has several Q&As here, it seems worthwhile posting a PSA here, too.

There's a whole lot more at Sustainability of new AI generated content policy and a fair number of other meta places, too, for the curious.

I am also informed (in comments to my preceding PSA) that https://jlericson.com/2023/05/31/mod_strike.html provides context and that anyone who performs curation duties, not just diamond moderators but anyone with the privileges of voting to delete/close/undelete/approve edits/whatever, and who wishes to show Stack Exchange Inc. what the world will be like once most people are overwhelmed in these tasks by a deluge of ChatGPT Q&As, is welcome to participate in this action, across the entire network.

17
  • 24
    I'm definitely not fond of being told that we are not allowed to try and figure out if people are using ChatGPT which as far as I'm concerned is essentially plagiarising some idiot on the Internet who just writes garbage for a living. The worst offenders are generally more active than normal users and could quickly turn the sites into a mess of wrong answers to worse questions. All they have to do now is tell us not to investigate or prevent sockpuppet voting and all the humans might as well pack up and let the chatbots talk and vote at each other.
    – Mokubai Mod
    Jun 2, 2023 at 20:44
  • 2
    I would rather just downvote the OpenAI garbage. It will take longer to prevent, additional contributions from plagiarist that use OpenAI derivatives, but it’s more effective than doing nothing. As always I will judge the contribution not the user. A perfectly worded answer with hundreds of characters, proofread by Grammarly, has less than 0.01% OpenAI detection.
    – Ramhound
    Jun 3, 2023 at 6:38
  • 4
    Ugh. There's a lot more going on. I might need to fill in the other mods
    – Journeyman Geek Mod
    Jun 3, 2023 at 9:56
  • 17
    My experience with ChatGPT answers on our site has shown that it has just about the same chance of coming up with a good answer as an infinite number of monkeys typing randomly on typewriters for an infinite time. I detect such answers by the fact that they simply make no sense, although they seem well-written.
    – harrymc
    Jun 3, 2023 at 16:39
  • 1
    ChatGPT answers can be good/perfect, but mostly for questions that are easily googleable by humans as well anyways. What this changes is mainly causing more plagiarism to stay on the network, as bad/inaccurate answers can be removed for other reasons anyways.
    – Destroy666
    Jun 3, 2023 at 19:07
  • 4
    @Destroy666 - “ChatGPT answers can be good/perfect, but mostly for questions that are easily googleable by humans as well anyways.” - I seriously question your ability to detect quality contributions if your experience is that OpenAI generated contributions are “good” or “perfect” because I haven’t seen a single generated contribution by OpenAI that even answers the question that was asked let alone the fact it certainly wasn’t “good” and wasn’t “perfect” and I have identified and reported hundreds of generated contributions.
    – Ramhound
    Jun 4, 2023 at 0:47
  • 1
    @Ramhound I seriously question if you have any knowledge about AI models if you think they can't answer lots of well-formulated software or hardware related questions in 2023... If you think you've been catching most of them before the rule change, you're also likely wrong, it's very easy to get an answer and change the wording, if you can speak English. I've seen a bunch of potential coding answers like that on StackOverflow when reviewing. It's pretty much impossible to detect then if it's an inaccurate answer due to lack of user's knowledge or because they copied code from ChatGPT
    – Destroy666
    Jun 4, 2023 at 1:06
  • 7
    @Destroy666 - Everything I know about OpenAI (specifically ChatGPT) is from flagging the absolute trash answers generated by ChatGPT. I also know it takes a lot of effort for OpenAI to generate production ready code, specifically enough knowledge, to actually just write the code yourself so it’s not that helpful. The code I have seen on SO that’s been generated by ChatGPT hasn’t been impressive
    – Ramhound
    Jun 4, 2023 at 5:34
  • 4
    I have seen on SU a user (account since removed) who answered in a very short time 5 posts, all answers full of weakly related information but with no solution for the poster or just plain wrong. They even got upvoted because they seemed informative. Allowing ChatGPT will mean that anybody can answer as many posts as he has energy for doing copy-paste, and the upvotes will surely exceed the downvotes. This for sure will mean pure data pollution for SU.
    – harrymc
    Jun 4, 2023 at 10:48
  • 8
    @harrymc "They even got upvoted because they seemed informative." - Exactly. It's far too likely for a bad GPT answer to get upvoted just because it sounds helpful or informative. I've seen bounties awarded to GPT answers even though they didn't answer the question, because no one else provided an answer and the OP was just "thankful" for the "attempt to help." Jun 4, 2023 at 11:39
  • 2
    Every ChatGPT answer has something in common. It parses the question and will answer every element of the question. Including addressing irrelevant parts of the question. Most of the time, the information is "accurate" but utterly irrelevant to the actual question. I see the current iteration of AI language models to be the equivalent of 3D movies in the 1980s, it's basically as impressive as those glasses with three panes of specific colors(red, green, and blue) made out of plastic. Reading how "AI" is going to be used to full HR jobs at IBM, i fully expect, a full reverse course in a year
    – Ramhound
    Jun 4, 2023 at 18:38
  • 5
    @Ramhound: I have seen answers where ChatGPT got creative and invented commands that don't exist. Its algorithm of chaining words selected by frequency of appearance can give strange results.
    – harrymc
    Jun 4, 2023 at 20:16
  • 2
    @Ramhound I feel like you are skipping over both the "can" and the "questions that are easily googleable" part of what you quoted. Go ask chatGPT for say, a bash one-liner that counts the number of unique lines in a file that include the word "tomatoes"; and you'll get a "good" or "perfect" answer. But these questions shouldn't be on SU/SO/SE in the first place, as, like said, they're easily googleable anyways.
    – mbrig
    Jun 10, 2023 at 3:10
  • 1
    @mbrig - Most of the questions only take about 5 minutes to research, but it has taken a lifetime of knowledge regarding Windows to answer most of those questions so the individual asking the question understands the answer.
    – Ramhound
    Jun 10, 2023 at 10:32
  • 3
    Wow. I had no idea that this was going on and I support all of you moderators to the fullest. Not only is the output from ChatGPT deceptive guessing but nobody seems to realize that we will all be put out of work by "less than" garbage. Jun 12, 2023 at 22:34

1 Answer 1

-17

In this specific instance I am against moderators unilaterally handing out sanctions for perceived1 ChatGPT usage2.

In this instance I am also against the use of a strike to achieve the end-goal.

But First ... I am not pro ChatGPT answers

Do not misconstrue this as pro-ChatGPT answers. I remain in favour of the current ban and my words here should not be misunderstood to disagree with it. Users posting many ChatGPT answers should be banned.. However moderators are not the right way to handle the scourge of AI content.

Leaving moderators to go toe to toe with bots will only end badly including, ironically, moderator burn out.

Moderators are good at moderating the humans. Let SE themselves work on stopping the bots4, because bots are best stopped by technology not humans.

Only Confirmation Bias says you're good at determining this

I've been seeing "ChatGPT hallucinations" in people's posts for the last 10 years! This includes made up quotes linking to non-existent articles... Only ChatGPT hasn't been around for 10 years and users are capable of generating bad content in the most weird forms.

I know that you may have seen many posts which look certainly like ChatGPT output. The problem is that there's no direct feedback on whether you're getting it wrong or right. So you're going to be at serious risk of extreme conformation bias.

Even if you are personally good at making this judgement call, kindly remember how large SE is and consider the fact that SE has no means to test your personal ability on this topic.

High risk racial bias

Unfortunately people are so bad at this that their practical assessment is likely to expose a significant unconscious racial bias.

Make no mistake, this is not about people from some countries having "bad English". That's not remotely what I'm discussing and would actually be quite racist of me to suggest! The reality is much more nuanced than that. Common English teaching in different countries varies producing markedly different styles, some more formal than others. Where someone learns English can easily influence the extent to which their language is perceived as "perfect English".

In the days since the ban was declared, I've personally become increasingly disheartened with the resulting witch hunt that has ensued. It is absolutely counter cultural to SE. Too many times I've seen accusations of ChatGPT where those accusations were based on use of English. Command of the English language is isn't generally an indicator for ChatGPT both positive or negative!

What catches a serial ChatGPT user?

SE are not opening the door to every ChatGPT bot. There are other ways to detect it's usage that is much better than moderator intervention.

The best information for catching such users is actually buried in information that is not available to moderators. This might include such features as

  • the timing and evolution of saved drafts while forming answers
  • the click through time between first viewing a question through to posting fully formed answers complete with citations
  • cross SE answer rate

There are also multiple sources of information that moderators might have access to, but can be simply applied programmatically:

SE do not need moderators to detect AI content!

ChatGPT is an AI model designed to be very good at authoring content3. It is not remotely helpful with mimicking human behaviour.

Should there be a flagging process?

Probably yes. But it's unclear what precisely could be done with those flags, other than letting them accumulate and try to spot a pattern across separate SE sites. For reasons mentioned above these flags are likely to involve a high degree of false-positives and possibly be racially biased.

There is no point in showing SE what the effect of this policy will be

The logic here doesn't stack up. SE will find out the effect of this policy in good time when it goes into force and the real life effects can be seen.

Strike action here actually does nothing but throw toys out of the pram.

The effect of this strike will not even be a fair demonstration of the effect of SE's policy.

Moderators are not the community

A strike here can only be action by moderators, it cannot be action taken by the community. Forgive my directness but in other corners of SO I've already encountered moderators who think they are entitled to dictate to the community at large. That's not the order of how stack exchange is built.

I empathise with the fact that moderators have genuine concerns and feel let down over this. I also see there is some good community support for the view point. But a strike here will hurt many who have not taken an interest in this particular topic.

Historically strikes turn those hurt most against the cause. What I fear most here is a level of toxicity building between the moderators and the community. There will inevitably be many who feel that moderators should either do their best to fulfil their role or otherwise move out of the way for someone else who will.

In short nuclear options always have massive collateral damage and invariably unintended consequences. I'd urge you to reconsider your approach.


1"perceived" here meaning the conclusion was made from the content of the post, or the behaviour of the user without conclusive proof such as a signed confession.

2I am also against handing out sanctions where the poster has explicitly stated their ChatGPT usage because honesty should not be penalised and such error is better dealt with by politely informing the offender of the rules.

3I deliberately make no comment here about it's success

4Bot or part-bot. Ctrl-c Ctrl-v on repeat is largely the same problem as a bot.

19
  • 22
    All answers that I suspected of being AI-generated were characterized by perfect English. In fact they read as if they were written by an English major who knows nothing about computers and is rephrasing top Google results into a bullet list. If a new account is spewing these out at a humanly impossible rate… That's pretty convincing evidence, if you ask me. If someone thinks an answer was written by AI because the language isn't great, then they probably don't know much about the current state of AI.
    – gronostaj
    Jun 5, 2023 at 19:00
  • 2
    @gronostaj it isn't so convincing to me because not only did I see similar answers long before ChatGPT, I had the misfortune of hiring someone liable to write them. A truly weird mix of being able to paraphrase and reword anything without any capability to understand a word of it. As I say, confirmation bias plays terrible tricks on people on this topic.... where as impossible answer rates don't really need moderator attention. SE should be perfectly capable of identifying those programmatically. Jun 5, 2023 at 19:37
  • 15
    "Extreme racial bias" - 1. no LLM has bad English. Perfect English with clear lack of understanding subject is usual indicator of AI answer, not the opposite. 2. Correlation does not mean causation. I doubt any mod before issuing ban or even mod message checked out where this particular user is from.
    – markalex
    Jun 5, 2023 at 19:40
  • 6
    "What catches a serial ChatGPT user?" How about users who come on streaks of multiple answers on unrelated topics separated by minutes? Here is an example. Should we allow such users pollute answers with senseless garbage?
    – markalex
    Jun 5, 2023 at 19:43
  • 1
    @PhilipCouling, I have merely stated that only possible indicator of AI content in regards of English level is if said level is perfect. Nothing to do with culture, race, account location. Your insinuations about "those who against chatGPT answers are racially biased" same as those made by official policy announcement - are most outrages statements in regards of current situation.
    – markalex
    Jun 5, 2023 at 19:52
  • 5
    @PhilipCouling, "Neither SE not I am suggesting such users be left to pollute." this is exactly what SE suggested. But they felt ashamed to share this stance in public announcement, and left it in private rooms. Moderators are effectively prohibited from taking ANY action against AI answers, unless they are breaking other policies or rules. No rule prohibits posting 6 answers in 12 minutes. What should be done to user who does this in your opinion?
    – markalex
    Jun 5, 2023 at 19:56
  • @markalex then see my final two sections of this answer. If this is truly the case then the truth of it won't be hidden. By taking this action you don't give SE opportunity to deal with the problem in another way that they haven't told you about. Jun 5, 2023 at 19:58
  • 2
    Even if we suppose that everything you said is correct, that we cannot accurately pinpoint that these BS answers are coming from AIs and they are actually coming from humans... shouldn't we still delete them? How long did you keep that employee you mentioned in your comment, the one who generated large volumes of paraphrased and reworded text without any grasp or understanding of it? How often did you ask them for help when you were stuck? That's the point. Moderators want to remove low-quality content from the site. And the larger community wants that, too. Jun 6, 2023 at 6:19
  • @Ramhound no I didn't but I'll rephrase.. Moderators seem to be under the impression they are the only ones who can fix this problem. That's not the case. Moderators are still being asked to act on VLQ content just as they always have. But the decision to go further than that by either banning users or deleting content specifically because it is AI generated is in question. SE were asking moderators to take further action, they are now asking moderators not to because that backfired. NOBODY is saying we should leave VLQ content on the site. If it's VLQ then act as we always have done. Jun 6, 2023 at 9:53
  • @PhilipCouling: You are also wrong about not having tools for catching ChatGPT answers. Have a look at GPTZero and ZeroGPT. These are the tools that I use to verify my impression that the nonsense answer came from ChatGPT, before I flag it for the moderators.
    – harrymc
    Jun 6, 2023 at 10:00
  • @harrymc how precisely to SE need Moderators to be a part of the process when such lovely tools exist that can be applied pragmatically by SE themselves? SE's decision to stop moderators taking this issue into their own hands is only strengthened in light of such tools! Let SE do the work using better tools and stop using moderators from performing tasks which they are unlikely to be good at. Jun 6, 2023 at 10:05
  • 2
    @PhilipCouling - The issue is, and I have witnessed it, users who are using LLM to answer questions submit dozens of low-quality answers. Usually, this wouldn't be a problem; real users only submit a few answers daily. Very quickly, they would notice that their contributions are not well received.
    – Ramhound
    Jun 6, 2023 at 10:15
  • 2
    Additionally, the problem with submitting dozens of answers at once is the community can't moderate them without assistance from the moderators. If the moderators are not allowed to delete, low-quality AI-generated garbage, then the AI-generated garbage will be left on the website for a while. It will then require users to downvote (not upvote) this garbage, and vote to delete the AI-generated answers, which is more work on everyone's part. Moderators should handle the users, the community should handle the contributions, submitting dozens of low quality contributions is a user problem.
    – Ramhound
    Jun 6, 2023 at 10:16
  • 2
    Sorry to have to say this, but there's no point arguing with any semblance of logic in this case. Recently there's a mob of people throughout stackexchange that will down vote any post that even slightly supports leniency to anything generated by AI. Some people are pushing the narrative of "AI bad, ban AI", and will go around downvoting anything that doesn't support it. Even though your post clearly says "chatgpt based spam should be removed", since it doesn't say "anything chatgpt should be removed" the "ban AI" mob is going to come and downvote this.
    – user13267
    Jun 7, 2023 at 10:15
  • 1
    @user13267 - Removing only spam generated by OpenAI isn't enough, there have been users who submitted several answers in a short amount of time, but the content wasn't "spam" it was just low quality. In those cases only a moderator can delete, contributions like that, because only contributions with a negative vote can be deleted. It's impossible for an engaged community user, to downvote multiple contributions, from a user who chooses to submit garbage generated by OpenAI.
    – Ramhound
    Jun 9, 2023 at 3:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .