-2

Every so often, we review some questions, close questions, reopen questions etc, some of which do take a bit of time, thought and consideration, to suddenly be told

"Congratulations, this was a test! "

This is a nuisance... To open / close posts already requires 5 votes, why add this patronising and time consuming test - it's superfluous?

(Updated based upon allquixotic excellent comments) So, can we Remove it (or greatly reduce it) after a user reaches a certain reputation (certain number of review badges )... Whether that be 10k, 25k, 50k doens't matter - the point is, it's a high number and it becomes very unlikely a user with such a high rep score would be so evil.

2

5 Answers 5

Reset to default
12

After thinking about this some more, I think it might be a good idea to reduce the frequency of review audits by 80% or so, for users who have a gold review badge. If you've earned a gold review badge, while suffering through the fairly frequent audits the whole way and passing a majority of them so you don't get review-banned, that's much better evidence than any amount of rep that you can probably be trusted to do good reviews.

My proposal is thus:

  • Review audits should never be completely disabled for any user. There should always be a non-zero probability of getting an audit.
  • Reviewers with a gold badge for the review queue they're currently reviewing are 80% less likely to get a review audit than those without a gold badge in that review queue.
  • If you fail a review audit while you are in the privileged "80% less often" category, you will be silently (without informing the user) bumped back into the normal review audit frequency category for 30 days (or 15, or 60, or whatever SE feels is prudent).

So, yeah, -- but not quite as Dave Rook envisioned it. :)

3
  • That would be sweet, to reduce the percentage some, based on the trustworthyness of the persons doing the reviews.
    – Psycogeek
    Commented May 22, 2014 at 14:19
  • If they find this is successful, they could even do a 50% reduction for people with a silver badge in that queue.
    – allquixotic
    Commented May 22, 2014 at 14:20
  • 1
    "passing a majority of them so you don't get review-banned" There is no permanent review ban last time I checked, so achieving a gold badge is quite possible even for bad reviewers.
    – Daniel Beck Mod
    Commented May 22, 2014 at 14:44
7

This is the review audit feature. It was added in response to people (or scripts) mindlessly clicking "No change needed" or "Close ..." or whatever (just to get badges, for example). You could easily block the behavior if they always clicked the same button, but if they had their script click the buttons available randomly, it would be impossible to distinguish this from normal user behavior.

That's why the audits were added. It's like a CAPTCHA. It strongly discourages users seeking to get a cheap gold badge in an automated way from harming the system by answering randomly, when they should be making educated judgments instead.

It's unlikely to be removed since the need for it was identified and it was implemented to stop bad behavior.

Putting aside the scripting issue for a moment: it also prevents people from manually, repeatedly, making poor decisions when they are processing the review queue. From the perspective of SE, there is no reason to think that, just because someone has very high rep, they are automatically trustworthy, and will always make good decisions. For instance, they could get drunk and go on SE, and in their drunken stupor they might think that voting to close every question in the close queue would be a good idea. The review audit will catch them.

The only people SE really trusts more than a typical user are diamond moderators. Everyone else gets more "permissions", but there are still checks and balances in place to make sure those privileges aren't abused. It's much more difficult for SE to automatically stop a bad diamond moderator; that kind of situation is usually handled manually (which is fine because there aren't a great many of them, but there are thousands of people with 10k+).

9
  • Are you saying the scripts can be run when some one isn't logged in?
    – Dave
    Commented May 22, 2014 at 13:56
  • The issue is orthogonal to whether someone is logged in or not. That's not relevant. You can very easily use a framework like Selenium to get a "logged in" session to your user account, and then automate it in arbitrary ways, including automatically answering randomly to review edits. There are many countermeasures to bot-like behavior on the site, and they're there on purpose. The site is meant to be used by humans, not by bots.
    – allquixotic
    Commented May 22, 2014 at 13:58
  • Oh... Interesting. So, based upon this, your saying 'one' could fake the login, and their reputation points?
    – Dave
    Commented May 22, 2014 at 13:59
  • No. You misunderstand. Selenium is an automation framework. That means, anything you can do in your normal web browser by clicking or typing, Selenium can do the equivalent automatically by writing code to tell Selenium to do that. So, since you can login to your SE account by typing your username and password at the appropriate prompt, then you can review stuff by clicking on the appropriate link and then clicking on the buttons, it is possible (trivial, even) to automate this using Selenium (or a myriad of other frameworks).
    – allquixotic
    Commented May 22, 2014 at 14:01
  • This has nothing to do with faking/spoofing login or reputation points. It is about, assuming you have successfully logged in an automated bot/script as your account, then using the script to get "free" badges or seemingly positive contributions to the site, such as the review edit badges, without contributing positively at all (in fact, random button-clicking in the review queue is detrimental to the site for obvious reasons).
    – allquixotic
    Commented May 22, 2014 at 14:02
  • That is very clear. But this indicates that SU (SE) are concerned that people who worked hard and got a high enough reputation would do this? See my point 2 in my question, which is ideally the goal... I understand about the automation, but the reputation surely (although not guaranteed) means that a person won't (very unlikely) to be bad. Even if a user had a rep of over 10k SE greatly reduced this test question
    – Dave
    Commented May 22, 2014 at 14:02
  • The Access Review Queues privilege only requires 500 rep. You have 13k; you should know that isn't a lot, and doesn't take much effort or "hard work" to achieve. It's even less work on StackOverflow, since just one FGITW answer could easily get 50 upvotes or a bounty. There's a reason you don't get Trusted User until 20k -- everyone under that is implicitly "untrusted" ;-)
    – allquixotic
    Commented May 22, 2014 at 14:04
  • OK, that is actually an excellent answer. For me 13k was hard work but thankfully, Google had most of my answers he he, but the point you make is fantastic... Yes, this is now very nice and clear... Although, despite being only on 18.7k I will trust your answer and information is correct and you won't spam my posts :)
    – Dave
    Commented May 22, 2014 at 14:07
  • Edited with more info.
    – allquixotic
    Commented May 22, 2014 at 14:11
3

As Allquixotic mentioned this is here to assure the quality of reviews in general and to help regulate the community.

There is such a thing as a review ban which is similar to the bans that come in when people post too many low quality questions or answers.

This is an important tool to prevent people abusing the system and assure that the quality of your reviews is good in general and also to potentially show you as a reviewer where you might be going wrong when reviewing questions.

1

I agree that it is usefull so the review is not abused, but I am often wondering if the way it comes up could not be changed.

When I do 3-4 reviews in a day, I can get on average 1 of these, that seems like a lot. If I review 12 items in a stretch , I might get only 1 of them also. (There is more to it than that, between skipping thing I can not review, and working all the sections, so this is a very loose estimate)

Without complaining, does it seem odd to anyone that I can have an average of 4-1 ratio, and an average of a 12-1 ratio?

Sometimes it seems , if I am picking up a few reviews in low times, I get to play this game with the machine more? Are smart people leaving these stupid things there :-) So i get to deal with them .

0

If you have mod tools, then do it through there. You can click on Tools, and see the closed votes, in order of votes. These can't be faked I believe because there is no way to disagree (although it does mean you can't disagree, but you can at least just ignore those posts)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .