5

For some reason posts or comments with some Korean characters inside aren't allowed. For example in this question it must be worked around using images How to search a Korean word made up of two or more syllables in Windows Explorer and if I edit this question Displaying Korean characters properly on a computer with English Windows XP I also get a message that says Body cannot contain "표준". When I commented with Korean characters I also get an error

I've seen this meta question Cannot ask question with Korean in it? but it's been too long ago. Shouldn't the ban be lifted off? One answer there says

The restriction on what characters are no longer allowed has been relaxed.

You should be able to post Korean characters now and the error message has been updated to be more helpful.

but I still can't use Korean characters in a post. If the spam still happens then at least count the ratio of Korean characters per post, because those spam posts contain all Korean characters

| |
  • Agreed. This was/is weird. No idea how the content filtering works, but if the question is — let’s say — 90% western characters and 10% in the non-Western character range it should be a valid post. Maybe this I a better question for the larger Meta Stack Exchange site? – Giacomo1968 Sep 1 at 2:48
  • 2
    It's iirc a block put in place as needed when we get hit with Korean spam waves.... – Journeyman Geek Sep 1 at 3:49
  • While it was previously relaxed it appears to have been switched back to a blanket ban again. I would therefore assume that the ban is still relevant and necessary and that the reason for the ban being switched back is because spammers were finding ways around the blocks. – Mokubai Sep 1 at 6:12
3

There was a second spam wave in October of 2019, so the ban was added back again. We can disable it again or we can try to make it smarter since these waves seems to happen more than once. I looked at the entry and this is what I'm seeing - the [\uAC00-\uD7AD]+ is part of the unicode range for Hangul.

Screenshot of SEDE query results listing the date the block was put in, the text of the block "[\uAC00-\uD7AD]+" (a subset of Hangul characters) and the date it was last hit (2020-09-01 and the number of hits -226.

Over the last 11 months there have been a bit over 300 hits on the block -

Screenshot of graph of the times the block has been hit

Based on this, it doesn't seem like there's a ton of attempts happening currently, so it's probably fine to drop it again. Confirm if this is what y'all would like and I'll get a dev to remove the block. If you have other thoughts, let me know.

I'll check back in a week unless someone pings me that it's more urgent and decided.

| |
  • This is good info. Thanks for sharing! As I mentioned in my original comment, what about doing some weighted comparison between Western characters and non-Western characters. In cases like what is linked, it seems to me that 90% of the text is Western and the rest is not. If somehow non-Western characters were something like more than 50% of the content that should be blocked. – Giacomo1968 Sep 1 at 15:15
  • 1
    That'd be super awesome but our system isn't set up to do that. It's a simple block/allow so we can't count content or anything like that. Y'all could certainly make a FR for it but I don't know that the need is high enough that we'd build it in the near future. :) – Catija Sep 1 at 19:03
  • Thanks! So what you are saying is the simple block for Hangul is just this regex, correct? [\uAC00-\uD7AD]+ – Giacomo1968 Sep 1 at 21:20
  • 2
    That's correct. It's not the full Korean character set but it seems to be enough that it catches most of the content. We can rewrite the regex matching but we can't have it check for a percentage. We do this for other languages, too, like Cyrillic. :) – Catija Sep 1 at 21:22
  • 1
    The problem I can see with removing the block is that what could simply be "testing the waters" with a couple of minor attempts a day could quickly avalanche into a full-throttle tsunami. It's impossible to know it is safe to unblock until we do and either nothing happens or the spam wave starts again. It's Schrödinger's Spam.... just because we don't see it, doesn't mean it isn't there waiting to happen the moment we open the door to have a peek. I think whatever we do needs to be a bit smarter than just leaving the gates open. It's happened several times before and probably will again. – Mokubai Sep 2 at 13:04
  • @Mokubai So... yeah. The fact that there's one minimum (no weeks have zero attempts) does seem a bit odd... it could be a bot testing the block weekly... but lots of sites get foreign-language spam without it being a "spam wave" that needs an outright block on the language being posted. Charcoal tests for posts that are primarily non-English characters, so any such spam should get quickly caught and deleted by them. The fact there's spam is kinda less important than that it's deleted and if there are users who want to post content that uses Korean characters, we should allow that if possible. – Catija Sep 2 at 14:59
  • I'm all for allowing the content as it is a bit unfair to exclude a large part of society that may happen to use any given language, but I can't help to think that the response needs to be more nuanced than an all-or-nothing approach. Being an English speaking site we expect questions to be primarily English, but there are cases where occasional foreign characters are required. I agree with Giacomo in that we need to have a threshold of characters before a block occurs. Yes that means processing, but spammers have proven they can't be trusted even though we need to allow other languages. – Mokubai Sep 2 at 15:12
  • I willing to accept that this is another "can we remove remove the block yet" attempt, but I'd much rather it be a learning experience in what we can do better than simply yet another opening of the floodgates followed by immediate shuttering some months later. We have the tools to do it properly, the question is do we have the time and effort as well? If not, then are we doing it for the sake of giving one more chance or do we have an end game? Insanity is meant to be performing the same ritual and expecting a different outcome. Our opponents (spammers) are debatably insane, are we as well? – Mokubai Sep 2 at 15:43
  • Sorry, I was trying to get a more detailed explanation from someone that would be able to estimate the work required for what you're asking but I seem to have failed - @Mokubai I understand what you're saying and if it were possible, I'd do it in a second but, right now, the choices are between the two sides of "allow entirely" and "block entirely" - as far as I can tell. We could try blocking Korean in titles only - which is an option but if the spammers figure that out, they can get around it by writing English titles. – Catija Sep 4 at 13:27
  • @Catija and that is fine, I just wanted there to be some consideration to both sides of why we have the block vs needing to let people do stuff. I am completely happy if we want to remove the ban and hope that the spammers have found something productive to do with their time. That would literally be my best case scenario: that it is no longer needed. If you've at least started some kind of discussion about what might happen then we're cool and can see what may come. I only posted another answer so that we could gather a bit more community involvement in whether we remove the block or not. – Mokubai Sep 4 at 13:50
1

Catja's answer is effectively advocating that the block is outright removed and I wanted to simply garner Community opinion on whether that is the right course of action. To that end this is a counter-argument saying that the block should remain in place until such time as an effective permissive, yet still spam-blocking, alternative can be put in place.

Note: I am generally in favour of the removal of the block, as no language or person should feel that they are unwelcome here and in its current form the block is more of a hinderance than a help, but it needs to be done in a considered and deliberate fashion.

As such this answer is intended to play the devils-advocate.


Those parties choosing to bombard our site with spam are an ever present background hum of activity on not only the Stack Exchange group of sites but the internet in general.

One of those groups are a set of Korean language spammers who are technologically savvy users who are able to evade conventional spam-fighting tactics such as user and IP blocks and several previous incarnations of regex based filtering (might have previously been removed and reinstated, but I do not know) as evidenced by the fact that attacks persist even when fed into the existing spam prevention system by both community members and moderators alike. This puts them above the more mundane spammers that frequent our site to advise us of their nice floral boutique shops never to be heard from again.

These attacks have happened before [1], [2] and we have tried to relax the system after some time only to be met with another influx.

Lacking a system that is flexible enough to meet the demands of the community I would argue that the current block, while heavy-handed, is necessary.

Unless the current block can be relaxed while simultaneously retaining a sufficiently strict edge to prevent future waves I do not see simply removing it as a solution.

The current regex based system may not allow sufficient flexibility to suit a large number of use cases. If it is not up to the task then perhaps a new system needs to be put in to use.

As a primarily English speaking site we already enforce rules that the majority of a post should be in the English language, but there will always be situations where users need to use their native alphabet to demonstrate a problem correctly and succinctly. We shouldn't be afraid of one set of bad actors

Such a system was requested in the past Meta SE: Enable a percent-based block of CJK characters on Travel so this isn't a particularly original request, but it is a request nontheless.

While I am willing to relax again, I would prefer that it be done with a goal in mind: at an absolute minimum it should be to finally give evidence of whether or not enhancement is needed to the current content filtering system. If the spam has subsided then nothing needs to be done, but if it happens again we need to know that something will be done going forwards.

| |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .