Existing list of garbage "new" TLDs? - ORF Forums

Hello,

We're starting to receive a ton of spam from these new "fad" TLDs like:
.rocks
.xyz
.bar
.website
...

Rather than looking at the logs and blocking a new one each day, does anyone have a text file listing the new TLDs added in 2014?

I don't mean for country codes like .es, but the ones created just to sell more domain names, that no mail would likely be legit from

Here is a list of the TLDs we block because of past spam. Typically a few times a day I load my log viewer with 24 hours worth of events and then futher filter (The Message field contains "Email passed checks") and (The Time field is from the last 2 hours) thus showing me a recent snapshot. I then sort by IP and look for trends. The TLD domain stuff tends to cycle, so the ones we have in our list are hand picked by myself. Blocking an entire domain can be dangerous unless you're absolutely certain your organization doesn't expect traffic from there.

asia
at - austria
click
club
co - colombia
country
eu - European Union
in - India
link
me - montenegro
mobi
rocks
website

@felipe.garcia: I agree that it is risky to tag top level domains, but it can be helpful to use that as part of a pattern. For example, you could look for 2 or more keywords to match any suspicious TLD along with a suspicious keyword. Example Sender rule:
.*((cambogia|forskolin|\.mobi|\.rocks).*){2,}

I understand the risk at the TLD level, but it's all relative... a fortune 500 company has a lot more to lose by whacking those, but someone with 100 users is probably ok blocking anything from for example .rocks (i doubt any legit business would use that as a sending domain - at least the clients we work with anyway)

So back to the question, is there an actual list of all newly added TLDs around that anyone knows about?

Also - i'm greatful to the spammers who decide to use links that end in .rocks or .link or .click, it's that much easier to block them based on body patterns

@Bryon: If your userbase is that small, perhaps it would be easiest to simply whitelist only TLD domains you want, starting with *.com I presume. Any new TLD domain that comes out would automatically be blacklisted. It's less work.

@Sam

I sense some sarcasm, well i hope at least - can you imagine the effect of whitelisting *.com?

Why even have a filter at that point

@Bryon: No sarcasm intended - although I see how it read that way.

You can add a Sender Blacklist rule like this:
(?!.*(\.com|\.gov))(?=.*)

Then any sender not from .com or .gov TLDs would be blacklisted. It is a very wide swath and you will surely have a false positive eventually when there is a legitimate need to add a new TLD to your userbase contacts but it would do the job if your TLD email contact domains are relatively static.

Here is a nice list of newly registered TLDs:

https://www.name.com/account/ntld/watcher

Here's an example that just started filling our logs today - .democrat

All these stupid TLDs that have no business existing, other than to sell registrations

Sam thanks for the regex. Every day we add filters for new tlds, and many are as bogus as everything else in the headers of the email. DN's have enterer anarchy.

I just block any TLD longer than four alpha characters: .*\@.*\.[a-z]{5,20}$

Have been doing that for over a year - no legitimate messages have been lost, that I've ever seen. I've been blocking .info addresses and several other 4-character TLD's for 6 years, also without any problems.

I like that idea, it's pretty good!

Just started getting floods of email claiming to be from *.ninja

Really, I know the resellers want more money by selling new TLDs, but find another way.

I'm putting in that '3 or less' rule right now

My advice is if you are legit company don't use .ninja domain. I send it all to junk. all I get from .ninja domains is crap so filter to junk.

@chris.cunningham: @Chris: I like this idea but be careful, as new top level domains open up, more and more we are likely to see legit domains using long alpha sequences. I am using your rule for about a week and it works well but today I noticed a legit mail from a .london domain so I needed to start adding an exception list.

I block anything that doesn't deal with the specific countries my company interacts with.

(?!.*(\.com|\.gov\|\.sg|\.hk|\.cn|\.ph|\.jp|\.org|\.net))(?=.*)

anyone else can find themselves manually whitelisted.

@felipe.garcia: .at? Wat? are you stupid? It’s the normal Austrian domain.
And .in and .co too?

if you’re not some weird kind of racist, you’re at least as stupid as one.

@BAReFOOt: Wow BAReFOOt - sorry you don't like blocking countries as a whole

I would point out "countries" are not the same as "races"... someone who has no dealings with Australia, India, or Columbia and chooses to have no email from servers in those countries is their own choice... what "race" are they descriminating against? The Australian race? Think about that for a minute.

If you look into it, you'll see that some countries have no laws against spam, and providers are happy to take money from spammers and allow them to send millions of emails. Why should anyone "have to" receive those emails from those countries, or be labeled a "weird kind of racist"?

Seems like "stupid as a weird kind of racist" pretty much discredits any use for you here

http://arstechnica.com/security/2015/09/many-new-top-level-domains-have-become-internets-bad-neighborhoods/

Interesting article about how many of these are just pure spam havens. I'd be fine blocking toplevel domains that product 50% or higher spam. These new domains were always a bad idea.

@Sam Russo: Along these lines, what would you guys suggest as an efficient regex for BODY text to capture anything containing links to these garbage domains (i can fill them in just looking for the pattern, i'm not great with regex)

i was thinking a body search for something like:

(?i).*(http://.*/.(review|racing|faith|truth|ninja|democrat|site|win|name|whatever-else).*)

i know that's probably not right, which is why i'm asking you :)

@Bryon: or even "any link in a body which contains 4 or more characters after the last period after http or https" ?

@Bryon: There is no quick answer here. I have spent many days tuning ORF as well as adding custom external agents and a database to get what we needed that ORF could not offer directly. For most this would not be practical.

I just looked over my logs to double check and here's how I catch these bad spam TLDs, in no particular order:

- a custom external agent that catches a complex pattern from a known spammers/toolkits that blast spams to quickly for lagging indicators to work
- a custom external agent that catches snowshoe spam (not Vamsoft's)
- RBLs and SURBLs like Spamhaus
- Sender blacklist rule for long domains (shown below)
- Sender blacklists that catch known spammy word combinations (stop snoring)
- Subject blacklist rules that catch common spam phrases (80% off...)
- Firewall vendor identified IP address as spammy, ORF rule picks this up from the header
- IP Blacklist rules for bad Hoster/ISP networks that have unacceptable volume of spam

Some of these methods (like tagging some host networks) are not likely to be endorsed by Vamsoft but I feel like the risk is warranted based on what I've seen in the logs.

To answer your question, this rule below will help if you are willing to risk false positives. I've added one domain ".london" as an exception so it is not tagged.

Sender Blacklist rule (risk of false positives)
"Suspicious Top Level Domain, 5+ alpha chars 20150424"
(?!.*\.london)(?=.*\@.+\.[a-z]{5,20}$)

@Bryon: For a Keyword blacklist rule, looking for links to specific garbage TLDs, this may work:
.*https*\://[a-zA-Z0-9\-\.]+\.(review|racing|faith|truth|ninja|democrat|site|win|name|whatever-else)

@Sam Russo: Thanks for these two examples

Looking for a keyword blacklist pattern, i borrowed some from the sender-blacklist suggestion, but the keyword-blacklist example seems to catch even things like "http://update.windows.com" (.win)

The sender example had the $ at the end, but the link isn't always at the end of a line, and since we're looking at links and not emails i dropped the @ sign

what do you think about something like this to catch "links with 5 or more characters after the last period"

(?=.*.+\.[a-z]{5,20})

I can see the danger here where if someone types a lengthy email and forgets to hit "space" after the end of one sentence and the next sentence starts with a word of 5 or more characters :)

"Hey Wendy, I'll meet you on friday.Mcdonalds is open late."

maybe we could put "http" or "https" in there somewhere to mitigate that?

@Bryon: I'm glad you are testing things - you'll need to keep a close watch on it whenever you go live.

We could change the regex to require that the link be followed by space, a greater-than sign which could close an html link, or new line characters. Perhaps this would work for you:
.*https*\://[a-zA-Z0-9\-\.]+\.(review|racing|faith|truth|ninja|democrat|site|win|name|whatever-else)[ >\n]

You can develop and test you regexes on refiddle.com

Good luck!

@Sam Russo: Oh i see what you did there

so taking it even a step further, so as to not have to keep updating the list of domains, one could do something like this:

.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}

testing that as a keyword blacklist, it appers to grab only "5 or more characters after a period after something after http-or-s"

looks pretty good, i think i'll set that to tag/redirect and see what happens :)

do you see anything i'm missing false-positive-wise? I can't think of any instances where something legit would inadvertantly trigger this, nor can i think of any legitimate use for a 5+ TLD (in our users normal course of action)

@Bryon: I think you'd still want to include the trailing space, < or new line after the link, so probably:
.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}[ >\n]

Be careful with it but you know your users. I've been tagging emails senders (rather than links) with 5+ alpha char domains for months now and it has not been a problem for us. (I have a very short exception list for legit long domain names.)

@Sam Russo: I like the Ideas of blocking many of the new TLDs and the ones over 4 Characters as senders addresses.

Have you also thought about blocking sending server Names with the same TLDs in the 'Sender Host Name Blacklist' ? Adding the same (?!.*\.london)(?=.*\@.+\.[a-z]{5,20}$) to that blacklist?

I received emails from host 'change.dermalogica.link' though the From addresses are like 'Ventilation Deals <>'

@scooter133: That's an interesting thought - your "from" address is just words, i bet the actual smtp-envelope address (logged by orf) was a .link address anyway though

We have enabled the setting where if the "from" email address doesn't match the sending server name, treat it as hostile (mx/a lookup i think)

@Bryon: Sorry, the From Address didn't all get Copied into the reply.
It was: 'Ventilation Deals <>'

So it was Different than the Mail Host. We cannot Block Mail hosts that don't match email addresses. So many of our customers us Hosted SMTP servers that many of them don't even come close to matching.

@scooter133: ha, Maybe it did... the < and > around it probably make it disappear?
See if this Works.
'Ventilation Deals < dahlia . sanford @ wsxbg i. me >'

Ventilation Deals - dahlia . sanford @ wsxbg i. me

Oh we should tweak this a little, just found that the expression i use actually blocks .stupid domains properly, but also blocks these legit formats:

http://twitter.com/somethingelse

how could we change this to say stop matching if there is a / somewhere after the last period?

.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}

@Bryon: Are you sure that is the link that matched? If there are other links in the message it could be another link that matched.

You can copy/paste the email text into the ORF rule sample area to try things out. If you start deleting characters until your rule no longer matches that will show you which part matched.

As an additional look, I've saved an refiddle page to show that your /somethingelse example should not match, but .stupid would. I suppose it could be possible that ORF's flavor of Regex's behave differently.

You'll need to do the legwork on this to find out what the real problem is.

See http://refiddle.com/2ioi for the examples.

@Sam Russo: Thanks - filtering it down i see the link that triggered it (was hidden in a clickable image) was:

http://blog.dhlecommerce-usa.com/

ORF must handle it a little differently than refiddle, refiddle doesn't match but orf does

Don't forget to add something to the end of the regex to require a space, > or new line so that the URL link is clearly terminated, as we discussed above. Does this work better?

.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}[ >\n]

@Sam Russo: or this, to allow for the first forward slash if present

.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}[ >\n/]

@Sam Russo: hmmm with that:

http://something.rocks - no match
http://something.rocks/ - matches

http://blog.dhlecommerce-usa.com/ - no match (good)
http://blog.dhlecommerce-usa.com - no match (good)

@Bryon: Then:
.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}[\S>\n/]

Actually, I like this better
.*https*\://[a-zA-Z0-9\-\.]+\.[a-z]{5,20}[^a-zA-Z0-9\-\.]

That's the way it goes with a Regex, you need to test alot to be sure...

@Sam Russo: That works really well :)

I use the following restrictions in my postfix. It blocks all of these domains (racing, faith, top, review, etc.

smtpd_helo_restrictions =
reject_rhsbl_helo zen.spamhaus.org,
reject_rhsbl_helo dbl.spamhaus.org

Figured I'd trow in my Regexs for sender and domain. They have been great, though I wish there was an early warning when the junk comes in from a new one. We got hit with 100's of .wang this morning.

So after going though the few hundred thousand email addresses in the company Database, there were no Domains with 5 or more, only a Couple at 4, though the 4 are becoming more common.

Sender BL:
[a-zA-Z_0-9.-]+@[a-zA-Z_0-9.]+?\.(asia|click|club|country|date|gq|help|in.net|kim|link|pw|rocks|top|win|wang|work|xyz|zip)$

DNS BL:
(?!.*\.london)([a-zA-Z0-9\-\.]+\.[a-zA-Z]{5,20}$)

Scott<-

Thanks for this - it's really good

I just noticed the .wang garbage today... and also a lot of ".LOL" (lol)

you should also consider .faith, .site, .racing, .review, .space, and the newest one - .uno

Great, I added a few of those.

With the two I listed We have filtered out about 18,000 emails in 7 Days!

We actually have had greylisting turned on, a fair amount still got thru though. Looking closer at the logs those that did still get thru were all surrounded by dns errors, a broken dns blacklist, so i've updated those per the vamsoft recommendations

Good tip!

@scooter133: Looks like there was a whole slew of new ones today... .cf, .tk, .ga

though some old ones too as my regex didn't accommodate a '-' in the domain name part. many of them today were @com-duffl.<something> and @com-oct1022.something

Here is my new Sender BL Regex:
[a-zA-Z_0-9.-]+@[a-zA-Z_0-9.-]+?\.(asia|cf|click|club|country|date|faith|ga|gq|help|in.net|kim|link|pw|racing|review|rocks|tk|top|uno|win|wang|work|xyz|zip)$

I was glad to stumble on this forum... it didn't solve my problem directly, but it pointed me in the right direction!

Here's the regex I came up with:

if /.*\@.*\.[a-z]{3,20}$/
!/.*\@.*\.(com|net|org|gov|biz|info)$/ REJECT Mail not accepted. Likely spammer domain.
endif

This is for Postfix, but says to capture any domains that are 3 characters long or longer, and drop them if they *aren't* .com, .net, .org, .biz, or .info. I might have to tweak it later, and this is in addition to other rules (like SPF checks, etc.), but so far, it has greatly reduced the spam coming in.

@Clay: Hi,

would this blocking f.i. tld .Information as well or is it whitelisting them because of your exceptions? Would be easy to do for spammers, using tld beginning with your whitelisted "old" tlds.
I modified this for ORF Fusion:
(?!.*(\.com|\.net$|\.org$|\.gov$|\.biz$|\.info$))(?=.*\@.+\.[a-z]{3,20}$)
Thanks for Suggestion given in this thread.

Regards
Norbert

Just used this as a filter and found a "legit" Domain ".direct" ;) So maybe crawling the logfiles is a good advise.

@n.fehlauer: Thanks for the comment. I believe you're asking if fake/spammy domains like ".coms" would pass because I am allowing ".com". (If not, I apologize for not understanding the question.)

Certainly test it in your environment, but on Postfix, it works as expected:

root@mail:/home/clay# postmap -q pcre:/etc/postfix/sender_access
(This one yields no output, which means it passes this particular check.)

root@mail:/home/clay# postmap -q pcre:/etc/postfix/sender_access
REJECT Mail not accepted. Likely spammer domain.
(This one (with .coms, instead of .com) prints the REJECT message as expected.)

Hi Clay,

thanks four your explanation. In ORF I had to put the $-sign at the end to prohibit things like .coms and now it seems like it's working the same as your sendmail config.
Btw. I forgot to enter the .edu domain. ;)

regards
Norbert

@NorbertFe: Ack! Good catch on the .edu TLD! I forgot it as well, but have added it now.

Has anyone had any legitimate mail from .eu?

We have been getting hit by many Garbage TLD ending in .eu. To many to block just the domain name as they change all the time. Though given that its .eu and we do deal with vendors in Europe (have yet to check if any of the emails end in .eu) I didn't want to blanket drop all .eu.

Is it reasonable thinking that the Servers that are being used for this type of Junk mail are all sending junk and pretty much nothing legitimate or have legitimate customers as they severs would be getting added to SPAM Lists? I've been ending up blocking the entire Class C address space for these as they are coming from at least 6 different servers in the Class C.

@scooter133: We do get legit mail from .eu domains but the vast majority is spam

There is no one best strategy - we use a combination of:
IP range (based on spam friendly hosts hitting us)
Keyword content rules (based on historical spam messages)
Spamhaus and our firewall vendor IP/URL reputation

There is one Sender rule we use that could help you a little, use at your own risk:
(?!.*\@edu\.)(?=(.*\@.+\..+\.eu$)|(.*\d.*\.eu$))
EU (numbered or subdomained) 20141019

Thanks Sam, Have to love that there are a gazillion ways to do the same thing. (-;

I Finally convinced management to allow me to filter the subdomained .eu senders. I weren't with this and its Similar to the others floating around on here:
[a-zA-Z_0-9.-]+@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(eu)$


I like this Addition, |(.*\d.*\.eu$) Though not sure if we have had any come though.


I was thinking of doing the same Regex for .co to and filtering out the subdomained versions too.

@felipe.garcia: Well, blocking senseless gTLDs that are used to 90% by spammers and other malware slings does make sense. Their reputation and brand value is so bad that no respectable company will use them and also for privateers they are not attractive. False positive rate likely very low if you block them.

What is rather insane and makes no sense is blocking country TLDs of relatively high reputation such as .at or .eu. These are used by real companies and persons with spammers likely a very small fraction.
If you feel a need to block these TLDs then your spam detection is seriously lacking I would say.

Personally I currently get away without hardblocks on TLD. I only heavily greylist some of the more notorious gTLDs such as .xyz. Their throwaway domains are sufficiently shortlived that they do not survive 24h greylisting anyway. This is safer and does the job as well.

Hi,
as it happens sometimes that there are some legitimate TLDs with more than 2 letters, I thought I'd share my updated filter here, which filters all TLDs with more 3 or more letters with the execption of
.com
.net
.org
.gov
.biz
.info
.direct
.edu
.int

(?!.*(\.com|\.net$|\.org$|\.gov$|\.biz$|\.info$|\.direct$|\.edu$|\.int$))(?=.*\@.+\.[a-z]{3,20}$)

HTH
Norbert

@NorbertFe: FYI:
Legit flight booking related emails coming from .tix.work
seed4.me VPN service sending from .xyz

@Bryon: Just saw this looking for a tld file format for spamdyke to block .xyz, .stream and a few others being used solely for spam it seems lately. Saw your comments about .rocks. I chuckled. I currently own qmail.rocks and we are legit. But you're also correct in that I don't do business with you - LOL just thought I'd chime in

@Steve at qmail.rocks: Interesting about qmail.rocks... all i see there is "It Works!" - i guess it works, but what does it do? no www-dot, mail-dot, etc... must be very exclusive

Interested to see my first example of a legit service operating on such non-standard tld's - maybe one day we'll all see more legit than spam, but for now it seems like 999999 out of 10,000,000 are junk

Well - a buddy of mine in Maine and I do qmail research and use FreeBSD vm's to test configurations of qmail and add ons. Right now I am using all the usual suspects with the addition of Spamdyke, PF and a few other goodies. Amazing feature set, both manual and auto blocking of spam. Cannot believe how little gets through now. Qmail.Rocks was a fit since it was available and well - qmail rocks!!! So that's the story behind it. The work is all posted at William's site freebsdrocks.net. The man is a guru. And I am seeing .US, .STREAM .TOP and a few other that are just 99.9999 percent spam - and I block them!!!! {except .US - schools and others use this}

@Bryon: Nope - a fortune 500 company is big enough to say - "if you want us to get your email you will send from a legitimate TLD like .com, .net, .org, .gov, .uk, .ca, etc." TLDs with garbage names may eventually EARN their way into widespread and legitimate usage, but until then it's the bit-bucket for you. With my exchange server I have added a LOOONG list of TLDs that simply get deleted. I've run into only one business with a TLD that was on my list and took it off the list. As soon as I did my spam quadrupled. I put it back in and notified the company that unless they could send their notifications via an acceptable TLD then I would no longer be able to do business with them. Now I am a very small company, but they actually agreed and began sending email out on a .net address. I'm certain I wasn't the entire reason for their change. My point is, these crap TLDs are just for mass mailing companies and any service that does not allow you to block them cart-blanch is out-of-touch or in league with them.

@Des: I totally agree here - i think the root of the problem is the registrars who are selling these TLDs. I'm sure almost nobody asked for them to be created, and even less are actually buying them for legit uses... the only customers of these TLDs are spammers and maybe some teenage kids.

What self-respecting company would actually do business under any of these is beyond me... it just seems childish or unprofessional.

Sorry, world, if you're trying to do business as MyAwesomeCompany.Ninja, good luck.

But, as long as your favorite registrar gets their $5, that's all that matters.

@Des: While being a local builder and administrator of email/web servers I agree with you Des, the internet community as a whole does not. Including Google. There are many legit companies whose preferred domain .com .net whatever name has already been registered either by profiteers or other companies and see alternatives. Just 1 Ref of many - https://www.wired.com/2015/08/alphabet-rewrites-the-domain-name-game/

Daniel Negari is the founder of .XYZ, the company that acts as the registry operator for the .xyz domain. When I interviewed him for WIRED’s April issue, he told me, “We end the alphabet in ‘xyz’ and we should end domain names the same way.”

It turns out someone agrees with him. Yesterday, Google’s new holding company, Alphabet, revealed that it is making its online home at abc.xyz—a move that could signal an end to .com dominance for good.

Negari launched .XYZ last year to get into the generic top-level domain business. What are gTLDs? Whatever follows the “dot” in a URL: Most notably, “.com,” a term that has come to symbolize a whole lot more than a punctuation mark and three letters might suggest.

@Steve at qmail.rocks: I read the article, and in my opinion Google didnt "validate" .xyz as being the next big thing - they simply wanted a cool TLD for their company Alphabet. Since the alphabet is abc thru xyz, it makes perfect sense to own and brand abc.xyz

however, think about random examples:
www.intel.xyz
www.microsoft.xyz
www.hp.xyz

No, it will never, ever happen.
a. xyz is harder to type than com
b. saying verbally "email me, joe at microsoft dot ecks why zee"

Just because google thought it was snazzy doesn't necessarily mean they endorse it as the de-facto standard for new domains

Surely the internet community as a whole doesn't agree one way or the other, it never will (hundreds of millions of people can't ever agree on one thing)

If a company wants their domain name for their registered business name and it's already taken, all they have to do is file a lawsuit and take it based on copyright (cybersquatter act of the 90's, copyright infringement, etc)

.garbage .tld's .are .still .garbage

@scooter133: I like this scooter133:
"[a-zA-Z_0-9.-]+@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(eu)$
I like this Addition, |(.*\d.*\.eu$) Though not sure if we have had any come though."

I'd like to do this on sendmail but with .us so perhaps:
[a-zA-Z_0-9.-]+@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(us)$

or |(.*\d.*\.us$)

Does this go in the access file or sendmail.cf?

For the last few months all my spam seems to be coming from .us addresses so contemplating setting filter to ban all of them but don't know how widely used this domain is by legit companies..... any suggestions?
Cheers

@Goatherd: Yep seeing the same we're blocking any .us that have a subdomain so subdomain.anything.us but not just anything.us

@RobbieTheK: Thanks for that...... would you then use *.*.us to put into filter?

@Goatherd: Since I use sendmail I have to use its R command so my regex looks like this (adding .info as well as .us):
LOCAL_CONFIG
# this will block subdomain.domain.us or subdomain.domain.info but not domain.us or domain.info
Kcheckaddress regex -a@MATCH
[a-zA-Z_0-9.-]+<@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(us|info)

HMessage-Id: $>CheckMessageId

SCheckMessageId
R< $+ @ $+ > $@ OK
R$* $#error $: "553 Header error"

LOCAL_RULESETS
SLocal_check_mail

R$* $: $>Parse0 $>3 $1
R$+ $: $(checkaddress $1 $)
R@MATCH $#error $: "553 Your Domain is Blocked for Unsolicited Mail"

@RobbieTheK: is there a way we could do this using Spam Assassin? I'm having tough time fighting .US Domain Spammers. I have many customers using .US Domain and my current setup is based on Mail Server (Postfix) by Synology.

@Bony Buche: See https://serverfault.com/questions/728641/blacklisting-tld-in-postfix/728658

And we are blocking any subdomains with both .info and .us seems to be rejecting a lot of spam. So subdomain.domain.us would be blocked and domain.us would pass.

@RobbieTheK: @RobbieThek Thank you Mate. I read methods described on the link but problem is we have a GUI based interface where all we do is write *@*.asia or *@*.us to mark any domain as spam and then any incoming from those lands right into Spam/Junk.

It works wonder and I've marked about 20-25 such domains including the notorious (*@*.top yes) and boy ain't that heaven for spammers? though recently the .Us (*@*.us yes) seems to haven over taken .Top in terms rapid spamming. This has really affected my legit clients; their messages are getting lost into pool of .Us, .Top, .Win, .Steam and so on spam.. Spam Assassin is doing its job by curbing majority of spam but still some Alzheimer, Hair Transplant, Enlargement related garbage makes its way into the Inbox and to my surprise the sender has all policies met including spf, dkim etc. and this also includes those .Us level spamming.

Do you think using (*@*.subdomain.us yes) would work? is there a way we can mark everything from .Us as Spam except few selected .Us domains? this would definitely resolve my constant struggle of fighting spam.

Any tips on DNS based BlackLists? I'm currently using Zen.Spamhause, SpamCop.. God knows if I take them off how much crap would land in me Inbox then lol.

I would appreciate any help and please forgive any ignorant mistake I made above while explaining this sad situation.

Regular Expressions can have exceptions. Something like this should work:
(?!.*\.(except1|except2|except3)$)(?=.*\@.+\.(us|top|win)$)

This has not been tested, test it in the GUI as you add it. Monitor before you trust it and to look for new exceptions to add.

@Bony Buche: Not only did we block @anysubdomain.*.us and @anysubdomain.*.top but I added:
info|to|br|bid|cn

For RBL's we use: psbl.surriel.com, bl.spamcop.net, zen.spamhaus.org, & b.barracudacentral.org.

I also block the TLD's listed in https://www.autonarcosis.com/2015/10/14/vanity-top-level-domains-how-to-block-them-using-sendmail/ and then some!

We have been using the following to block multi-part .us domains:
[a-zA-Z_0-9.-]+@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(us)$

Though recently a *@ci.<city>.<state>.us tried to email us.

I'm not that great at RegExs and wanted to be able to alter the above so if it contains @ci. to allow it through but block anything else *@*.*.us

Thanks!

@scooter133: This should be what you want:
Sender rule, regex:
(?!.*\@ci\..+?\.us$)(?=.*\@.+\..+\.us$)

Remember to test it in the ORF GUI as you add the rule and then watch it closely as you go live.

@Sam Russo: Thanks, I maybe missing something those combinations didn't work, what usually works is as shown in the picture link below, I only write *@*.us and it does the trick. I think it'd be something similar to cater sub domain though nothing has worked so far :( .Us is filling our Servers especially on weekends when I'm away lol

http://i.imgur.com/1vOfOfl.png

@RobbieTheK: Thank you, I've added PSBL.Surriel to the list; hope to see some hammering. too bad I can't use BaracudaCentral on shared DNS, gotta have our own first to be able to use it. Do you know any RBL's that are reputable other than the ones you mentioned? I wonder if there are any other ways to optimize Spam Assassin cause sometimes it's allowing even the most obvious of spam which should be put to Junk straight away but unfortunately not happening. smh at this random behavior.

@Bon: I'm not sure what software was shown in the screenshot you linked to but I think you may be showing a wildcard filter instead of a regular expression (regex). My sample above was a regex. If you are using ORF, you can use either type and when you enter it in the GUI be sure to set the type ("simple text" or Regex). Also, you can test the filter with samples and adjust until it works the way you like.

@Sam Russo: Okay great news the *@*.*.us string is working pretty well so far with only exception of 1 legit domain that's getting trapped in SPAM because it falls under *@*.*.us defined category. I wish there was a way to block everything from this domain except 1 or 2? right now I've whitelisted that particular domain but still its landing in the SPAM so it means the first rule takes priority over any whitelisting or acceptance rules.

@Bon: be careful using '*@*.us' or *@*.*.us you are going to filter out all City Governments and Schools. user@ci.<city>.<state>.us and user@<schoolDistrict>.<city>.<state>.us

We deal with Cities and Schools in our state so I have been using the Following to allow the @ci. and the @*.ca.us

(?!.*\@ci\..+?\.us$)(?!.*\@*\..+?\.ca.us$)([a-zA-Z_0-9.-]+@[a-zA-Z_0-9-]+?\.+[a-zA-Z_0-9.-]+?\.(us)$)

I might tweak it to allow other States, but I think for our Customers we are pretty much covered.

@scooter133: I think just did that lol.. one of my important client is affected by this rule and all their Emails are landing into SPAM with *SPAM* tag even though I added them to safe/accept list be forehandedly but like I said before the 'mark as spam' rule takes priority over this. (Domain Example: jackson.k12.ms.us).

I couldn't make any of those strings to work, I think they're platform specific so might not be effective on the GUI side of Spam Assassin maybe? looking at above example of domain I'm not sure how to properly tackle down the insane amount of spam coming from .us domain except to either use *@*.*.us and compromise on few valid incoming or simply look into buying a paid platform of spam handling? I was though wondering if there's any such possibility to put forward a rule with block everything *.potato* with exception of this "mash.potato"

@Bon: A couple of things:

You keep bringing up SpamAssassin so I assume that is what you are using. The SpamAssassin user forums may be a better place to post questions specific to that software.

I'm not sure you have a clear understanding of wildcard expressions vs Regular Expressions (regex) for pattern matching. The kind of problem that you describe (tagging all .us domains except for a few) is best handled by a regex.

If you are supporting anti-spam systems you should learn what regex's can do for you. The effort is well worthwhile.

Good luck,
Sam

Learn about Regex:
https://regexone.com/

Test Regex:
https://regex101.com/
http://refiddle.com/

@Sam Russo: @Sam Russo, @scooter133, I know this isn't a regex support forum but it appears negative look-aheads are not supported in sendmail and the R & K commands (see http://www.xiitec.com/blog/2009/02/25/using-regular-expressions-in-sendmail/). Would you know how to write your regex (omitting cities and schools) in a sed-friendly syntax?

@RobbieTheK: Sorry but I'm not familiar with Sendmail commands and don't use sed much.

ORF regex's are very standard except that I've found I need to add .* in front which I don't normally do for other regex uses. I never pursued this but I think ORF needs it for it's regex implementation. It should not take much work to take the examples above and adapt them to your environment.

For what it's worth, I just got a legitimate email from a .cc domain and have added them so they can get further in the checking (it was a payroll company, as it happens). It is only the 5th 2 letter TLD that I've had to add (and all the rest were countries... .us, .uk, .fr, and .de). In the case of .cc, I just added the specific domain, not the entire TLD.

Obviously, your mileage may vary, but it pays to be flexible and periodically check the logs.

@Clay: My new mortgage lender also uses a .CC TLD - incredibly irresponsible of a business, in my opinion, to use something like that. It's asking to be blocked and they don't even realize it

Obviously i'd much rather add the one or two legit .cc domains per year than block 50 new ones every day

@Bryon: First, there are no kangaroos in Austria :-) FYR, Austria is a smallish country in the center of Europe, capital Vienna, while Australia (.au) is way south, much larger, and has kangaroos.

But sure - if you expect no legitimate mail from India, Austria, or Colombia, then blocking from those countries may work *for you*.
I am currently blocking all traffic from Viet Nam - I have nothing against the Vietnamese, having been to the country three times privately. But I receive a huge amount of connections from mostly Vietnam Telecom subscriber lines - I presume it is a botnet. I had barely an option but to block the whole country's IP ranges.
I still monitor the situation, and once the flood ebbs, I'll release the block. This is important to avoid too many false positives. Also, in my blocking message I point at a web form where legitimate senders can ask to be unblocked; I think I get one or two requests like that per year.

@Bon: i see this post is from much earlier, but not much info on the topic. bon, what did you finally come up with for blocking TLD's with multiple subdomains that worked with the school systems (i noticed you were from mississippi as well.)

@Bon: In this case you WHITELIST *.mash.potato and also BLACKLIST *.potato

Whitelist always overrides blacklist

True, spam on the Internet is becoming more and more

I think you can just use regex to block all emails from a particular tld except certain ones.

Example:
Block all .io
Except whova.io
^(?=.*(\.io$))(?!.+[@.]whova\.io).*