how to block this type of spam RSS Back to forum
@Jeff Zell: the domain in the email body is now listed in uribl.com, Spamhaus DBL, and SURBL: Combined - I guess it was not when you received the email and ORF checked the SURBls via DNS, or the lookup failed. Unfortunately, these spam have to hit the honeypot/spamtrap addresses of these blacklist provider in order to get listed (so they are are always one step behind), but you can set up your own honeypot to deal with these: http://vamsoft.com/r?hporf
@Krisztián Fekete (Vamsoft):
That's consistent with what I see in the logs. The first few attempts will pass, then they'll be blacklisted by the SURBLs. For whatever reason, they hit my email first, then it gets blacklisted, go figure.
How about increasing the greylisting timeout to 60 or 120 or more seconds? The theory being the longer it takes for redelivery, the more likely the URL will be listed in the SURBLs.
@Jeff Zell:
That will not have any effect: as most MTAs will re-attempt delivery when encountering a temporary failure 15 minutes later first by default. So if these emails are Greylisted first, they are likely to get delivered 15 minutes later after anyway.
Instead, I recommend Honeypot, it is excellent for this scenario.
The honeypot is a lot of work setup and propagate, plus creates additional load on my server. I'll keep it in mind though.
Bumping up the greylisting timeout seems to be more effective than not -- no spam since I increased the timeout and unchecked the Skip Greylisting if SPF true option. The additional time for redelivery allowed enough time for URL to be listed in the SURBLs.
What's implied when an email passes through on the first delivery attempt? Another way of asking, what tests did it pass and what tests were not tried because it met certain criteria? Much of the spam that made it through passed through unchallenged, but neither whitelisted or authenticated.
@Jeff Zell: When an email passes tests, it means it passed all applicable tests (that is, all of those enabled). The thing with blacklists is that they are often working from spam they receive and thus have with little or no predictive power. It all boils down how fast a spam outbreak hits the spamtraps of a blacklist and how fast the blacklist acts on that. With massive botnets, a spammer is capable of distributing millions of emails in a very short window of time, so a spammer properly setting up new domains for each outbreak can be pretty successful against SURBLs.
This guy is signing up with new hosting companies every day and snowshoeing thru IPs, using new domains in links so RBL's, SURBL's don't work. Honeypot is not likely to help since he already has his list of recipients and I don't see any attempt of a dictionary attack. A simple Regex is not going to work as his content is dynamic.
I created this Powershell script that currently works, use at your own risk, hope this helps:
#
# Run-CustomSpamTests.ps1 runs custom Custom spam tests, to be used as an ORF External agent
#
# Usage:
# .\Run-CustomSpamTests.ps1 {filepath}
#
# Results:
# Exit code 0 - no problem
# 1 - spam
#
# 20141002 adjusted for hexcode surrounded by base10 numbers in messageID
# 20141001 original
#
Param( $filePath = "" )
# exit if no file specified
if ($filePath.length -eq 0) {
Write-Output "No EML file specified"
Exit(0)
}
$file = Get-ChildItem $filePath
# get content (whole EML), header and body
$content = [IO.File]::ReadAllText($filePath)
if (-not($content -match "(?ms)(.+?)\r?\n\r?\n(.+)")) {
Write-Output "Could not split header from body"
Exit(0)
}
$header = $matches[1]
$body = $matches[2]
# get MessageID
$messageID = ""
if ($header -match "(?ms)(^Message\-ID\:.+?>)") {
$messageID = $matches[1]
}
#
# Jumper Oct 2014 - this spammer quickly jumped from hoster to hoster, evading RBL lists so pattern recognition was needed.
# The patterns:
# a) 32 digit hex code from the Message-ID: header line that was repeated in the body
# b) giberish header fields repeating the hex code
#
$hexCode = ""
if ($messageID -match "[\-<>.@\d]([0-9a-fA-F]{32})[\-<>.@\d]") {
$hexCode = $matches[1]
if ($body -match "(?ms) $hexCode ") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
if ($header -match "(?ms)^[A-Za-z]{3,5}\-[A-Za-z]{3,5}\: $hexCode") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
# hexcode in Message-ID may be surrounded by base10 numbers. We'll need to pick it up from the body first
# and then test again.
if ($body -match "(?ms) ([0-9a-fA-F]{32}) ") {
$hexCode = $matches[1]
if ($messageID -match $hexCode) {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
if ($header -match "(?ms)^[A-Za-z]{3,5}\-[A-Za-z]{3,5}\: $hexCode") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
}
}
#
# Default return: no spam found
#
Write-Output "No spam found"
Exit(0)
You're welcome. Don't know if anyone is using this script but I have an update below (just the one section is changed). I won't post about this again unless someone requests it.
#
# Jumper Oct 2014 - this spammer quickly jumped from hoster to hoster, evading RBL lists so pattern recognition was needed.
# The patterns:
# a) 32 digit hex code from the Message-ID: header line that was repeated in the body
# b) giberish header fields repeating the hex code
# find 32 char hex code in messageID, check for that in body and other header fields
$longHexString = ""
if ($messageID -match "([0-9a-fA-F]{32,})") {
$longHexString = $matches[1]
for ($i=0; $i -le($longHexString.length - 32); $i++) {
$hex32Char = $longHexString.SubString($i, 32)
if ($body -match "(?ms) $hex32Char ") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
# find all header lines matching 2..5 chars, -, 2..5 chars
$myMatches = [regex]::Matches($header,"(?sm)(^[A-Za-z]{2,5}\-[A-Za-z]{2,5}\: [^\r\n]*?$hex32Char.*?\r?\n)")
foreach ($m in $myMatches) {
if ($m.Value -match "Reply-To\:") { }
elseif ($m.Value -match "http\:") { }
else {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
}
}
}
+1 for using PowerShell for an External Agent and the well documented code.
@Sam Russo: Let me contribute this code snippet to your project http://pastebin.com/G5KA12nJ. This is a PowerShell function that loads an EML file into a CDO.Message object (http://msdn.microsoft.com/en-us/library/ms526453(v=exchg.10).aspx). The value it adds is that you can access email MIME in decoded form, so you can easily scope your regexs and you can work with regular .NET Unicode strings, regardless the email charset and encoding (i.g. BASE64).
@Péter Karsai (Vamsoft): Thanks Péter for the MIME decoding function. I will definitely make use of that!
@Sam Russo: You're welcome. PowerShell can come handy around ORF in a number of ways -- for instance, I'm just running a script that parses a yearly archive of ORF PowerLogs to get sender FQDNs. It can also be used to access ORF real-time statistics or to author agents, as your example demonstrates. We should probably blog about these, I guess with more administrative tasks being PowerShell-first or PowerShell-only, our users are getting increasingly familiar with PS.
Hi. Looks like a great addition to the filters. Will you post up how the external agent should be defined in ORF? I've tried it a bunch of ways and all I get are errors that bounce everything. Thanks.
@Jeff Zell:
Hi Jeff,
The Agent Executable should point to your PowerShell binary (typically \Windows\System32\WindowsPowerShell\v1.0\powershell.exe) and Parameters should be something like this:
-NoLogo -ExecutionPolicy Bypass -File c:\orf\Run-CustomSpamTests.ps1 {EMAILFILESPEC}
Disclaimer: I have trouble accessing our test VMs right now, so I could not test if this works in an agent definition.
Thanks for the assist; I was close, but couldn't quite get it. It appears to be testing email without blacklisting everything, we'll see how it goes.
And thanks again for the code Sam!
Hi Jeff,
Not sure where you are starting from and whether or not you are comfortable with Powershell and Regular Expressions. Here are some basic tips for working as an ORF external agent:
1) We run Windows Server 64bit. ORF likes to launch external agents as 32bit. In order for scripts to run you must manually set the execution policy for scripts.
a) open the 32 bit Powershell window "Windows Powershell (x86)"
b) Set-ExecutionPolicy RemoteSigned
I would recommend the same for the 64 bit Powershell too.
2) Try something easy first such as a script that always returns code 0, but perhaps makes a copy of the EML file passed to it. In this way you can see if you script is running without affecting email.
3) A sample XML definition for an external agent is shown below. This sample assumes a directory is created in advance: c:\orfscan - where your script lives
(I don't know how to format things in this forum so apologies in advance for that).
4) This spammer you wrote about has been very active and changes his pattern frequently. I've adjusted my script along the way but it is still a cat-and-mouse game. I don't currently know of any other way to catch it, other than tagging IP ranges for sketchy hosting firms.
==============================================
Here is a sample XML of the External Agent:
<?xml version="1.0" encoding="utf-8"?>
<AgentDefinitions><Agents><Enabled>Yes</Enabled><Name>Run-SpamTests.ps1</Name><Description>A custom powershell script for deep spam tests.#CR##LF##CR##LF#svr 20141001#CR##LF#</Description><AgentFileSpec>C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe</AgentFileSpec><Parameters>c:\orfscan\Run-SpamTests.ps1 {EMAILFILESPEC}</Parameters><MaxWaitTimeSecs>30</MaxWaitTimeSecs><LogStdoutOnHit>Yes</LogStdoutOnHit><LogStdoutOnError>Yes</LogStdoutOnError><LogEmailfileName>No</LogEmailfileName><RoleIsAntiVirus>No</RoleIsAntiVirus><DefaultAction><Type>TagHeader</Type><DropResponseCode>550</DropResponseCode><DropResponseText>5.7.1 Message rejected.</DropResponseText><SubjectTag>[SPAM]</SubjectTag><SubjectTagOnRight>No</SubjectTagOnRight><HeaderTag>X-Text-Classification: spam</HeaderTag><RedirectionTarget></RedirectionTarget></DefaultAction><ExitCode><Enabled>Yes</Enabled><CodeRanges>1</CodeRanges><Comment></Comment><Category>Normal</Category><ExecuteAction>Yes</ExecuteAction><UseDefaultAction>Yes</UseDefaultAction><Action><Type>TagHeader</Type><DropResponseCode>550</DropResponseCode><DropResponseText>5.7.1 Message rejected.</DropResponseText><SubjectTag>[SPAM]</SubjectTag><SubjectTagOnRight>No</SubjectTagOnRight><HeaderTag>X-Text-Classification: spam</HeaderTag><RedirectionTarget></RedirectionTarget></Action></ExitCode><ExitCode><Enabled>Yes</Enabled><CodeRanges>255</CodeRanges><Comment>Error</Comment><Category>Error</Category><ExecuteAction>No</ExecuteAction><UseDefaultAction>No</UseDefaultAction><Action><Type>TagHeader</Type><DropResponseCode>550</DropResponseCode><DropResponseText>5.7.1 Message rejected.</DropResponseText><SubjectTag>[SPAM]</SubjectTag><SubjectTagOnRight>No</SubjectTagOnRight><HeaderTag>X-Text-Classification: spam</HeaderTag><RedirectionTarget></RedirectionTarget></Action></ExitCode></Agents></AgentDefinitions>
========================================================================
Here is my current script:
#
# Run-SpamTests.ps1 runs custom spam tests, to be used as an ORF External agent
#
# Usage:
# .\Run-SpamTests.ps1 {filepath}
#
# Results:
# Exit code 0 - no problem
# 1 - spam
#
# svr 20141014 adjusted for header fields without dashes
# svr 20141007 adjusted for header fields 2..5 - 2..5 instead of 3..5
# svr 20141006 pattern adjustment
# svr 20141002 adjusted for hexcode surrounded by base10 numbers in messageID
# svr 20141001
#
Param( [string]$filePath = "" )
# exit if no file specified
if ($filePath.length -eq 0) {
Write-Output "No EML file specified"
Exit(0)
}
$file = Get-ChildItem $filePath
# get content (whole EML), header and body
$content = [IO.File]::ReadAllText($filePath)
if (-not($content -match "(?ms)(.+?)\r?\n\r?\n(.+)")) {
Write-Output "Could not split header from body"
Exit(0)
}
$header = $matches[1]
$body = $matches[2]
# get MessageID
$messageID = ""
if ($header -match "(?ms)(^Message\-ID\:.+?>)") {
$messageID = $matches[1]
}
#
# Jumper Oct 2014 - this spammer quickly jumped from hoster to hoster, evading RBL lists so pattern recognition was needed.
# The patterns:
# a) 32 digit hex code from the Message-ID: header line that was repeated in the body
# b) giberish header fields repeating the hex code
#
# find 32 char hex code in messageID, check for that in body and other header fields
$longHexString = ""
if ($messageID -match "([0-9a-fA-F]{32,})") {
$longHexString = $matches[1]
for ($i=0; $i -le($longHexString.length - 32); $i++) {
$hex32Char = $longHexString.SubString($i, 32)
if ($body -match "(?ms) $hex32Char ") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
if ($body -match "(?ms)\-$hex32Char\-") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
if ($body -match "(?ms)\=$hex32Char\=") {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
# find all header lines matching 2..5 chars, -, 2..5 chars
$myMatches = [regex]::Matches($header,"(?sm)(^[A-Za-z]{2,5}\-[A-Za-z]{2,5}\: [^\r\n]*?$hex32Char.*?\r?\n)")
foreach ($m in $myMatches) {
if ($m.Value -match "Reply-To\:") { }
elseif ($m.Value -match "http\:") { }
else {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
}
# find all header lines matching 3..10 chars: [base10-]hex32Char[-base10]
$myMatches = [regex]::Matches($header,"(?sm)(^[A-Za-z]{3,10}\: [^\r\n]*?(\d*[\-\.\= ])*$hex32Char([\-\.\= ]\d*)*.*?\r?\n)")
foreach ($m in $myMatches) {
if ($m.Value -match "References:") { }
elseif ($m.Value -match "http\:") { }
elseif ($m.Value -match "ENVID\:") { }
else {
Write-Output ("Jumper 20141001 " + $file.Name)
Exit(1)
}
}
}
}
#
# Default return: no spam found
#
Write-Output "No spam found"
Exit(0)
@Sam Russo:
Right on time, he's at it again this morning. I should add that I've also been adding keyword subject and sender rules in ORF to help. I guess its about multiple lines of defense. It's currently catching with this email subject regex in ORF (substitute your company domain as appropriate):
Use at your own risk. Hope this helps.
Keyword, Subject, Regex:
.*(anti\-aging|auto(finder|warranty)|belly(burner|fat)|card\s*(egift|reward|special)|Dr\..*Oz\b|(\bharp|medicare).*(answ|enroll|ins|regis)|miracle cure|\@yourdomainhere\.com)
Hi Sam,
I'm not fluent with PowerShell or RegEx at all. I can figure out how the scripts work and modify existing code to fit my needs, but that's about it. Full-time IT was my career years ago and I've got a few clients remaining that I support, so my goal is usually to fix things and not get stuck in the deep end.
The XML for the external agent won't import, ORF spits out "Access violation at address 004A15A7 in module 'orfeeadm.exe'. I'm assuming I have to format the text somehow to make it work, however my external agent works already, so I'll probably take the lazy way out.
Yeah, I'm doing what I can, including blacklisting those sketchy hosting companies. I call it the whack-a-mole game, if you recall those days.
Thanks again for the code.
Jeff
The XML tags were cleaned (removed) during upload. You could create your own. Here's the fields as I've set them:
========================
Description tab:
Ext. Agent Name: Run-SpamTests.ps1
========================
Run tab:
agent executable: C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe
cmd line: c:\orfscan\Run-SpamTests.ps1 {EMAILFILESPEC}
timeout: 30 secs
========================
Role tab: spam filter
========================
Logging tab:
Enable both Catch checkboxes...
========================
Exit Codes tab:
Current Action: Tag email header with "X-Text-Classification: spam"
Exit code 1: Hit (default)
Exit code 255: Error (no action)
========================
That should do it.
Good luck, you're welcome and yes, I do remember whack-a-mole ;)
Hi. I'm getting inundated with this type of spam. I don't know exactly what it's called; the format is generally the same, but content varies between emails. The sender IP changes, as does the domain and unsubscribe link. I've got the best practices recommendations implemented and I'm using every DNSBL and SURBL in the list (or close to it). Looking for any potential solutions besides blacklisting everyone and whitelisting what I want.
I do get some SERVFAIL errors, which I've followed the instructions to stop, but it doesn't seem to affect the filtering. Otherwise, the logs are fairly clean. TIA.
----------------------------------------
Please do not waste anymore time.
---------------------------------------
Search No. 4848876
---------------------------------------
Right now you have the chance to find out which of your background records are visible online to prying eyes as we speak.
To find out immediately, please go here now: http://view.instantprofilescheckbetter.com
*You better hurry, because we can't promise this will be available for much longer.*
Your background records generation time: 55.33 seconds
Be removed here: http://leavehere.instantprofilescheckbetter.com
P.O. Box 749, 4040 Quisque Avenue Hastings 9334 New Zealand
e30db6543ea3d835815ab860d0b482fa
48488761215636
e30db6543ea3d835815ab860d0b482fa
48488761215636
4848876
4848876