SURBL uribl.com blacklisting almost everything RSS Back to forum
@Matt Reed:
Do you use your own, local DNS server for ORF queries or a public one (e.g., ISP's DNS server, OpenDNS, see http://blog.vamsoft.com/2009/03/31/opendns-uribl/)? Though in that case, the returned IP would be 127.0.0.255 afaik...
@Krisztian Fekete (Vamsoft):
Also see this thread:
http://www.vamsoft.com/forum/topic/show/MULTI-SURBL-SURBL-issues/10
If you use the DNS server of your ISP, it might be incorrectly configured to return an IP address to ORF (which is considered a hit) even if the queried service (uribl.com in this case) returns NXDOMAIN (non-existent domain), which indicates the queried URL "techweb.com" is not listed.
To solve this, use a local DNS server for the ORF lookups.
DNS servers should/must meet the following requirements:
1. They must support recursion
Recursion means the DNS server returns the query result in a single step instead of redirecting ORF to the root DNS servers. This feature in enabled by default in Microsoft® DNS servers.
2. They should be on the local network or on the ORF computer
Using ISP DNS servers and third-party DNS resolution services (such as OpenDNS or Google Public DNS) is discouraged: you have no control of the configuration of such third-party party DNS servers, and these are usually banned by free DNS and URL Blacklist lookup services (i.e., they will not reply to the queries or return false data).
3. They should not use forwarders (e.g., ISP DNS servers)
Forwarders usually work with a large cache, which is usually no problem in simple name resolution, but will cause inaccurate (outdated) query results to be returned when checking online DNS and URL blacklists, causing degraded filtering performance.
4. They should not be the DNS server authoritative for your own domain (and supports your Active Directory)
Occasionally, ORF may need to query the records of your own domain (e.g., for the SPF test). The authoritative DNS server may not see your public MX or A/CNAME record if your internal AD domain name is the same as your public domain name (e.g., domain.com, instead of domain.local or domain.internal), resulting in false positives (see http://blog.vamsoft.com/2010/04/08/tales-from-tech-support-part-13-dns-issues-with-own-domain/).
@Krisztian Fekete (Vamsoft):
Kristian,
I have only one server in house, a MS sbs2003. It is running DNs, is pointing upstream to a public dns server. My LAN domain is a .local
I have had this same setup for over 2 years. Never a problem before.
Do I need to install another DNS server?
My Firewall routher seems to be acting strickly as a forwarder. If I try to point nslookup at the router and lookup an address, I get a timeout everytime. So I assume it can not handle the job of being the dns server.
My SURBL list is now only pointing towards "Combined SURBL list", and my Spam-Ratio for ORF is down to 64% from a historic value of at least 85%.
More spam is now getting in to all my users.
@Matt Reed:
So the router/firewall IP is configured as a forwarder in your DNS setup, if I understand correctly. I suggest removing the forwarder from your DNS configuration, so your local DNS server will forward its queries directly to the root servers (using root hints), instead of asking the router/firewall first.
Also, I recommend importing the definition for Spamhaus DBL and using that along "SURBL: Combined" (though using the root DNS servers should fix the queries sent to uribl.com as well):
http://www.vamsoft.com/spamhaus-dbl.asp
For the complete list of recommended DNS and URL blacklists, please consult our best practices guide at
http://www.vamsoft.com/downloads/getmostguide.pdf
The same problem here with URI-Bl. almost everything is denied on 127.0.0.1 when checking at the website itself the domain are not blacklisted. Had this setup for years and never had issues. I use googles dns forwarders (8.8.8.8) and (8.8.4.4). But i dont think that this is related to that.
@Henri:
I think they changed the response indicating that the DNS server is blocked from 127.0.0.255 to 127.0.0.1 to be compatible with SpamAssassin, at least some parts of their website makes me suspect that (see "Blocked Query Testing" in the "Abuse" section on their "About" page at http://www.uribl.com/about.shtml)
When I look techweb.com up (techweb.com.uribl.com) using our local DNS server, I get NXDOMAIN back. When I use the DNS servers of Google (8.8.8.8), I get 127.0.0.1 back, so I still believe uribl.com is blocking public DNS servers because of excessive queries.
By default, uribl.com is configured in ORF to drop the email if it receives any IP address back to its query. Until we find out why uribl.com returns 127.0.0.1, I suggest changing the configuration as follows:
1. Start the Administration Tool
2. Navigate to the Configuration / Filtering - On Arrival / URL Blacklists page using the navigation tree on the left
3. Double-click "uribl.com Blacklist"
4. Click the "Lookup results" tab, uncheck the "Blacklist if DNS record exists (regardless record data)" option
5. Click New
6. Add 127.0.0.2, click OK
7. Save your configuration to apply the changes by pressing Ctrl + S
This will ensure uribl.com will block the email only if it returns a response explicitly indicating a hit (127.0.0.2).
I had the same problem with uribl.com returning 127.0.0.1. Following Krisztian's config changes fixed the problem for now.
Check http://uribl.com/. I bet you are using some high frequented DNS server like the google-dns. (8.8.8.8)
I think they were blocked because they generated more than 300k requests per day.
@Krisztian Fekete (Vamsoft):
When modifying the uribl.com Blacklist configuration,
which boxes should be checked on the "Lookup" tab?
Currently, only the Reverse IP address for lookup is checked.
Matt
Same problem here, it would randomly block legit domains. I have come to learn their service is terrible. This has been an ongoing issue for some time where they randomly erroneously flag domains.
When not using forwarders everything will work ok. But not using forwarders is not always preferable. But the best is to still change the uribl setting to 127.0.0.2, so when you later on place forwarders or you reach the maximum limit you wont have false positives.
@Henri: We discourage using forwarders, because they work with a large cache (ORF works with outdated data --> degraded filtering performance), and using public DNS servers as forwarders may cause problems like this (i.e., DNS/URL blacklist providers block them because of the large number of queries they receive from them). If you insist using forwarders, we recommend setting up conditional forwarding, so DNS/URL blacklist lookups are sent to the root servers directly, while other lookups go to the forwarders first.
can you explain how to set this up. I used conditional forwarders only for specific domains, how to do this only from one source? The DNS server for the workstations and servers are all on the same machine. ORF is also installed on that machine.
@Tom: actually, they do not randomly blacklisting domains, they just returning a 127.0.0.1 response to public DNS servers to tell them "we do not accept your queries, go away". Unfortunately, by default ORF will think this response indicates a hit, while it does not. Their only fault is changing this behavior without prior notice.
@Henri:
I never tested this myself, but I think you should simply find the authoritative server for uribl.com, and setup a conditional forwarder to forward the query to this authoritative server when resolving anything ending with uribl.com (instead of going through the forwarder first).
We will eventually write a complete tutorial for setting up a Microsoft DNS server for ORF, we will add this as well.
Update: uribl.com added the following news:
January 23rd, 2012: Blocked due to excessive queries?
If you are receiving a bounce message saying your email was blocked due to excessive queries, you should contact your email provider, as they have not correctly implemented URIBL lookups. In the event a high volume nameserver is blocked, a 127.0.0.1 response may be received to indicate the nameserver is sending high volume queries. Service providers who have implemented URIBL lookups outside of SpamAssassin should read http://www.uribl.com/about.shtml#implementation and correctly implement URIBL lookups. Those effected should also read http://www.uribl.com/about.shtml#abuse for more information. The limits in effect are by nameservers, not individual mailservers, as the DNS requests will be coming from your resolvers.
@Krisztian Fekete (Vamsoft): Can you attempt to follow this advice and create a conditional forwarder to uribl.com? I know how to do it but I can't seem to find any servers for the uribl.com domain that work, they all timeout.
Their name servers are:
O.ICUDP.NET 76.191.255.82
K.ICUDP.NET 64.29.157.250
H.ICUDP.NET 207.150.196.144
P.ICUDP.NET 94.228.131.217
I attempted a domain query using the first server via nslookup, I received the following list:
- b.uribl.com
216.49.92.249
216.49.92.250
216.143.70.134
216.143.70.135
62.189.112.181
91.208.173.193
black.uribl.com
- c.uribl.com
64.73.0.55
81.17.240.208
128.214.205.82
128.255.17.26
204.152.149.12
216.37.76.65
62.58.50.220
black.uribl.com
- d.uribl.com
black.uribl.com
- e.uribl.com
black.uribl.com
- f.uribl.com
black.uribl.com
- g.uribl.com
black.uribl.com
- a.uribl.com
81.91.242.10
195.141.89.161
196.14.182.21
212.25.179.127
64.73.128.23
72.29.90.182
black.uribl.com
Then I queried a domain name from a spam sample (as "domainname.com.black.uribl.com") using 216.49.92.249 and I got 127.0.0.2 back indicating a hit. I also tested a legit domain, the server responded with NXDOMAIN, so I guess that works, I did not test the rest.
I can do that via nslookup as well. What I can't do is create conditional forwarders in DNS for URIBL that actually work right. Those name servers never respond to requests coming from my DNS servers when I set them up in conditional forwarders, when I look at the firewall logs I see the connection to those servers being closed after 90 seconds. The DNS server is simply falling back to 4.2.2.1 and 4.2.2.2 forwarders I have setup for the DNS server when they don't reply.
I question the assertion in your blog post in the section about the Quick Fix:
"This will ensure uribl.com will block the email only if it returns a response explicitly indicating a hit (127.0.0.2). We will also change this in the default definition file shipped with future ORF versions."
My understanding is that the Quick Fix setting just tells ORF not to respond to anything other than a 127.0.0.2, it does not change what uribl is sending back.
So, if we just put in that change and nothing else are we just wasting bandwidth and time?
Are we still getting a response of 127.0.0.1 on every inquiry to uribl?
The reason I ask is that since I have made that change I have gotten back no hits from this list.
Matt R.
@Matt Reed:
Yes, the quick fix will only ensure no legitimate email will be blocked in ORF due to the change uribl.com introduced. To eliminate the problem, you should not use public DNS servers as forwarders when querying their servers, or disable uribl.com entirely so you will not waste any bandwidth (if you wish to continue using public forwarders).
Actually, using "Spamhaus DBL" and "SURBL: Combined" alone should be sufficient for SURBL testing, so disabling uribl.com should not have a significant impact on your spam catch rate.
@Krisztian Fekete (Vamsoft):
@Krisztian - You say "Their only fault is changing this behavior without prior notice."
But you realize the implementation guidelines (which have been in place for years) specified URIBL BLACK as returning bit 2 (ie 127.0.0.2). No where did it say any noerror value should be considered a positive result. So it was your fault for not following implementation guidelines. Dont try to shift the blame here.
@russel lug:
Indeed, their implementation guidelines does not say that any noerror value returned indicates a hit, I'm just saying they did not return 127.0.0.1 in the past to servers performing excessive queries, and now they do, and I still not see this properly documented on their site. There is only an announcement on their front page from January 23rd which I mentioned above.
This is what I meant by "changing this without prior notice".
By default, uribl.com is configured in ORF to drop the email if it receives any IP address back to its query. Until we find out why uribl.com returns 127.0.0.1, I suggest changing the configuration as follows:
1. Start the Administration Tool
2. Navigate to the Configuration / Filtering - On Arrival / URL Blacklists page using the navigation tree on the left
3. Double-click "uribl.com Blacklist"
4. Click the "Lookup results" tab, uncheck the "Blacklist if DNS record exists (regardless record data)" option
5. Click New
6. Add 127.0.0.2, click OK
7. Save your configuration to apply the changes by pressing Ctrl + S
This will ensure uribl.com will block the email only if it returns a response explicitly indicating a hit (127.0.0.2).
I have the same problem as SURBL uribl.com is blocking most emails. I'have seen the steps above to fix this, I am using SBS 5.0, can anyone give me the above sequence that works in V5 please?
@Sean:
The easiest way to solve this in ORF 5 is updating your current SURBL definitions (the one linked in our KB article has correct SMTP response configured):
http://vamsoft.com/support/docs/knowledge-base/update-dnsbl-surbl
Can I use this update for ORF 4.4? Will this fix this problem?
Best regards
@paweł.rozkoszny: Yes, see http://vamsoft.com/support/docs/knowledge-base/update-dnsbl-surbl-oldversion
I'm using SBS 2008 and our DNS server uses Forwarders to OpenDNS. Yesterday I tried setting up a Conditional Forwarder for the uribl.com domain and so far it seems to be working.
Under Microsoft DNS, Conditional Forwarders, I added "uribl.com" and then specified the following authoritive servers (obtained from http://www.dnsstuff.com/tools#dnsReport|type=domain&&value=uribl.com):
o.icudp.net.
k.icudp.net.
p.icudp.net.
So far every email we've received has been checked with uribl.com and we're still using OpenDNS.
@Matt Reed:
This is your solution:
http://blog.vamsoft.com/2012/01/24/ub-black-uribl-com-url-blacklist-started-to-block-everything/
Update your lists using this information: http://vamsoft.com/support/docs/knowledge-base/update-dnsbl-surbl-oldversion
Sample:
Blacklisted by the UB-BLACK SURBL (domain: "techweb.com", DNS lookup result: 127.0.0.1).
Is there something going on with their service or is this a configuration error on my end?
Matt