ORF 101 Guide

This guide describes the basics of email delivery in general and the way ORF integrates in your setup. Highly recommended for new ORF users.

Email Delivery in a Nutshell

The great thing about ORF is its ability to catch spam during email transmission. Alas, this also means we have to get a bit intimate with the email transmission process to get a solid understanding of ORF.

Let's see how this works by a simple example.

Email delivery from [email protected] to [email protected]

Email delivery scheme

The SMTP protocol

Internet email transmission uses a protocol called SMTP.

This works in a command-response model: the sending party issues a command and the receiving party either confirms the command or rejects it.

The SMTP conversation transcripts below demonstrate how a successful and a failed email transmission looks like:

SMTP conversation transcripts for a successful and a failed delivery. The difference is highlighted with blue and red colors. SMTP command responses always begin with a three-digit number (e.g. 250), which determines the status of the command. Response codes beginning with 1, 2 and 3 indicate success. Response codes starting with 4 or 5 indicate transient or permanent failure.

There are two things worth noting here:

• The receiving party can choose to reject a command. This cancels the transmission process (see the line marked with red) and will trigger a bounce report (it is the responsibility of the sending party to generate the report).
• The sender and recipient addresses are specified twice, first in the MAIL FROM: and the RCPT TO: commands (called the envelope addresses), then in the email header (called the MIME addresses). While email servers rely on the envelope addresses for delivery and ORF will also use the envelope addresses for its email-address based tests, email clients like Outlook display the MIME addresses. Spammers tend to use different envelope and MIME addresses to confuse the recipients (ever had users complaining about spam from themselves?), so keep this in mind when investigating email delivery and spam issues.

Now that we have a solid background on email delivery basics, let's see how ORF comes into the picture.

ORF Essentials

Your email server and ORF

The actual SMTP transmission involves the sender and the recipient servers only: ORF does not perform any relaying or rejects any emails by itself, it only communicates with your underlying server and tells it to halt the SMTP transport (i.e., reject the email), or allow it to continue (based on its test results).

Imagine ORF like advisor of your Microsoft® Exchange / SMTP server, telling what to respond to the sender server during the SMTP conversation. This method of integration has the benefit that you do not have to open any additional ports or configure any addition relaying after installing ORF, it can start the "counselling" basically right away.

Filtering points

The phases of the SMTP transport when ORF tells your server what to do are called filtering points. There are two of them: the Before Arrival filtering point and the On Arrival filtering point. These occur when the sending party has just issued a command and waits for your server to respond.

• Before Arrival filtering happens after the sender server tells which recipients it wants to send emails to (RCPT TO command). At this point, the email contents are not transmitted yet, but a great deal is already known about the email (such as the sender IP address). This is usually enough clue for ORF to decide about the email. Unwanted emails can be rejected by refusing to accept the recipient, which in turn causes the whole delivery process to stop.
• On Arrival occurs when the sending party has finished transmitting the email and waits for acknowledgement (end of DATA command). At this point, the email contents are available. This enables ORF to use its full filtering arsenal and also enables you to keep a copy of unwanted emails.

Tests

ORF goes through an evaluation process for each email to decide the email status. The evaluation has three possible outcomes:

• The email gets blacklisted: the email turns out to be unwanted. ORF acts accordingly, e.g., rejects the email.
• The email gets whitelisted: the email is excluded from filtering and it is allowed through ORF.
• The email passes the tests: the email gets neither blacklisted, nor whitelisted and it is allowed through ORF.

Testing process and its possible outcomes

ORF makes the decision using a series of tests. There are several tests in ORF and they all fall into two categories:

• Whitelists tests decide if the email is excluded from filtering by the administrator. Any hit on a whitelist test will cause the email to be automatically allowed through ORF.
• Blacklists tests decide if the email is unwanted. Any hit on a blacklist test will cause the email to get blacklisted.

Tests are carried out in a specific order and once the email status is determined (i.e. blacklisted or whitelisted), the testing stops. If the testing completes without a hit, the email receives the "passed" status and it is allowed through ORF.

Whitelist tests are generally performed before blacklist tests. This way they can stop the email testing before any blacklists would get the chance.

ORF tests are assigned to filtering points. Most tests can work at either or both of the filtering points. The email status is evaluated independently at filtering points – an email that passes Before Arrival might get blacklisted On Arrival.

Actions

ORF performs an action of your choice on blacklisted emails. These actions can be:

• Email Rejection (available at Before Arrival and On Arrival)
• Email Subject Tagging (available On Arrival only)
• Email Header Tagging (available On Arrival only)
• Email Redirection to another address (available On Arrival only)

Email Rejection is the most frequently used (and default) action in ORF. As the rejection happens during the email transmission, your server is spared from processing and storing the junk email. However, this also means you cannot recover a rejected email. Rejection also triggers a bounce report, so any rejected legitimate sender is very likely to be notified of the failed delivery (remember, bounce reports are generated by the sending party – ORF cannot make guarantees about the bounce report).

Email Tagging allows placing a customizable tag text on the email subject and/or inserting a custom header field in the email header. These tags then can be used to move tagged emails into the Junk Mail folder of end-users. As senders are not notified about the blacklisting, the end-users may have to be educated to periodically review the Junk Mail folder contents.

Email Redirection allows routing all junk emails into catch-all mailbox where the administrator can review and recover emails if needed. (Be sure to check the privacy laws of your country to see if this is compliant with your local regulations).

ORF components

Start discovering ORF by its management tools:

• ORF Administration Tool
  Allows reviewing the system status and configuring ORF settings such as whitelists and blacklists. This is where you start setting up ORF. More information...
• ORF Log Viewer
  ORF logs can be browsed and searched using the Log Viewer. If you think it is hard to get excited about logs, think again: ORF has the industry's best logs with a similarly great tool for working with them. Whether you are troubleshooting an issue or just trying to find out what happened to an email and why, turn to the Log Viewer: it has the answer for you. More information...
• ORF Reporting Tool
  This tool is for generating informative and beautiful reports about the activity of ORF. Give it a few days after deployment so that it can pick up enough information to be visualized. More information...

The rest of the ORF components you should know about:

• ORF Service
  The heart of ORF, which hosts the filtering logic, does the logging, etc. When this service is down, ORF is not filtering emails. More information...
• ORF Transport Agents and the ORF SMTP Module
  These components connect the ORF Service with Microsoft® Exchange or the Microsoft® IIS SMTP Service.

What's next?

It is time to start discovering ORF. We recommend starting with the ORF Administration Tool. If you are hungry for more information, check out the documentation available below.

If you are before installation

• Deployment Guide — Prepare for ORF deployment. No surprises.
• Installation Guide — Help with installing ORF.

If you are after installation

• Best Practices — Get the most out of ORF.