White Paper
Message Systems Evolution of Email Technologies
Abstract
"Email is the foundation for communication on the Internet. All business rely on this communication tool. Several billion emails transit the Internet in a single day, not including corporate intranets. This volume has been consistently increasing over the years but the technology was not keeping pace."
- Theo Schlossnagle
Chief Technology Officer
Message System
Message Systems represents a monumental step forward after a twenty-year lull in the evolution of email technologies. Message Systems is an "intelligent" email gateway designed to deliver millions of messages per hour, coupling standards-based Internet messaging with innovative design.
Background
In 1982, the mechanism for transferring mail from one organization to another was standardized. However, the Message Transfer Agents (MTA's) that were created to provide that mechanism did not evolve much over the next twenty years.
A number of factors have developed that require an evolution of email systems:
- Email systems are more aggressive now with throughput measured in millions of messages per hour instead of thousands per hour.
- The sprawl of the Internet has introduced millions of new identities. Large mail systems must be able to support tens of thousands of concurrent connections.
- The "spam/virus" and "phishing" threats are a serious concern for all parties on the Internet. Battling these attacks drain valuable resources, diminish productivity, and threaten security.
The era of using old and antiquated tools to perform the demanding tasks of today is over. Message Systems' innovative design and clean-room implementation break the mold of yesterday's systems and yield a truly "enterprise" messaging system which addresses the challenges of today and the internet growth of the future.
There are several popular MTA's commonly used today, both open-source and proprietary. The most popular are Sendmail, Exim, Postfix and Qmail; however, these software products lack the enterprise features needed in today's messaging systems - most notably scalability and performance.
For many years, companies have deployed such messaging solutions and simply accepted that a single machine was not capable of delivering in excess of 50,000 or 100,000 messages per hour. In most cases, the machine is more than capable, but the software is not and that has been due to poor architectural design. Message Systems makes light work of tasks that are difficult or impossible with other tools. Designed from the ground up with flexibility, scalability and manageability as core requirements, with Message Systems organizations now have the tool to address the evolution of email.
Technology
Message Systems is a fully standards-compliant Message Transfer Agent (MTA). It is designed to accept Internet email messages and deliver them to their final destination (called a mail exchange) in a timely manner. As a software solution, it does not require proprietary hardware or a proprietary operating system to run, thus reducing cost of deployment.
Message Systems uses a tried-and-true event driven scheduler to maximize systems resource utilization. Working inside the framework of any of today's operating systems, Message Systems leverages best-of-breed mechanisms for performing network I/O (talking to remote mail exchanges) and journaling messages to disk (for data integrity and assured delivery).
Integrated DNS Cache
Message Systems incorporates a full caching name resolution subsystem to reduce the quantity of DNS queries required to deliver mail. This means fewer resources are spent on resolving mail destinations and these resources can be reallocated to validation or delivery.
Best-of-Breed Scheduler
Message Systems uses a hybrid thread/event model to drive its operations. By utilizing the best event scheduling system available on each supported platform, Message Systems is capable of managing tens of thousands of concurrent incoming and outgoing sessions. By using a hybridized threaded model as well, the system can utilize multiple system processors to parallelize processor and disk intensive tasks such as message validation (spam/virus detection and policy enforcement) and message journaling (for assured delivery) that would impede the performance of a pure event system.
Often mail systems are compared using performance numbers acquired from a lab test environment. While these test are valuable and provide a controlled setting for operation (fair comparison), they do accurately represent how the system will behave when subjected to the stresses of real-world use. Message Systems' core engineering team has designed real-world mail systems that are responsible for sending in excess of 1 trillion messages per year. This architecture allows Message Systems to deliver millions of messages per hour on commodity hardware - in the real world, not a test lab!
MultiVIPT Virtual Interface Support:
Message Systems applies a variant on the standard networking technique called NAT (network address translation) to allow a single Message Systems instance to present itself as multiple MTA's. Messages that enter the system can be 'tagged' to come from a specific IP address (or pool of addresses) when they are delivered to the appropriate mail exchange. Message Systems can also support thousands of virtual interfaces and messages can be bound to any virtual interface based on their sender, recipient or any aspect of their content.
In the ISP setting, this feature allows for the delivery of each client's email from different IP addresses so that IP-based filters that affect one client will not affect other clients. This prevents any blocking issues incurred by one client from adversely affecting clients using the same services. Each MultiVIP interface appears as a unique MTA, and can have its own hostname and resource limits.
Flexible and Extendible Modules
It is quick and easy to extend Message Systems to meet your exact needs by writing modules that can be added and removed without changing the core program. Modules can be written to:
- Write logs in a custom format or directly to a database.
- Store messages in an archive for auditing purposes.
- Validate messages on arrival for spam/virus detection or policy management.
- Modify messages any way you see fit (e.g., adding disclaimers).
Message Systems' unique modular extension architectures provide virtually unlimited integration options, allowing all of the information inside Message Systems to be leveraged by your existing environment.
Painless Management
Managing a high-volume mailing system poses several challenges. First and foremost, with slower technologies many physical machines are required to handle the workload. The Message Systems performance advantage often eliminates the need for dozens of existing mail servers and thus greatly reduces systems maintenance requirements thus reducing cost.
One of the key challenges in building a system that allows for quick and painless online management was having all information available in a single location. Message Systems design provides a central information store that is used both to drive the system and to provide information to operators in a timely efficient manner.
Another feature, the advanced clustering techniques make the management of 100 instances as easy as managing a single install.
Detailed Reporting
Understanding your deliverability is vital. Message Systems console allows you to see global data, like reception and delivery rates, as well as specific information about messages, domains and connections. You can inspect live TCP connections, DNS records, queues, individual email messages, and a plethora of other statistics. The web-based console presents up-to-the-minute graphs of delivery, reception and failure rates with flexible and convenient access to historical data as well. No longer do you need to search for information it is delivered to you via the Message Systems console.