*
Microsoft.com Home|Site Map
Microsoft TechNet*
Search Microsoft.com for:
|TechNet Home
Commercial Internet System
Community
Downloads
Internet Explorer
Internet Information Server 3.0
Internet Information Server 4.0
Interoperability and Migration
IT Solutions
IT Tasks
Microsoft Mail
MS-DOS
Office
Personal Web Server
Project 98
Proxy Server
Security
SNA Server
Systems Management Server
Transaction Server
Visio
Windows 95
Windows 98
Windows 2000 Server
Windows for Workgroups
Windows ME
Windows NT Embedded
Windows NT Server 4.0
Windows NT Terminal Server
Windows NT Workstation

Microsoft's Migration to Microsoft Exchange Server - The Evolution of Messaging within Microsoft Corporation

*
On This Page
Executive SummaryExecutive Summary
Microsoft Corporate EnvironmentMicrosoft Corporate Environment
Microsoft Corporate Messaging EnvironmentMicrosoft Corporate Messaging Environment
Preparation for Microsoft ExchangePreparation for Microsoft Exchange
Migration ConsiderationsMigration Considerations
Migrating to Microsoft Exchange 4.0Migrating to Microsoft Exchange 4.0
Upgrading to Microsoft Exchange 5.0Upgrading to Microsoft Exchange 5.0
Upgrading to Microsoft Exchange 5.5Upgrading to Microsoft Exchange 5.5
Planning for Exchange "Platinum"Planning for Exchange "Platinum"
ConclusionConclusion
Appendix A: Microsoft Exchange BasicsAppendix A: Microsoft Exchange Basics
For More InformationFor More Information

Executive Summary

Businesses today rely on their messaging systems more than ever before. And the nature of e-mail communications has been rapidly changing. From early text-only messages to current combinations of rich text, embedded objects, intelligent forms, file storage and document management, increasing demands are being made upon messaging systems. Businesses need a comprehensive messaging platform that includes the tools necessary to create rich collaboration applications.

Since its release, Microsoft Exchange has been enthusiastically received and has taken its place as a cornerstone of the Microsoft BackOffice family of products.

Customers have been intrigued by the complexity and scope of Microsoft's simultaneous development and migration to Exchange. Microsoft realized that, in order to remain competitive, it needed a solid messaging foundation that was powerful, flexible and could be migrated into their existing environment. But how is such a Herculean task accomplished?

This technical case study examines the evolution of Microsoft's legacy XENIX-based messaging system into to the present Microsoft Exchange environment. Although providing the most direct benefit to IT professionals deploying Microsoft Exchange, it will also be of interest to anyone deploying an enterprise-wide client/server technology, especially messaging. This case study is not intended to be a project model, project planning tool or training aid for implementing Exchange.1Microsoft Official Curriculum courses are offered at numerous Authorized Technical Education Centers. Training and certification course schedules are available at http://www.microsoft.com/train_cert. The plans, decisions and actions in this case study are being presented to foster new ideas, generate further discussion and help identify potential planning issues when considering an enterprise level messaging system deployment.

Microsoft's migration from its XENIX operating system-based mail to the Exchange client/server-messaging environment was driven by two primary goals: to be the first, definitive production environment for Exchange and to help shape, support and make full use of the evolving nature of electronic communication in all its endeavors.

The bulk of research and planning for the migration from the legacy XENIX system began in 1993 and was completed just before Microsoft Exchange 4.0 shipped in April 1996. Exchange 5.0 and 5.5 upgrades were immensely simplified by lessons learned during the preceding migration.

Although Microsoft's Information Technology Group (ITG) began migration with a substantially homogenous environment, it faced many issues familiar to most medium to large enterprises:

Developing administrative models and processes

Managing migration complexities inherent in legacy systems

Estimating end-user training requirements

Determining hardware specification criteria

Assessing capacity planning and management

Planning for the upgrade to Exchange 5.0 began within months of Exchange 4.0's implementation. At this point ITG had a functional Exchange system in place and had a fairly good idea what to expect in upgrading the Exchange servers to Exchange 5.0. At this time the Internet was fast gaining popularity and had established itself as a major communication vehicle. To take advantage of the rapidly growing interest in Internet-based messaging, Microsoft added support for several native Internet protocols to allow enhanced Internet connectivity.

The upgrade from Exchange 5.0 to Exchange 5.5 replaced the Exchange client with the Outlook messaging and collaboration client. Servers running Exchange 5.5 could support many more users than earlier versions of Exchange because of the expanded information store size limit. This meant fewer servers and hence, reduced server maintenance and management.

The next version of Exchange, code-named "Platinum," is currently in development and ITG is working on developing the tools necessary for an efficient migration to the Windows NT Operating System 5.0 Active Directory. As "Platinum" development matures beta versions and release candidates will become available, allowing Microsoft to forge ahead with representative aggressiveness, moving employees to "Platinum" before it ships.

The implementation of Exchange at Microsoft was an ambitious undertaking in one of the most complex messaging environments for deploying a messaging system. While other Exchange designs and implementations may differ substantially, the comprehensive scope of Microsoft's deployment of Exchange reveals fundamental design and planning elements that are sure to be of interest to all messaging professionals.

Microsoft Corporate Environment

Microsoft believes in being the first "production environment" for all company products. This invaluable test platform builds a solid user experience base.

An organization's corporate "culture" dictates the blueprint of its migrations. Microsoft's deployment of Microsoft Exchange has been both praised and questioned. Therefore, a prerequisite to this case study is understanding the distinctive corporate environment at Microsoft specifically during the initial Microsoft Exchange migration.

Factors affecting the migration and upgrades were:

Implementation as corporate initiative unconditional backing from the Office of the President meant that the only decision left to the departments was scheduling when they would move to the new platform. This mandate allowed Microsoft's Information Technology Group (ITG) to make swift strategic and budgetary decisions. Given Microsoft's crucial reliance on e-mail for day-to-day operation including legacy applications and business systems tied directly into the messaging system research and planning decisions were unilaterally supported by all groups.

Homogenous messaging environment ITG has always engaged in research and design initiatives for Microsoft's infrastructure at the enterprise, rather than departmental level. This was a significant asset. ITG faced simpler system research and planning issues than a company approaching migration with multiple, disparate messaging systems, since Microsoft possessed a primarily single, homogenous messaging environment.

Ample resource designation corporate initiative and executive management support assured necessary resources would be available for migration. In fact, more than 100 project team members (far above comparable industry standards) participated worldwide at the peak of the migration schedule.

Technically focused user base Microsoft employees have a relatively sophisticated technical understanding of computers and software. Any Microsoft employee is free to select and install whatever Microsoft products are necessary for their job. This policy produces end users with an "above average" understanding of software installation and operation. These experienced employees gave Microsoft a huge advantage over most other companies.

End users willing to test new software Microsoft employees are familiar with encountering some level of functionality and/or productivity loss during the introduction of beta products and are often called upon to install beta software products on their computers to help find and fix bugs before release.

Microsoft Corporate Messaging Environment

Microsoft's messaging environment was as important a factor as its corporate environment in setting the course and execution of Microsoft Exchange deployment initiatives.

When Microsoft was a new, relatively small company, the Programming Language developers used Digital 2020's, the Finance Department used a VAX and the remainder of the company, including application developers, relied on Microsoft XENIX, a personal computer version of UNIX.

Microsoft's team-oriented projects, varied work hours and aggressive production schedules favor electronic messaging as the most efficient and therefore primary means of communication. By the time voice messaging was available as a communication option (1990), e-mail was firmly established and remains the preferred mode of in-house communication to this day. (Employees are commonly referred to by their e-mail aliases and it's not unusual to e-mail someone working in the next office.)

The original e-mail system was limited to an eight-character alias and, although Exchange has eliminated that restriction, there are still several associated internal applications that are designed around the eight-character limitation.

From early character-based messages to the current combination of rich text, embedded objects and intelligent forms, Microsoft's messaging system evolved rapidly to accommodate the need for high-performance, scalable and robust communication architecture. Concurrently, the nature of e-mail communications, throughout the industry, has changed to include file storage and document management, placing even greater demands on the messaging environment and necessitating a high-performance messaging platform.

Microsoft knew, while developing this client-server messaging system, that its own internal operations provided an ideal "test platform" environment to prove the effectiveness of an enterprise-ready messaging solution. As Exchange developed, it became necessary to implement it internally.

Exchange Product Group Messaging Environment

"Eating your own dog food" is a significant part of Microsoft's product development process. It refers to the practice of implementing and using your own products before they are considered ready for release.

With Exchange, an operational environment code named "Dogfood" was established for development and testing purposes. The Dogfood organization originally supported a total of 10 users, but has since grown to more than 1700 supported on 18 servers. This bipartisan arrangement gives both the Exchange development team and ITG a chance to evaluate Exchange product builds before creating a corporate-wide environment.

Product functionality and support concerns grew along with the number of Dogfood users. Despite consciously operating in a test environment, the possibility of losing essential, work-related messages or suffering decreased functionality is nonetheless frustrating. Product Group managers saw this as an added stimulus to ensure the Exchange group's attention to detail. It was a stern, but reasoned, stimulus.

To accommodate both testing and end-user needs, the Dogfood environment has several servers that do not host user mailboxes. These specially configured servers host the product builds on a more frequent basis to test stability and functionality.

As the Dogfood organization increased in size and functionality, it also assumed greater responsibilities. The first crucial goal of establishing message connectivity between the Dogfood and Exchange production organizations was achieved by using Internet Mail Connectors (IMC) to act as Simple Mail Transport Protocol (SMTP) gateways between the two groups.

The next step was to extend the SMTP gateway functionality to the Internet. Dogfood had its own Internet Mail Service (IMS) connectivity to the Internet and its own domain, "exchange.microsoft.com". ITG reworked the corporate SMTP routing so messages and replies could be directed to both organizations through the Internet. In addition, Dogfood and ITG users were set up as custom recipients in the respective directories so that each environment could accept messages from one organization and pass them on to the other.

Figure 1: Custom Recipient Scenario between the ITG Exchange and Dogfood Exchange systems.

Figure 1: Custom Recipient Scenario between the ITG Exchange and Dogfood Exchange systems.
See full-sized image.

To explain the custom recipient a bit further, every mailbox in the Dogfood environment had a corresponding custom recipient item in the ITG Exchange Directory. Likewise, every mailbox in the ITG Exchange environment had a corresponding custom recipient item in the Dogfood Exchange Directory. This strategy produced important advantages that:

Allowed the Dogfood Exchange organization to act as a separate entity the Microsoft Exchange Organization (MEO)

This rigorous environment served ITG's desire to test builds of Exchange, prior to introducing them into the corporate servers.

Eliminated Microsoft e-mail address confusion

An "outside" e-mail user may have been confused when receiving e-mail from user@exchange.microsoft.com, assuming that the domain name for Microsoft (traditionally @microsoft.com) had been changed. ITG's strategy eliminated this confusion by allowing user@microsoft.com or user@exchange.microsoft.com as a valid e-mail address for delivery to a specific Microsoft employee.

Provided an excellent working example of Exchange

Companies could observe Exchange managing communications with subsidiaries or interacting with vendor / partner companies while maintaining autonomy between the two.

The third step in Dogfood functionality was synchronizing Address Books between the two organizations. The same process that updated the custom recipients for SMTP routing was used successfully for the Address Book Synchronization. The product group solved the challenge of sharing free/busy-calendaring information by creating a tool that synchronizes free/busy objects between two Exchange Public Folder organizations.

The constant and effective cross-communication required to run two organizations within one company helped the Dogfood team understand their environment's impact on production and helped the production team anticipate changes.

Preparation for Microsoft Exchange

Most of the research and planning for the evolution of Microsoft's migration and subsequent upgrades of Microsoft Exchange took place during the initial migration to Exchange from the legacy XENIX system. This research began in 1993 and was completed just before Microsoft Exchange shipped in April 1996. Exchange version 5.0 and version 5.5 upgrades were aided immensely by lessons learned during the preceding migration. This section discusses the issues that ITG faced before it undertook this large and complex project.

Rollout Plans

While research continued on test releases of Exchange and evaluations of message routing on XENIX, ITG reviewed the system specifications, ran initial modeling on the first test releases and provided feedback to the Exchange product group.

Because ITG was working with a non-released code base, the rollout plans had to be adapted to the maturing functionality of the product. The number of end-users hosted on Exchange who used it as their primary electronic messaging platform grew with the increased functionality and stability of newer product revisions. Product maturity also had an effect on the topology and Site Connector strategy because some decisions were based on when the move from X.400 connectors to Exchange Site Connectors would occur. This affected the Public Folder design and system infrastructure.

ITG recognized that coordinating server upgrades and client installations would become especially challenging since the Windows NT version 3.51 and Windows 95 operating systems were still under development. Most early adopter beta users were members of Windows NT 3.51 and Windows 95 development teams who often had to install and use a particular build of the Windows software they were developing. Running a beta product on a beta operating system was especially challenging.

Pacing development to maintain product compatibility often meant halting development of one product when another was a step behind and needed to "catch up."

ITG needed to choose a hardware platform for global introduction of the product internally before release. Since the logistics of procuring, deploying and installing new hardware in more than 90 locations worldwide were formidable and time-consuming, the choice of an appropriate hardware platform became a crucial decision very early in the development phase. Product specifications often change during the development cycle, so ITG understood clearly that it needed to move forward, based on product specifications many months in the future.

Phased Rollouts

The first phase was designed to confirm the ability to install local and remote servers, move users to the new environment, manage the Exchange servers, synchronize directories between the legacy XENIX and new systems and route messages between both environments.

Additional goals included: hosting 1000 users on ITG-administered Microsoft Exchange servers, hosting another 1000 users in the Dogfood environment and routing 800 Windows 95-based clients that had been hosted on Microsoft Mail post-offices through the Microsoft Mail connector to ITG's Exchange servers.

As a baseline, ITG committed to maintaining operational levels of Exchange consistent with the existing messaging environment. At that time, the message environment managed:

650,000 to 750,000 messages daily

10 percent to 15 percent of all messages delivered to/from Internet

Average user received 30-35 messages per day

Average message size was 7.2 KB

Average daily volume of about 4.5 GB

The second phase was designed to stress the communication capabilities of Exchange and validate the domain, site and administrative models. This phase was not originally targeted to support a maximum number of users, but instead focused on finding areas with communication challenges and extending the system to them. This included connectivity to Microsoft subsidiaries with slow speed network links, asynchronous links and multi-national client interaction (localized clients).

After extensive research and analysis, ITG expanded the scope of this phase and extended the objectives from 2,500 users at 10 locations to 7,500 users at 35 locations and introduced expansion beyond one messaging site.

The third phase concluded the global product implementation by replacing the existing messaging backbone. ITG planned to complete the entire company conversion within 90 days of the final Exchange release candidate's availability. With the encouragement and unconditional support of Executive Management, the entire company completed its worldwide conversion on the same day Microsoft Exchange was released to manufacturing.

Project Team Initiatives

The considerable logistical challenges faced in achieving this aggressive rollout plan required ITG to communicate objectives, timeframes and schedules to numerous groups on a global scale. Six key groups participated on a daily basis in the migration from the legacy transport to Exchange:

ITG Messaging Research

The ITG Messaging Research team (messaging architect, research lead and six system analysts) analyzed product specifications, tested administrative and site strategies, and devised an overall implementation strategy for approximately two and a half years. This same group, furnished with a release product version, would have only needed an estimated four to six months to fulfill their responsibilities.

ITG Exchange Messaging Operations

The 14 member Messaging Operations team provided 7x24 server administration and troubleshooting for the production servers during the first and second phases. Messaging Operations also generated case studies that guided the design of ITG's related training courses. As the project progressed, these team members returned to the operational team. This provided first-hand knowledge of Exchange administration procedures including then-new Message Transfer Agent (MTA) services, directory services, connector services and Public Folder management.

ITG Operations Engineering (Messaging)

The ITG Operations Engineering Messaging team (5 members) was created during the latter half of the second phase to provide analytical input and strategic planning to the Messaging Operations team. This released Messaging Operations to focus solely on operations and Messaging Research to focus on delivering the final strategy, while working together to create policies and daily administrative procedures.

Regional ITG

Regional ITG (beyond Microsoft's Redmond campus) provided necessary support for the messaging systems in their location/region, in addition to providing other non-messaging ITG-support duties throughout the U.S. and around the world. These individuals contributed heavily to the success of the migration, due in part to their ability to multitask. In Redmond, formal groups assumed specific areas of migration responsibility, but Regional ITG participants were called upon, and able, to assume project manager, administrator and analyst roles as needed.

Product Support Services (PSS)

PSS and Messaging Research maintained effective communications for the Exchange product group, which reinforced ITG's surrogatecustomer status, comparable to an early adopter company migrating to Microsoft Exchange.2

During the first phase, PSS members were added to assist ITG by tracking bugs and evaluating software test initiatives crucial elements of preparing a support structure and training other PSS engineers.

End-user Client Support

This specialized Help Desk related team provided communication between ITG and the end-user community. They also organized "lunch hour" training sessions for corporate campus end-users during the initial worldwide Exchange client introduction. Server training was designed for both Regional IT and Redmond-based administrators. End-user Client Support (ECS) helped the rollout process by lessening end-user confusion, which reduced the impact on the corporate Help desk.

Migration Considerations

Although ITG began their migration with a substantially homogenous environment, it faced many familiar enterprise issues:

Developing administrative models and processes

Managing migration complexities inherent in legacy systems

Estimating end-user training requirements

Determining hardware specification criteria

Assessing capacity planning and management issues

Gateway Management

Managing gateways was also necessary to provide better support for the IMC. To ensure a consistent set of user addresses and distribution lists (DL) were available to all internal e-mail users, ITG designed an in-house MAPI-based (Messaging Application Programming Interface) tool capable of synchronizing XENIX and Microsoft Exchange directories daily.

Message Transport

The Microsoft Exchange Messaging Transfer Agent (MTA) introduced many configuration options that needed review to optimize the system for the Microsoft messaging environment. ITG's initial concern focused on legacy coexistence, specifically because inter-system communications would depend on the native transport and new gateways necessary to bridge the Exchange system and the legacy XENIX system.

After resolving legacy communications issues, ITG focused on optimizing message routing. Analysis showed that during initial migration, X.400 site connectors offered an advantage in non-spoke-and-hub topologies due to their greater control over route selection costing. Determining which combination of connectors to use and the placement of gateways attached to external service providers with the necessary proxy addresses to support them, was significantly influenced by the administrative model used in Microsoft's case, central, as opposed to distributed and/or shared.

After evaluating message routing hubs and Directory Service bridgehead server (the first server in line after a gateway) usage, inter-site communications were off-loaded to dedicated servers to lower replication times. This increased the performance of the store servers and dedicated messaging servers.

Public Folders

In additional to adding primary e-mail functionality, ITG also used and evaluated Microsoft Schedule+ and Public Folders; and initiated review and analysis of Exchange Server sample applications but, at this early stage of product development, there were only a few available.

Tools and practical experience on the Exchange system were limited, so ITG monitored and maintained the environment until a standard operating procedure could be established. ITG decided to make Public Folders available to users to determine their needs, desires and the resulting administrative adjustments. Although it was assumed that administrator feedback would be reactive rather than proactive, this experience would prove to be beneficial for planning Public Folder use.

Another Public Folder challenge was creating an intuitive directory navigation system for the folder hierarchy. Exchange version 4.0's directory service maintained a flat namespace, making it necessary for ITG to restrict the inclusion of Public Folders due to the likelihood of lower level folders inadvertently sharing the same name as folders higher in the hierarchy. Users posting or sending e-mail to such a folder would find multiple folders listed with the same name in the address book without indicating a parent folder. This would have forced users to select and review the properties page to identify the correct folder.

ITG faced challenges of evaluating and creating procedures for replicating information; establishing dedicated Public Folder servers; and defining and managing folder quotas.

The E-Forms Designer (EFD), an Exchange utility capable of generating basic electronic forms applications and using Visual Basic development system to extend the application if required, presented issues for the administrating research team. These were:

Should users be able to independently register and/or replicate forms?

What business practices should be altered with use of this technology?

What procedures might be used to test, validate and accept end-user generated solutions for inclusion in the enterprise set of tools?

The Public Folder component of Microsoft Exchange (Groupware) provided so many new offerings that ITG established a dedicated evaluation team to focus on possible usage, administrative and client issues.

Directory

Directory population and maintenance presented a challenge for ITG. At Microsoft, the only directory attribute "owned/managed" by ITG is the login ID. All other attributes title, manager, office location, phone number are managed by Human Resources and the Registrar with separate databases and user interfaces. ITG consolidates this information and feeds it to Exchange via database API's.

Administration

The administrative model dictates many design decisions. For ITG, designing the migration strategy plan raised the following questions:

Who should oversee service level agreements with end user groups?

What store quotas are enforced?

What are the conditions that warrant a restore?

How frequently are backups performed?

When is off-line compacting and integrity validation of the store performed?

All these issues were considered carefully when designing the Exchange migration plan.

Topology

When planning for Exchange, it was vitally important for the Messaging Research team to analyze and thoroughly understand the enterprise network and systems management structure. The Messaging Research team identified company office locations, the type of connectivity that existed between these offices and where IT administration resources existed.

The Messaging Research team concluded that it was vitally important for the domain strategy to be robust and capable of supporting the Microsoft Exchange site architecture. These two elements, understanding the enterprise network and designing a strong domain strategy, were designed early and concurrently because, once configured, it would be difficult to make changes to either without notable effort and possible service interruptions.

Also, when determining the Windows NT domain and Exchange site strategies, the Messaging Research team considered the underlying network, bandwidth, protocols, network costs and traffic patterns. These factors helped decide where to set site boundaries and when to use Site Connectors rather than X.400 connectors.

The Microsoft Corporate Windows NT Domain

In Microsoft's Windows NT domain strategy, all user accounts are located in multiple "Master User Domains" (MUDs) nearest the served user community. They have trust relationships between each other to allow for local administrative control privileges.

One level below the MUDs were one or more "Business Unit Domains" (BUDs), resource domains where SQL Server-based servers, print servers and file servers are located. The BUDs trust only one of the first tier MUDs its regional parent domain providing optimized access authorization.

Figure 2: Exchange Interaction within the Corporate Windows NT Domain Structure

Figure 2: Exchange Interaction within the Corporate Windows NT Domain Structure
See full-sized image.

For the Exchange implementation, corporate Windows NT Domain practices were changed to define a special, global domain in the second tier. All Microsoft Exchange servers would be located here to ease identification of messaging servers, to provide for consistently applied "global groups" and to simplify server management. It was only the second tier domain that spanned geographical regions.

ITG designed the Exchange site topology to reflect network connectivity between regions and subsidiary sites within a region and the corresponding distribution of support responsibilities at Microsoft. In addition, ITG attempted to minimize the number of sites while providing bandwidth consumption management over dedicated links and regional support of the system.

Message Site Rules

These criteria eased administration of the corporate Exchange system. They provide a reasonable model for a highly centralized organization like Microsoft, but may pose challenges for distributed environments with cellular management.

Fewer Message Sites are better.

Message Site must adhere to the centralized administration model.

Message Sites should manage routing and replication where network bandwidth dictates.

Naming Conventions

With Exchange 4.0, there was no process to change an enterprise name, domain name, site name, or server name without rebuilding the servers. ITG realized that it was substantially easier to "split" rather than "merge" an existing messaging site after its creation, so administration of the enterprise messaging system during these early implementations favored a topology with fewer rather than more messaging sites.

User login ID's were also carefully considered by ITG. ITG realized that identifying the limitations imposed by existing systems was the first step in creating adaptations to those limitations, e.g., determining if the messaging system is tied to human resources or other internal systems and planning accordingly. If either is true, then recognizing the limitations imposed by the existing systems is necessary and adaptations to the model created to reflect those limitations.

At Microsoft, the site and server names were designed to reflect generally static information that is unlikely to change over time. Avoiding use of departmental names led to use of regional names, which are not likely to require modification. Consistent application of names makes it easier for users to move about and for administrators to manage the system.

Hardware

It was necessary for ITG to determine the hardware requirements and deploy the hardware globally at a very early stage to complete the migration to Exchange on schedule.

The Exchange implementation was based on leveraging the server information store. It also took advantage of the single-object/multi-pointer message store and allowed for definition of a universal Inbox.

Microsoft's heavy e-mail usage prompted ITG to configure servers with high-end systems. This decision ensured a viable platform for growth and a high level of support from local vendors. Three standard configurations were adopted to ease troubleshooting and guarantee a high level of operational capability.

SIZESYSTEMMEMORYHARD DRIVEOTHER

Large

2 Pentium

128 MB

16 GB

RAID 5,
24 GB array

Medium

2 Pentium

64 MB

8 GB

RAID 5,
10 GB array

Small

Pentium

32 MB

4 GB

RAID 5,
6 GB array

There were additional challenges obtaining technical support for both delivery and maintenance of hardware in foreign countries where regional customs and tax laws impede the distribution of high-end technology. If a deployment plan calls for upgrading hardware, the procurement and placement of this equipment is a concurrent sub-project within the overall deployment of Microsoft Exchange.

Further Considerations

ITG invested in excess of 10 work-years in developing tools for their messaging environment. The endeavor was successful due only to an early start in research and analysis.

Other implementations may face:

Integration or extensions into existing administrative methods

Tools that may be proprietary or entrenched in legacy applications or systems

Extending the system with workflow forms, such as form-based DL management

Migrating to Microsoft Exchange 4.0

The migration from XENIX to Exchange was driven by the need to meet future messaging requirements while using Microsoft's flagship messaging product. This section examines the Exchange 4.0 implementation at Microsoft.

Pre-implementation

Microsoft's migration from XENIX mail to the Exchange client/server-messaging environment was the beginning of a bold journey a journey that would introduce the company's largest infrastructure project to date. But in a greater sense, Microsoft moved from simply using an increasingly out-moded messaging technology to conceiving and creating a vastly powerful, yet adaptable new messaging environment. This meant every Microsoft desktop around the world would be affected, so ITG chose planning and research initiatives that ensured uninterrupted interaction between this new system, existing line-of-business applications and the legacy email system.

The Network

Initially, the Microsoft network was a flat Ethernet global broadcast network where NetBios Enhanced User Interface (NetBEUI), Xerox Networking Service (XNS) and Internetwork Packet Exchange / Sequenced Packet Exchange (IPX/SPX) were the primary network protocols. Although some of these protocols are routable, ITG had not implemented routing services but would be doing so using Transmission Control Protocol / Internet Protocol (TCP/IP) in conjunction with the Exchange deployment. Eventually all Microsoft servers and clients became TCP/IP hosts.

Microsoft had a sizable WAN in place and extensive analysis showed that sufficient bandwidth existed to support Exchange without circuit upgrades.

Figure 3: WAN connectivity prior to Exchange Implementation and regional bandwidth capabilities.

Figure 3: WAN connectivity prior to Exchange Implementation and regional bandwidth capabilities.
See full-sized image.

This was a topology with three primary hubs. Redmond, the largest, served North America and all locations not served by the European and Far East hubs. Bandwidth ranged from speeds exceeding T1 to a mere 64K in some Latin America locations.

Microsoft Mail and XENIX

Although Microsoft migrated to Exchange from the nearly homogenous legacy XENIX system, a small Microsoft Mail system existed as well, serving the Microsoft Mail product team.

The Microsoft Mail client provided the optimum Windows client for electronic messaging, prompting ITG to create an environment where the Microsoft Mail client could communicate with existing XENIX mail servers. In this hybrid system, XENIX was the backend server with a customized Microsoft Mail as the user interface. ITG wrote custom functionality into the Microsoft Mail client that allowed users access to their XENIX mailbox. The system consisted of the following components and processes:

Approximately 160 XENIX servers supported 60 to 250 users each.

The XENIX servers were 386/486 class computers with 16 MB of RAM and 300 to 600 MB of disk space.

Microsoft Mail clients downloaded users' messages from XENIX to a local Microsoft Mail-based Microsoft Mail File (.MMF) file.

Microsoft Mail clients downloaded the XENIX address book, which was stored on the client in a XENIX Address Book (XAB) file.

Remote users accessed their mailboxes via Remote Access Services (RAS) using the Microsoft Mail client or a telnet session with the XENIX server.

Execmail addresses were maintained on the XENIX servers in an alias file. Execmail was the primary message-handling daemon, which processed e-mail from the inbound spool to the user's mailbox.

Migration Preparations

ITG had to overcome several fundamental challenges before moving any user mailboxes to Exchange.

The first was the lack of a common communication protocol between XENIX mail and Exchange. XENIX servers communicated using XNS and there was no plan to support XNS on Windows 95 the platform for the Exchange client.

The solution, therefore, was to port a version of the XENIX Message Transport (XMT) that could run on Windows NT Server and communicate with XENIX using XNS and Execmail. XMT could also interact with Exchange (running on the same Windows NT computer) using TCP/IP and SMTP. Once XMT was implemented, Microsoft Mail Connectors provided connectivity with business partners and test/development Microsoft Mail post offices. This created a hardware migration path from XENIX to Exchange and, allowed XMT to "burn in" new hardware that would be used to run Exchange services on. This reduced hardware concerns while ITG became familiar with the operations of the new Exchange system.

Figure 4: XENIX / XMT and Microsoft Mail Connectivity

Figure 4: XENIX / XMT and Microsoft Mail Connectivity
See full-sized image.

XENIX systems moved Internet messages to Windows NT-based servers running XMT, which would route SMTP messages to UNIX hosts for routing to the Internet and to the newly migrated users through Exchange servers.

Migration Strategy

The migration strategy was to move users quickly from the legacy XENIX mail system to Exchange. Network protocol and Windows 95-based client considerations drove the need for this rapid deployment between XENIX mail and Exchange, as well as Microsoft's desire to have Exchange fully deployed internally before releasing the product to manufacturing.

This strategy entailed moving users logically in large groups from XENIX to XMT initially and then to Exchange ultimately. To do this, users in each group were sent three e-mail notices of the pending change:

Notification Plan for Migration from XENIX to XMT

Seven days prior to the migration from XENIX to XMT a notice was sent to each effected user announcing the pending migration and providing information about what the change would entail.

The night of the server migration a final notice was sent to each user. This was the last message delivered to their XENIX host. All new mail was now routed to their XMT host. The final message provided instructions for changing the HOST association from the XENIX server to the XMT server. The only thing required of the user was editing the name of the HOST in their MS Mail configuration file.

Notification Plan for Migration from XMT to Exchange

Seven days prior to mailbox migration, a first notice was sent to users announcing the pending migration to Microsoft Exchange and providing tips for early preparation and what the change would entail.

Three days prior, a second notice was sent to users advising them to install the Exchange client and create a Personal Store File (.PST). It also included instructions on how to import the Microsoft Mail .MMF file into the Exchange client's .PST file.

The night of the mailbox migration, a final notice was sent to each user. Careful communications and synchronization between the Messaging Operations and the client support teams ensured that this final notice was the last e-mail that users would receive on their old messaging system. When users arrived at work the next morning, their active mailbox was now on the Microsoft Exchange system. The last message on the XMT system contained final instructions for moving to Exchange. Additional e-mail would no longer be delivered to the XMT mailbox and users would be unable to send e-mail. Any e-mail received after this last ITG delivered message could be found in the user's new mailbox on Exchange. By following the instructions of the last e-mail on XMT, users configured their Exchange client for access to the Exchange private store server and moving their remaining e-mail .MMF/.PST over to Exchange.

Using this plan, ITG established a very aggressive migration schedule of approximately 2,000 Redmond users per week. This pace was mandated by a rapidly approaching release date of Exchange (April `96) and Microsoft's goal to have all users on a single messaging platform. Even at this ambitious pace, the bulk of user migration took approximately five months due to delays caused by the time involved in resolving unforeseen issues, which often brought the migration process to halt. For example, server upgrades to a newer beta version of the product frequently slowed down the migration. With each new software upgrade, more servers were in place; requiring more upgrade time before the migration could begin again.

Remote sites had different migration schedules. In Europe, 300 to 400 users were migrated per night, over 8 weeks, in 35 countries. This aggressive migration strategy was necessary to move users as quickly as possible from XMT to Exchange, but caused some network bandwidth problems because of the volume of replication and message traffic generated by both XMT and Exchange hosting users in the same office/region. These new mailboxes, coupled with those being created in Redmond, produced many directory updates that in turn caused a large amount of Exchange replication. While most new mailbox recipients were replicated throughout the organization within an hour, replication of new mailboxes in some locations took as long as two days. ITG was forced to halt user migrations to allow the system to synchronize the directory.

Mailbox Management

From the outset, successful e-mail account creation required collaboration between ITG and the Human Resources department. At that time, Windows NT and Exchange systems interacted with the Human Resource database system through an automated synchronization process that generated approximately 5,000 directory changes per day.

Today, when mailbox accounts are created, each new mailbox is created with three storage limits. Exchange's rich configuration capabilities provide the administrator the ability to set these limits to adhere to the specific policies or implementation concerns of their company:

Warning At 45 MB, users receive a message that they are approaching their mailbox size limit. These messages appear at a regular interval in the mailbox, until the user reduces the size of their mailbox.

Prohibit Send At 50 MB, user can receive new mail, but is unable to send any messages while continuing to receive size limit notification messages.

Prohibit Send and Receive At 75 MB, users are unable to send or receive any messages while continuing to receive size limit notification messages.3

Although the overall functionality has increased significantly since the initial migration to Exchange, there has not been a significant increase in the size of an average mailbox. At the time of the Exchange rollout, the average size of a mailbox was 25 MB. With Exchange 5.5, the average size of a mailbox is 28 MB. Although users are allowed to exceed the average mailbox size (with the above limit restrictions), they are encouraged to use personal folders (.PST's) to store mail. Currently, heavy messaging users at Microsoft store approximately 50 pieces of mail per day.

Software Distribution

The only Exchange client available during initial phases of the Exchange 4.0 migration was a 32-bit client. This posed challenges for ITG because neither the Windows 95 nor Windows NT 3.51 operating systems had shipped. It was common to require end users to simultaneously upgrade their desktop operating system and their messaging client, every time Messaging Operations upgraded the Exchange servers. ITG client support teams faced enormous support issues since users were new to both Windows 95 and Exchange. Additional support staff was added on a temporary basis and end user training was made available to help alleviate this support issue.

In North America, users were willing and able to install their own Exchange client with the proper training, so installation of the Exchange client was left up to them. However, in certain parts of Europe, where a complete Microsoft Systems Management Server (SMS) implementation was in place, Systems Management Server was used to successfully deploy the client.

Training

In addition notifications and instructions sent to user mailboxes, organized lunch sessions a popular training method were held to familiarize users with the new features and functionality of the Exchange client.

ITG administrators were trained in Redmond and at the regional sites in five-day sessions more formal than the lunch sessions where administrators learned server administration features, functionality and how to perform day to day administrative tasks.

PSS created a 10-day Instructor course for PSS engineers, who then traveled to various PSS sites and trained other PSS engineers worldwide.

Exchange 4.0 Infrastructure Design

This section outlines the final infrastructure design and the reasoning behind particular architecture decisions.

ITG's goal was to replace their currently stable, heavily utilized messaging system, XENIX/XMT mail, with a product that had not yet supported users in a production environment. This design process was unique because ITG had no data to help anticipate how the product would perform in an enterprise the size of Microsoft.

While ITG was converting the XENIX messaging systems to the XMT platform, the Exchange product group was constructing an enterprise-class client/server application from the ground up. This was a first among an industry full of store-and-forward shared file system products

The Exchange Topology

At the time of the Microsoft Exchange 4.0 migration, many site locations had 64K connections to Redmond and much of that bandwidth was allocated for other use. Early plans for ITG's Exchange Site architecture was designed to accommodate as many as 36 sites, but after further research, ITG decided to reduce the number of Exchange sites to simplify management of the system. For ITG, fewer large sites were easier to manage than many small ones. However, attention remained on the issue that limited bandwidth between locations in an Exchange Site could possibly cause intra-site communication problems.

ITG's first Exchange implementation called for 15 sites. X.400 connectors were used during the initial phase of the implementation because X.400 was an established ITU standard. But the final rollout of Exchange 4.0 replaced the X.400 connectors with Site Connectors (still in development during the initial stages of the rollout) for greater efficiency and management capabilities.

The Information Store

ITG wanted to locate a dedicated mailbox server in each site. These servers contained only a private information store for that location's users, but not a public information store. The initial design specification of Exchange called for a public information store on every server. However, ITG pushed to change this design specification to reduce the overhead of the information store service on an Exchange server. Without a public information store, the service does not have to manage both the public and private information stores.

Only one location within each site contained a public information store to supply that site's users with the public information store hierarchy and provide access to their free and busy objects.

Protocols

TCP/IP was the network protocol routed between physical locations. The Site Connectors used Remote Procedure Calls (RPCs) routed by TCP/IP to connect the Exchange sites. To facilitate the migration to TCP/IP, Microsoft provided client connectivity to Exchange servers through only one network protocol, TCP/IP.

Connectors and Gateways

As explained previously, tests were conducted with the X.400 connector and, once available, the Site Connector was chosen to connect all the Exchange Sites. The Site Connector was configured for a bridgehead relationship with the connected site. ITG housed all of the bridgehead servers for each site in Redmond.

Message Transfer Agent (MTA) at the remote location opens an RPC association with the corresponding bridgehead server in Redmond and transfers the message to that MTA. The site's bridgehead server in Redmond then opens an RPC session with a server in the Redmond site and delivers the message.

The table below highlights the magnitude of the Microsoft Exchange 4.0 migration and implementation project by presenting the actual number of servers, physical locations and number of mailboxes per site.

Site NameServers Per SitePhysical Locations Per SiteApproximate Mailboxes Per Site

North America

118

31

29000+

Asia, India, Middle East

7

7

500+

Latin America

20

20

1000+

Far East

13

6

2600+

Europe

25

13

5000+

Central Europe

8

5

1700+

Nordic

4

1

550+

South Pacific

10

8

840+

Corp-Hub

4

Redmond

0

Message Routing Analysis

ITG's research analysis indicated that the planned topology was feasible because most messages were either sent between users within the same remote location or directly to Redmond. There was limited inter-site messaging traffic, except that which went to Redmond. If the existing messaging culture had included many users sending several messages a day across the organization, this topology may not have been practical.

In addition to Exchange Site Connectors, there were two external messaging connectors providing public network connectivity. One was an X.400 connector to MCI, which provided Microsoft users with access to the X.400 global network.

The other was to the Internet via several IMC servers in Redmond and Europe. All incoming SMTP mail came in through Redmond for worldwide delivery. Outbound SMTP messages took the lowest cost route to the Internet. The Europe IMC delivered Europe Internet messages and the Redmond IMC's delivered Internet messages from all other sites.

Public Folders

ITG's strategy for designing Exchange Public Folders began by observing users. To better understand how Microsoft employees might use this new feature, a semi-flat Public Folder hierarchy was created and allowed to grow with few restrictions. This showed ITG's research teams how end users would use and adapt to this new area of functionality in Exchange. To retain some control of this process, ITG and the user community acknowledged that ITG could purge and redesign the Public Folder system at any time. Given the large number of mailboxes and the clients' need for quick access to the Public Folder hierarchy, ITG developed strategies using two types of servers:

Hierarchy and Site Folders (Free/Busy and Offline Address Book): These contained only Public Folders pertaining to the free and busy Public Folder which supports scheduling and organizational forms library, allowing a maximum number of users per server. The hierarchy and Free/Busy Public Folder server did not manage any content replication or storage. These servers may also contain the Offline Address Book. In Redmond we have a number of public folder servers that are dedicated to hierarchy and free/busy. Some servers, outside of Redmond, the addition of the Offline Address Book.

Public Folder Content Servers: These provided Public Folder content to clients, but did not manage the 40,000+ users requesting the hierarchy or users' Free/Busy times.

Every user has an object in the Free/Busy Public Folder that contains the user's free and busy times. When a user launches the Exchange client, it immediately contacts the Public Folder server associated with the user's private information store. The decision to place users' Free/Busy times in the hierarchy Public Folder servers meant faster client start time and greater reliability.

Accessing Public Folders

Each mailbox server with a private information store was configured to direct its clients to a designated hierarchy Public Folder server. This enabled ITG to balance the number of users hitting each hierarchy Public Folder server and balancing the number of users' whose free and busy times were stored on each server.

For sites other than North America, at least one Public Folder server existed. This Public Folder server satisfied the hierarchical requests of users in that site and in some cases depending upon the number of users served hosted the content of Public Folders specific to that site. To give clients access to non-local Public Folder content, affinity was configured among all sites and North America, which allowed travel across the WAN. Here is how it worked:

1.

The Exchange client contacted the user's private information store to find the associated Public Folder server.

2.

The Exchange client contacted the user's public information store where it gets the Public Folder hierarchy.

3.

If the requisite content was not found in the site's public information store, the client searched all other servers on its site, then searched the Public Folder servers in the North American site to find the Public Folder content.

Unfortunately, this initial implementation of Public Folders caused problems for the smaller remote sites. Users accessing their free and busy Public Folder upon startup encountered poor performance due to the limited WAN bandwidth at that time. However, today this same placement of Public Folders helps to accommodate the tremendous growth of Microsoft's Public Folder system (77,000 Public Folders as of September, 1998.)

As the initial rapid expansion of Public Folders subsided, between 20 to 30 MB of Public Folder content was being replicated between content Public Folder servers each day.

System Administration

XENIX was in place as a mature product for several years, so it required only 12 ITG administrators, but, upon initial implementation of the new, unreleased Exchange messaging platform, this number grew to more than 40.

The support resources were based on three levels of escalation:

Tier 1Messaging Operations: This group monitored electronic links and servers, and conducted performance monitoring. Each physical location with an Exchange server maintained an IT (Helpdesk) technician, whose responsibilities included Windows NT support, Line-of-Business application support, and general PC desktop support. With the introduction of Exchange that support was extended to providing local emergency Exchange server maintenance and administration if a problem occurred. The technician provided hands-on access for Messaging Operations to resolve any problems.

Tier 2Messaging Platforms and Gateways: This temporary group supported Exchange, XNT, and gateways as well as escalating issues for Message Ops.

Tier 3 Product Support Services (PSS): PSS provided final escalation for server problem resolution. This tier eventually replaced Tier 2 after PSS became more familiar with the product and the support issues.

Every regional Exchange site such as the North America Site had their own support teams. Some geographical locations within a site such as Redmond, New York, Atlanta, San Francisco with either the appropriate human resources or more than two servers, also had support teams. The Exchange support teams for sites other than North America escalated issues to the North American Messaging Operations team and PSS when needed.

Exchange support at remote locations varied based on the number of people involved and physical location within the WAN. European Exchange Operations had its own Tier 1 group, a team of five people who supported Exchange in the three European sites as well as Africa, India and the Middle East.

The Exchange Client

Although the Exchange client shipped with Windows 95, a much more feature-rich client, specific to Exchange, was shipped with Microsoft Exchange Server. This Exchange-specific client added functionality including the Exchange Server Information Store and Address Book information service. Managing the rollout of the Exchange client component was difficult during the Exchange 4.0 rollout because during migration, many client versions required a specific build of Windows 95 or Windows NT and both systems were still under development during that time. Sometimes the Exchange client had to be uninstalled before updating to a new build of Windows 95 or Windows NT. The ITG client support team spent a lot of time managing these Exchange client issues.

Exchange Security

For the Exchange 4.0 migration, ITG relied on the Windows NT security domain and Remote Procedure Call (RPC) encryption of the existing Windows NT Domain infrastructure for its authentication and transport security. The Internet protocols and the methods to secure them were not implemented into Exchange until later versions.

Collaborative Workgroup Applications (Scheduling)

At the time of the Exchange 4.0 migration, the only collaborative workgroup application in use was Microsoft Schedule+ 7.0. Exchange was developed as a platform on which collaborative applications can be built, but these applications were not developed until Exchange development was complete.

Another one of the many reasons ITG performed such an aggressively timed migration was the lack of coexistence between the Exchange calendar and the Schedule+ 1.0 calendar used by XMT and Microsoft Mail users. Users of either system could not easily view the calendar of users on the other system. Again, due to Microsoft's heavy reliance upon electronic scheduling, this posed problems for user groups who regularly shared schedule information specifically users of a large group who were not moved to Exchange at the same time. Therefore, ITG attempted to migrate users in large, logical working groups (such as the Windows NT group) to minimize the impact of this inconvenience. While alleviating the problem somewhat, this lack of coexistence did affect user productivity during the move process. ITG aggressively migrated users as quickly as possible to minimize this period of "calendar blindness."

Unified messaging

Very early in the Exchange 4.0 migration planning, ITG included requirements for voice and fax objects in the private information store. ITG went so far as to run a pilot test of voice mail with Exchange, but performance and reliability concerns postponed the production implementation of this service. Fax services have since been successfully integrated in the Exchange production environment, and ITG continues to test ISV integrated voice mail products.

Upgrade Benefits

While complexities of the messaging architecture and the deployment of a new product required successful management by ITG, both users and administrators reaped tremendous benefits from this new, more manageable client/server messaging architecture:

A considerably feature-rich client

Decreased delivery times

Active monitoring of messaging traffic

Enhanced scheduling

A versatile development platform

Challenges and Solutions

ITG's design innovations while creating the new Exchange environment were guided by the need to meet conflicting implementation challenges on several levels, simultaneously. This concert of coordinated strategic solutions is reflected in the "organic evolution" of Exchange.

This section summarizes those challenges and solutions.

Windows 95 and XENIX/XNS coexistence: Many of the challenges ITG faced during migration resulted from an absence of support for the XNS protocol in Windows 95. To deal with this challenge, ITG developed the following process for migration:

Develop the XMT system on Windows NT Server, which allowed the use of TCP/IP.

Move XENIX mailboxes temporarily onto XMT system running on Windows NT.

Move XMT users to the new Exchange system.

Schedule+ coexistence

XENIX/Microsoft Mail Schedule+ 1.0 users needed to seamlessly coexist and interact with Schedule+ 7.0 users. To accommodate this, ITG developed an aggressive schedule that moved members of large (500+ members) logical groups.

Valuable feedback to the Exchange product group

ITG moved users to Exchange but also needed to provide feedback to the product group. However, the rapid deployment schedule did not leave much time to provide intricate, detailed feedback to the product group. To remedy this, ITG created additional teams on the messaging operations side to provide vital feedback to the product group. These specialized groups worked closely with teams representing PSS, client support, research and the product group. This allowed Messaging Operations to focus entirely on the daily tasks of moving users and maintaining servers.

Message volume throughput and bottlenecks between XENIX/Microsoft Mail and Exchange

A bottleneck developed in the XENIX/Microsoft Mail / Exchange throughput when there were nearly equal numbers of Microsoft employees on both sides of the connector. ITG, anticipating this message-handling backlog, accelerated the migration schedule to push past this mixed system equanimity quickly, by delaying Exchange server product revision upgrades that could stall user moves. Once there were significantly more Exchange users than XMT users, the servers were upgraded.

Message format translation between systems

The Exchange rich text format was lost when messages were transferred to the XENIX/Microsoft Mail system. There was no workaround for this issue, nor was it a showstopper for corporate migration. Therefore, the inconvenience was simply endured until all users were migrated to Exchange.

Using an unreleased code base

This was a unique aspect of Microsoft's Exchange implementation and a requirement of the Exchange product group. To remedy the challenge, the product group provided continuous developer support for ITG. A pager system was established for emergencies and the product group was on constant stand-by ready to send a developer anywhere in the world to fix critical e-mail stoppages.

Code version incompatibility during development

Another unique aspect of Microsoft's Exchange implementation was that, despite processes put in place to thoroughly test a new build before it was put into production, it was not always possible to test the interaction between the new build and the existing build. Often, ITG had to move forward with a new build after a successful run in the product group's system with only the option of calling on the product group developers, rather than consulting a manual or calling upon experience, should a problem arise.

Automated application migration

Microsoft had several business applications and processes that relied heavily upon the XENIX mail system. These systems needed alteration to function with Exchange. To accomplish this, the ITG research team set and achieved the goal of identifying all applications and systems that would be affected by the move to Exchange. Although there were several applications found that would be affected by migration, some of the groups responsible for these systems were reluctant to spend the time and money necessary for conversion. Subsequently, long after Exchange was deployed, a small number of XENIX servers remained which allowed these systems to continue functioning.

ITG had the advantage of being located near the product group on Microsoft's corporate campus. Many challenges were resolved in close collaboration with the product group and these solutions came in many forms, from simple architectural design changes to significant changes in the Exchange product.

For example, the product group included Directory API (DAPI) functionality in the product code after IT presented the value-add to the overall Exchange administration. DAPI allowed ITG to automate synchronization of changes in the HR database with the Exchange Global Address List. Due to the fact that it was very likely other companies would also have this same need, it was added to the product.

Project Completion

Preliminary planning of the Exchange 4.0 migration began in April 1993.

January 1995 - 500 users running on Exchange Beta 1.

September 1995 - 5,000+ users running on Exchange Beta 2A.

April 1996 - 32,000 mailboxes successfully migrated to Exchange and Microsoft Exchange ships.

ITG accomplished the corporate goal of moving all employees at Microsoft to the Exchange platform before the product was released.

Upgrading to Microsoft Exchange 5.0

ITG Messaging Services wanted to continue developing their relationship with the Microsoft Exchange team. Now that Microsoft owned an Exchange client/server environment, this intimate working relationship was important to both ITG and the Exchange product group.

ITG also wanted to continue providing reliable messaging services to users, while the product group desired ITG's continued efforts in expanded real-world beta testing of new Exchange versions. To fulfill these obligations, ITG began planning the rollout to Exchange 5.0 a scant two months after the conclusion of the Exchange 4.0 migration, with initial server migrations scheduled within seven months of the 4.0 conclusion. As an integral part of the validation cycle for the Exchange development team, it was necessary for ITG to rollout Exchange 5.0 as the product neared release.

ITG support staff requirements dropped significantly in the months following the 4.0 implementation, from a staff of 45 to 33. The reasons for this reduction in support staff requirements were:

It was no longer necessary to have support staff for both the legacy XENIX environment and the Exchange environment.

Support staff and end-users became comfortable with the Exchange 4.0 messaging environment and made fewer help requests.

ITG prepared for the 5.0 upgrade by increasing the support staff to 36, then reducing the staff to 29 after the 5.0 product was implemented and stable. The following chart shows the staff support cycle.

Figure 5: Staff Support Resources During Exchange 4.0 and 5.0

Figure 5: Staff Support Resources During Exchange 4.0 and 5.0
See full-sized image.

The 4.0 implementation was much more resource intensive than the 5.0 upgrade due to the relative complexity of a new product implementation versus a version upgrade. Although the staffing requirements were significantly different, each rollout required thorough planning and management to implement the new product and functionality while providing an ever-improving level of service to the end user.

New Features and Services

Exchange 5.0 provides several improvements over Exchange 4.0, such as:

Network News Transfer Protocol(NNTP) service: NNTP allowed an Exchange server to participate in the USENET news hierarchy and made it possible to view Exchange Public Folders with an NNTP client.

Lightweight Directory Access Protocol (LDAP) access: LDAP allowed read access to the Exchange server directory by using an LDAP client.

Post Office Protocol version 3 (POP3) access: POP3 enabled access to the Inbox of a client mailbox by using a POP3 client.

Web access: Combining Active Messaging 1.1 (the collaborative computing component of Exchange 5.0), Windows NT Server's built-in Web server, Internet Information Server (IIS) and Active Server Pages made it possible to access a client mailbox or Public Folder via a web browser.

Address Book Views: A new Address Book view feature allowed the global address list to be sorted into views based on directory information.

Internet Mail Service: IMC was enhanced and renamed the Internet Mail Service (IMS), providing a more robust SMTP mail transport. In addition, the functionality of the IMS was extended to route SMTP messages and act as a smart host.

Microsoft implemented the LDAP and POP3 access early in the upgrade process. This expanded client access to the Exchange organization was only available within Microsoft's internal network, because of the firewall.

A partial deployment of the Outlook Web Access (OWA / HTTP) was implemented by ITG as well, although, owing to ITG's integral role in the Exchange product cycle, a few "showcase" servers were created to let marketing and sales employees demonstrate this feature. ITG decided that the feature was not necessary on a wide-scale basis within Microsoft.

Infrastructure and OS Improvements

Between the rollout of Exchange 4.0 and the migration to Exchange 5.0, ITG upgraded the standard server operating system to Windows NT Server version 4.0. Although a few servers running Windows NT 3.51 Service Pack 5 are still operating in the worldwide organization, the majority of servers were upgraded. As a result, the Exchange 5.0 planning and implementation team benefited from a consistent operating system version on which to load the Exchange messaging product for the first time.

Server Upgrade Strategy

The Exchange 5.0 upgrade process adhered to traditional Microsoft-defined implementation methodology. The ITG planning team worked closely and continuously with the product development team to understand the product feature set and architect an effective product design.

Testing and release proceeded on a structured basis with the product development team testing every build of the product using Build Verification Tests (BVTs). All product versions were tested in the Dogfood environment and at pre-defined project milestones; the product development team negotiated with ITG to implement particular builds. Those builds were subject to BVTs in the ITG test environment, including a 72-hour stability test, to ensure version reliability before introduction in the production environment.

In addition to the general implementation process, several clear principles and agreements between ITG, the product development team and end-users characterized the upgrade process at Microsoft. Guiding principles for the upgrade to version 5.0 were:

A Major Impact to the messaging environment was defined as either a delay of Internet or inter-site messages of more than 8 hours, or downtime on a mailbox server of more than 4 hours. This definition made determining escalation procedures easier when server or client issues arose.

If any upgraded server experienced a problem that caused a Major Impact, the entire migration schedule was suspended until the system could be stabilized.

If ITG missed a scheduled upgrade opportunity, the project schedule was extended rather than increasing the number of servers upgraded to meet the schedule objectives. In other words, the pace of the schedule was not increased to achieve a fixed calendar deadline.

The upgrade process would start at Redmond and then expand to the other sites.

By following these principles and adhering to its structured project methodology, ITG effectively developed a plan and an effective schedule for the migration.

Upgrade Process

ITG used three major builds in the deployment of Exchange 5.0: Pre-Release Candidate 1 (Pre-RC1), Release Candidate 1 (RC1) and Pre-Release To Manufacturing (Pre-RTM). This approach to migration committed the Messaging Operations team to performing at least three upgrades on servers used in the earlier phases of the project. With over 100 servers in the Redmond environment, time and resources necessary to support the migration process were substantial.

The upgrade plan for the Redmond servers called for seven phases, designed to test the different components of Exchange messaging services, connectors and gateways and Public Folders. The following table shows the number and type of servers for each phase.

PhaseMessaging Servers AddedConnector / Hub ServersPF ServersTotal # of Servers

1

1

2

2

5

2

4

2

 

11

3

2

3

 

16

4

2

3

 

21

5

14

6

 

41

5.5

31

 

 

72

6

22

2

 

96

After Phase 6 was complete, the majority of the Exchange servers in Redmond had been migrated and ITG aimed to upgrade servers across Microsoft's international subsidiaries at a rate of approximately 20 servers every weekend.

The new Exchange 5.0 information store format required the servers to be upgraded. The processing time for 4.0 to 5.0 information store conversions averaged one-GB of data per hour. Given an information store limit of 16 GB, the maximum upgrade time for a single store was approximately 16 hours. This time constraint defined the migration schedule. ITG selected the servers with the smallest stores for migration early in the process to rigorously test the 5.0 environment with a higher number of upgraded servers at the outset of the project.

Client Migration Strategy

A mandatory upgrade of the Exchange client was not necessary in the 5.0 migration for several reasons.

The aggressive server rollout schedule required substantial resource support. By making the 5.0 client an option, that issue was partially relieved.

The product development team requested that product testing processes stay focused primarily on testing server functionality, with the intent of concentrating on the server's new features. Normal testing procedures within the product group were sufficient to ship a problem-free client.

The Exchange 4.0 client was compatible with the 5.0 server, making a client upgrade unnecessary. For users who wanted to migrate to the Exchange 5.0 client, ITG made the RC1 client available on a network share. Those users choosing to upgrade the Exchange client between RC1 and RTM were informed that limited support was available there and loss of mail was possible with pre-released clients.

Unexpected Challenges / Waiting for Solutions

Project challenges for the Exchange 5.0 upgrade centered on control and identification of code fixes. The first release candidates of Exchange 5.0 experienced functionality problems with the IMS and permissions on Public Folders. ITG proceeded with builds lacking code fixes for these issues and the Exchange development team eventually resolved them. Consequently, ITG decided to postpone implementing 5.0 on the IMS and Public Folder servers until a build was available that fixed the issues and was tested by ITG. Forethought, flexibility, coordination and patience, used appropriately to support the goal.

Summary

The Exchange 5.0 upgrade provided numerous benefits for the messaging environment at Microsoft, including:

Expanded client accessibility: Access to the Exchange 5.0 information store and directory were expanded to accommodate more clients, including many Internet standards such as POP3 and LDAP clients and limited HTTP access.

A test platform for the new product: The Exchange 5.0 migration allowed the Exchange product development team, ITG and PSS to continue their joint implementation and support of the product in a production environment before product release.

Robust Internet messaging support: The new IMS provided more robust Internet messaging.

Lower support staff: ITG also benefited by the reduced number of staff needed to implement and support Exchange. The upgrade process allowed ITG to plan, implement and support the second commercial version of Exchange without significantly increasing their staffing level.

The Exchange 4.0 to 5.0 migration process was a success because of extensive planning and efficient communication throughout the project. The research, planning and implementation teams worked closely with the product development team and support staff, planning and implementing an upgrade project with minimal impact to the end-users.

Upgrading to Microsoft Exchange 5.5

Exchange 5.0 was a tactical release for Microsoft that allowed ITG to add the latest Internet protocol standards to the corporate messaging platform. Once completed, the Exchange product group began developing a more strategic release.

Microsoft Exchange 5.5, code named "Osmium", had several server enhancements that made Exchange more scaleable for very large organizations. Exchange, in the form of a publicly available Microsoft product, had been on the shelves and in production at Microsoft for over a year. During this time, Exchange early adopters and other customers implemented the product and provided constructive feedback to the Product Support teams and directly to the product group. Listening to their customers, the Microsoft Exchange product group designed features in the server that would ease management of the product.

By now, ITG was an integral and steadfast partner in the product release process. ITG committed to upgrading the production Exchange organization from Exchange 5.0 to Exchange 5.5 to provide realistic testing of Exchange 5.5 in a real world environment.

Once again, a project plan was devised, along with a rollout schedule that outlined the phases necessary to successfully upgrade the Microsoft organization.

New Features and Services

The centerpiece feature of Exchange 5.5 was the Hardware Limited Store that increased the information store size limit -- from 16 GB to 16 TB. Although this was a feature that ITG had longed for, the increased capacity in the information store also resulted in added stress to other, fundamental Exchange server functions (store backup scenarios, catastrophic failure restore, etc.) With a much larger information store, supporting services such as these needed close scrutiny and careful resolution.

Other new features to Exchange 5.5 included:

Additional Internet standards included: Internet Mail Access Protocol version 4 (IMAP4) client access to mailboxes and LDAP v3 became features of Exchange 5.5. These new Internet standards enabled reads and writes to the Exchange directory.

Additional Outlook Web Access (OWA / HTTP) features: Outlook Web Access users were now able to view their schedule, request meetings and view other users' free and busy times via a Web browser.

Enhanced fault tolerance: New clustering support allowed for enhanced fault tolerance in the event of a hardware failure.

Security: Security features in Exchange 5.5, looked to support for the latest encryption standards including Secure Sockets Layer (SSL), Enhanced SMTP (E/SMTP), Simple Authentication Security Layer (SASL) and digital signatures.

Deleted Item Recovery: This new feature allowed users to undelete item(s) in a server side private or Public Folder.

Exchange Scripting Agent: This agent enabled event-driven server based scripts to be used as part of messaging applications.

Real-time collaboration: Chat Service and Internet Locator Service (ILS) allowed for real-time communication such as virtual meetings and real-time application sharing.

The new features and services of Exchange 5.5 allowed ITG to lower the number of servers in the Microsoft organization and, consequently, how they were managed. The increased information store size, coupled with new store management features and appropriate hardware configurations, made it possible for ITG to greatly increase the number of users per server. With the upgrade to Exchange 5.5, ITG geared up for this change and new hardware standards and operating procedures for the Exchange servers.

New Server Size Analysis

The Exchange product group proposed an increase from 300 users per server to 2,000 4,000 users per server, but based on capacity planning and disaster recovery analysis, ITG chose ratio on approximately 1,000 users per server 3 times more users per server. Although this was less than half of the number that the product group wanted, this decision satisfied ITG's concern about the number of users impacted if a server failed. However, the 1,000-user figure addressed both requirements: the Exchange development group could test a large number of users in a production environment and ITG met its goal of managing the risk of possible challenges.

At 1,000 users per server, Microsoft was able to reduce the number of user mailbox servers as much as 66%. To accommodate 1000 users per Exchange server, the Exchange product group recommended quad DEC Alpha or Compaq 5000 servers with 512 MB of memory and 150 GB of drive space on high-speed SCSI arrays. After comparative analysis, ITG chose the Compaq 5000's that are still in place today.

Side Effects of a Larger Information Store

The implementation of the deleted item recovery feature of Exchange 5.5 increased the size of the information store. While user mailbox quotas were set at 50 MB at Microsoft, the size of the retained deleted items was not counted as part of this mailbox size limit. This meant the potential size of the private information store could exceed the amount of hard drive space set aside for 1000 users with 50 MB private information store limits. Research at Microsoft indicated that deleted item recovery accounted for approximately 2 GB on a server with 1,000 users.

ITG also revised operating procedures to accommodate the number of users and the size of the information store. For example, with the information store functionality providing a much larger server, backup and restore technology also needed improvement. The digital linear tape (DLT) devices specified for the Exchange 5.5 servers wrote 20 to 60 GB per hour and held from 20 to 65 GB per tape. This was adequate for the size of the information store each server was expected to house. So ITG ensured that their backup technologies and procedures were sufficient to handle the new Exchange server configurations.

The ability to restore a server was also an issue. ITG's disaster recovery procedures were designed around a 16 GB information store. With servers in excess of 50 GB, new procedures and end-user support agreements needed to be developed for the obvious longer turnaround times and new backup hardware.

Along with the increased number of users, came the need for not only a large disk array, but also additional processor and memory requirements. ITG found that the following server configuration met those requirements:

ComponentConfiguration

Memory

512 MB RAM

Processor(s)

Quad Pentium Pro 200

System Disk

Mirrored 2 GB FAT partition

Log File Disk

Mirrored 6.6 GB NTFS partition

Database Disk

RAID5 array: 14, 9 GB drives

Backup Device

DLT 7000 Tape Drive

This specification varies in locations outside North America based on the number of users and server functionality, where offices are responsible for their own hardware budget and purchasing and have limited flexibility around the standard Exchange server configuration they choose.

Server Upgrade Strategy

Major changes in the database structure of Exchange 5.0 meant each server database would have to be upgraded. In addition, more users per server meant fewer total servers. Therefore, users would have to be consolidated from the numerous Exchange 5.0 servers to fewer Exchange 5.5 servers. Both the database upgrade and relocation of users concerned ITG and the builds ITG would deploy were in development. To limit ITG's exposure to beta code, pre-deployment tests were run in the Exchange product group's messaging environment. To help keep their commitment to upgrading to Exchange 5.5 without a major impact to the user community, ITG devised the following rollout guidelines to assure product stability before deployment:

The rollout would start in Redmond alone, with expansion to other sites close to the release of Exchange 5.5 to manufacturing. This plan was developed using experience gained from the rollout of Exchange 4.0 and Exchange 5.0. Although it was important to present the Microsoft enterprise as a product showcase on the product release day, it was more important to demonstrate how well the features of each new version of Exchange operated in a large environment like Microsoft.

Rollout would occur in a phased approach. Each phase required that the software build earmarked for introduction into the ITG environment had to be tested using BVTs in both the ITG environment (on limited servers) and the Dogfood environment. Each build had to run for 72 hours successfully and pass all BVTs. Some of the BVT criteria were: no mail service shutdown for a continuous 48 hours and no lost mail.

If the build passed with no severe problems, or more than 4 hours of downtime, ITG expanded the rollout to other servers and additional users as planned.

ITG's rollout milestones coincided with the Exchange 5.5 product development milestones. This fostered better cooperation and synchronization among the two groups' projects overall.

Support was provided through normal PSS channels; however, several product development group developers carried pagers, to provide immediate response to serious issues.

Upgrade Process

The upgrade plan consisted of four phases. The details of these phases allowed ITG to gradually rollout successive builds into the production environment before the product was released to manufacturing. The timeline for the project was set early in the project lifecycle, but was ultimately dependent on the Exchange product group's release dates.

The selection of servers and users for the phased migration was based on the size of the information stores. Servers nearing the 16 GB Information Store limit were identified and users on those servers moved to the newly built Exchange 5.5 servers. A hardware upgrade was necessary to support the new store size, so the most efficient upgrade method was building new Exchange 5.5 servers on the new hardware and then moving users to them.

Client Migration Strategy

The Exchange client was not upgraded from Exchange 5.0 to Exchange 5.5. The Microsoft Outlook messaging and collaboration client replaced the Exchange client with this version.

The product development group performed supportability verification testing on both Outlook and the Exchange 4.0 client to assure there were no outstanding issues. This testing was required because an estimated 60-70% of all Redmond users were using Outlook (it was an integral part of Microsoft Office 97). Data from this testing was provided regularly to ITG.

ITG's experience with the first version of Outlook had shown that the Outlook client could have an enormous impact on server performance. The Personal Information Management System (PIMS) of Outlook offered a personal address book, task management and journal entries. The biggest advantage of Outlook was combining e-mail and calendaring in one application for simplicity of access.

Challenges and Solutions

For ITG to upgrade Exchange servers from Exchange 5.0 to Exchange 5.5 and support mailbox consolidation, the following requirements required new procedures be established:

The ability to easily move users from Exchange 5.0 servers to Exchange 5.5 servers had to exist. This was necessary to move users from one server to another as defined in the migration plan. Prior to this point in the evolution of Exchange, ITG had always upgraded servers in place. ITG plans for Exchange 5.5 required that users be moved from smaller, older hardware to new, larger, faster machines.

Any upgrade lasting more than 12 hours would be restricted to weekends only and the maximum allowable time for any upgrade in Redmond was 24 hours. This allowed a period for replication to "catch up" and stabilize after each upgrade.

ITG needed additional configurable limits for individual mailbox quotas (i.e., refuse delivery after 75MB). To further control mailbox size, the product group added this feature to the release product. This also allowed ITG to better manage the size of the information store. Limits were set at 50 MB on the private information store that would restrict users from sending e-mail, but there were no limits set to restrict incoming e-mail. With this new feature, a maximum of 75 MB per user was assured.

Offline database maintenance had to be optional. The potential Exchange 5.5 database sizes and the associated downtime required for offline maintenance dictated that no offline maintenance could be required. Therefore, the Exchange product group enhanced some of the servers' automatic maintenance features, which reduced the need for offline database maintenance on Exchange servers.

Risk Factors

Additional challenges came in the form of risks. The following risks were identified during the Exchange 5.5 upgrade:

More users per server meant more users affected if a server failed. In productivity hours, this meant that 1000 users would be affected versus 300 users prior to Exchange 5.5. Entire sites might be affected by downtime, specifically for locations that had less than 1,000 users and were housed on a single server. If that server were to fail, then all users at that location would lose access to e-mail. In total dollar cost to the corporation, the higher number of users had a direct connection to the server size limit set by ITG.

While item recovery was meant to alleviate the need for some database restores due to user error, the need was not completely eliminated. There was always a chance that other scenarios might require a server restore e.g., possible catastrophe related information store corruption or user error. Therefore, full database restores were still necessary and would take an extended amount of time.

Summary

As with the Exchange 4.0 to 5.0 upgrade, the 5.5 upgrade was successful due to extensive planning and communication throughout the project. The Exchange 5.5 upgrade provided numerous benefits for the Exchange organization at Microsoft. The primary and most visible benefits were:

Reduced number of servers: This reduced monitoring obligations and replication traffic, increased use of the single-instance store feature and provided the product development group with thorough testing of enterprise-level features before commercial release. The reduced number of servers had impacts beyond the messaging environment. For example, network operations, facilities, staffing and licensing all benefit from a lower number of servers deployed.

Reduced number of mailbox restores: The introduction of the deleted item recovery feature reduced the amount of administrator intervention often necessary when supporting recovery services.

Planning for Exchange "Platinum"

Microsoft ITG is preparing for the next version of Microsoft Exchange, scheduled to ship shortly after Windows NT 5.0. The new version of Exchange, code-named "Platinum", represents significant changes to the Exchange product. These changes are meant to compliment the changes in Windows NT 5.0 and other BackOffice solutions. The process of migrating from Exchange 5.5 to "Platinum" mirrors the careful planning and testing process of the past Exchange migration projects.

"Platinum" will be the fourth significant release offered by the Exchange team. Leveraging technologies from BackOffice and Office products, "Platinum" will depend on a number of BackOffice technologies, such as the Microsoft Management Console (MMC) for administration, Windows Web Service (formerly IIS) for the transport, Site Server, Certificate Server and numerous other elements of Windows NT Server 5.0. Most significant with "Platinum", is the migration towards the new Windows NT 5.0 Active Directory, a BackOffice service that will offer directory services across the family of BackOffice products.

With this level of fundamental changes forthcoming, ITG is well underway with their research to prepare for the challenges the new Exchange product will introduce.

The ITG Design Team

To better handle the likely infrastructure changes to come, Microsoft IT has refined the research, administrative and support organizations working alongside the product group to preparing for "Platinum". This new Microsoft ITG Design Team for "Platinum" is a multi-faceted group of technologists who contribute to the research, design, implementation and support processes. A project manager who will direct the organizational effort heads this team and a technologist directs the research, design and implementation efforts.

The team's project manager will focus primarily on:

Team resources

Budget issues

Logistical analysis

Cause and effect on ITG organizations and their process and procedures

Migration schedules

To focus research and design efforts, the design team's technologists have segmented the "Platinum" release objectives into the following areas:

Advanced security

Transport and gateways

Store & protocols

Public Folders

System administration

Directory services

Application development

Real-time collaboration

Unified Messaging Service (UMS)

Client Services

Challenges Ahead for Microsoft ITG

Today at Microsoft, ITG Operations is composed of functional groups that focus on specific tasks such as Exchange administration, Windows NT administration and network administration. Each of these teams are able to focus on their particular task using tools and applications geared toward their functional area. Given the new direction of the BackOffice and Office strategies, Microsoft ITG will need to reevaluate its support and administrative models to ensure continued efficiency in administrating the corporate infrastructure. Three primary challenges for Microsoft are apparent when combining Windows NT 5.0 and "Platinum":

Directory Services: Today Exchange Server supports its own directory service. However, with the introduction of Windows NT 5.0, the directory service (Windows NT 5.0 Active Directory) is a shared service, for which Exchange will be one of many applications that share that directory service. The challenge therefore is how the ITG organization will balance administrative processes to support the Windows NT Active Directory, while still meeting the needs of Exchange.

System Administration: Exchange is administrated via the Exchange Admin application. With "Platinum", Exchange Admin is replaced with the Microsoft Management Console (MMC) in BackOffice. Microsoft ITG will need to reevaluate their current Exchange procedures since the MMC will allow administrative services across the BackOffice family of products. What used to be handled by multiple administrative groups in ITG will now be possible from one or more groups using a single application user interface the MMC.

Public Folders: Today, the current corporate Public Folder environment at Microsoft consists of over 70,000 Public Folders. With the introduction of the "Platinum" feature set, ITG will have the opportunity to fine tune that environment and introduce a more scalable, user-friendly environment. The challenge is building the environment to promote workflow solutions and support of other product interdependencies such as indexing, publishing and knowledge management initiatives without adversely affecting the current business use of the feature.

Coexistence: The migration of users from Exchange 5.5 to Windows NT 5.0 / "Platinum" and the Active Directory is dependent on the stability of the Active Directory Connector (Synchronization Agent). For Exchange 5.5 and "Platinum" to coexist during migration, the synchronization agent must accommodate the Exchange 5.5 directory service and the Windows NT 5.0 Active Directory so the two systems maintain a virtual global address book of the entire organization. (See illustration below).

Figure 6: Active Directory Connector

Figure 6: Active Directory Connector
See full-sized image.

Once the first "Platinum" server is installed and Exchange 5.5 users are migrated to "Platinum", the synchronization agent will ensure that users of each system have access to a complete global address book.

Summary

As more research is completed and implementation plans solidify, ITG will make appropriate changes to their tasks and schedules. The project manager is focusing on organizational issues to ensure that human and budgetary resources are available for the project, while the ITG design team's technologists focus on functional areas to determine how new features in each area will affect the way the infrastructure is run today.

Currently, plans are underway to design tools necessary for efficient migration to the Windows NT 5.0 Active Directory while providing full backward compatibility. At this point is, much of that work is research and planning, but as the product development matures, betas and release candidates will become available, allowing Microsoft to forge ahead with representative aggressiveness, moving much if not all, employees to "Platinum" by the time the product ships.

Conclusion

The implementation of Microsoft Exchange at Microsoft was an ambitious undertaking, representing one of the most complex messaging environments for deployment of a new product with requirements for high performance and high availability for sites around the world. The lessons learned from this project are important for Microsoft, when moving forward with subsequent deployment of Exchange and for other companies contemplating deployment of Exchange. The criteria for successful deployment are important to remember because these criteria apply equally to other organizations. Future trends for the product are important considerations. Being positioned to adapt to new technology and integrate that technology as quickly as possible provides competitive advantages.

Lessons Learned

When ITG began planning to move from XENIX to Exchange, the challenges were different than those they faced when migrating from Exchange 4.0 to Exchange 5.0 and from Exchange 5.0 to Exchange 5.5. The lessons learned by Microsoft in the initial migration were in two areas: how Exchange actually functioned in a large enterprise and what ITG must do to support this new version of Exchange.

The challenges ITG faced and surmounted in the migration from XENIX laid the groundwork for what was to come for future upgrades. The client deployment, inter-group cooperation, end user support and version testing would become components of future upgrades in the coming years.

Within months after Exchange 4.0 was implemented, the process of planning for the upgrade to Exchange 5.0 began. At this point ITG had an Exchange organization in place and knew what was involved in upgrading the Exchange servers to Exchange 5.0. The database conversion from Exchange 4.0 to Exchange 5.0 was a major part of the server upgrade.

ITG had developed enough practical experience with Exchange to understand the importance of planning each step of the project, ensuring that end users were able to continue their daily work as efficiently as possible. The Outlook client and how it interacted with the Exchange server raised performance issues that challenged the product group. Again, BVT testing proved to be the key to avoiding problems associated with implementing new versions of a product in a production environment.

The upgrade from Exchange 5.0 to Exchange 5.5 entailed reducing the number of servers in the Exchange organization. Exchange 5.5 servers could support many more users than earlier versions of Exchange because of the expanded information store size limit. This enabled ITG to implement larger servers that would host more users and thereby reduce the total number of servers in the environment. ITG approached this reorganization of users using a phased approach to the upgrade.

Large Exchange servers were implemented and users were moved from the Exchange 5.0 servers to these large Exchange 5.5 servers. This allowed ITG to complete two objectives with one action, adding more users to the new servers and implementing newer hardware.

The events that took place at Microsoft during the Exchange implementation and upgrades taught ITG the critical components necessary for a successful Exchange project. The lessons learned have survived and are now integrated in a set of best practices for the organization.

ITG's Elements for Success

Exchange represented a new type of application for Microsoft ITG: a mission-critical application, not customized specifically for Microsoft. This meant additional focus and attention was necessary for the project to be successful. The key elements are:

Communicating with Product Development: To provide the testing ground for the product development team while continuing to provide the messaging performance for the end-users required constant contact between the ITG team, the Dogfood messaging team, product support services, client support teams and the product development team. Each phase and component of the project had a clearly defined owner who had responsibility for that component and final decision authority. This communication cycle started early in the development cycle and gave ITG the opportunity to gain the expertise needed for rollout.

Communicating with End Users: Communication between ITG and the end users was as important to the project's success as inter-group communication. For users to be able to view the project as a success, it was necessary to clearly inform end-users of the pending changes and the affect of those changes.

Planning: The complex processes of a messaging rollout, especially when they included several different builds, required meticulous planning. Each stage of the rollout included steps that were clearly defined and identified the target servers and services.

Testing: ITG needed a testing process independent from the testing performed by the product development group. ITG wanted a process that would identify the issues in their specific environment. ITG also wanted to give their testing perspective back to the product development team to foster enhancements and improved features. The relationship, established early in the process and maintained today with the product group, was unique at the time that the project began. Although testing Exchange in both the product group and ITG testing environments increased the costs in time and resources of upgrading, overall it helped to assure a constant level of service to end users. This satisfied ITG's objective of providing fast and efficient messaging services to every employee at Microsoft.

Cross Department Coordination: ITG, the product development team, client support, PSS and the end-users themselves, all participated and cooperated in the rollout of Exchange. Again, this was essential because all groups were working with a new product. Members of each department, from development, through ITG, to PSS, had to become familiar with the latest features and their effect on the Exchange environment to help assure a constant level of service.

Service priority: ITG's first priority was to provide reliable, efficient messaging services to the end-users. Although important, testing new versions of the product must not impact the ability to deliver quality services. ITG's testing assured the reliability of the new version of Exchange for the end-users. In contrast, the testing done by the Dogfood team was primarily to validate the build stability and functionality before proceeding with development efforts. While both groups used many of the same testing methods, their motives were often at odds.

Future Trends

The evolution of Microsoft Exchange at Microsoft will continue past version 5.5. The product, ITG best practices and the underlying infrastructure continue to change and the design of the Exchange environment continues to adapt to those changes. ITG is anticipating the following changes and is currently making design and planning decisions around them:

Product enhancement: Exchange is tightly integrated with Microsoft Windows NT and other BackOffice technologies. With the introduction of Windows NT 5.0 Server, Exchange "Platinum" is scheduled to follow closely, taking advantage of the new operating system and BackOffice features and functionality. These new features will center mostly on the Active Directory and integrating with Internet standards. In addition, this next generation of Windows NT Server and "Platinum" will mean major changes in the ITG administration structure and the services offered by ITG. ITG is preparing early for these changes by using the same planning and communication processes developed in previous rollouts.

ITG best practices: ITG is establishing relationships with the Windows NT product group to understand how maintaining an enterprise based on Windows NT 5.0 may affect the current administrative policies and procedures. For example, today ITG has separate groups maintaining functions such as Windows NT accounts, mailboxes and employee data such as manager name and office number. As these functions integrate further into the Windows NT administration model, it may identify overlap in functional administration groups. In addition, as Microsoft Exchange continues to mature and features are expanded, migration details, environmental issues and policy decisions will continue to change. The migration infrastructure that ITG has today will ensure ITG is ready to administer the new Microsoft Exchange into the next millennium.

Infrastructure: The network services group is currently changing to public ATM services for the majority of the Microsoft WAN. This change will result in a significant increase in service and available bandwidth for all of the network applications such as Exchange. This increased bandwidth between physical locations gives ITG the opportunity to revisit the Exchange architecture. It may be determined that a server is not necessary in every location, or that more Public Folder servers can exist in remote sites. By periodically revisiting the Exchange architecture, ITG is assured that they are taking advantage of new technologies and product functionality.

Microsoft Exchange: Built, Tested and Ready To Serve

At Microsoft, the volume of messaging traffic, the integral role of users in product development and high user sophistication and support expectations makes Microsoft a unique environment for the Exchange product. However, all these factors were managed successfully with communication, planning, testing and cooperation. Although there have been many issues for ITG to resolve during each project, the messaging environment in place today is full-featured, stable and capable of supporting expanding needs.

Appendix A: Microsoft Exchange Basics

Microsoft Exchange is the communications platform for business-critical messaging that enables one of the widest ranges of collaborative solutions. In a true enterprise fashion, Exchange scales to fit into very large and complex organizations. Exchange also serves small companies well as a messaging platform for their finite, but expanding, environments.

For the remainder of this document, we will try to segregate Exchange into its functional components. Each of those components is defined here and is based on the Microsoft Exchange 5.5 product feature-set.

The Directory

The Exchange directory contains all objects and attributes that define an Exchange organization. The directory is contained in a separate database, as are the Information Stores and is managed by a Windows NT service the Exchange Directory Service. The directory service provides access to the directory attributes. These directory attributes form a hierarchical tree that originates from an organizational root.

How Directory Service functions:

It contains addresses, mailboxes, Public Folders, Distribution Lists (DLs) and configuration information about sites and services.

It maintains the directory information for an Exchange organization and automatically replicates this information to all servers in a site.

It utilizes the messaging connectors between sites to maintain replicas of the directory on all servers in the organization.

The Directory Service Global Address List can be accessed through any of the following methods:

Exchange clients using the MAPI.

Lightweight Directory Access Protocol (LDAP) clients.

The Exchange implementation of the directory service is a multi-master implementation. When a client queries an Exchange server for directory information, the server is able to answer the request from its local copy of the directory without having to forward the request to a central server.

The Exchange directory is loosely based on the X.500 standard. Unlike X.400 counterparts, X.500 is becoming a preferred directory structure for client-server environments. While X.400 has lost ground to SMTP, X.500 use has grown by providing directory services to the once directory-less SMTP environment. Furthermore, its common access protocol, LDAP, is proving to be the protocol used by Internet clients, such as clients that support POP3 and IMAP4, to provide directory services. X.500 is also the directory model planned for Microsoft Windows NT 5.0 operating system.

Information Stores

The Exchange Information Stores are databases maintained by the Exchange Information Service running as a service of the Windows NT operating system. Exchange Server databases are based on the Joint Engine Technology (JET) format, which uses log files to track and maintain information. Microsoft JET is an advanced 32-bit multithreaded database engine that combines speed and performance with other advanced features to enhance transaction-based processing capabilities. For the Standard edition of Exchange Server, each database can grow to a maximum of 16 GB. The Hardware Limited Store feature is a theoretical limit for the databases in the Enterprise edition of Exchange Server. At Microsoft, that limit has been set to 16 TB.

The Information Store service provides server-based storage of e-mail messages and other information, such as files, faxes, or voice mail. The information is maintained in two databases: the Public Information Store and the Private Information Store.

Public Information Store: Maintains information stored in Public Folders. Public Folders are containers for messages and other information that are used in a sharing or collaboration environment.

Private Information Store: Maintains all messages sent to a specific individual or a selected group of people in private folders. E-mail within the private folders is available only to those who have access to the mailbox associated with the private folders.

The Information Store has the following characteristics:

Messages in the Information Store are stored in Rich Text Format (RTF). For clients that are not RTF capable, a separate process, the IMAIL process, converts the messages from RTF to the preferred format of the client.

A single instance store is employed that enables a single message sent to multiple users on one server to be stored once in the Information Store. Instead of replicating copies for each recipient of a message, each recipient receives a pointer to the message. This allows Exchange to store each message only once in the Information Store database.

The maintenance of previous transaction logs enables incremental and differential backups and restores.

The Information Store can be accessed through:

MAPI clients

POP3 or IMAP4 clients

NNTP clients

Outlook Web Access (OWA ) also known as HTTP)

A server can have one private Information Store or one public Information Store or both. Providing control over the creation of stores allows load-balancing of the servers. Future versions of Exchange may allow for multiple Private and Public Information Stores resident on a single Exchange server.

Protocols

Exchange Server supports SMTP, POP3, IMAP4, LDAP, NNTP and indirectly, HTTP.

SMTP is a protocol used to deliver messages across the Internet over TCP/IP. SMTP is used to deliver messages from the Internet client to an SMTP host. The SMTP host then forwards the message, again using SMTP, to the destination SMTP host and mailbox.

POP3 and IMAP4 protocols provide access to an individual mailbox. POP3 provides access to a user's Inbox folder. IMAP4 extends access to include Exchange Server-based personal and Public Folders and is the most recent Internet mail access protocol added to Exchange Server.

LDAP, as mentioned in the directory service section above, is a protocol for clients to search against the Exchange Server Directory Service to provide limited Address Book functionality.

NNTP is a protocol for Network News posting and distribution across the Internet and is widely used. Clients access newsgroups as Exchange Server Public Folders.

From a technical standpoint, Exchange Server does not implement HTTP as a protocol. Instead, Exchange Server uses it as a flag within the directory that can be used to determine if a Web-based client can access a server. Web-based clients actually send the HTTP commands to a computer with the Windows NT Server technology, Internet Information Server. Once it has these commands, the IIS computer translates them into MAPI commands and submits them to the server on behalf of the client.

MAPI is the traditional protocol used by Microsoft messaging products. It is a powerful and extensive messaging API relied upon by Microsoft to provide messaging capabilities in an efficient and reliable manner. As Internet protocols mature and evolve, Microsoft may begin to rely more on Internet protocols such as SMTP and IMAP4 or HTTP.

Connectors and Gateways

Exchange Server uses messaging connectors to connect two sites for the purpose of message transfer. There are three types of connectors: messaging connectors that move Exchange messages between sites, the Directory Replication connector that utilizes the messaging connectors to replicate directory information and gateways that enable Exchange to coexist with foreign messaging environments. The following connectors and gateways are included in the enterprise edition of Microsoft Exchange.

Site Connector: A high-speed connector for use across high bandwidth connections between sites. The Site Connector is typically the easiest way to configure two Exchange Server sites to interchange data. To use the Site Connector, the sites must be connected to each other on a network. This is because the Site Connector uses Remote Procedure Calls (RPC), which require a network protocol such as TCP/IP, NetBIOS Extended User Interface (NetBEUI), Internetwork Packet Exchange (IPX), or some other transport-level protocol. The Site Connector assumes a high-speed, permanent network connection. The Site Connector is approximately 20 percent faster than the X.400 Connector is. This makes it more efficient and easier to configure than the X.400 connector. The available bandwidth of the environment can limit the usability of the Site Connector. Typically, it is used in a medium to high bandwidth environment. It can work over low network bandwidth depending on the amount of data being transferred.

X.400 Connector: This connector is probably the most common connector implemented today throughout the messaging industry. The X.400 connector is both fast and reliable across limited bandwidth network connections. The X.400 Connector can be used to connect two Message Transfer Agents (MTAs) with the TCP/IP, TP4 and X.25 network transports and provides control over the use of the link. It provides the capability to restrict when the connector is used, specify which users can send e-mail over the connector and control the size of messages.

Dynamic RAS Connector: The Dynamic RAS Connector is the only connector that allows two Exchange sites to communicate over a private dial-up connection. It provides message transport between sites when no permanent LAN or WAN connection exists. It uses the RAS functionality of Microsoft Windows NT to provide the temporary transport necessary for two Message Transfer Agents (MTA) to communicate.

Internet Mail Service: A RFC compliant SMTP host with re-routing capabilities. This "smart-host" enable Exchange to interact with the Internet's messaging environment. The IMS takes advantage of key Microsoft Exchange Server components including the Information Store, Directory Service and the MTA to send messages directly to another SMTP host.

Directory Replication Connector: This connector is not a simple messaging connector like the others. The Directory Replication Connector assures directory replication between sites in an Exchange organization. To establish directory replication between sites, e-mail connectivity must be established between the sites. Directory replication relies on the message connectivity that is provided by the Message Transfer Agent (MTA). Once a messaging connector has been configured, a Directory Replication connector can be configured between the sites.

Gateways: The ability of Microsoft Exchange to coexist with its competitors, either during migration or indefinitely, makes Exchange one of the most dynamic messaging applications on the market. The MSMail, Lotus Notes and cc:Mail connectors (gateways) allow for both message and directory synchronization between foreign messaging systems.

Public Folders

Public Folders are repositories for many different types of information that can be shared among users in an organization. When Public Folders are used in combination with customized forms, they become the basis for collaboration applications such as bulletin boards, discussion groups and customer tracking systems. Public Folders can only be created with a client, but can be administered using either the client or the Exchange Administrator Program.

The Public Folder hierarchy is a directory-like organization of the Public Folder system. Every Exchange server in the organization that contains a Public Information Store maintains a copy of the Public Folder hierarchy.

To give users access to the contents of Public Folders across the organization, Exchange allows administrators to replicate the content of Public Folders between sites. Exchange can also be configured to use "affinity." Public Folder affinity defines the order in which a client will search through sites other than its own for the content of a Public Folder.

Larger organizations may choose to house their Public Folders on dedicated servers. These dedicated servers only manage the Public Folder hierarchy and public Information Store and do not have a private Information Store.

Within each message site, there is a concept of Site Folders. Site folders include the Offline Address Book and Schedule+ free/busy. Of the specialized Public Folders the free/busy Public Folder contains an object for each user. The object contains user's free and busy times. When an Exchange user schedules a meeting with another Exchange user, the free/busy Public Folder gives the meeting planner access to the attendee's free and busy times through the objects in the Public Folder. This allows the meeting planner to schedule the meeting during a time when the attendee is not busy.

System Administration

Exchange benefits from single seat administration, which makes Exchange far simpler to maintain and administer than the traditional store and forward messaging system such as Microsoft Mail and cc:Mail. The Exchange Administrator application gives Exchange administrators a view into the Exchange directory. Based on permissions, Exchange administrators can manage the parts of the organization applicable to them, including the entire organization.

In addition to single seat administration, the system administrators can backup the Exchange server while the databases are on-line. This allows the most critical data to be saved without impacting the functionality of the server. Furthermore, Exchange can be configured to enable incremental and differential backups and restores.

For optimization, the Exchange Performance Optimizer, a tool included with Exchange, automatically analyzes the local hard disks and then makes recommendations for placing certain Exchange Server components on different drives based on hard disk performance criteria. Additionally, the Performance Optimizer modifies the Windows NT registry settings to improve the server's performance.

For maintenance, Exchange will run scheduled on-line maintenance to sustain its databases. There are also database tools for doing off-line defragmentations of the stores and to check the consistency between the various databases.

Another area specific to system administration is security. The Exchange directory is divided into three separate Exchange security contexts. Permissions flow from one security context to the other, but not beyond a single context. These security contexts, the Organization, Site and Configuration contexts, allow for Exchange administrators to have varying permissions of different contexts throughout the organization.

Figure 7: Organizational Representation in Exchange Admin

Figure 7: Organizational Representation in Exchange Admin
See full-sized image.

The Exchange Client

A client is the user's interface through which you can access a computer running Microsoft Exchange Server. Exchange Server supports numerous clients. For example:

Microsoft Outlook (32-bit, 16-bit and Macintosh versions): A full-featured MAPI-based client that works with Exchange Server.

Microsoft Outlook Express: An Internet standards-based program that supports functions such as e-mail and newsreader functionality.

Microsoft Outlook Web Access: Provides the capability to securely access e-mail, personal calendars and collaboration applications by using a World Wide Web browser.

Through the use of an Exchange client profile, the client knows which mailbox to access upon startup. The Exchange client profile is typically stored in the user's section of the registry. If the registry is configured to support roving users, the user's Exchange profile will be available when a user roves around an organization.

Collaborative Workgroup Applications

One of the standard workgroup applications to be implemented with Exchange is the calendaring functionality in Microsoft Outlook. With calendaring, Outlook and Outlook Web Access users can view, manipulate and schedule each other's times and resources.

Beyond calendaring, Exchange provides the foundation for a solid messaging-based application environment.

Security

The Key Management Server (KMS) provides both encryption and digital signatures to Exchange users. Through the use of these security technologies, Exchange users can help assure messages are not altered or decoded in transit between the originator and recipient.

Outlook communicates to the Exchange server using Remote Procedure Calls (RPC). These RPCs can be configured on the Outlook client to be encrypted. Furthermore, the RPCs between servers in a site, or servers communicating via a Site Connector between sites, enjoy the benefits of RPC 40-bit to 128-bit encryption, depending on the version of Windows NT, export restrictions and the Service Pack applied.

Unified Messaging

It is the goal of Microsoft Exchange and Exchange compatible third-party products to help turn data into information that is relevant and accessible to the knowledge worker. Unified messaging is the goal for tying a user's e-mail mailbox, voice mail mailbox and fax inbox into one retrieval location.

For voice mail, unified messaging means that as voice mail messages arrive for a user, they are routed into user's Exchange mailbox. Once stored there, users can retrieve them as a sound file, forward them in e-mail, reply to them into e-mail. Additionally, users can then also use the phone to retrieve e-mail messages.

The fax component of unified messaging allows users to send and receive faxes from the desktop via the Exchange mailbox.

The unified inbox theme that has evolved at the client and the server enables the Exchange server, with the addition of Exchange compatible third-party products and the client to be a single depository for multiple sources of information.

For More Information

Latest information on Microsoft Exchange can be found at http://www.microsoft.com/exchange

To view additional IT Showcase material, please visit
http://www.microsoft.com/technet/itsolutions/msit/default.mspx

For any questions, comments or suggestions on this document, or to obtain additional information about Microsoft IT Showcase, please send email to showcase@microsoft.com

1Several books are available on design and administration of an Exchange environment at
2For further information and case studies on these Early Adopters, see
3The Prohibit Send and Receive feature was introduced with Microsoft Exchange 5.5. See "Upgrading to Exchange 5.5" in this document for more information


© 2005 Microsoft Corporation. All rights reserved. Terms of Use |Trademarks |Privacy Statement
Microsoft