Background process for Microsoft Outlook before ESTABLISHED message

What are background process for Microsoft Outlook before ESTABLISHED message for DIRECTORY AND MAIL?

Query:

Below are the JPG photograph of Printscreen of Outlook /rpcdiag

switch .

We see 2 things

1) Directory ------- Established

2) Mail --------Established

What happens in behind outlook and then the ESTABLISHED message comes out. FOR BOTH DIRECTORY AND MAIL

The reason to ask is some times my PRCDIAG shows

Directory ----established and for

Mail ----------connecting

Even though I am able to connect TCP/IP communication port (static )--4002

+ 53+88+389+3268 ( both tcp /udp ------UDP checked through PORTQUERY software)

still some times MAIL is in connecting state.

So ultimately my querry is

1) What is required to go into ESTABLISHED state

for both MAIL and Directory.

SOLUTION:

Remote Procedure Call Fundamentals

When an Outlook client views the content of a mailbox or peruses items in a public folder, the requests go to the Exchange server in the form of Remote Procedure Calls, or RPCs.

If you're accustomed to thinking of single-purpose network protocols such as ftp, DNS, or LDAP, each with its own well-known ports, it can be a little difficult to visualize the operation of RPCs.

Consider the Exchange Information Store, for instance. An Outlook client connects to the Information Store service using MAPI. The Outlook client uses a set of commands calledprocedure calls designed to communicate instructions to the Information Store and to handle the responses. Because the service responding to these procedure calls runs on a separate machine, they're implemented as remote procedure calls, meaning that the underlying network gets involved to ferry the procedure calls to and from the pair of machines involved in the transaction.

Here's where the process gets interesting. An RPC-based application does not use a well-known TCP port. It registers a port dynamically when the service starts on the server.

But that presents a problem for the client. How does it discover the port registered by the server?

The client enlists the help of a special service running at the server called the RPC Endpoint Mapper, or EPM. The EPM acts like a receptionist with an attitude. When someone calls and asks for an employee by name, the receptionist says curtly, "Here's her extension. Thank you for calling. Goodbye." It's up to the caller to dial the extension.

Here's a more concrete example. Refer to Attched Figure 1.

When an Outlook client first starts up, it makes its initial connection to the Referral service on the server so it can get the name of a Global Catalog server. Outlook doesn't know the port used by the Referral service on that particular Exchange server, so it first makes a connection to the RPC Endpoint Mapper service using well-known TCP port 135.

Outlook asks EPM for the TCP port number registered by the msExchangeRFR service. (I'm simplifying this just a bit. In reality, Exchange services register a variety of interfaces, each identified by a Globally Unique Identifier, or GUID.)

EPM replies to the request with the port number registered by the Referral service. Outlook breaks the connection to the EPM and immediately connects to the port registered by the Referral service. What happens after that depends on the nature of the service.

You don't need an in-depth knowledge of Remote Procedure Call technology to deploy or troubleshoot RPC over HTTP, but it helps to know a few key elements of RPC communication.

RPC-Based Service Identifiers

When an RPC service initializes, it identifies itself with a Universally Unique Identifier, or UUID. Each interface used by the service gets a different UUID, and a particular service can register several interfaces. (The word "interface" in this case is a logical interface, the combination of a protocol and a port number.)

You can use the RPCDump utility in the Windows Server 2003 Resource Kit (also in the Windows 2000 Resource Kit) to list the RPC services and their UUIDs. Here are the UUIDs registered by the Exchange Server Store Admin interface for a server with the IP address of 192.168.0.13

192.168.0.13[6001] [99e64010-b032-11d0-97a4-00c04fd6551d] Exchange Server STORE ADMIN
 

Interface

192.168.0.13[6001] [89742ace-a9ed-11cf-9c0c-08002be7ae86] Exchange Server STORE ADMIN 
Interface
192.168.0.13[6001] [a4f1db00-ca47-1067-b31e-00dd010662da] Exchange Server STORE ADMIN

Interface

RPC Client Protocols

If you picture the ISO networking model, an RPC-based service sits right up there at the top, at the Application layer. It makes use of the underlying network services to move procedure calls across the network, in the same way that the network redirector relies on the TCP/IP drivers to move file system calls around the network.

Windows has several network services, and a particular RPC service can use any or all of them. Microsoft calls them RPC client protocols.

Exchange uses five RPC client protocols, each of which has an identifier called a "protocol sequence." I'll list them here because you'll need to configure an RPC Proxy server to recognize these protocols when an Outlook client requests them:

• TCP (ncacn_ip_tcp)

• UDP (ncadg_ip_udp)

• Named Pipes (ncacn_np)

• HTTP (ncacn_http)

• Local Procedure Call (ncalrpc)

RPC Endpoints

When an RPC service launches, it registers the client protocol (or protocols) it supports and an entry point for each protocol. The combination of a client protocol and an entry point is called anendpoint. Of the endpoint types used by Exchange, only those with wire protocols that require port numbers concern us. Unlike standard Internet services such as ftp or DNS, which use specific, well-known ports for communication, an RPC service selects its port based either on a Registry entry or by choosing a port dynamically from the non-registered ports above 1024.

Because of this dynamic port registration, an RPC client needs a way to determine the port number registered by the RPC service. It makes this determination by querying the Endpoint Mapper (EPM) service on the target server.

The EPM listens on two ports: TCP port 135 and TCP port 593. (The original DCOM standard assigned TCP port 593 specifically to ncacn_http requests, but Exchange and most other applications register their ncacn_http interfaces with TCP port 135.)

You can use a tool from the Windows Server 2003 Support Tools called Portqry to list the ports that have registered with the Endpoint Mapper. Microsoft Knowledge-Base article 310298 has a good list of the RPC ports. You'll get a fairly long listing so pipe the output to a text file using this syntax:

portqry -n  -p tcp -e 135 > portqry.txt  

The Portqry output resembles Rpcdump, but you can run Portqry against any server. Here's a sample listing showing the ports registered by the Exchange Referral (RFR) service:

C:\>portqry -n w2k3-ex3 -p tcp -e 135  

Querying target system called:   w2k3-ex3    

Attempting to resolve name to IP address...  

Name resolved to 192.168.0.13  

TCP port 135 (epmap service): LISTENING  

Querying Endpoint Mapper Database...  

Server's response:    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncacn_np:\\\\W2K3-EX1[\\pipe\\E60E3840067D1C87]    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncalrpc:[LRPC00000784.00000001]    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncacn_ip_tcp:192.168.0.6[1067]    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncadg_ip_udp:192.168.0.6[1068]    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncacn_http:192.168.0.6[6002]    

UUID: 1544f5e0-613c-11d1-93df-00c04fd7bd09 MS Exchange Directory RFR Interface  ncalrpc:[OLE86D8B80CA5B44FDB8E1B7FF4C784]    ==== End of RPC Endpoint Mapper query response ====  

More Detailed End-Point Mapper Operation

When an Outlook client connects to an Exchange server, it asks the EPM for the port associated with a particular service. To continue with the example of the Exchange Referral (RFR) service, the client directs a query at TCP port 135 on the Exchange server, a query that contains these elements:

• UUID of the target service: 1544F5E0-613C-11D1-93DF-00C04FD7BD09

• Type of client protocol: Connection Oriented

The EPM replies with the port number registered by the service. In the previous Portqry listing, for example, the RFR service registered the ncacn_ip_tcp client protocol to use TCP port 1067, so the EPM returns that port number to the Outlook client.

Outlook repeats this process for the Information Store Admin interface and the Information Store mailbox interface. Then, armed with the port numbers for those services, Outlook proceeds to open the user's mailbox and obtain messages.

RPC Endpoints Used By RPC over HTTP

Each of the Exchange services required by a client registers an ncacn_http endpoint. In other words, these services listen for HTTP requests at specified ports. The ports are not dynamically determined.

Instead, they are statically mapped in the Registry to simplify setup at the RPC Proxy server. These ports are as follows:

• Information Store: TCP port 6001

• System Attendant: TCP port 6002

• Referral (RFR) service: TCP port 6002

• NSPI (Name Server Provider Interface): TPC port 6004 (As a reminder, this service provides access to address lists such as the GAL)

Sample RPC over HTTP Process

Attached Figure 2 shows a simplified set of transactions that describes how an Outlook client configured to use RPC over HTTP communicates with the user's Exchange mailbox server. The actual process is a bit more complex, as we'll see in a minute, owing to the need for firewalls and global catalog servers and so forth. For now, though, let's take a look at the essentials of the process.

1. First, the user launches Outlook. This results in a series of RPC messages that contain instructions for the Exchange server to open the user's mailbox and deliver any new message headers. Because Outlook has been configured to use RPC over HTTP, it embeds the RPC message into a standard HTTP message with a header that contains instructions for the RPC Proxy service.

2. Outlook will not send a bare RPC over HTTP message due to the possibility of interception in transit. Instead, Outlook establishes an SSL session with the RPC Proxy service. This service is a feature of Windows Server 2003, not of Exchange, so the proxy server does not need to be running Exchange. As part of the SSL handshake, Outlook obtains a session key that it uses to encrypt the contents of the RPC.

3. Outlook sends the encrypted HTTP traffic to the RPC Proxy server using TCP port 443, the SSL port. Think of this as a SSL "tunnel" for the RPC traffic.

4. The RPC Proxy server decrypts the traffic, yielding a stream of HTTP messages that contain an RPC payload. These messages specify the name of the final destination, which in this case is the Exchange server hosting the user's mailbox.

5. The RPC Proxy server establishes an HTTP connection to the Exchange server and uses it to send the RPC over HTTP traffic to the server.

6. The Exchange server extracts the RPC instructions from the HTTP data stream and performs the requested operations.

RPC Limitations for Internet Use

Although RPCs are ideally suited for use in a LAN environment where the machines share a reliable, high-speed data communication path, when you try to use them across slow and unreliable WAN, they have a few distinct disadvantages:

• Sensitive to network latencies. When an RPC process initiates a transaction with its partner, it expects a quick response. If network latency delays the reply longer than a few tenths of a millisecond, the RPC connection could time out. After a few timeouts, RPC applications get fed up and break connections, and you have to restart the application or otherwise refresh the connection.

• EPM no longer available on the Internet. If you lived through the MSBlast worm and its many variants, you're aware that the RPC protocol driver that originally shipped with Windows had several unchecked buffers that made it the target of buffer overflow exploits. These vulnerabilities have long since been patched, but ISPs blocked TCP port 135 at their firewalls and blocked it will stay. It is now virtually impossible to make RPC connections across the Internet.

• Chatty. RPCs make a tacit assumption that bandwidth is highly available. If you look at a packet trace for a typical RPC transaction, you'll see many, many small packets generated as the client and server trade configuration information, status checks, parameter changes, and so forth. Many of these transactions involve synchronous communications, requiring either side to ACK the message before the next transaction can proceed. This causes extremely slow performance if the WAN connections in between get congested.

• Open to inspection. There is nothing inherently secure in the operation of an RPC-based application. Sure, an application developer can decide to encrypt the contents of the messages traded between the RPC client and server, but you cannot guarantee that every application does this. By the same token, RPC transactions perform only rudimentary consistency checking, so it's theoretically possible to modify the contents of an RPC message in transit without the two endpoint processes being aware of the changes. In a nutshell, you cannot guarantee the end-to-end confidentiality or integrity of RPC communications.

RPC over HTTP Resolves Limitations

The RPC Proxy service on Windows Server 2003 alleviates most of these problems. By embedding RPC messages into a standard HTTP stream, it's possible to create a tunnel between an RPC client and server that does not require connection to an Endpoint Mapper and is much more resilient in the face of uncertain network latencies.

When the RPC Proxy service is used to support Outlook, it acts just like an Exchange front-end server in that it passes messages to and from Outlook 2003 and Exchange 2003 without modifying the message contents.

The RPC Proxy service also overcomes the inherent insecurity of RPC transactions by using the encryption and integrity validation features of SSL. This ensures that the user credentials and mailbox content are not exposed over the Internet.

Figure 3

RPC over HTTP is a feature of Windows Server 2003 and XP SP2 or XP SP1 with QFE 331320. (QFE stands for Quick Fix Engineering, a Microsoft term for product updates issued between service pack releases.) You can obtain QFE 331320 by downloading it from Microsoft's Web site. Search for Knowledge-Base article 331320.

There is no general-purpose interface for RPC over HTTP, no "redirector" capable of nabbing standard RPCs and transferring them via HTTP. Both the client application and the server application must know how to construct RPC over HTTP messages.

In the messaging arena, only Outlook 2003 and Exchange 2003 know how to take advantage of RPC over HTTP, so you must first upgrade your messaging infrastructure. This includes Public Folder servers if you want your RPC over HTTP clients to view public folder content across the Internet.

You must run Exchange 2003 on Windows Server 2003 servers with IIS installed and RPC over HTTP enabled. Although you can run Exchange 2003 on Windows 2000, you cannot use RPC over HTTP in that configuration. (Windows 2000 has a rudimentary form of RPC over HTTP but it is too limited for production use and Microsoft has no plans for upgrading it.)

Because Outlook clients interact directly with Global Catalog servers to obtain address lists, you must also upgrade all Global Catalog servers to Windows Server 2003. Standard domain controllers can continue to run Windows 2000, if required.

Also check How Outlook APIs works with Exchange Store + Refer Inside MAPI book

http://msdn2.microsoft.com/en-us/library/aa167881(office.11).aspx