译者：胡章优
日期：2007-12-20
标题E：Windows Sockets 2.0: Write Scalable Winsock Apps Using Completion Ports
标题C：使用完成端口开发扩展winsock应用（程序）
关键字：IOCP Server windows socket 服务器 完成端口 服务器开发 服务器开发设计 服务器设计开发 服务器设计
说明：部分英文不需要翻译的，引用原文，版权属于微软，但是翻译由笔者完成，欢迎转载，必须著名译者。

定义：
A completion port is a queue into which the operating system puts notifications of completed overlapped I/O requests. Once the operation completes, a notification is sent to a worker thread that can process the result. A socket may be associated with a completion port at any point after creation.

翻译：完成端口是一个由操作系统来通知完成重叠端口请求的queue,一旦操作完成，一个通知事件将发送到worker thread，这个workerthread能够处理结果。在一个socket创建以后，应该关联一个完成端口。

说明：
Typically an application will also create a number of worker threads to process these notifications. The number of worker threads depends on the specific needs of the application. The ideal number is one per processor, but that implies that none of these threads should execute a blocking operation such as a synchronous read/write or a wait on an event. Each thread is given a certain amount of CPU time, known as the quantum, for which it can execute before another thread is allowed to grab a time slice. If a thread performs a blocking operation, the operating system will throw away its unused time slice and let other threads execute instead. Thus, the first thread has not fully utilized its quantum, and the application should therefore have other threads ready to run and utilize that time slot.

翻译：一般来说，应用程序同时需创建一定数量的worker threads来处理这些通知事件。Worker Threads的数量由应用程序特殊的应用所决定。理想的数量是一个处理器一条线程。[见注一]。 但是这意味着任何一个这种线程也不执行一个阻塞的操作，比如同步的read/write，或者等待一个事件。任何一个线程给了一定数量的CPU事件片，称为quantum(量子),她能在另外一个线程允许抢去一个时间片之前处理(翻译可能有所不通，请谅解)。
当某线程处理一个阻塞的操作时，操作系统将会throw away 目前不使用的事件片，让其他线程处理工作。因此，第一条线程没有全部用完她的时间片，应用程序因此应该使用其他线程准备执行同时利用这个时间片。

使用：
Using a completion port is a two-step process. First, the completion port is created, as shown in the following code:

翻译：使用完成端口需要两步，第一，创建完成端口，如下代码：
HANDLE    hIocp; hIocp = CreateIoCompletionPort(     INVALID_HANDLE_VALUE,     NULL,     (ULONG_PTR)0,     0); if (hIocp == NULL) {     // Error }

使用：
      Once the completion port is created, each socket that wants to use the completion port must be associated with it. This is done by calling CreateIoCompletionPort again, this time setting the first parameter, FileHandle, to the socket handle to be associated, and setting ExistingCompletionPort to the handle of the completion port you just created.
      The following code creates a socket and associates it with the completion port created earlier:
翻译：一旦完成端口被创建，任何想使用完成端口的socket必须关联，这可以通过再次调用CreateIoCompletionPort 完成。通过设置FileHandle参数，同时设置ExistingCompletionPort 参数为上一步创建的完成端口句柄。
SOCKET    s; s = socket(AF_INET, SOCK_STREAM, 0); if (s == INVALID_SOCKET) {     // Error if (CreateIoCompletionPort((HANDLE)s,                            hIocp,                            (ULONG_PTR)0,                            0) == NULL) { // Error } ••• }


相关说明：At this point, the socket s is associated with the completion port. Any overlapped operations performed on the socket will use the completion port for notification. Note that the third parameter of CreateIoCompletionPort[见注二] allows a completion key to be specified along with the socket handle to be associated. This can be used to pass context information that is associated with the socket. Each time a completion notification arrives, this context information can be retrieved.
      Once the completion port has been created and sockets have been associated with it, one or more threads are needed to process the completion notifications. Each thread will sit in a loop that calls GetQueuedCompletionStatus each time through and returns completion notifications.
      Before illustrating what a typical worker thread looks like, we need to address the ways in which an application keeps track of its overlapped operations. When an overlapped call is made, a pointer to an overlapped structure is passed as a parameter. GetQueuedCompletionStatus will return the same pointer when the operation completes. With this structure alone, however, an application can't tell which operation just completed. In order to keep track of the operations that have completed, it's useful to define your own OVERLAPPED structure that contains any extra information about each operation queued to the completion port (see Figure 1).

翻译：现在，Socket已经关联了完成端口。任何在这个socket上的重叠操作将通知使用这个完成端口。说明CreateIoCompletionPort的第三个参数允许a completion key特殊关联socket handle 。这可以用来传递关联Socket的context information ，每次完成端口通知来临，这个context information 将被带回。

一旦完成端口被创建同时已经关联，一个或者更多的线程需要去处理完成通知事件。每个线程将在一个调用GetQueuedCompletionStatus 的死循环中，同时返回完成通知事件。
在说明Worker thread 之前，应该说一下重叠操作。当一个重叠操作调用了，一个指向重叠结构的指针将被当作一个参数传递出来，
GetQueuedCompletionStatus 将会在操作 完成以后返回这个指针。应用程序无法获悉哪个操作是否完成，为了知道哪个操作已经完成，定义自己的OVERLAPPED结构包含关于每个操作完成端口的queue附加信息是有必要的。
图一的代码如下：
typedef struct _OVERLAPPEDPLUS {     OVERLAPPED        ol;     SOCKET            s, sclient;     int               OpCode;     WSABUF            wbuf;     DWORD             dwBytes, dwFlags;     // other useful information } OVERLAPPEDPLUS; #define OP_READ     0 #define OP_WRITE    1 #define OP_ACCEPT   2

相关说明：Whenever an overlapped operation is performed, an OVERLAPPEDPLUS structure is passed as the lpOverlapped parameter (as in WSASend, WSARecv, and so on). This allows you to set operation state information for each overlapped call. When the operation completes, the OVERLAPPED pointer returned from GetQueuedCompletionStatus will now point to your extended structure. Note that the OVERLAPPED field within the extended structure does not necessarily have to be the first field. After the pointer to the OVERLAPPED structure is returned, the CONTAINING_RECORD macro can be used to obtain a pointer to the extended structure.
      Take a look at the example worker thread in Figure 2. The PerHandleKey variable will return anything that was passed as the CompletionKey parameter to CreateIoCompletionPort when associating a given socket handle. The Overlap parameter returns a pointer to the OVERLAPPEDPLUS structure that is used to initiate the overlapped operation. Keep in mind that if an overlapped operation fails immediately (that is, returns SOCKET_ERROR and the error is not WSA_IO_PENDING), then no completion notification will be posted to the queue. Alternately, if the overlapped call succeeds or fails with WSA_IO_PENDING, a completion event will always be posted to the completion port.
  

翻译：当一个重叠操作执行完毕，一个OVERLAPPEDPLUS 结构将被通过lpOverlapped 参数传递出来（比如WSASend, WSARecv），
这允许你为每个重叠调用设置操作状态信息。当操作完成以后，OVERLAPPED 指针从GetQueuedCompletionStatus 调用中返回，并指向你的扩展结构。注意OVERLAPPED 域并不必须在扩展结构的第一个域。当指向OVERLAPPED 的指针返回了，CONTAINING_RECORD 宏可以用来获取指向这个扩展结构的指针。

图二的代码如下：
DWORD WINAPI WorkerThread(LPVOID lpParam) {         ULONG_PTR       *PerHandleKey;     OVERLAPPED      *Overlap;     OVERLAPPEDPLUS  *OverlapPlus,                     *newolp;     DWORD           dwBytesXfered;     while (1)     {         ret = GetQueuedCompletionStatus(             hIocp,             &dwBytesXfered,             (PULONG_PTR)&PerHandleKey,             &Overlap,             INFINITE);         if (ret == 0)         {             // Operation failed             continue;         }         OverlapPlus = CONTAINING_RECORD(Overlap, OVERLAPPEDPLUS, ol);          switch (OverlapPlus->OpCode)     {     case OP_ACCEPT:         // Client socket is contained in OverlapPlus.sclient         // Add client to completion port             CreateIoCompletionPort(                 (HANDLE)OverlapPlus->sclient,                 hIocp,                 (ULONG_PTR)0,                 0);         //  Need a new OVERLAPPEDPLUS structure         //  for the newly accepted socket. Perhaps         //  keep a look aside list of free structures.         newolp = AllocateOverlappedPlus();         if (!newolp)         {             // Error         }         newolp->s = OverlapPlus->sclient;         newolp->OpCode = OP_READ;         // This function prepares the data to be sent         PrepareSendBuffer(&newolp->wbuf);            ret = WSASend(                 newolp->s,                 &newolp->wbuf,                 1,                 &newolp->dwBytes,                 0,                 &newolp.ol,                 NULL);                  if (ret == SOCKET_ERROR)         {             if (WSAGetLastError() != WSA_IO_PENDING)             {             // Error             }         }         // Put structure in look aside list for later use         FreeOverlappedPlus(OverlapPlus);         // Signal accept thread to issue another AcceptEx         SetEvent(hAcceptThread);         break;     case OP_READ:         // Process the data read             // •••         // Repost the read if necessary, reusing the same         // receive buffer as before         memset(&OverlapPlus->ol, 0, sizeof(OVERLAPPED));         ret = WSARecv(               OverlapPlus->s,               &OverlapPlus->wbuf,               1,               &OverlapPlus->dwBytes,               &OverlapPlus->dwFlags,               &OverlapPlus->ol,               NULL);         if (ret == SOCKET_ERROR)         {             if (WSAGetLastError() != WSA_IO_PENDING)             {                 // Error             }         }         break;     case OP_WRITE:         // Process the data sent, etc.         break;     } // switch     } // while }  // WorkerThread
PerHandleKey变量将会返回任何传入为CompletionKey的参数，该参数为通过CreateIoCompletionPort关联socket句柄而设置的。Overlap参数返回一个指向OVERLAPPEDPLUS结构的指针，记住如果一个重叠操作日记失败（返回SOCKET_ERROR，同时错误不是WSA_IO_PENDING），将没有完成通知传递到queue,相反地，如果重叠操作调用成功或者失败返回WSA_IO_PENDING，完成事件将被传递到完成端口


注一：网路上流传的数量众说纷纭，有的说2*processors，还有的说2*processors +2，一些朋友的讨论时候，发现由于单CPU的时候，系统始终应该是串行的，而并不是并行的，所以微软所认为的一条线程一个CPU应该是正确的，同时微软也说了，这个应该视特殊的应用决定。
注二：
HANDLE WINAPI CreateIoCompletionPort(   __in          HANDLE FileHandle,   __in          HANDLE ExistingCompletionPort,   __in          ULONG_PTR CompletionKey,   __in          DWORD NumberOfConcurrentThreads );