C++ TCP/IP IOCP接收的数据有时已损坏-Visual C++；在窗户上

我正在编写一个简单的测试ICOP客户端和服务器，以确保正确使用API，并且服务器正确接收客户端发送的数据。我已经包含了这个问题的所有代码

这就是我遇到一些问题的地方，即接收缓冲区中的数据有时似乎已损坏（有时缓冲区中的数据块可能会无序或丢失）。要明确的是，这是单个接收缓冲区中的数据，我不是说由于线程调度问题，多个缓冲区之间的顺序出现了问题。我之前发布了一个与此相关的问题。不过，我已经做了更多的工作，以获得一个适当的代码示例，所以我张贴了一个新的问题，并将链接到这个。我希望其他人能够运行这段代码并体验同样的怪异行为

测试代码

/************************************************************************
*                                                                       *
*  Test IOCP Client and Server - David Shaw                             *
*                                                                       *
*  There is limited error handling here and it assumes ideal conditions *
*  Some allocated objects are not freed at the end, this is a test only *
*                                                                       *
************************************************************************/

#include "stdafx.h"
#include <iostream>
#include <string>
#include "IOCPTest.h"
#include <Windows.h>

void printUse()
{
    std::cout << "Invalid arguments" << std::endl;
    std::cout << "This test app has very limited error handling or memory management" << std::endl;
    std::cout << "Run as client or server (run the server first) e.g." << std::endl << std::endl;
    std::cout << "To run as server listening on port 3000 with 2 pending receives:" << std::endl;
    std::cout << "> IOCPTester.exe server 3000 2" << std::endl << std::endl;
    std::cout << "To run as client connected to 127.0.0.1 on port 3000 with 2 pending sends:" << std::endl;
    std::cout << "> IOCPTester.exe client 127.0.0.1 3000 2" << std::endl << std::endl;
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

int main(int argc, char *argv[])
{
    if (argc < 4)
    {
        printUse();
        return 0;
    }
    std::string mode(argv[1]);
    if ((mode.compare("client") != 0) && (mode.compare("server") != 0))
    {
        printUse();
        return 0;
    }

    IOCPTest::IOCPConnectionManager *manager = new IOCPTest::IOCPConnectionManager();

    bool server = mode.compare("server") == 0;
    if (server)
    {
        std::string listenPort(argv[2]);
        std::string postedReceiveCount(argv[3]);

        manager->listenPort = atoi(listenPort.c_str());
        manager->postedReceiveCount = atoi(postedReceiveCount.c_str());
        manager->postedSendCount = 1; // Not really used in this mode
        manager->startListening();
    }
    else
    {
        if (argc < 5)
        {
            printUse();
            return 0;
        }

        std::string host(argv[2]);
        std::string port(argv[3]);
        std::string postedSendCount(argv[4]);

        manager->postedReceiveCount = 1; // Not really used in this mode
        manager->postedSendCount = atoi(postedSendCount.c_str());

        IOCPTest::IOCPConnection *connection = manager->createConnection();

        connection->host = host;
        connection->port = atoi(port.c_str());
        connection->connect();
    }
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

测试应用程序可以在两种模式下运行，客户端和服务器。运行服务器，它开始侦听，运行客户端并连接到服务器，一旦连接，它将以允许的速度开始向服务器抛出数据。然后，服务器验证调用WSARecv后从GetQueuedCompletionStatus返回的每个缓冲区中的数据。每次WSASend完成时，我都会将结构的重叠部分归零，并使用原始数据缓冲区再次调用WSASend

客户端发送的每个数据缓冲区都是一个字节序列，一个接一个地递增到指定的最大值。我不发送完整的范围0..255，以防大小以倍数整齐地放入数据包，并以某种方式隐藏问题，因此在我的示例中，代码字节的范围为0..250。对于构造的每个发送缓冲区，我重复该模式numberOfGroups次

这种格式应该意味着我可以有多个WSARecv的未完成，然后验证返回的缓冲区中的数据，完全独立于任何其他缓冲区，这意味着不需要同步或重建顺序。i、 e.我可以从第一个字节开始，验证它们是否依次递增到最大值，然后重置为0。一旦我的测试没有问题，我就可以进入更复杂的领域，对接收到的缓冲区进行排序，并验证更复杂的数据

您可以在命令行上指定同时有多少个未完成的WSASend和WSARecv调用。当有2个或更多未完成的WSARecv调用时，此问题似乎会更频繁地发生。对于1，它可以在偶尔检测到问题之前运行一段时间

我已经在Windows 7上进行测试，使用Visual Studio 2010 C++。 客户端和服务器中同时调用的次数似乎都有影响。将2用于两者似乎比某些组合更容易产生损坏的数据

Sockets和IOCP似乎需要相当多的样板代码才能启动和运行非常基本的客户端和服务器应用程序。接收缓冲区的实际代码只有几行，涉及调用WSARecv和处理来自GetQueuedCompletionStatus的已完成调用

这段代码调用WSARecv

void IOCPConnection::postRecv(PTestOverlapped overlapped)
{
    DWORD numberOfBytesTransferred = 0;
    DWORD flags = 0;
    if (overlapped == nullptr)
    {
        overlapped = new TestOverlapped(receiveBufferSize);
        overlapped->connection = this;
    }
    else
    {
        overlapped->reset();
    }
    overlapped->operation = soRecv;
    auto returnCode = WSARecv(socket, &(overlapped->buffer), 1, &numberOfBytesTransferred, &flags, (LPWSAOVERLAPPED) overlapped, nullptr);
}

当WSARecv调用完成时，它们由工作线程处理——我已经删除了与从这个代码段接收数据无关的行

void IOCPWorker::execute()
{
    bool quit = false;
    DWORD numberOfBytesTransferred = 0;
    ULONG_PTR completionKey = NULL;
    PTestOverlapped overlapped = nullptr;
    while (!quit)
    {
        auto queueResult = GetQueuedCompletionStatus(manager->iocp, &numberOfBytesTransferred, &completionKey, (LPOVERLAPPED *)&overlapped, INFINITE);
        if (queueResult)
        {
            switch (overlapped->operation)
            {
                case soRecv:
                {
                    IOCPConnection *connection = overlapped->connection;
                    connection->onRecv(overlapped, numberOfBytesTransferred); // This method validates the received data

                    connection->postRecv(overlapped);
                    overlapped = nullptr;
                    break;
                }
                default:;
            }
        }
    }
}

对connection->onRecv的调用是我验证数据的地方。这里有什么明显的问题吗

我已经包含了完整的代码供参考，如果你喜欢冒险，应该编译

---

供参考的完整来源

/************************************************************************
*                                                                       *
*  Test IOCP Client and Server - David Shaw                             *
*                                                                       *
*  There is limited error handling here and it assumes ideal conditions *
*  Some allocated objects are not freed at the end, this is a test only *
*                                                                       *
************************************************************************/

#include "stdafx.h"
#include <iostream>
#include <string>
#include "IOCPTest.h"
#include <Windows.h>

void printUse()
{
    std::cout << "Invalid arguments" << std::endl;
    std::cout << "This test app has very limited error handling or memory management" << std::endl;
    std::cout << "Run as client or server (run the server first) e.g." << std::endl << std::endl;
    std::cout << "To run as server listening on port 3000 with 2 pending receives:" << std::endl;
    std::cout << "> IOCPTester.exe server 3000 2" << std::endl << std::endl;
    std::cout << "To run as client connected to 127.0.0.1 on port 3000 with 2 pending sends:" << std::endl;
    std::cout << "> IOCPTester.exe client 127.0.0.1 3000 2" << std::endl << std::endl;
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

int main(int argc, char *argv[])
{
    if (argc < 4)
    {
        printUse();
        return 0;
    }
    std::string mode(argv[1]);
    if ((mode.compare("client") != 0) && (mode.compare("server") != 0))
    {
        printUse();
        return 0;
    }

    IOCPTest::IOCPConnectionManager *manager = new IOCPTest::IOCPConnectionManager();

    bool server = mode.compare("server") == 0;
    if (server)
    {
        std::string listenPort(argv[2]);
        std::string postedReceiveCount(argv[3]);

        manager->listenPort = atoi(listenPort.c_str());
        manager->postedReceiveCount = atoi(postedReceiveCount.c_str());
        manager->postedSendCount = 1; // Not really used in this mode
        manager->startListening();
    }
    else
    {
        if (argc < 5)
        {
            printUse();
            return 0;
        }

        std::string host(argv[2]);
        std::string port(argv[3]);
        std::string postedSendCount(argv[4]);

        manager->postedReceiveCount = 1; // Not really used in this mode
        manager->postedSendCount = atoi(postedSendCount.c_str());

        IOCPTest::IOCPConnection *connection = manager->createConnection();

        connection->host = host;
        connection->port = atoi(port.c_str());
        connection->connect();
    }
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

服务器示例侦听端口3000，最多有2个未完成的WSARecv调用

> IOCPTest.exe server 3000 2

客户端示例连接到端口3000上的127.0.0.1，最多有2个未完成的WSASend调用

> IOCPTest.exe client 127.0.0.1 3000 2

该程序由少量的类组成

IOCPConnectionManager

这个类负责监听连接，并启动工作线程

IOCPConnection

只需跟踪套接字和一些处理异步调用的方法。当WSARecv返回并验证缓冲区中的数据时，将调用IOCPConnection:：onRecv。它只打印一条消息，如果发现数据顺序错误，则返回消息

IOCPWorker

工作线程。IOCPWorker:：execute（）是调用GetQueuedCompletionStatus的位置

TestOverlapped

所需的重叠结构

您还需要为链接器包含Ws2_32.lib和Mswsock.lib

主cpp文件

/************************************************************************
*                                                                       *
*  Test IOCP Client and Server - David Shaw                             *
*                                                                       *
*  There is limited error handling here and it assumes ideal conditions *
*  Some allocated objects are not freed at the end, this is a test only *
*                                                                       *
************************************************************************/

#include "stdafx.h"
#include <iostream>
#include <string>
#include "IOCPTest.h"
#include <Windows.h>

void printUse()
{
    std::cout << "Invalid arguments" << std::endl;
    std::cout << "This test app has very limited error handling or memory management" << std::endl;
    std::cout << "Run as client or server (run the server first) e.g." << std::endl << std::endl;
    std::cout << "To run as server listening on port 3000 with 2 pending receives:" << std::endl;
    std::cout << "> IOCPTester.exe server 3000 2" << std::endl << std::endl;
    std::cout << "To run as client connected to 127.0.0.1 on port 3000 with 2 pending sends:" << std::endl;
    std::cout << "> IOCPTester.exe client 127.0.0.1 3000 2" << std::endl << std::endl;
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

int main(int argc, char *argv[])
{
    if (argc < 4)
    {
        printUse();
        return 0;
    }
    std::string mode(argv[1]);
    if ((mode.compare("client") != 0) && (mode.compare("server") != 0))
    {
        printUse();
        return 0;
    }

    IOCPTest::IOCPConnectionManager *manager = new IOCPTest::IOCPConnectionManager();

    bool server = mode.compare("server") == 0;
    if (server)
    {
        std::string listenPort(argv[2]);
        std::string postedReceiveCount(argv[3]);

        manager->listenPort = atoi(listenPort.c_str());
        manager->postedReceiveCount = atoi(postedReceiveCount.c_str());
        manager->postedSendCount = 1; // Not really used in this mode
        manager->startListening();
    }
    else
    {
        if (argc < 5)
        {
            printUse();
            return 0;
        }

        std::string host(argv[2]);
        std::string port(argv[3]);
        std::string postedSendCount(argv[4]);

        manager->postedReceiveCount = 1; // Not really used in this mode
        manager->postedSendCount = atoi(postedSendCount.c_str());

        IOCPTest::IOCPConnection *connection = manager->createConnection();

        connection->host = host;
        connection->port = atoi(port.c_str());
        connection->connect();
    }
    std::cout << "Hit enter to exit" << std::endl;
    std::cin.ignore();
}

---

我希望这只是我做错的简单事情。在发送之前，我在数据缓冲区上运行了与接收时相同的验证，以确保我没有在那里做傻事，但它通过了检查。我用另一种语言编写了一个服务器，它没有使用IOCP，似乎正确地接收了数据。我还用另一种语言编写了一个客户机，IOCP服务器似乎也能检测到这种情况下的损坏。但也就是说，客户端和服务器可能都有问题。我很感激任何人愿意花时间在这上面。

好的，我可能已经发现了你的问题。如果你看一下你收到的数据，所有的字节都是有序的，但是突然间序列跳了起来，就好像它被另一个调用打断了一样。现在，从和上的MSDN文档：

如果您使用的是I/O完成端口，请注意对WSASend的调用顺序也是填充缓冲区的顺序。不应同时从不同线程在同一套接字上调用WSASend，因为它可能导致不可预测的缓冲区顺序

如果您使用的是I/O完成端口，请注意对WSARecv的调用顺序也是填充缓冲区的顺序。不应同时从不同线程在同一套接字上调用WSARecv，因为它可能导致不可预测的缓冲区顺序

就这样。我现在真的不知道你想要什么的好方法，但你所做的可能不是

Invalid data. Expected: 169; Got: 123
Invalid data. Expected: 114; Got: 89
Invalid data. Expected: 89; Got: 156
Invalid data. Expected: 206; Got: 227
Invalid data. Expected: 125; Got: 54
Invalid data. Expected: 25; Got: 0
Invalid data. Expected: 58; Got: 146
Invalid data. Expected: 33; Got: 167
Invalid data. Expected: 212; Got: 233
Invalid data. Expected: 111; Got: 86
Invalid data. Expected: 86; Got: 153
Invalid data. Expected: 190; Got: 165
Invalid data. Expected: 175; Got: 150
Invalid data. Expected: 150; Got: 217
Invalid data. Expected: 91; Got: 112
Invalid data. Expected: 95; Got: 162
Invalid data. Expected: 207; Got: 182
Invalid data. Expected: 222; Got: 243
Invalid data. Expected: 126; Got: 101
Invalid data. Expected: 157; Got: 132
Invalid data. Expected: 160; Got: 89
Invalid data. Expected: 205; Got: 180
Invalid data. Expected: 113; Got: 134
Invalid data. Expected: 45; Got: 20
Invalid data. Expected: 113; Got: 201
Invalid data. Expected: 64; Got: 198
Invalid data. Expected: 115; Got: 182
Invalid data. Expected: 140; Got: 115