Not able to send 10 Gbps

[biz98]

I was developing a simulator for a sensor that I am working with. I already had it made Linux. I was using AF_PACKET socket and I was able to achieve 10Gbps data transmission. I started with the same approach on windows I was using Winsock but then I realised they don't allow me to create AF_PACKET socket. The sensor output is a custom ethernet II packet. which I cannot share. I am sending Jumbo packets. I cannot share my code also. But I can share the code that I used to benchmark pcap. Please tell me is there anything I can do to increase the data rate. I am not that familiar with windows so if I can change something in windows settings that can help boostup the datarate, share that also. The base data rate I am able to acheive with this is 5476.43 Mbps. The server I am testing this in has close to 400GB ram and 48 CPU cores. I don't care about efficiency I just need the best datarate.

#include <pcap.h>
#include
#include
#include

void sendLargePackets(const uint8_t* interfaceMAC, const char* interfaceName) {
char errbuf[PCAP_ERRBUF_SIZE];

pcap_t* handle = pcap_create(interfaceName, errbuf);
if (handle == nullptr) {
    std::cerr << "Unable to create the adapter: " << errbuf << std::endl;
    return;
}

if (pcap_set_buffer_size(handle, 10485760) != 0) {
    std::cerr << "Error setting buffer size: " << pcap_geterr(handle) << std::endl;
    pcap_close(handle);
    return;
}

if (pcap_activate(handle) != 0) {
    std::cerr << "Error activating pcap handle: " << pcap_geterr(handle) << std::endl;
    pcap_close(handle);
    return;
}

if (pcap_setmintocopy(handle, 8192) != 0) {
    std::cerr << "Error setting minimum to copy: " << pcap_geterr(handle) << std::endl;
    pcap_close(handle);
    return;
}


const int payload_size = 8002;
const int packet_size = 14 + payload_size;  // 14 bytes Ethernet header + payload
uint8_t* packet = new uint8_t[packet_size];

uint8_t dest_mac[6] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
uint8_t src_mac[6];
std::memcpy(src_mac, interfaceMAC, 6);
uint8_t ether_type[2] = { 0x12, 0x34 };

std::memcpy(packet, dest_mac, 6);
std::memcpy(packet + 6, src_mac, 6);
std::memcpy(packet + 12, ether_type, 2);
std::memset(packet + 14, 0x00, payload_size);

struct pcap_pkthdr header;
header.caplen = packet_size;
header.len = packet_size;
header.ts.tv_sec = 0;
header.ts.tv_usec = 0;

auto start = std::chrono::high_resolution_clock::now();

const int num_packets = 1248000;

for (int j = 0; j < 4000; j++) {

    pcap_send_queue* sendQueue = pcap_sendqueue_alloc((unsigned)1024 * 1024 * 1024 * 2);

    for (int i = 0; i < 312; ++i) {
        /*if (pcap_sendpacket(handle, packet, packet_size) != 0) {
            std::cerr << "Error sending the packet: " << pcap_geterr(handle) << std::endl;
            break;
        }*/
        int queueResult = pcap_sendqueue_queue(sendQueue, &header, packet);
        if (queueResult != 0) {
            std::cerr << "Error queueing bthe packet: " << pcap_geterr(handle) << std::endl;
            break;
        }
    }

    if (pcap_sendqueue_transmit(handle, sendQueue, 0) != sendQueue->len) {
        std::cerr << "Error transmitting the queue: " << pcap_geterr(handle) << std::endl;
    }

    pcap_sendqueue_destroy(sendQueue);
}

auto end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double> elapsed = end - start;
std::cout << "Sent " << num_packets << " packets in " << elapsed.count() << " seconds." << std::endl;

// Calculate the data rate
double total_data = static_cast<double>(num_packets) * packet_size * 8;  // in bits
double data_rate = total_data / (elapsed.count() * 1e6);  // in Mbps
std::cout << "Data rate: " << data_rate << " Mbps" << std::endl;

delete[] packet;
pcap_close(handle);

}

int main() {
uint8_t interfaceMAC[6] = {0x00, 0x0c, 0x29, 0x4f, 0x8e, 0x35}
char interfaceName[256] = "\Device\NPF_{random}";

sendLargePackets(interfaceMAC, interfaceName);

return 0;

}

[biz98]

Can someone at least confirm if it is possible to send 10 Gbps data or not?

[dmiller-nmap]

Your timer is measuring the time it took you to create and destroy the packet queue, not only how long it took to send. Move the start and end times to just before and just after the call to pcap_sendqueue_transmit() for a more accurate measurement. With so many cores available, you will likely get higher throughput if you have separate threads for creating and sending the packets, otherwise the packets will transmit in a burst, followed by a wait for the next batch of packets to be queued.

Also, if you are not intending on capturing any of these packets on the same handle, set a buffer size of 0 to avoid doing any work related to queueing packets for capture.