-
Notifications
You must be signed in to change notification settings - Fork 129
NIC Driver
This NIC driver accompanies the contributed NIC project.
The host driver is located in netfpga-10g/contrib-projects/nic/sw/host/driver and it consists of four separate c files (and header files that go along with each of them).
Also available are three sample applications that show how to read/write AXI registers and read stats. Those apps can be found in netfpga-10g/contrib-projects/nic/sw/host/apps.
Top level file for the nic driver. Contains functions that are called when module is loaded/unloaded to initialize/remove PCIe device, as well as allocate/free necessary NIC data structures.
This code creates the /dev/nf10 file that can be used to read/write AXI registers through ioctl calls which are also implemented in this file. Because the fifo that keeps AXI registers reads/writes in hardware is 64 entries deep, and there is no back pressure from that fifo to the PCIe core (this was necessary so DMA requests never get blocked behind AXI requests), the driver implements ad-hoc flow control. There is a special hardware register that indicates the status of the AXI fifo (empty/almost full/full). This register is read by the driver every time there is a potential for overflow of the AXI fifo. Because AXI reads require a response the buffer will always be empty after a read reply comes back. The overflow problem is only with AXI writes, however the driver takes care of it and completely hides it from end users who can use ioctl calls to read/write AXI registers.
This file implements functions that create and present Ethernet interfaces nf0-nf3 to linux. Minimum functionality is implemented here, as interrupt handler and transmit functions are only a wrapper around device specific implementations in nf10priv.c.
These functions control the card tx/rx operation.
nf10priv_xmit -- gets called for every transmitted packet (on any nf interface)
work_handler -- gets called when the interrupt handler puts work on the work queue
nf10priv_send_rx_dsc -- allocates and sends receive descriptors to the NIC
There also exists a LOOPBACK_MODE (enabled by defining constant LOOPBACK_MODE) that allows the driver to be tested on a single machine. This mode, at receive, flips the last bit in the second to last octet of the source and destination IP addresses. (e.g. address 192.168.2.1 is converted to 192.168.3.1 and vice versa).
An example configuration that has been tested with loopback between interfaces 0 and 3 (must add static ARP entries, because ARPs aren't fixed by the LOOPBACK_MODE):
ifconfig nf0 192.168.2.11;
ifconfig nf3 192.168.3.12;
arp -s 192.168.2.12 00:4E:46:31:30:03;
arp -s 192.168.3.11 00:4E:46:31:30:00;
"ping 192.168.2.12" -- should now work with packets going over the wire.
Example C application that shows how to use the driver to read AXI registers from the NetFPGA card.
Usage example: ./rdaxi 0x7d400000
./wraxi 0x7d400000 0xa5a5
NIC project contains 8 AXI registers that can be read/written for test purposes (they have no effect on the NIC). Check hw/system.mhs file to find the the AXI addresses (under dma section look for C_BASEADDR; the register addresses are C_BASEADDR...C_BASEADDR+7)
Example C application that shows how to read stats off of the NetFPGA card. All counters are 32 bit wide (64bit counters created some timing closure issues), even though 8 bytes of address space are reserved for each counter. Lower 128 counters are reserved, and the reset are listed here. (For more details look at DMA SystemVerilog source code file stats.v)
128+ 1: pcie rx number of 8-byte words
128+ 2: pcie rx number of writes
128+ 3: pcie rx number of reads
128+ 4: pcie rx number of completions
128+ 5: pcie rx number of errors
128+ 8: pcie tx number of 8-byte words
128+ 9: pcie tx number of writes
128+10: pcie tx number of reads
128+11: pcie tx number of completions
128+12: pcie tx number of errors
128+17: mac tx number of 8-byte words
128+18: mac tx number of packets
128+21: mac rx number of 8-byte words
128+22: mac rx number of packets
128+23: mac rx number of errors
NOTE: reading the stats is a PCIe event in itself and that itself will affect the PCIe stats.