MonitoringSystem
-- Main.GianniAntichi - 02 Aug 2011
This is a High Performance Passive Monitoring Platform for NetFPGA providing accurate timestamping. NetFPGA takes care of filtering the traffic of interest (i.e., classifying the flows based on the 5-tuple) and takes the timestamp of the packets in a nanosecond accurancy. Only the flows that match the rules are passed to the CPU-HOST. All the traffic received is restransmitted out of the board so it can be used as a bump in the wire monitoring platform.
- Status :
- Version :
- Authors :
- Design :
- NetFPGA source :
- Select traffic of interest (5-tuple) using up to 32 different rules.
- Accurate timestamping (lose less than one millisecond in 6 hours with respect DAG 4.3ge SX).
- NO NEED for additional hardware.
- Take statistics from the traffic of interest.
- NAGIOS plugin provided.
TIMESTAMPING MODULE:
The timestamping module was added before the RX Queues in order to timestamp the incoming packets as they are received by the hardware. In this way, it is possible to obtain the lowest possible jitter - lower than if the timestamp were sampled after the kernel receives the packet from the hardware.
We have also inserted two registers that pass to the CPU the timestamp of the packets dropped in the input queues (a packet could be dropped if the associated input queue is full or if a bad Ethernet CRC is received).
We avoid to use as reference clock a GPS or a CDMA receiver to correct the oscillator drift. At the same time we avoid to use the HostPC because of the reference clock would be affected by inaccurancies introduced by the interrupts that are not necessarily handled as soon as they arrive. In this way we do not leverage the low-cost NetFPGA hardware with extra hardware, obtaining at the same an easy--to--use system (just download and run the gatware is needed). The correction is made using another clock of the FGPA.
FILTER MODULE:
All packets received are retransmitted. Packets that match one of up to 32 filter rules are also copied verbatim, with their timestamp prepended, to the host. The filter module is created by means of two TCAMs.
CLI:
We provide auxiliary command-line tool for on--the--fly TCAM rule management, and the initialisation of the hardware timestamp using the value returned by the NTP system call.
Our CLI (Command Line Interface) can lists the rules set in hardware, insert new rules, load a set of rules from a file or clear a specified rule.
A rule is presented as the 5-tuple with the associated masks for each field.
NANOSECOND LIBPCAP and KERNEL MODULE:
libpcap is the de facto standard capture API but, without any change in the NetFPGA kernel module, libpcap applications cannot yet directly be used with our monitoring solution. Simple packet recorders should work, but the 8-byte timestamp prepended to each packet will confound protocol analysis. For this reason we changed the NetFPGA kernel module running on the PC-Host in order to remove the timestamp prepended and store the value in the related structure in the sk--buff. In order to access nanosecond granularity timestamps, we modified libpcap to return struct timespecs instead of struct timevals.
STATISTIC COLLECTOR:
A multi--threaded software collects statistics about the received traffic of interest and, if required by the user, stores the received traffic in a PCAP--compatible file with nano--second resolution for later analysis.
The software is able to collect:
- number of packets received
- number of byte received
- number of IP packets
- number of non-IP packets
- number of UDP packets
- number of TCP packets
- number of SYN,FIN,RST received
- number of TCP options received
- mean interarrival time
- mean bit-rate.
- First of all compile the new Kernel driver for the NetFPGA
- Go in /root/netfpga/lib/kernel
- Update the nf2_control.c file with the one provided and run make
- Remove the old kernel driver (rmmod nf2)
- Insert the new one (insmod nf2.ko)
- Install the provided nanoseconds libpcap
- Go in monitoring_system/sw
- Apply the provided patch to the libpcap-1.0.0
- Donwload the bitfile of the Monitoring System
- If needed, synchronize the timestamp module with the NTP server
- Make sure that the command "ntptime" from terminal does not return any error.
- Go in monitoring_system/sw/ntp_synchronization
- Run make
- Run SyncMonitoring.sh
- Run the CLI to select the traffic of interest
- Go in monitoring_system/sw/cli/
- Run make
- ./monitoring
- options are vailable. Type "help" to have the list of all the available commands.
- Run the Statistic Collector
- Go in monitoring_system/sw/monitoring_system_sw
- Make sure that in monitoring_system.c you have included the correct path to the new nanosecond libpcap
- run make
- Select the interface to monitor through the '-i' option
- If you want that NAGIOS display the statistics run the '-N' option (the software will become a daemon). Otherwise you can choose to store the received packets in a PCAP nanosecond file and you will see the statistics during the passing of the time on the terminal.
- Install NAGIOS on your system
- Compile the C file in the req_statistic directory and put it in /usr/lib/nagios/plugins/local
- Copy the check_statistic.cfg in /etc/nagios-plugins/config
- Update the file /etc/nagios3/conf.d/services_nagios2.cfg with the one provided in this directory.
- Restart NAGIOS (/etc/init.d/nagios3 restart)
- You should be able to see the statistics as NAGIOS services.
Sources for the project can be downloaded from netfpga_MonitoringSystem_2_1.tar.gz
The original tcpdump shows the time in microsecond resolution. We provided a modified version of TCPdump in order to exploit the nanosecond granularity provided by our libpcap patch.
We have seen that tcpdump have the "-o" feature free. So we inserted the "-o" as option an then a word for the option: in this case we use the word "nano".
In this way we can decide the granularity of the time of the interface we want to monitor:
example: tcpdump -i eth0 - o nano tcpdump -i eth0
With the first example the software will show a nanosecond granularity (newer kernel support nanosecond granularity of the timestamp). The second example will work as the common TCPdump, showing the timestamp in microsecond granularity.
Sources for the patch can be downloaded from tcpdump.patch
A first part of the work is described in:
- Antichi G., Miller D., Moore A., Giordano S., "An Open Source Hardware Module for High-Speed Network Monitoring on NetFPGA", European NetFPGA Developers Workshop 2010, Cambridge, UK, 9-10 Sept
Gianni Antichi antichi.gianni<at>gmail.com
http://netgroup.iet.unipi.it/people/gantichi.shtml
Andrew W. Moore andrew.moore<at>cl.cam.ac.uk