Data Processing Module
The Data Processing Module is a component of the CogNet Controller. This component, written in PHP, is responsible for calling ModifiedTCPDump to extract packet header values from the uploaded capture file, determining the table in the CogNet Repository into which to insert the extracted values, and finally performing this insertion. A new table is created for every month of packet captures.
The module is passed one parameter via the HTTP GET method, a filename. The filename is assumed to contain the name of a capture file located in the /home/cognet directory on the server, the location to which the Nodes upload via FTP. Basic error checking to performed to determine whether the file exists, and then ModifiedTCPDump is executed with the –r (read) option, followed by the full path to the capture file referenced by the filename parameter. This results in the creation of one or more tab-separated values (TSV) files in the /tmp directory. Since the module runs with the permissions of the webserver, this user must, on some systems, be given explicit permission to execute ModifiedTCPDump with root privileges, in which case an entry must be created in /etc/sudoers using the visudo command.
ModifiedTCPDump outputs the names of the TSV files it creates to stdout during execution, and the Data Processing Module in turn parses these filenames to determine the respective database table into which they should be inserted. It then creates a new table if one does not already exist. (The naming of the TSV files is described in greater detail in the following section, dedicated solely to ModifiedTCPDump.)
For each of the TSV files, the Data Processing Module connects to the MySQL database (defined as a constant at the top of the PHP file, along with various file-paths and the login credentials for the MySQL database) and calls the LOAD DATA LOCAL INFILE command to insert the values in the file into the appropriate database table. MySQL automatically interprets the Horizontal Tab (ASCII 0x09) characters in the file as separating columns and the Line Feed (ASCII 0x0A) characters as separating rows. If the file is successfully inserted, the module moves it to /home/cognet/completed and deletes the TSV file in the /tmp directory.
A debug mode is provided that outputs information and error messages. It can be enabled through a Boolean constant defined at the top of the PHP file.
The PHP file can be downloaded here: data_ack.php