Automating Security Control Compliance Assessment with Sagacity MATTHEW SHUTER, University of Colorado, Colorado Springs Conducting vulnerability assessments on Department of Defense systems is subject to numerous levels of requirements to assess for compliance. We help to develop, test, and stabilize the open source tool Sagacity, which aims to automate much of the effort associated with correlating data from multiple vulnerability assessment tools, vulnerability databases, and compliance frameworks. We extend Sagacity to include the capability to map findings to NIST 800-53 security controls and identified 55 bugs in Sagacity, remediating 13 of them. 1 Introduction The US Department of Defense (DoD) considers the security of information systems to be of paramount importance to national security. As such, the DoD mandates compliance with DoD Instruction (DoDI) 8510.01, which, among other requirements, directs the use of the Risk Management Framework (RMF) and the Committee for National Security Systems Instruction (CNSSI) 1253 to select, implement, evaluate, and monitor compliance with the NIST 800-53 security controls (hereafter, “NIST Controls”) [1, 2, 3]. Furthermore, it directs implementation of the DoD Information Systems Agency’s (DISA) Security Requirements Guides (SRGs) and Security Technical Implementation Guides (STIGs), hereafter collectively referred to as Security Guides (SGs) for brevity unless more specificity is needed [4]. The implementation of RMF has proved to provide an overwhelming quantity of data to manage for DoD programs, with the 800-53 controls being decomposed into a minimum of 1,376 Control Correlation Identifiers (CCIs) – a number which only goes up depending
Transcript
Automating Security Control Compliance Assessment with Sagacity
MATTHEW SHUTER, University of Colorado, Colorado Springs
Conducting vulnerability assessments on Department of Defense systems is subject to numerous
levels of requirements to assess for compliance. We help to develop, test, and stabilize the open
source tool Sagacity, which aims to automate much of the effort associated with correlating data from
multiple vulnerability assessment tools, vulnerability databases, and compliance frameworks. We
extend Sagacity to include the capability to map findings to NIST 800-53 security controls and
identified 55 bugs in Sagacity, remediating 13 of them.
1 IntroductionThe US Department of Defense (DoD) considers the security of information systems to be of
paramount importance to national security. As such, the DoD mandates compliance with DoD
Instruction (DoDI) 8510.01, which, among other requirements, directs the use of the Risk Management
Framework (RMF) and the Committee for National Security Systems Instruction (CNSSI) 1253 to select,
implement, evaluate, and monitor compliance with the NIST 800-53 security controls (hereafter, “NIST
Controls”) [1, 2, 3]. Furthermore, it directs implementation of the DoD Information Systems Agency’s
Provided code comments for developers and documentation for users of Sagacity.
2 Analysis and DesignThe architecture of Sagacity is depicted in Figure 2-1. This shows how administrative command line
scripts are used to import open source vulnerability data in a variety of formats to build the database.
Once the database is built, users may begin using the web interface to import the various data types
generated by vulnerability assessments. All data types are normalized and correlated before persisting
into the database. Finally, checklist files can be exported for analysis and in the future, compliance
reports will be exported as well.
Additionally, Figure 2-1 shows the portions of Sagacity that will be modified to implement the RMF
control mapping. New capabilities are depicted in blue, while modified capabilities are depicted in
green. Details of how these capabilities will be modified are described futher in the remainder of this
section.
Figure 2-2 - Sagacity Architecture
2.1New RequirementsShortly after the initial pre-beta release of Sagacity (version 1.2) the DoD migrated from the DIACAP
framework to the Risk Management Framework (RMF). In light of this change, Sagacity must be
adapted such that export and reporting functions are capable of correlating scanner and checklist
findings to NIST SP 800-53 controls (hereafter “NIST controls”) instead of legacy DIACAP controls. A
principle objective of this project is to lay the foundation for RMF compliance assessment by
implementing the following two new requirements:
1. Import eMASS Control Correlation Identifier (CCI) to NIST control mapping spreadsheet. eMASS
is used by agencies to report compliance to the Authorizing Official for approval. It has a
unique control numbering scheme which is slightly different than that implemented by the CCI
XML or the NIST SP 800-53 publications themselves. The control mapping spreadsheet maps
the NIST controls to the CCIs that correlate to STIG findings.
2. Export eChecklists with eMASS/NIST SP800-53 security control identifiers to facilitate analysis
and reporting of control compliance.
Figure 2-2 is an example of an eChecklists before modification of the export algorithm to map the CCIs
in the STIGs to the NIST SP800-53 controls. Our objective is to replace the CCIs in column D (“IA
Controls”) with the NIST control identifier so that non-compliant STIG settings can be used to identify
their associated non-compliant control(s).
Figure 2-3 - CCIs in eChecklist Exports
2.2Database Schema Modifications
To store the mapping of NIST controls in the database, we need to identify a reasonable place to
store the mapping. In analyzing the database schema, we see that there is an existing table named
rmf.cci which holds the CCIs. This table is intended to be populated from the DISA-provided, XML-
formatted CCI list which maps NIST controls to CCIs. Since CCIs are intended to be atomic and the
eMASS CCI list is meant to provide a one-to-one mapping of subcontrols to CCIs, this initially seemed
like a perfect place to store the eMASS control identifier.
However, upon attempting to update the rmf.cci table from the DISA CCI list, we discovered that
there is an issue with using this table. Specifically, the parser for the CCI list, parse_cci.php was written
in 2014 and DISA has since changed the schema of the CCI list XML, therefore it is not currently
functional. Although the table in the database baseline is currently populated in the baseline from an
older, compatible version of the CCI file, with no feasible way of making regular updates it seemed
unwise to use the table to store the eMASS control identifiers until CCI import is working again. Upon
first importing, it was confirmed that this was a wise decision, as the rmf.cci table contained 2,734
records, while the emass_cci table contained 2,921 records, confirming that there would have been
unpopulated CCI records created as a result of importing the eMASS control map, with no effective way
to update them.
Therefore, we chose to read in the eMASS CCI table to a new table named rmf.emass_cci. The table
contains two fields, id and control. The id field is used to correlate the controls to the CCI specified in
STIG checklists. The control field is used to export NIST controls that are associated with STIG checks
for compliance reporting as required by eMASS.
2.3Required New CodeTo implement the new requirements, there are three primary components which must be created:
1. Create a parser to read the eMASS control to CCI mapping spreadsheet
2. Create a function in the database class to save the records in the table
3. Create a function in the database class to retrieve the records from the table
In Sagacity, parsers of the various data formats required to build the initial database are
implemented in command-line scripts contained in the <wwwroot>/exec directory. By convention,
they are named parse_<file_type>.php. In keeping with this convention, we will implement the parser
in a script titled parse_emass_control_map.php. Sagacity is using the PHPOffice library for interfacing
with Microsoft Excel, so this script will use the same library to read in the eMASS-to-CCI map file,
iterating over rows in the spreadsheet and saving them to the database table. Table 2-1 shows an
excerpt of the eMASS-to-CCI map file to indicate its structure.
Table 2-1 - Exceprt of eMASS-to-CCI Mapping Spreadsheet
AC-1.1 CCI-
002107
AC-1.2 CCI-
002108
AC-1.3 CCI-
000001
AC-1.4 CCI-
000002
AC-1.5 CCI-
000004
AC-1.6 CCI-
000005
AC-1.7 CCI-
000003
AC-1.8 CCI-
001545
AC-1.9 CCI-
000006
AC-1.10 CCI-
001546
AC-2.1 CCI-
002110
AC-2.2 CCI-
002111
AC-2.3 CCI-
002112
AC-2.4 CCI-
000008
AC-2.5 CCI-
002113
AC-2.6 CCI-
002115
AC-2.7 CCI-
002116
AC-2.8 CCI-
002117
AC-2.9 CCI-
002118
AC-2.10 CCI-
002119
AC-2.11 CCI-
000010
AC-2.12 CCI-
002120
… …
All of Sagacity’s interaction with the database is implemented in the ‘db’ class in database.inc.
Functions in this class follow the convention get_<Table_Name> and save_<Table_Name> for the
functions that save complete records. Again, adhering to convention, we name our functions
save_EMASS_CCIs() and get_EMASS_CCIs() functions. It should be noted that although there are
already functions for save_CCIs() and get_CCIs(), these functions are intended for use in the
unsupported parse_CCI.php script previously mentioned. Because of the decision create a new table,
there is no conflict with function names.
2.4Required Code ModificationsIn addition to creating the parsing script and database interface functions, existing functionality
must be modified to enable Sagacity to export the NIST controls made accessible through the new
database table in lieu of the CCIs it was previously exporting as references for the STIG settings.
Sagacity currently exports the CCIs to a column in eChecklists at two different points. The first is
when STIG checklists are imported, and the second is when eChecklists are exported. Upon importing
STIG checklists, the parse_stig.php script creates CSV-formatted eChecklist templates in the
<wwwroot>/reference/stigs directory. These CSVs can then be used during the assessment to quickly
create eChecklists for evaluation of hosts and platforms that are not automatically populated based on
scan data (in lieu of using the interface to create targets, assign checklists, and export them). Listing
2-1 is an example of the XML being parsed by the parse_stig.php script, and following that is Listing 2-
2 which shows the relevant sections of code that extracts the CCIs and writes them to the CSV.
Listing 2-1 - CCI Reference in STIG XML
Note the tag <ident system="http://iase.disa.mil/cci">CCI-000366</ident>. This is the CCI value
that we are looking for to enable correlation to the NIST controls.
<Group id="V-1115"><title>Rename Built-in Administrator Account</title><description><GroupDescription></GroupDescription></description><Rule id="SV-25023r2_rule" weight="10.0" severity="medium"> <version>4.022</version> <title>The built-in administrator account must be renamed.</title> <description> <VulnDiscussion>The built-in administrator account is a well-known account subject to attack. Renaming this account to an unidentified name improves the protection of this account and the system.</VulnDiscussion>… </description> <ident system="http://cce.mitre.org">CCE-8484-8</ident> <ident system="http://iase.disa.mil/cci">CCI-000366</ident>…</Rule></Group>
Listing 2-2 - CCI Processing Code
In the above code, the $groups variable is the collection of <Group> DOMElement nodes of the XML
STIG checklist. The loop simply iterates over each <Group> element, capturing the data that is desired
for storage in the database. In addition to saving the data in the database, select data is assembled
into a CSV-formatted eChecklists as well, which is written in the call to fputcsv() in the final line of code
above. To make this function write controls as opposed to CCIs, we must modify the $ias variable so
that it is populated with the correlated controls from the rmf.emass_cci table.
The second place that Sagacity currently exports the CCIs is when exporting eChecklists via the
export.php script. This script behaves in much the same way, however instead of creating a CSV file
from the STIG XML, it retrieves the findings from the database and writes them to an Excel eChecklist.
Listing 2-3 shows this in action. In this code, like before, we must replace the $ia_controls_string
variable with a list of NIST controls rather than a list of CCIs.
// Get the collection of STIG rules i.e., <Group> elements$groups = getValue($xml, '/x:Benchmark/x:Group', null, true);…foreach ($groups as $group) {… // Get the Rule DOMElement $group_rule = getValue($xml, 'x:Rule', $group, true)->item(0);… // Get the ident DOMElement $rule_ident = getValue($xml, ".//x:ident", $group_rule, true); // Loop over ident elements, looking for CCIs for ($x = 0; $x < $rule_ident->length; $x++){ // If text looks like a CCI, save it if (substr($rule_ident->item($x)->textContent, 0, 3) == 'CCI') { $split_ia = explode("-", $rule_ident->item($x)->textContent); if (isset($split_ia[0]) && isset($split_ia[1])) { // Add cci string to the array that will be inserted into the csv $ias[] = "{$split_ia[0]}-{$split_ia[1]}"; // Add cci to the array that will be inserted into the ia_controls table $new_controls[] = array( $pdi_id, $split_ia[0], $split_ia[1] ); } }… // Insert the all the STIG check attributes into the STIG csv. fputcsv($csv, array($rule_stig_id, $vms_id, implode("", array_fill(0, $cat, "I")), implode(" ", $ias), $rule_title, "Not Reviewed", "", $rule_check_content, $sv_rule, $oval_id));}
Listing 2-3 - CCI Exporting Code
Finally, to enable testing of the new RMF export functionality without breaking current functionality,
we will want to wrap all modified code in a block which executes conditionally depending on the
setting of a Boolean global variable which we will call “do_rmf”.
3 Implementation3.1Database Schema Implementation
Upon installation, Sagacity uses the install.php script to read in the db_schema.xml file, which it
uses to create the baseline database schema. Therefore, to add our table to the schema, we modified
the file to include an XML description of the table, its fields, constraints, and attributes.
We chose the ‘id’ field representing the CCI identifier as the primary key. Since CCI identifiers are
fixed length and all begin with the string “CCI-“ followed by a six-digit unique integer, we chose the
mediumint data type to efficiently store the ID. The “CCI-“ string and leading zeroes will be prepended
in code where CCIs are used. For example, the first CCI in the imported file is “CCI-002107”, which will
be stored as a mediumint value 2107.
The second field named ‘control’ is used to represent the NIST control identifier used by the eMASS
database. These identifiers are alphanumeric and consist of a two-character control family, followed by
a dash and control number, optional parenthetical control enhancement number, and finally a decimal
CCI identifier to uniquely identify each atomic CCI element of the control. An example control ID with
no enhancement would be “AC-1.1.” This tells us this is the first control in the Access Control (AC) family, and the first CCI
// Iterate over checklist items ($stig_id) and populate spreadsheet with status foreach ($data['stigs'] as $stig_id => $tgt_status) { $notes = ''; $ia_controls_string = $tgt_status['echecklist']->get_IA_Controls_String(); $sheet->setCellValue("A{$row}", $stig_id) ->setCellValue("B{$row}", $tgt_status['echecklist'] ->get_VMS_ID()) ->setCellValue("C{$row}", $tgt_status['echecklist'] ->get_Cat_Level_String()) ->setCellValue("D{$row}", $ia_controls_string) ->setCellValue("E{$row}", html_entity_decode($tgt_status['echecklist'] ->get_Short_Title())); …}
of that control. An example of a control ID with enhancement would be AC-2(1).1, which would be first enhancement to control
AC-2, and the first CCI of that control enhancement. Since control numbers, enhancements, and CCI numbers are currently at
most two digits, the field size for the control string must be 12 characters with punctuation.
Finally, both fields are assigned the attribute of “Not Null”, since this is a junction table and there is
no reason to have a record if both fields are not populated. The following is the modification to the
db_schema.xml file required to implement the control-to-CCI mapping in the database:
Listing 3-4 - Extending the Database Schema
3.2New CodeThe primary new code is in the parse_emass_control_map.php script. This script does the work of
importing the mapping of the CCIs to the eMASS-formatted NIST control IDs. Listing 3-2 shows the
Listing 3-5 - Parsing the eMASS Control Map Spreadsheet
The main code is the loop which reads in a line and cell at a time and captures the mappings in an
array. It then pushes each mapping onto the $ccis array, which is then written to the database using
the call to save_EMASS_CCIs.
The save_EMASS_CCIs function is defined in databases.inc, and is reproduced in Listing 3-3. The db
class acts as the interface between the objects Sagacity uses and the database. To import the new
mapping to a junction table, we defined the function so that it takes the array of mappings created in
the above code and inserts them into the database with an extended insert. It does this using the
extended_insert function of the db objects helper class, which abstracts the SQL INSERT statement to
allow for insertion of an arbitrary number of records into the table passed in the first argument with
the columns being inserted passed as a second argument and the array of records to insert passed in
as the third argument.
// Create reader for file$Reader = \PhpOffice\PhpSpreadsheet\IOFactory::createReaderForFile($cmd['f']);$Reader->setReadDataOnly(true);$objSS = $Reader->load($cmd['f']);
// Get the first and only worksheet$wksht = $objSS->getSheet(0);
$col1 = true; // Bool to determine which column we're in within the loop$ccis = array(); // Array to hold cci-control mappings
// Main loop to read in the Excel spreadsheetforeach ($wksht->getRowIterator() as $row) { foreach ($row->getCellIterator() as $cell) { if ($col1){ // First column is the control $ctrl = $cell->getValue(); } else { // Second column is the cci $cci = $cell->getValue(); } // Change the col1 to iterate between the two columns $col1 = !$col1; } // Truncate the CCI to just the integer portion $cci_id = substr($cci, strpos($cci, "-") + 1); // Push the pair on the $ccis array array_push($ccis, array($cci_id, $ctrl));} // Save the completed array to the database$db->save_EMASS_CCIs($ccis);
Listing 3-6 - Function save_EMASS_CCIs
Finally, we need a way to retrieve the mappings from the database. Fortunately, the table is small
enough that we can load the entire array into memory for very fast lookups. Therefore, we need define
only a function to retrieve the entire table and return it into an array. Listing 3-4 shows the
get_EMASS_CCIs function, which is called in the sections of code we modify to populate with controls in
lieu of CCIs.
Listing 3-7 - Function get_EMASS_CCIs
3.3Modified CodeThe modifications we made to replace CCIs with RMF controls in parse_stig.php are shown in Listing
3-5. Immediately before the script begins looping over the <group> nodes in the XML, we retrieve the
mapping of controls to CCIs from the database using the database class get_EMASS_CCIs function
defined and described in Section 3.2. Within the loop, after the CCIs relevant to the currently
processing <group> node have been extracted to the $ias variable, we simply loop over the CCIs, and
/** * Function to save an array of eMASS-CCI mappings * * @param array:cci $cci_in * * @return boolean */ public function save_EMASS_CCIs($ccis_in) { $ret = false; $columns = array('id', 'control'); $this->help->extended_insert('rmf.emass_cci', $columns, $ccis_in, true); if ($this->help->execute()) { $ret = true; } else { $this->help->debug(E_ERROR); } return $ret;
/** * Get eMASS CCI Map * * @return array * Array of CCI-eMASS control mappings */ public function get_EMASS_CCIs() { $ret = array(); $this->help->select("rmf.emass_cci"); $ret = $this->help->execute(); return $ret; }
build an array of the correlated controls in the $rmf_controls array variable. Then in the fputcsv call
that writes the row to the CSV file, we replace the $ias array used previously with our newly created
$rmf_controls array.
Listing 3-8 - Substituting Control IDs for CCIs on STIG Checklist Creation
The behavior of export.php is similarly modified as shown in Listing 3-6. We begin be retrieving the
mapping of CCIs to controls, and this time, instead of looping over STIG XML, we are looping over
actual results that have been retrieved from the database. For each result, we retrieve the list of CCIs
using the get_IA_Controls method of the echecklist object. Then we declare an array to hold the
$rmf_controls, and loop over the CCIs (the $ia_controls variable) and replace them with RMF controls
looked up from the $emass_ccis array and expanded into a string in the $ia_controls_string variable.
And finally, the $ia_controls_string is inserted into the appropriate spreadsheet cell with the
setCellValue function.
// Get mapping of eMASS controls to CCIs from DB$emass_ccis = $db->get_EMASS_CCIs();// Iterate over each group element processing the attributes/childrenforeach ($groups as $group) {… // MAS replace CCIs w/ eMASS RMF Control
$rmf_controls = array(); foreach($ias as $ia){ // Remove 'CCI-' and leading zeros $id = ltrim( substr( $ia, strpos($ia, "-") + 1) , '0'); // lookup cci in $emass_ccis $key = array_search($id, array_column($emass_ccis, 'id')); // Push the control onto $rmf_controls array_push($rmf_controls, $emass_ccis[$key]['control']); }
These functions get_Regex_Array() and get_Software() are member functions of the database
object. Each parser object, including the nessus_parser object implemented by parse_nessus.php, has
a ‘db’ variable which holds a database object to abstract the parser’s connection to the database.
Since the ‘db’ variable is inherited from the scan_xml_parser object, the fix was to use $this->db-
>get_Regex_Array() or $this->db->get_Software().
4.1.3 Targets Not Populating in Multiple Screens (Bug #19)Once targets were confirmed to be loading into the database successfully, an issue remained where
the targets would not be displayed on the AJAX-enabled screens. In Figure 4-1, note the Operations
screen, where the “Unassigned” category has 23 hosts in the group (indicated by the “(23)” at the
right, and the category is expanded (indicated by the “-“ icon), however no hosts are listed. Similar
issues are exhibited in other pages wherever hosts are listed with AJAX calls, which indicates a
common root cause.
Figure 4-5 - AJAX Control Bug
Reloading the Operations screen with the Chrome developer tools console open, we get the
following error:
Listing 4-13 - Internal Server Error in ajax.php
The apache error log shows the following error repeated multiple times:
jquery-1.11.3.min.js:5 POST http://localhost/cgi-bin/ajax.php 500 (Internal Server Error)send @ jquery-1.11.3.min.js:5ajax @ jquery-1.11.3.min.js:5get_hosts @ ste_script.js:296collapse_expand @ ste_script.js:239onclick @ (index):327
Listing 4-14 - "Not Executable" Error in Apache Error Log
Adding a “#!C:\xampp\php\php.exe” fixes the server error, but now we get the following in our php
4.1.4 Errors Displayed When Adding and Editing Categories (Bug #23)When adding or editing a category on index.php, the interface provides a multiselect box to choose
the scan source data types that are expected for that category of targets (e.g., Nessus, SCC,
eChecklists, etc.). This selection is then used for tracking completion of those scan sources. When the
page is first loaded, however, none of these scan sources are selected, so a 'foreach' loop that loops
over them to associate them to the category was producing errors since the _REQUEST object
'scan_sources' key was initially unset.
Listing 4-18 - Undefined Index Errors in index.php
This was fixed by simply wrapping the 'foreach' loop in a test to ensure the key is set:
<Directory "C:/xampp/cgi-bin"> AllowOverride All Options None Require all granted</Directory>
<Directory "C:/xampp/cgi-bin"> AllowOverride All Options +ExecCGI Require all granted</Directory>
[20-Nov-2016 16:04:00 America/Denver] PHP Notice: Undefined index: scan_sources in C:\xampp\www\ste\index.php on line 48[20-Nov-2016 16:04:00 America/Denver] PHP Stack trace:[20-Nov-2016 16:04:00 America/Denver] PHP 1. {main}() C:\xampp\www\ste\index.php:0[20-Nov-2016 16:04:00 America/Denver] PHP Warning: Invalid argument supplied for foreach() in C:\xampp\www\ste\index.php on line 48[20-Nov-2016 16:04:00 America/Denver] PHP Stack trace:[20-Nov-2016 16:04:00 America/Denver] PHP 1. {main}() C:\xampp\www\ste\index.php:0
Listing 4-19 - Testing for the scan_sources Request Parameter
4.1.5 Undefined Offset Notices When Importing CPEs (Bug #54)Using the parse_cpe.php script to import the CPE dictionary resulted in undefined offset errors. This
was because the parse_cpe.php script was reading CPE string fields into an array and attempting to
operate on the array parts without first verifying that the array parts were not empty. As such, blank
fields in the CPE resulted in attempts to operate on null array elements, causing a large number of
errors on import. The array with empty elements (printed via var_dump) and the error can both be
seen in Listing 4-11.
if(isset($_REQUEST['scan_sources'])){ foreach($_REQUEST['scan_sources'] as $idx => $id) { $ste_cat->add_Source($db->get_Sources($id)); }}
Listing 4-20 - Undefined Offset Warnings in parse_cpe.php
To fix this, we added a check to verify if the array element is set prior to operating on the element
using:
Listing 4-21 - Testing for Presence of CPE Array Elements
4.1.6 Issues Importing Nessus Plugins (Bugs #262 and #270)
When attempting to import Nessus plugins with the --nasl argument to update_db.php, the following
errors were reported for each disabled plugin:
Listing 4-22 - Errors Deleting Disabled Plugins
Running update_db.php yields countless errors of the following sort:
To debug, we used the PHP getcwd function to verify that the working directory was the document
root, whereas the files to be deleted were in tmp/nessus_plugins/. This was fixed by passing absolute
paths to the unlink command.
While debugging #262, we additionally observed that plugins that were being skipped were not
actually disabled and opened Bug #270. As shown in the following output:
XX.XX% Plugin file C:/xampp/www/tmp/nessus_plugins/xtux_server.nasl is DISABLEDXX.XX%Warning: unlink(xtux_server.nasl): No such file or directory in C:\xampp\www\exec\update_db.php on line 251Call Stack:0.0000 194456 1. {main}() C:\xampp\www\exec\update_db.php:02472.9520 13934816 2. unlink() C:\xampp\www\exec\update_db.php:251
Listing 4-23 - Plugins Incorrectly Identified as Disabled
The first plugin processed (CSCdx17916.nasl) is properly disabled in the NASL; however, the second
plugin processed (CSCeb56909.nasl) is not.
Listing 4-24 - Searching NASL Files for the "@DEPRECATED@" Keyword
The first plugin processed (CSCdx17916.nasl) is properly disabled in the NASL; however, the second
plugin processed (CSCeb56909.nasl) is not.
The keyword used to indicate deprecated plugins is “@DEPRECATED@”. The regular expression
used to match the deprecated plugins was case insensitive and did not include the “@” character
before and after the capitalized “DEPRECATED” keyword, therefore it was incorrectly matching
instances of the string “Deprecated” in vulnerability comments as well as the NASL function
deprecated_version. The corrected regular expression is:
Started at 15:21:18Found 86679 NASL filesStarted at 2017-06-26 15:21:18 Plugin file C:/xampp/www/tmp/nessus_plugins/CSCdx17916.nasl is DISABLED
0.00%Warning: unlink(CSCdx17916.nasl): No such file or directory in C:\xampp\www\exec\update_db.php on line 263
Listing 4-25 - Corrected Regular Expression to Identify Disabled Plugins
4.1.7 Repeating Port Banners in the Database from Nessus Imports (Bug #280)When parsing Nessus scan results, the parser captures interface IP addresses, host names, and
open ports, as well as the banner associated with each port (the banner field of the targets.pps_list
table). The script appended the banner text to the existing text in the field if the port already exists in
the database. The problem observed was that a.) multiple plugins may report the same port and
banner and b.) multiple scan files sometimes scan the same interface. After several plugins or scan
files identifying the same port were imported, the field became quite large, and was causing database
timeouts when the script attempted to save the port to the database.
The fix was to add a check to ensure that the banner text from the plugin was not already in the
database field before appending it. A similar check was also added to the notes field. The following is
the revised update_TCP_Port function with comments. Similar updates were also made to the
public function update_TCP_Port($tcp_port) { // Get pointer to current port by reference so updates persist upon return $cur_port = &$this->tcp_ports[$tcp_port->get_Port()]; // Get current and new port banner and notes to determine if we need to update. $cur_banner = $cur_port->get_Banner(); $cur_notes = $cur_port->get_Notes();
// Only update banner if new banner is not already in current banner if (strpos($cur_banner, $new_banner) === false){ $cur_port->set_Banner($tcp_port->get_Banner()); } // Only update notes if new notes is not already in current notes if (strpos($cur_notes, $new_notes) === false){ $cur_port->append_Notes($tcp_port->get_Notes()); } }
After an update to the DISA STIG Viewer tool, the .ckl file format exported from Sagacity no longer
imported into STIG Viewer. The latest version was found to validate a UUID before importing. The uuid
parameter was being set to an empty string by export-ckl.php, which had worked in the past but now
did not. After testing, it was found that a pseudo-random UUID is sufficient to pass the validator.
To fix, we created a file uuid.inc in <wwwroot>/classes/ and added a UUID class found in the PHP
documentation forums. UUID.inc was then included in export-ckl.php. Then, on line 140 in export-
ckl.php we changed the array that creates the the uuid parameter to use a value generated by the
UUID class as shown in Listing 4-18.
Listing 4-27 - Populating the UUID Value
4.2Details of Significant Debugging WorkThis section details significant issues we identified and debugged to the point of isolating the
problems but stopping short of fixing the issue. Typically, this was due to the need to discuss the
proper approach with the development team or due to the need for additional insight into the design
and intended software operation, or otherwise because the fix needs additional testing or development
deferred to future releases.
4.2.1 Trigger Permissions Issues (Bug #17)Upon completion of Bug #14 (Section 4.1.2), importing Nessus data still did not work properly. The
script appeared to complete and did not report any runtime errors; however, the information was not
present in the database as shown in Figure 4-2 by the absence of hosts on the Operations screen.
This issue was accompanied by an error visible in the php_error_log repeated multiple times:
Listing 4-28 - SQL Syntax Errors in PHP Error Log
Searching the code for the SQL string ‘AND `pdi_id`’ yields a SQL string being built dynamically in
the db object’s get_Finding method in the database.inc file.
We also observed that the application sql_log, which logs all SQL statements, shows the following:
Listing 4-29 - Invalid Statement in SQL Log
Note that the `tgt_id` parameter is not assigned to anything in the WHERE clause. This indicates
that the target id is null for some reason. Also we see the “AND `pdi_id`” string referenced in the PHP
error log.
The call to get_Finding in parse_nessus.php is in the callback function that gets called when
processing the NessusClientData_v2\Report\ReportHost\ReportItem closing tag in the Nessus xml. The
call is as follows:
Listing 4-30 - get_Finding Function Call in parse_nessus.php
So the method is passed the parser’s ‘tgt’ variable, which is a target object. To determine why it is
null, we must determine where the parser assigns the ‘tgt’ variable. Using var_dump on $this->tgt, I
noticed that the id attribute is ‘false,’ which is indicative of an error setting the ID value. So I searched
for calls to the $this->tgt->set_ID method, and found it is assigned in this line of code, belonging to
[13-Nov-2016 17:08:34 America/Denver] You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AND `pdi_id` = 0' at line 1
2016-11-13T17:34:29-0700 - NOTICE - SELECT `id`,`tgt_id`,`pdi_id`,`scan_id`,`findings_status_id` as 'findings_status',`notes`,`change_id`,`orig_src`,`finding_itr`,`cat` FROM `scans`.`findings` WHERE `tgt_id`= AND `pdi_id` = 0
TRIGGER command denied to user 'web'@'%' for table 'target'
CREATE DEFINER=`web`@`%` TRIGGER `targets`.`target_BEFORE_INSERT` BEFORE INSERT ON `target` FOR EACH ROWBEGIN
SELECT MAX(`id`) INTO @newid FROM `targets`.`target`; SET NEW.`id` = COALESCE(@newid + 1, 1);END
Listing 4-34 - Target Data Correctly Populated
4.2.2 Undefined Offset Error When Exporting (Bug #20)Sagacity provides a feature to organize hosts in the target network into arbitrary categories (e.g., a
user might have a “Windows Workstations” category and a “Windows Servers” category to group
similar systems for analysis). Until they are categorized, it places hosts into the “Unassigned”
category. When clicking the Export button for the Unassigned category, the error in Figure 4-3 was
shown.
MariaDB [(none)]> select id,os_id,ste_id,os_string from targets.target;+------+-------+--------+----------------------------+| id | os_id | ste_id | os_string |+------+-------+--------+----------------------------+| 1 | 76373 | 1 | Red Hat Enterprise Linux 5 || 2 | 76373 | 1 | Red Hat Enterprise Linux 5 || 3 | 76373 | 1 | Red Hat Enterprise Linux 5 || 4 | 76373 | 1 | Red Hat Enterprise Linux 5 || 5 | 76373 | 1 | Red Hat Enterprise Linux 5 || 6 | 76373 | 1 | Red Hat Enterprise Linux 5 || 7 | 76373 | 1 | Red Hat Enterprise Linux 5 || 8 | 76373 | 1 | Red Hat Enterprise Linux 5 || 9 | 76373 | 1 | Red Hat Enterprise Linux 5 || 10 | 76373 | 1 | Red Hat Enterprise Linux 5 || 11 | 76373 | 1 | Red Hat Enterprise Linux 5 || 12 | 76373 | 1 | Red Hat Enterprise Linux 5 || 13 | 76373 | 1 | Red Hat Enterprise Linux 5 || 14 | 76373 | 1 | Red Hat Enterprise Linux 5 || 15 | 76373 | 1 | Red Hat Enterprise Linux 5 || 16 | 76373 | 1 | Red Hat Enterprise Linux 5 || 17 | 76373 | 1 | Red Hat Enterprise Linux 5 || 18 | 76373 | 1 | Red Hat Enterprise Linux 5 || 19 | 76373 | 1 | Red Hat Enterprise Linux 5 || 20 | 76373 | 1 | Red Hat Enterprise Linux 5 || 21 | 76373 | 1 | Red Hat Enterprise Linux 5 || 22 | 76373 | 1 | Red Hat Enterprise Linux 5 || 23 | 76373 | 1 | Red Hat Enterprise Linux 5 |+------+-------+--------+----------------------------+23 rows in set (0.00 sec)
Figure 4-7 - Undefined Offsest Error When Exporting
Line 43 of export.php at this time was:
Listing 4-35 - The Source of the Undefined Offset Error in export.php
Debugging the $_REQUEST object with print_r, we get the following:
Listing 4-36 - The _REQUEST Object Variable at Runtime
We observed that the ‘cat’ value passed to get_Category is 0. The SQL generated by
get_Category(0) is:
Listing 4-37 - SQL Statement Generated by get_Category(0)
This query, however, returns no results because there are not ste_cat records with id of 0. The
returned array is then null, which explains the Undefined offset error since there are no offsets to a null
array.
By default, newly imported targets are not assigned to a category, i.e., the target table cat_id field is
null. The link on the Operations page to the “Unassigned” category was passing the value of “0” to
the request object, as shown by the argument “0” to the open_echecklist function below:
SELECT `id`,`ste_id`,`name`,`analysts` FROM `targets`.`ste_cat` WHERE `id`=0;
Listing 4-38 - JavaScript to Initiate the Export
This initiates the cascade of calls all the way to the query where the script fails. We confirmed the
bug is isolated to the unassigned targets by assigning some of the hosts to a new category. As seen
below, the data on the category summary page now populates with host data and the export function
completes.
On speaking with the project team, the decision was made to disable the hyperlink to this category
since targets would not have an associated category number until they were sorted.
4.2.3 Large Text Strings in Inserts Cause Timeouts (Bug #281)Some Nessus plugins generate large quantities of text data which the parse_nessus.php script
attempts to insert into the database. These inserts were observed to cause the database connection to
timeout before the insert could complete. The problem was isolated by truncating insert, update, and
replace statements at 1024 bytes and observing that the same inserts now succeed. This was
accomplished by modifying the _escape function of the database class, which is used to escape data
being passed to the database. Listing 4-31 shows the updated function.
Listing 4-39 - Truncating Large Text Inserts, Updates, and Replaces
A fix was proposed to create a configurable variable (MAX_VALUE_SIZE) which would allow users to
specify a maximum size for unbounded Nessus plugin data. The lead developer also proposed reducing
the number of records inserted at a time. Adjudication is still pending and further testing is needed to
determine the impact of these solutions.
4.2.4 Multiple Discrepancies Between Sagacity and STIG Viewer .ckl Files (Bug #320)Once Bug #319 was fixed, it was quickly observed that the formats of the XML .ckl files generated
by the STIG Viewer and Sagacity were significantly different. Figure 4-4 shows the export from
Sagacity, and Figure 4-5 shows the output from STIG Viewer for the same STIG checklist and SCC data.
Significant differences can be easily observed.
public function _escape($val) { if (is_null($val)) { return 'NULL'; } elseif (is_numeric($val) || is_string($val)) { if (strtolower($val) == 'now()') { return $val; } elseif (preg_match("/\.`\w+`/", $val)) { return $val; } // If it's inserting/updating/replacing values based on query type, // truncate the field length to prevent DB timeouts if (in_array($this->query_type, array( 'insert', 'extended_insert', 'update', 'extended_update', 'replace', 'extended_replace', )) && strlen($val)>=1024) { $val = substr($val,0,1004) . "**OUTPUT TRUNCATED*"; } return "'{$this->c->real_escape_string($val)}'"; } elseif (is_a($val, 'DateTime')) { return "'{$val->format(MYSQL_DT_FORMAT)}'"; } elseif (is_bool($val)) { return $val ? "'1'" : "'0'"; } else { print_r($val); }
throw(new Exception("Unknown datatype to escape in SQL string {$this->sql} " . gettype($val), E_ERROR)); }
Figure 4-8 - The STIG Viewer .ckl File After Import
Figure 4-9 - The Sagacity .ckl File After Import
There are four unique types of inconsistencies identified through manually comparing each finding
in the two files and adjudicating the reasons for differences. The first is that the STIG viewer rejects
SCAP data that has a rule number revision mismatch (i.e., it will not import results for SV-25018r2_rule
if the checklist uses SV-25018r3_rule. The second was due to a bug in the SCC parser which did not
reset the finding status variable if a check was determined not to exist in the version of the STIG being
processed (i.e. the rule had been removed from the STIG in the current version of the checklist).
Third, the procedure to create the XML that is used to populate the Check Content and Fix Text tabs
was simply omitted from the export_ckl.php script. Finally, the database field that was being used to
populate the Rule Name was actually the stig.stig_id field, which is a numeric identifier rather than the
STIG's GroupTitle tag content as used by the STIG Viewer.
The first issue was determined to be working as intended. Since a goal of Sagacity is to reduce
manual checks, the decision was to allow matching regardless of rule revision number because
revisions are often merely cosmetic. The second was fixed since it could lead to corrupted data by
changing finding status. The third was fixed to ensure all required tags were present in the XML. The
fourth was changed to use the pdi_catalog.short_title field, which is a textual short description of the
STIG. This is still inconsistent with the GroupTitle tag used by the STIG Viewer, however the GroupTitle
tag is not currently being imported by Sagacity, so the fix for this was deferred to future releases.
4.3Commented FilesIn making the updates we attempted to comment all sections of code that we debugged or modified
as we came to them to increase the maintainability of the project. In addition, significant portions of
the following files were commented.
parse_cpe.php: Comments added throughout the entire file
update_db.php: Commented the NASL parsing routine
parse_stig.php: Comments added throughout the entire file.
Additionally, the portions of code provided for the implementation of RMF controls as described in
Section 3 were all commented in their entirety.
5 Future Work5.1Bug Fixes and Design Changes
As a project under active development, there are a significant number of outstanding bugs that
have been deferred to future releases. We opened several of these in the course of this project effort
and they are summarized in Appendix A. We highlight several of the more significant ones here.
Refactoring is recommended for more intuitive and consistent object names. For example, ‘STIG’ is
used in various places either to refer to individual STIG settings or the STIG checklists themselves.
Also, variations of ‘category’ are used in different contexts to mean either categories of systems,
category (severity) of STIG findings. Overloading these terms can be confusing when reading the code
as it can be unclear as to which context variable names and database fields refer.
Refactoring of the db object is also highly recommended. The class currently extends over 10,000
lines of code and is used by essentially every parser and object for all interactions with the database.
This has been causing issues since bug fixes for methods of the db class often manifest unanticipated
bugs in other areas of code, which is significant since the class must be frequently updated – having
been modified in 59 of 208 commits since the projects’ inception or 28%.
We observed that many functions accept arrays or primitives as datatypes rather than
encapsulating them in objects. This makes type checking challenging and inhibits IDE Content
Assistance. Additionally, it leads to a propensity to attempt to pass unexpected data types to the
functions.
Finally, there is a significant issue with the implementation of CPE checking. Sagacity attempts to
identify software using direct lookups of CPE strings, which is not how the CPE specification was
intended to be used [21]. As such, sections of code that attempt to identify software often behave
unpredictably, leading to false positives and false negatives. Additionally, these sections of code
become convoluted in an attempt to account for different conditions encountered. Sagacity would
benefit from a full implementation of the CPE specification.
5.2Feature AdditionsFull RMF compliance reporting is obviously a major objective for the project. To implement this,
there are several things that must be done. First, a parser will need to be written to ingest the DISA
CCI list, which provides the most comprehensive NIST 800-53 control data available. Second, the
CNSSI 1253 control baselines must be imported. Of course, the underlying database schema must be
updated to hold the data and user interface elements must be updated to include the reporting
controls and configuration options. Finally, there will need to be some business logic to implement
control baseline generation and risk determination.
Several features could be added to enhance the quality of reports as well. Compliance metrics
would be useful for an executive summary. Another often-requested capability is comparison of
results between assessments to show regressions, improvements, and trending in compliance.
Furthermore, the ability to parse the NIST CVE database (in lieu of the MITRE CVE database currently
used) would provide a great deal more data about identified CVEs. This includes CVSS scores,
vulnerable software CPEs, and vulnerable configuration CPEs [22]. This information could prove useful
in reporting as well.
Currently, Sagacity does not provide any capability to export data other than to eChecklist or .ckl
flat files. A useful feature would be to export and import an entire ST&E. This is especially useful
because they are not delivering releases as patches, so new releases frequently require dropping and
rebuilding the database, which means data must be re-imported one file at a time if the version is
upgraded.
Finally, since a current objective is to recruit new developers, it would be useful to publish a
developers’ guide to explain the software architecture, database architecture, code conventions, and
other such useful information to familiarize contributors with the application.
6 Lessons Learned and Conclusion6.1Lessons Learned
Our primary reason for undertaking the project was to learn the architecture and code base of
Sagacity sufficiently to be able to apply the tool to real-world data and isolate and fix problems as they
arise. This goal has been achieved and put into practice with two different real-world data sets in the
scope of this project.
The project has been an exercise in the synthesis of many years of education, training, and work
experience across numerous domains. This includes software design, project management,
programming, IDEs, database design and query language, cybersecurity governance, and the
vulnerability assessment process. Specific tools, technologies, and techniques learned include:
Installing and configuring development XAMPP and LAMP environments
PHP development, SQL, and, to a lesser extent, JavaScript
The PHPOffice PHPSpreadsheet Library
The PHP XML Parser Library
Installing and configuring Eclipse PHP Development Tools and configuring the XDebug
debugger for use with PHP CLI and Web Applications
Eclipse Debugger, including limitations on variable size, breakpoints, conditional breakpoints,
and watchlist expressions
Using SourceForge and the SVN protocol to checkout and commit to community open source
projects, including powerful SourceForge features such as bug tracking, commit browsing, and
diffs, and file history
Using WinMerge to merge code changes
MySQL Workbench design tools, code editor, and reverse engineering
The CPE Specification
Chrome Developer Tools
Numerous components of the Sagacity application as well as its architecture and database
structure/relationships
Multiple security data file formats including MITRE and NIST CVEs, CPEs, Nessus report files,
SCC XCCDF files, and DISA STIGs
6.2ConclusionThe public release of Sagacity 1.3 was November 1, 2017 due in large part to the contributions of
this project [24]. The critical RMF capability and increased stability provided as a result of this project
will increase the software’s chances for adoption and hopefully serve to encourage others to contribute
as well. Additionally, the proficiency we’ve gained in supporting and developing for Sagacity by virtue
of executing this project enables us to support and extend Sagacity as future bugs are identified and
to help to add features to the software in future releases.
7 References[1] Defense Information Systems Agency. 2014. DoDI 8510.01, Risk Management Framework for
DoD Information Technology. DISA. http://www.dtic.mil/whs/directives/corres/pdf/851001_2014.pdf.
[2] National Institute of Standards and Technology. 2013. NIST SP 800-53 Revision 4, Security and
Privacy Controls for Federal Information Systems and Organizations. NIST.
[20] Apache HTTP Server Project. Apache Tutorial: Dynamic Content with CGI.
http://httpd.apache.org/docs/current/howto/cgi.html, Accessed November 14, 2017.
[21] Mary Parmelee, Harold Booth, David Waltermire, Karen Scarfone. 2011. NISTIR 7696, Common
Platform Enumeration: Name Matching Specification Version 2.3, 8-19. NIST. DOI:
http://doi.org/10.6028/NIST.IR.7696.
[22] National Institute of Standards and Technologies. National Vulnerability Database - Data Feeds.
https://nvd.nist.gov/vuln/data-feeds. Accessed November 14, 2017.
8 Appendix A – Table of Bugs ReportedBug No.
Title Status Date Created
LOE Rev Fixed
9 parse_nessus.php bombs when importing closed 10/23/2016 Opened 12Summary:Parse_nessus.php failed with "Fatal error: Cannot access protected property tcp_ports::$port in C:\xampp\www\classes\interfaces.inc on line 274". Problem was due to the interface class making direct calls to private variables of the port class.
14 Overwriting config files closed 11/10/2016 Fixed 27Summary:Sagacity ships with its own configuration files for Apache, MariaDB, and PHP. These configuration files are copied over the default files upon Sagacity installation by the 'install.bat' script. The my.ini file (used to configure MariaDB) that was included with Sagacity included a 'skip-federated' directive which had been supported in MySQL but is not supported in MariaDB. Therefore, when the XAMPP project migrated from MySQL to MariaDB, MariaDB would fail to start after Sagacity installation due to the invalid directive. The solution was to remove the 'skip-federated' directive from the my.ini file supplied with Sagacity.
Additionally, this bug included discussion with the team regarding whether it was a bad practice to overwrite users' configuration files on the chance that it might be a multi-application server. This was mitigated by editing the install script to back up the existing config files (renamed to '*.old') before copying the Sagacity configuration files.
15 Fix erroneous references to $db variable in parse_nessus.php closed 11/11/2016 Fixed 16Summary:Parsing Nessus results using parse_nessus.php failed with the a fatal error: c:\xampp\www\exec>php parse_nessus.php -s 1 -f ..\tmp\nessus_report_Linux_Vulnerability_Scan.nessus -d \xampp\wwwNotice: Undefined variable: db in C:\xampp\www\exec\parse_nessus.php on line 269Call Stack:0.0042 690312 1. {main}() C:\xampp\www\exec\parse_nessus.php:01.0967 3753024 2. scan_xml_parser->parse() C:\xampp\www\exec\parse_nessus.php:16331.4158 3767176 3. xml_parse() C:\xampp\www\inc\xml_parser.inc:3551.4300 3772368 4. scan_xml_parser->stopElement() C:\xampp\www\inc\xml_parser.inc:3551.4306 3772824 5. call_user_func:{C:\xampp\www\inc\xml_parser.inc:309}() C:\xampp\www\inc\xml_parser.inc:3091.4306 3775352 6. nessus_parser->NessusClientData_v2_Report_ReportHost_HostProperties_end() C:\xampp\www\inc\xml_parser.inc:309Fatal error: Call to a member function get_Regex_Array() on null in C:\xampp\www\exec\parse_nessus.php on line 269An examination of the code revealed the failing line 269 as follows: $os_regex = $db->get_Regex_Array("os");This and one subsequent line both referred to a non-existent $db variable. The second was line 279: $os = $db->get_Software($os_arr->get_CPE());The proper way to reference it would be to use $this->db->get_Regex_Array() and $this->db->get_Software().
16 Comments for parse_cpe.php closed 11/11/2016 Commented 17
Bug No.
Title Status Date Created
LOE Rev Fixed
Summary:Provided comments for parse_cpe.php
17 Fix DEFINER on target_BEFORE_INSERT Trigger closed 11/14/2016 Debugged 21Summary:"Permission is denied when doing operations that insert into 'targets' table due to the '%' wildcard in the target_BEFORE_INSERT trigger. Need to change:CREATE DEFINER=web@% TRIGGER targets.target_BEFORE_INSERT BEFORE INSERT ON target to:CREATE DEFINER=web@localhost TRIGGER targets.target_BEFORE_INSERT BEFORE INSERT ON target in db_schema.xml."
19 Targets not populating on ST&E Operations screen closed 11/14/2016 Fixed 32Summary:The httpd.conf file included unquoted ScriptAlias directory and was missing +ExecCGI from the directory options.
20 Export.php dies when attempting to export the "Unassigned" category closed 11/14/2016 Debugged 29Summary:Attempting to export spreadsheets from the "Unassigned" category hyperlink generated a PHP error because the hyperlink passess the category value "0" to the database query, and no hosts are ever assigned to category "0". Decision was made to disable the hyperlink to this category since targets would not have an associated category number until they were sorted.
21 E_DEBUG Undefined closed 11/20/2016 Fixed 24Summary:"Helper.inc errors with ""Notice: Use of undefined constant E_DEBUG - assumed 'E_DEBUG' in C:\xampp\www\inc\helper.inc on line 1124""E_DEBUG is not a predefined constant per http://php.net/manual/en/errorfunc.constants.php.I fixed this by adding: ""include_once 'error.inc';"" to helper.inc. Not sure if there is a better fix."
22 Typo in install.php closed 11/20/2016 Fixed 30Summary:Install.php errors with "Notice: Undefined variable: key in C:\xampp\www\install.php on line 345" due to comparison operator (==) where assignment operator (=) should be.Line reads:if(($key == array_search("sample_data.sql", $sql_files)) !== false) {should be:if(($key = array_search("sample_data.sql", $sql_files)) !== false) {
Bug No.
Title Status Date Created
LOE Rev Fixed
23 Error displayed in index.php if scan_sources is not set closed 11/20/2016 Fixed 28Summary:When adding or editing a category on index.php, the interface provides the ability for the user to use a multiselect box to identify the scan source data that is expected for that category of targets, which can then be used for tracking completion of those scan sources. When the page is first loaded, however, none of these scan sources are selected, so a 'foreach' loop that loops over them to associate them to the category was providing errors since the _REQUEST object 'scan_sources' key was initially unset. This was fixed by wrapping the 'foreach' loop in a test to ensure the key is set:
if(isset($_REQUEST['scan_sources'])){ foreach($_REQUEST['scan_sources'] as $idx => $id) { $ste_cat->add_Source($db->get_Sources($id)); }}
24 Non-intuitive and inconsistent path strings closed 11/27/2016 Opened 32Summary:Upon installation, the install.php form accepted configuration paths to files that were in non-standard Windows path formats (e.g., omitting the leading drive letter) and were inconsistent regarding the use of full paths or relative paths, and therefore the code that processed these configuration settings.
25 Remove "ENGINE=MyISAM" in install.php and update.php closed 12/8/2016 Opened 43Summary:Install and update scripts were setting the database engine to MyISAM in an attempt to enable full text indexing on the PDI Catalog. However, MyISAM does not support key constraints, which is dangerous, and furthermore does not support reverese engineering with MySQL Workbench, which is antithetical to the objective of recruiting new developers.
27 Need to LoadFile pthreadVC2.dll in httpd.conf wont-fix 12/9/2016 Rejected N/ASummary:Sagacity had been using the pthread library to track the PID of exec jobs. When the lead developer ported pthread calls to the native php getmypid() function, he removed the httpd.conf line to load the .dll file, but neglected to remove the line that loads the module. Rather than add the LoadFile line, the proper fix was to remove the LoadModule line.
28 Change Document Properties for Excel exports closed 12/10/2016 Opened 32Summary:The document properties that were generated for exported files were hard-coded strings and were not appropriate for general public use.
Bug No.
Title Status Date Created
LOE Rev Fixed
Recommended that the strings be added to the configuration variables upon install.
29 Multiple Copyright and SMC Remnants closed 12/10/2016 Opened 34Summary:Multiple files were still using an old and incorrect copyright notice and referenced the Space and Missile Systems agency which, being a government organization, cannot hold copyright.
52 Install.php redirect to home page after completion closed 2/4/2017 Opened UTDSummary:After completing database installation, the install.php script re-posts to itself. It should instead load the http://localhost/ste/index.php. Unable to determine which release fixed this issue.
54 Undefined offset Notices in parse_cpe.php closed 2/5/2017 Fixed 38Summary:The parse_cpe.php script was reading CPE string fields into an array and attempting to operate on the array parts without first verifying that the array parts were not empty. As such, blank fields in the CPE resulted in attempts to operate on null array elements, causing a large number of errors on import. Fix was to check if the array element is set prior to taking action.
94 Clean up if (false) instances accepted
3/14/2017 Opened N/A
Summary:The developers had been using the dead code if (false) { $arg = new objType() } as a hack to get Content Assist in Eclipse. The proper way of doing this is either to use type hints in the method declaration or to use PHPDoc for local variables. This is also indicative of a need for refactoring for increased abstraction in the many instances where arrays are passed as function arguments.
132 Import .nasl files from Nessus data directory open 4/9/2017 Opened N/ASummary:If Nessus installed on the same system as Sagacity, the script for parsing the .nasl files for plugin information should be read directly from the %ProgramData%\Tenable\Nessus\nessus\plugins directory. This saves substantial time copying the 90,000 plugins to the /tmp/nessus_plugins directory.
255 Remove cgi-bin directory copy from install.bat closed 6/14/2017 Fixed 173Summary:Developer removed all cgi-bin directory contents, however did not remove the installation script commands to copy the non-existent directory, leading to "No such file or directory" errors on install. Fix was to remove the copy lines in the install script.
256 Fix include of Excel library in import.inc closed 6/14/2017 Opened 174Summary:The PHPExcel library was deprecated and replaced with PHPSpreadsheet, however the includes for the PHPExcel library were generating
Bug No.
Title Status Date Created
LOE Rev Fixed
errors on any pages that attempted to load it. Fix was to remove include statement.
257 Site Management "Country" drop-down should populate all countries open 6/14/2017 Opened N/ASummary:Since Sagacity is going to be released publicly, the "Country" option should include all countries of the world. Currenly it only has a handful of locations.
Summary:The calendar control for selecting an ST&E start date and end date fields only allows the user to scroll back one month. It should allow scrolling back to an arbitrary date. 259 update_db should look in www/tmp for default file names if --po is
specified.open 6/14/2017 Opened N/A
Summary:If update_db.php is used with the --do option (download only), it downloads the cve and cpe database files and renames them, appending the current date. Therefore, when parsing the files on an offline system using the --po option (parse only) the script looks for files with a date appended to the name. However, a user should also be able to manually download and transfer the files as an option if they choose. If files have been manually downloaded and transferred to an offline system, the files will not have been renamed with the date appended to the end and also will not have been transferred to the tmp/cpe and tmp/cve directories, which is where the script looks for them. The script should be modified to allow copying these files into /tmp and the script should look for the default file names official-cpe-dictionary_v2.3.xml.zip and allitems.xml, respectively.
260 parse_stig.php hangs on U_VVoIP_V3R10_Manual-xccdf.xml closed 6/14/2017 Opened 172Summary:Special characters in the STIG xml were preventing the parser from working properly. Fix was to use str_replace to replace the special characters with there ASCII equivalents.
261 Scan import interfaces aren't working open 6/14/2017 Opened N/ASummary:This problem proved to be a result of cached JavaScript, and a force refresh fixed the issue. However, the ticket is remaining open to see if there is anything we can do to reduce the cache lifespan of the JavaScript files.
262 Update_db.php on nasl files gives multiple "No such file or directory" errors closed 6/14/2017 Fixed 173Summary:Calls to the unlink() function to delete .nasl files following processing were failing because the script sets the working directory the document root directory, and a relative path was being used. The fix was to create an absolute path to the files to use wherever needed.
263 parse_nessus.php dies b/c no parse_config.ini file open 6/14/2017 Opened N/A
Bug No.
Title Status Date Created
LOE Rev Fixed
Summary:The parse_nessus.php script requires a file called "parse_config.ini" to specify its operating parameters. When Nessus scans are imported from the GUI, it works because the scripts create the file from the selected options, and then delete the file upon completion. However, at the command line, the parse_nessus.php script fails if the user does not first manually create the ini file. Normally a command-line script would be passed options via arguments rather than an ini file, and furthermore the documentation and help text give no clues that the file is required.
264 Unable to expand categories on ST&E Operations page closed 6/14/2017 Opened N/ASummary:Details unknown.
265 add_scan interface "Delete File" link deletes file wont-fix 6/16/2017 Opened N/ASummary:In testing, it was observed that the interface for uploading files provides a button labeled "Delete File" that deletes the file on the disk as opposed to deleting the . This was thought to be a risk as it could
266 Sourceforge Project Readme can't display PDF wont-fix 6/20/2017 Opened N/ASummary:The SourceForge SVN code tree page is attempting to load the README.pdf document in the "Read Me" area beneath the code tree. Proposed renaming the file and creating a README.txt file that could properly load, but proposal was rejected. 267 CCIs are broke closed 6/20/2017 Opened 177Summary:The values for CCI numbers were being corrupted upon ingestion into the database. This was fixed by changing the datatype in the database from integer to string and modifying the parser accordingly.
268 database.inc: get_Software() bombs when passed a software object pending 6/21/2017 Fixed N/ASummary:The database class get_Software() function attempts to access an array index to the $software_In[0] variable before verifying to check that it is an array. This is a problem because the function argument $software_In is a "mixed" data type and can be a software object rather than an array. This was fixed by adding an is_array() check before executing the array code. Fix has not yet been committed.
269 eChecklist assignment not working in bulk edit closed 6/21/2017 Opened 195Summary:The "Bulk Edit" feature enables editing properties of multiple hosts in a category at a time. The feature was not working when attempting to add or remove checklists to targets.
270 update_db incorrectly identifying .nasl files as disabled closed 6/26/2017 Fixed 173Summary:The Nessus .nasl file parser was incorrectly identifying plugins as disabled and therefore skipping them. The regex that was used to match
Bug No.
Title Status Date Created
LOE Rev Fixed
the "deprecated" string was inadvertently matching the word "deprecated" in the descriptions of vulnerabilities as well as the deprecated_version() function call, generating false positives. To remediate this, the regex was replaced with a more specific string "@DEPRECATED@" to more exactly match the way Tenable .nasl files identify deprecated plugins.
273 Need to declare Sagacity_Error object in parse_stig.php closed 7/12/2017 Opened 178Summary:The $log variable was being referenced before it had been instantiated. The references to the $log Sagacity_Error object were function calls that should have been static function calls to the Sagacity_Error class rather than the $log object.
274 Remove or relocate IAControls tag logic in parse_stig.php closed 7/13/2017 Opened 178Summary:In r176, the code to parse the IAControls tag was moved to a line after the variable holding the IAControls tag had been reduced to a substring which no longer contained the IAControls tag. Code reducing the substring was removed.
280 Eliminate duplicate port notes in interfaces.inc closed 7/25/2017 Fixed 181Summary:When parsing Nessus scan results, the parser captures interface IP addresses, host names, and open ports, as well as the banner associated with each port (the banner field of the targets.pps_list table). The script appended the banner text to the existing text in the field if the port already exists in the database. The problem observed was that a.) multiple plugins may report the same port and banner and b.) multiple scan files sometimes scan the same interface. After several plugins or scan files identifying the same port were imported, the field became quite large, and was causing database timeouts when the script attempted to save the port to the database. The fix was to add a check to ensure that the banner text from the plugin was not already in the database field before appending it.
281 Large text strings in inserts causes timeouts open 7/25/2017 Debugged N/ASummary:Some Nessus plugins generate large quantities of data which the parse_nessus.php script attempts to insert into the database. These inserts were observed to cause the database connection to timeout before the insert could complete. The problem was isolated by truncating insert, update, and replace statements at 1024 bytes and observing that the same inserts now succeed. A fix was proposed to create a configurable variable (MAX_VALUE_SIZE) which would allow users to specify a maximum size for unbounded Nessus plugin data. The lead developer also proposed reducing the number of records inserted at a time. Adjudication is still pending and futher testing is needed to determine the impact of these solutions.
282 start_time overwritten when batch importing SCC xccdf results open 7/26/2017 Opened N/ASummary:If you observe the scan.scans table while importing a large volume of results, the start_time field timestamp is being overwritten upon job completion by the time that the script was first queued. Here is a detailed description of the process to reproduce:
Bug No.
Title Status Date Created
LOE Rev Fixed
1.) Copy a large number of SCC XCCDF results to /tmp, and click "Import" on the Results page.2.) SELECT * FROM scans.scans; 3.) Observe when scripts are queued, the "start_time" field is populated with the start time representing when the script was queued. 4.) Once a file is kicked off, status changes to "RUNNING" start_time is correctly updated with the time the given result is actually kicked off once it is dequeued5.) Upon completion, the start_time will be replaced by the original queue time.
283 perc_comp not updated when parsing eChecklist open 7/27/2017 Opened N/ASummary:Eventually it was determined that this symptom is indicative of a hung eChecklist import due to incorrect column order. However, this is indicative of a need for a way of monitoring job status, gracefully catching hung import jobs and terminating them, perhaps a timeout if no other means can be devised. Furthermore, there is an issue that if the job fails once, the same file cannot be imported again even after terminating and deleting it. Root cause still unknown.
286 Multiple issues importing eChecklists wont-fix 8/14/2017 Debugged N/ASummary:eChecklists were not importing and the jobs would hang. It was determined that the eChecklists had been corrupted by users and did not match the file format expectations of the parser. The file format was fixed but Bug #313 was opened to track the need for input validation, and Bug #310 was opened to have the "Flatten eChecklists" option enabled by default on installation. The reason for this is that if "Flatten eChecklists" is disabled, the two rollup columns are hidden by default, causing users to inadvertently insert additional host columns after them. This causes problems for the parser since the columns are out of order.
304 Cache metrics on Ops page open 9/20/2017 Opened N/ASummary:The "Not a Finding/Open/Not Applicable" finding counts displayed on the ST&E Operations Page take way too long to populate with large data sets, which means every time users navigate away and back, there is a long wait for the query to complete and page to load. The data should be cached after the first query or loaded via AJAX after the query completes instead.
305 eChecklist fields that are used by the import functionality should be locked open 9/20/2017 Opened N/ASummary:This is related to the need for input validation (bug #313). The input validation issues could be mitigated by locking on export any fields in eChecklists that are used to make associations upon import. At a minimum this would include host names, column headers, and STIG/VMS IDs. We should also look at possibly doing this for all fields that we don't expect the user to need to edit (i.e., populated directly from the STIGs - check content, title, cat, control number, etc. and Excel formulas).
306 Distinct fields in eChecklists for automated vs. manual notes open 9/20/2017 Opened N/ASummary:Currently, there is only a single notes field for each STIG finding, and it is appended with notes from vulnerability scanners and also the source for each finding, which is de-normalized since sources and notes should be per-finding and per-host. Re-import of the eChecklist
Bug No.
Title Status Date Created
LOE Rev Fixed
leads to replicating this information yet again, making the notes field very polluted. At a minimum, we should have automated data generated by Nessus, SCC, or nmap stored separately from manually generated data written by analysts during testing. Ideally, it would be completely normalized, however, the eChecklist format does not currently support this.
307 eChecklist export puts classification in "Date(s) Tested field" open 9/20/2017 Opened N/ASummary:The eChecklist export script is placing the classification in the wrong field.
308 Application Server SRG incorrectly identified as Application Development STIG
open 9/20/2017 Opened N/A
Summary:Symptom is apparent on the eChecklist tab as well as the Operations screen checklist icon. Problem appears to be a greedy regex issue.
309 Truncated Short Titles in Symantec Endpoint Protection 12.1 Managed Client AV STIG
closed 9/20/2017 Opened N/A
Summary:STIG item short titles were being truncated for one specific checklist. Unable to reproduce.
310 Set "Flatten eChecklists" to enabled by default. closed 9/20/2017 Opened 195Summary:When "Flatten eChecklist" option is disabled, the two rollup columns in the eChecklists are hidden, leading people to insert columns for additional hosts after the rollup columns, which prevents the script from importing properly. Fix was to enable it by default until better input validation is implemented.
312 Exported Win 7 eChecklist generates Excel error open 9/22/2017 Opened N/ASummary:"Opening an exported Win 7 eChecklist causes the error 'We found a problem with some content in “Win_7_-_x64_SP1-eChecklist-1.xlsx’. Do you want us to try to recover as much as we can? If you trust the source of this workbook, click Yes.'If you click 'Yes', then it displays 'Removed Records: Formula from /xl/worksheets/sheetX.xml part' repeated multiple times with different sheet numbers."
313 Input validation for eChecklist import open 9/22/2017 Opened N/ASummary:If columns in eChecklists are not in a specific order, importing of eChecklists fails and import jobs hang. Input validation is needed and the code needs to be re-written such that known columns are accounted for and the remaining columns are identified as hosts, regardless of the order in which they appear.
Bug No.
Title Status Date Created
LOE Rev Fixed
314 "Delete Scan" popup needs clearer warning text open 9/22/2017 Opened N/ASummary:The JavaScript alert that pops up when deleting scans prompts "Do you want to also delete associated targets?" What is unclear is that this action permanently deletes all targets and findings from every target contained within the deleted scan - even if some of that data comes from other scans. I suggested adding the following line to the prompt:"WARNING: This will delete all targets in this scan and all associated data even if it was imported from another scan. This action is irreversible."Also this dialog should be "Yes/No" instead of "OK/Cancel"
315 Compress Help File screenshots open 9/22/2017 Opened N/ASummary:Exporting the Help to PDF is 15+ MB, which is too big to easily pass around (email attachment, downloads from the website, etc.). Compressing the screenshots would probably go a long way toward reducing this.
319 Exported .ckl files aren't importing into STIG Viewer closed 10/17/2017 Fixed 199Summary:.Ckl files exported from Sagacity don't import into STIG Viewer. STIG Viewer apparently checks for a valid UUID before importing, whereas the uuid parameter is currently set to an empty string by export-ckl.php. A pseudo-random UUID is sufficient to pass the validator.To fix, I added the attached uuid.inc to <wwwroot>/classes/ and included it in export-ckl.php. Then on line 140 in export-ckl.php changed the array instantiation for the uuid parameter to be generated by the UUID class.
320 Multiple discrepancies between Sagacity and STIG Viewer .ckl output. open 10/17/2017 Debugged 203Summary:Exported .ckl files were inconsistent with the STIG Viewer .ckl files after importing the same source data. There were four reasons identified for this. The first is that the STIG viewer rejects SCAP data that has a rule number revision mismatch (i.e., it will not import results for SV-25018r2_rule if the checklist uses SV-25018r3_rule. Since the goal of Sagacity was to reduce manual checks, the decision was to allow matching regardless of rule revision number because revisions are often merely cosmetic. The second was due to a bug in the SCC parser which did not reset the finding status variable if a check was determined not to exist in the version of the STIG being processed (i.e. the rule had been removed from the STIG in the current version of the checklist). Third, the procedure to create the XML that is used to populate the Check Content and Fix Text tabs was simply omitted from the export_ckl.php script. Finally, the database field that was being used to
Bug No.
Title Status Date Created
LOE Rev Fixed
populate the Rule Name was actually the stig.stig_id field, which is a numeric identifier rather than the STIG's GroupTitle tag content as used by the STIG Viewer. The first issue was determined not to need fixing. The second was fixed since it could lead to corrupted data by changing finding status. The third was fixed to ensure all required tags were present in the XML. The fourth was changed to use the pdi_catalog.short_title field, which is a textual short description of the STIG. This is still inconsistent with the GroupTitle tag used by the STIG Viewer, however the GroupTitle tag is not currently being imported by Sagacity, so the fix for this was deferred to future releases.
