FDICís Information Technology Configuration Management Controls Over Operating System Software

September 2005
Report No. 05-031

AUDIT REPORT

FDIC OIG, Office of Audits

Background and Purpose of Audit


The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) contracted with International Business Machines (IBM) Business Consulting Services to audit and report on the effectiveness of the FDICís configuration management controls over operating system software. The results of this audit support the OIG in fulfilling its evaluation and reporting responsibilities under the Federal Information Security Management Act.

Configuration management is a critical control for ensuring the integrity, security, and reliability of information systems. Absent a disciplined process for managing software changes, management cannot be assured that systems will operate as intended, that software defects will be minimized, and that configuration changes will be made in an efficient and timely manner.

The objective of the audit was to determine whether the FDIC had established and implemented configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices.

FDIC, Federal Deposit Insurance Corporation


Results of Audit


The FDIC had established and implemented a number of configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. Such controls included a software patch management policy, a change control board, and periodic scanning of operating system software configurations. These actions were positive; however, control improvements were needed. Specifically, the FDIC needed to establish an organizational policy and system-specific procedures to ensure proper configuration of operating system software. The FDIC also needed to standardize and integrate the recording, tracking, and reporting of operating system software configuration changes to the extent practical.

Recommendations and Management Response

IBM recommends that the FDIC:

  • establish an organizational policy that defines roles, responsibilities, and overall principles and management expectations for performing configuration management of operating system software;
  • develop configuration management plan(s) that include system-specific procedures for managing the configuration of operating system software;
  • ensure that the certification and accreditation of the FDICís general support systems incorporate an evaluation and testing of the configuration management policy and plan(s) referenced above;
  • fully document the minimum required configuration settings for the operating systems covered in this review, and develop procedures to ensure that changes to baseline configuration settings are documented; and
  • standardize and integrate the recording, tracking, and reporting of configuration changes within and across operating system software platforms to the maximum extent practical.

FDIC management generally agreed with the reportís recommendations and has either initiated or plans to initiate actions to address them.


FDIC OIG letterhead

DATE:  September 8, 2005

MEMORANDUM TO:  Michael E. Bartell
 Chief Information Officer and
 Director, Division of Information Technology

FROM: Russell A. Rau [Electronically produced version; original signed by Stephen M. Beard for Russell A. Rau]
 Assistant Inspector General for Audits

SUBJECT:  FDICís Information Technology Configuration Management
 Controls Over Operating System Software
 (Report No. 05-031)

Enclosed is a copy of a report completed by the independent professional services firm of International Business Machines (IBM) Business Consulting Services. The firmís report is presented as Part I of this document.

The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) contracted with IBM to audit and report on the effectiveness of the FDICís configuration management controls over operating system software. The results of this audit support the OIG in fulfilling its evaluation and reporting responsibilities under the Federal Information Security Management Act of 2002. The objective of the audit was to determine whether the FDIC had established and implemented configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. The audit focused on four of the FDICís operating system software platforms used to support sensitive and mission-critical business applications. This report provides recommendations to strengthen configuration management controls over the FDICís operating system software.

Our evaluation of your response, a summary of your response and the status of corrective actions, and your response in its entirety is contained in Part II of this report. The response adequately addressed the recommendations in the report. We consider the reportís recommendations to be resolved, but they will remain undispositioned and open for reporting purposes until we have determined that agreed-to corrective actions have been completed and are effective.



TABLE OF CONTENTS

Part I:
Report by International Business Machines (IBM) Business
Consulting Services
FDICís Information Technology Configuration Management Controls
Over Operating System Software
Part II:
Corporation Comments and OIG Evaluation
Management Response to Recommendations
Corporation Comments



Part I

Report by IBM Business Consulting Services




IBM Business Consulting Services



FDICís Information Technology
Configuration Management Controls
Over Operating System Software


Report No. 05-031



Prepared for the
Federal Deposit Insurance Corporation
Office of Inspector General

Submitted by:IBM Business Consulting Services
  Security, Privacy, & Wireless
  12902 Federal Systems Park Drive
  Fairfax, VA 22033


FDICís IT Configuration Management Controls Over Operating System Software
Prepared for the Federal Deposit Insurance Corporation Office of Inspector General


TABLE OF CONTENTS

Section
  1. Executive Summary
  1. Background
  1. Detailed Finding
          Configuration Management Controls for Operating System Software
Appendix A:   Objective, Scope, and Methodology
Appendix B:   Additional Information on the Capability Maturity Model Integration
Appendix C:   Laws and Regulations
Appendix D:   Acronyms
Appendix E:   Glossary of Terms
 
List of Tables
 
Table 1:   Operating System Software Platforms Reviewed
Table 2:   CMMI Bodies of Knowledge
Table 3:   Capability Levels (Continuous Representation) Excerpts from the CMMI
 
List of Exhibits
 
Exhibit 1:   Software Configuration Management Principles of the CMMI
Exhibit 2:   Change Management Process Steps
Exhibit 3:   Configuration Identification
Exhibit 4:   Configuration Control
Exhibit 5:   Configuration Accounting
Exhibit 6:   Configuration Auditing
Exhibit 7:   Windows® Server and Desktop Software Configuration Changes


FDICís IT Configuration Management Controls Over Operating System Software
Prepared for the Federal Deposit Insurance Corporation Office of Inspector General


1. Executive Summary

The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) contracted with International Business Machines (IBM) Business Consulting Services (hereafter referred to as IBM) to audit and report on the effectiveness of the FDICís configuration management controls over operating system software. Federal agencies are required by the Federal Information Security Management Act of 2002 (FISMA)[ 1 ] to establish and implement minimally acceptable configuration requirements for their information systems. In addition, agency Chief Information Officers (CIO) and Inspectors General are required by Office of Management and Budget (OMB) policy to assess and report on the implementation of agency configuration management controls as part of their annual FISMA reviews and evaluations. The results of this audit support the OIG in fulfilling its evaluation and reporting responsibilities under FISMA. IBM conducted its work in accordance with generally accepted government auditing standards.

The objective of the audit was to determine whether the FDIC had established and implemented configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. The scope of the audit focused on four of the FDICís operating system software platforms: (1) Microsoft Windows» for servers, (2) Microsoft Windows® for desktop (and laptop) computers, (3) Sun Microsystems, Inc.ís Solaris™ for servers, and (4) Cisco IOS® for telecommunications. IBM chose these four platforms because they support many of the FDICís sensitive and mission-critical business applications. IBM used the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-53, Recommended Security Controls for Federal Information Systems, and the Capability Maturity Model Integration (CMMI)[ 2 ] developed by Carnegie Mellon Universityís Software Engineering Institute (SEI) as the primary criteria for conducting the audit. IBM chose the CMMI because it defines a generally accepted set of software configuration management principles, and the FDICís Division of Information Technology (DIT) has embraced the CMMI as a means of achieving process improvement. The recommendations contained in this report are designed to promote compliance with federal standards and guidelines and further the FDICís goal of achieving CMMI process improvements.

A detailed description of the auditís scope and methodology is contained in Appendix A. Appendix D contains a list of acronyms, and Appendix E contains a glossary of terms used in the report.

Results of the Audit

DIT established and implemented a number of configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. Such controls included a software patch management policy, a FDIC Infrastructure Change Control Board,[ 3 ] and periodic scanning of operating system software configurations. Such actions were positive; however, control improvements were needed. Specifically, DIT needed to establish an organizational policy and system-specific procedures to ensure proper configuration management of operating system software. DIT also needed to standardize and integrate the recording, tracking, and reporting of operating system software configuration changes to the extent practical. Collectively, these control weaknesses limited the FDICís assurance that its configuration management practices were efficient and effective and that systems were configured to minimize the risk of security vulnerabilities and service interruptions. A summary of IBMís recommendations follows.

Summary Recommendations

IBM recommends that the FDIC CIO:

  • Establish a policy that takes an enterprise approach to defining the roles, responsibilities, and overall principles and management expectations for performing configuration management on operating system software. The policy should address requirements for developing and maintaining configuration management plans and performing periodic self-assessments of configuration management processes and practices.

  • Develop configuration management plan(s) covering the four operating system software platforms addressed in this report consistent with federal standards and guidelines and industry-accepted practices. DIT should determine whether other operating system software platforms require configuration management plan(s) and develop such plans where appropriate.

  • Ensure that the certification and accreditation of the FDICís general support systems incorporate an evaluation and testing of the FDICís configuration management policy and plans referenced in recommendations 1 and 2 of this report.

  • Document the minimum required configuration settings for the Windowsģ server and desktop operating system platforms, and develop procedures to ensure that changes to baseline configuration settings are documented.

  • Standardize and integrate the recording, tracking, and reporting of operating system software configuration changes to the maximum extent practical. As part of this effort, DIT should consider using automated mechanisms to improve performance metric reporting for configuration changes from a system-specific and enterprise perspective.

2. Background

Key to ensuring the integrity, security, and reliability of any information system is implementing structured processes for managing the inevitable changes that will occur during the systemís life cycle. Such processes, collectively referred to as configuration management, include evaluating, authorizing, testing, tracking, reporting, and verifying both hardware- and software-related changes. Typical changes that should be subject to formal configuration management in the operating system software environment include the installation of security patches that address known vulnerabilities, programs that support system maintenance, and upgrades (referred to as ďservice packsĒ) that improve system performance, security, and functionality. Without disciplined processes for controlling software changes, management cannot be assured that its systems will operate as intended, that software defects will be minimized, or that systems maintenance will be performed in a cost-effective or timely manner.

A number of internationally recognized software configuration management standards are in wide use today. These include standards published by the SEI,[ 4 ] Project Management Institute, American National Standards Institute/Institute of Electrical and Electronic Engineers, and International Organization for Standardization. IBM selected SEIís CMMI as a key criterion for conducting the audit because the CMMI defines generally accepted software configuration management principles, and DIT has embraced the CMMI as a means of achieving information technology (IT) process improvement. In addition, the configuration management principles embodied in the CMMI are consistent with the configuration management security controls defined in NIST SP 800-53 for non-national security federal information systems. Exhibit 1 depicts the software configuration management principles of the CMMI. These four principles are based on the best practices of carefully chosen disciplines, including systems analysis and design and software engineering. IBMís audit results are organized around these four fundamental principles, which are described in greater detail below. Exhibit 1: Software Configuration Management Principles of the CMMI

Exhibit 1, is a pie graph of the CMMI organized around four fundamental principles: 1) Configuration Identification, 2) Configuration Control, 3) Configuration Accounting, and 4) Configuration Auditing.

Source: IBM Analysis of the CMMI.    [ D ]

Configuration Identification involves the establishment of baseline configurations for ďwork productsĒ that are subject to configuration management. Typical work products subject to configuration management within the operating system environment include software, such as the system itself and its component parts, and documents, such as server and desktop build procedures, systems inventories, and configuration management plans. Establishing and enforcing baseline configurations is critical to reducing software maintenance costs and ensuring secure and reliable systems. For example, without a baseline configuration for desktop operating systems, such as Microsoft Windows® XP Professional Service Pack 2, organizations would expend excessive time and resources patching, troubleshooting, and testing changes for earlier desktop versions of Windowsģ, such as Windowsģ XP Professional Service Pack 1. Baseline configurations serve as the basis for future development and should be modified only through the change control processes described under Configuration Control below.

Configuration Control is the heart of configuration management because it involves implementing a change management system to systematically control and monitor changes to the baseline configurations established under Configuration Identification. A change management system consists of policies, procedures, automated tools, and other controls for performing the change management process steps depicted in Exhibit 2. Configuration Control ensures that proposed software changes are properly evaluated for compliance with applicable standards, assessed and tested to avoid potential disruptions or compromise of system security, approved or disapproved by appropriate authorities, verified upon completion, and reflected in baseline configurations. Exhibit 2: Change Management Process Steps
Exhibit 2, is a graph that depicts a circular seven step change management process beginning with: Step 1: Change Initiation; Step 2: Change Classification; Step 3: Change Analysis; Step 4; Change Disposition; Step 5: Change Implementation; Step 6: Change Verification; and Step 7: Baseline Update.

Source: IBM Analysis of the CMMI.
[ D ]


Configuration Status Accounting is the recording and reporting of configuration management activities in sufficient detail to provide stakeholders with information needed to manage their work products. Such information can include metrics such as the number of in-process and completed configuration changes at a particular point in time, the average time spent processing high-priority or low-priority changes, and the number of changes that address new requirements or defects in work products.[ 5 ] By performing trend analysis of such metrics, management can identify potential problems in its configuration management processes. Configuration Status Accounting information is used to make important policy, resource, and budget decisions and assists managers in determining whether configuration management processes and practices are efficient, effective, and achieving intended results. Due to the complexity and volume of configuration changes associated with operating system software, many organizations use automated configuration management tools to centrally track, record, and report the status of configuration changes.

Configuration Auditing involves the self-assessment of an organizationís configuration management activities and processes to determine whether controls function as intended. Such self-assessments provide management with assurance that baseline configurations and related documentation are current, accurate, and complete and that implemented changes can be traced to original requirements. The results of configuration management audits can be subject to review by independent third parties.

Roles and Responsibilities for Operating System Software

DIT has overall responsibility for maintaining the configuration of the FDICís operating system software. Responsibility for maintaining the configuration of individual operating system software platforms is shared among two branches and several sections within DITís Infrastructure Services (IS). Table 1 below identifies the four operating system software platforms selected for the audit, along with a brief description of the platform and the IS section with primary responsibility for its configuration.

Table 1: Operating System Software Platforms Reviewed

Operating
System Software Platform
General Description IS Branch/Section With Primary Responsibility
Microsoft Windows® for Servers Windows® 2000 Advance Server is the standard operating system in the network server environment.* The Windows® server platform supports network-based applications, such as the New Financial Environment and FDICconnect, as well as IT services for managing the network. Software Support Branch, Server Software Section Operations Branch, LAN Management Section
Microsoft Windows® for Desktop (and Laptop) Computers Windows® XP Professional is the standard operating system in the desktop computer environment. Windows® XP Professional primarily supports user productivity tools, such as the Internet Explorer and Microsoft Office Suite. Software Support Branch, Client Software Section Operations Branch, LAN Management Section
Sun Microsystems, Inc., Solaris™ Solaris™ is the standard operating system in the UNIX® environment. Solaris™ supports core IT infrastructure services, such as the Public Key Infrastructure, and a number of business applications, such as the Overarching Automation System and Corporate Human Resources Information System   Time and Attendance system. Software Support Branch, Server Software Section Operations Branch, Telecommunications Section
Cisco IOS® Cisco IOS® supports the routing, message processing, and protocol interfaces needed to transfer data over  the FDICís LANs, metropolitan area networks, and wide area network. Operations Branch, Telecommunications Section

* At the time of our audit, DIT supported a limited number of servers operating the Windows NT® and            [ D ]
Windows® 2003 operating systems.

Recent Control Improvements

DIT has taken a number of recent actions to strengthen its configuration management controls over operating system software, and additional improvements were underway during the audit. Of particular note, DIT issued a formal patch management policy,[ 6 ] established the FDIC Infrastructure Change Control Board, and strengthened its security vulnerability scanning techniques[ 7 ] for network devices. DIT also established a new performance-based contracting structure in September 2004 that included a service-level agreement to achieve CMMI process improvement in infrastructure operations (including software configuration management). In addition, DIT was working to improve the integrity of patch information reported by the Microsoft Systems Management Server (SMS) 2003[ 8 ] and implement Symantecís Enterprise Security Manager on the FDICís IT infrastructure to better monitor the configuration of Windows® servers. While such improvements promote an enterprise-wide approach to performing configuration management, additional controls are needed to ensure proper management of operating system software configuration.

3. Detailed Finding: Configuration Management Controls for Operating System Software

CONDITION

DIT established and implemented a number of configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. Such controls included a software patch management policy, an FDIC Infrastructure Change Control Board, and periodic scanning of operating system software configurations. Such actions were positive; however, control weaknesses existed relating to each of the four configuration management principles defined in the CMMI. A description of these weaknesses follows.

Configuration Identification

DIT used a number of key work products, such as server and desktop build procedures, ghost imaging procedures, software image files, and system inventories, to manage the configuration of the four operating system software platforms that we reviewed. DIT had not subjected these work products to formal configuration management, as defined by the CMMI, to ensure that they were current, accurate, and complete. Specifically, DIT had not developed procedures for (1) identifying work products that should be subject to configuration management, (2) designating responsibility for maintaining current and historical versions of work products, (3) designating authority to approve changes to work products, and (4) determining when work products should be revised.[ 9 ] Such procedures are typically documented in a configuration management policy and/or system configuration management plan(s).

Exhibit 3: Configuration Identification

Exhibit 3 depicts a pie graph of the software configuration management principles of the CMMI organized around four fundamental principles with an expanded view of the first fundamental principle of the CMMI called Configuration Identification.

Source: IBM Analysis of the CMMI.     [ D ]

DIT defined configuration procedures in various documents but had not documented procedures for the ongoing identification and documentation of minimum custom baseline configuration settings applicable to the Windows® server and desktop operating system platforms. Custom configuration settings include, for example, registry permissions, Internet Explorer settings, local system security settings, and unnecessary computer services that should be disabled or uninstalled. In OIG Audit Report No. 05-016 entitled, Audit of Security Controls Over the FDICís Electronic Mail Infrastructure, dated March 31, 2005, IBM noted that DIT had not documented procedures for disabling unnecessary computer services on Exchange e-mail servers when appropriate. Based on a sample of 15 e-mail servers, IBM identified 6 computer services that had been enabled but were not required for processing e-mail. The six unnecessary computer services presented a potential security risk because they could have been exploited by a virus, worm, or other malicious program to damage the FDICís IT resources.

Configuration Control

DIT had not standardized or integrated its processes for tracking and recording configuration changes within or across its operating system software platforms to ensure that such changes were properly controlled, accounted for, and reported. For example, software-related changes[ 10 ] for one of four operating system software platforms reviewed (i.e., Windows® server) were documented using standard forms that were stored in a central location. However, hardware-related changes[ 11 ] for this same platform were neither recorded using a standard form nor stored in a central repository. In order to obtain information about hardware-related changes for this platform, such as when a change was made, who implemented a change, testing that was performed for a change, or who verified a change, IBM had to speak with systems personnel with first-hand knowledge of the changes. Configuration changes for another of the four operating system software platforms (i.e., Sun Microsystems, Inc.ís Solaris™ for servers) were either managed through an automated change management tool[ 12 ] or recorded in various Microsoft Word documents, depending on which IS branch and section had responsibility for the servers affected by the change.

Exhibit 4: Configuration Control

Exhibit 4 depicts a pie graph of the software configuration management principles of the CMMI organized around four fundamental principles with an expanded view of the second fundamental principle of the CMMI called Configuration Control.

Source: IBM Analysis of the CMMI.    [ D ]

Although DIT had developed test plans and roll-back plans[ 13 ] for its major operating system upgrades, test plans and roll-back plans were generally not documented for other configuration changes. Although IS personnel stated that they had tested configuration changes before they were implemented, test plans and test results were not documented for 22 of 25 judgmentally selected configuration changes to the four operating system software platforms. In addition, 21 of 25 configuration changes did not have a documented roll-back plan. Test plans, roll-back plans, and test results are important for ensuring that configuration changes function as intended and have no negative impact on IT operations. In January 2005, several hundred FDIC users were unable to access their Outlook e-mail for several days following an unsuccessful configuration change to a key e-mail server. IBM noted that DIT had not documented a test plan, test results, or roll-back plan for this configuration change. Test results also document system incompatibilities and managementís rationale for making specific configuration decisions. The level of documentation needed for test plans, roll-back plans, and test results should be based on the risk and complexity of a configuration change and could be as simple as a completed checklist or memorandum.

Configuration Accounting

DIT collected and reported configuration status information on the four operating system software platforms that we reviewed through a variety of means, including the FDIC Infrastructure Change Control Board meetings, SMS reports on patch deployments, and software scanning techniques. However, DIT did not track or report key configuration management metrics within or across its operating system software platforms. Such metrics could include, for example, the number of in-process and completed configuration changes, the status of in-process changes, or the average amount of time (i.e., speed) to implement high-priority versus low-priority changes. In addition, because DIT did not classify configuration changes as addressing new requirements or defects in work products, DIT was unable to determine the amount of effort expended to enhance software versus correct problems.

Exhibit 5: Configuration Accounting

Exhibit 5 depicts a pie graph of the software configuration management principles of the CMMI organized around four fundamental principles with an expanded view of the third fundamental principle of the CMMI called Configuration Accounting.

Source: IBM Analysis of the CMMI.    [ D ]

Configuration Auditing

DIT used various automated tools to evaluate the configuration of its operating system software,[ 14 ] but had not developed self-assessment procedures for determining whether configuration management controls functioned as intended. Developing self-assessment procedures is a recognized practice in configuration management and could be used to detect or prevent the types of process weaknesses identified in this report. Common self-assessment procedures include evaluating the integrity of key work products subject to configuration management; determining whether change request documentation is current, accurate, and complete; inspecting configuration changes for compliance with applicable policies, procedures, and guidelines; and verifying that configuration changes have been implemented as intended

Exhibit 6: Configuration Auditing

Exhibit 6 depicts a pie graph of the software configuration management principles of the CMMI organized around four fundamental principles with an expanded view of the fourth fundamental principle of the CMMI called Configuration Auditing.

Source: IBM Analysis of the CMMI.    [ D ]

CAUSE

Several causes contributed to the control weaknesses in DITís configuration management processes and practices as discussed below.

Organizational Policy and System-Specific Procedures

Circular 1320.4, FDIC Software Configuration Management Policy, dated July 8, 2003, establishes key roles and responsibilities, management expectations, and requirements for configuration management (including the need for configuration management plans, reviews of configuration management processes, and the use of automated configuration management tools). However, the circular is limited to application software and does not address operating system software. An organizational configuration management policy should define overall principles and expectations for performing operating system software configuration management and the associated roles and responsibilities of key personnel and organizational components, such as system engineers, administrators, and the FDIC Infrastructure Change Control Board. The policy should also address, and be a critical component of, the certification and accreditation process for operating system software.[ 15 ] An organizational policy is an important component of an enterprise approach to software configuration management.

DIT documented a number of its configuration management practices in various policies and procedures and established the FDIC Infrastructure Change Control Board to oversee configuration changes to the FDICís operating system software. However, DIT had not developed configuration management plan(s) for any of the four operating system software platforms that IBM audited. Configuration management plans are a fundamental control for maintaining proper configuration of information systems because the plans define procedures for evaluating, classifying, authorizing, testing, documenting, and verifying configuration changes. These plans also define the system-specific roles and responsibilities (including controls for ensuring appropriate separation of duties) of key stakeholders and procedures for identifying, maintaining, and updating work products subject to configuration management. In addition, configuration management plans can describe the type and frequency of configuration status information to be tracked and reported to management, training requirements for key personnel, and requirements for conducting self-assessments of configuration management controls. Configuration management plans are an important component of system certification and accreditation. Because systems vary in complexity and design, configuration management plans should be tailored to the requirements of individual systems.

Automated Workflow Processes

The lack of standardization and integration in DITís practices of recording, tracking, and reporting system configuration changes was caused primarily by a lack of automated workflow processes using configuration management tools. System change requests for three of four operating system software platforms reviewed were not managed with an automated workflow process tool. Although DIT used a workflow process tool to manage change requests on the remaining platform (i.e., Sun Microsystems, Inc.ís Solaris™ for servers), the tool was not used to track all change requests on the platform. In addition, system change requests for two other platforms (i.e., Windows® server and desktop) were generally stored in Outlook e-mail folders and Microsoft Word documents rather than in a central repository that could be used for tracking, reporting, or interfacing with other corporate systems, such as REMEDY®.[ 16 ] The lack of automated workflow processes using configuration management tools contributed to the absence of meaningful configuration management metrics discussed earlier.

Exhibit 7 illustrates the combined number of application and infrastructure configuration changes affecting two of the four operating system software platforms that IBM reviewed.[ 17 ] An automated workflow process tool to manage such a large number of configuration changes offers many advantages, such as defined business processes built into the tool that promote consistent, auditable, and repeatable change control practices. Configuration requirements can vary from system to system. Therefore, a single automated workflow process tool may not satisfy all configuration management needs. Selecting a workflow process tool should be based on an assessment of DITís existing information systems and commercially available software products. Exhibit 7: Windowsģ Server and Desktop Software Configuration Changes

Exhibit 7 is a bar graph that illustrates on the left-side vertical axis ranging between 0 and 3000 the ďNumber of ChangesĒ and on the horizontal axis the years 2000 through 2004, the ďYearĒ actual change occurred.
Source: Analysis of DITís Windows® Implementation Announcements Outlook E-Mail Folders.

    [ D ]

CRITERIA

The CMMI, NIST, and OMB have established a number of guiding principles for effective software configuration management as discussed below.

Organizational Policy and System-Specific Procedures

The CMMI identifies an organizational configuration management policy as an important control for effectively planning and performing software configuration management. According to the CMMI, the configuration management policy defines organizational expectations for establishing and maintaining configuration baselines and tracking and controlling changes to work products subject to configuration management. Additionally, NIST SP 800-53 references a ďformal, documented, configuration management policy that addresses purpose, scope, roles, responsibilities, and compliance [i.e., self-assessments]Ē as a recommended security control for protecting federal information systems.

A key practice in the CMMI is performing periodic audits (i.e., self-assessments) of configuration management activities and processes to ensure the integrity of baseline configurations and related documentation. Self-assessments evaluate compliance with applicable configuration management standards and procedures and verify the integrity of items in the configuration management system based on requirements documented in the configuration management plan. In addition, NIST SP 800-53 recognizes audit activities associated with configuration changes to federal information systems as a recommended security practice.

The CMMI recognizes the importance of establishing and maintaining a configuration management plan. According to the CMMI, an important configuration management practice is identifying work products that will be subject to configuration management based on documented criteria and designating individuals responsible for work product maintenance. NIST SP 800-53 identifies a configuration management plan as a recommended security control for protecting federal information systems. The NIST publication also references ďformal, documented procedures to facilitate the implementation of the configuration management policy and associated configuration management controlsĒ as a component of a security control structure. In addition, NIST SP 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, notes that configuration management plans are a key security-related document that can be part of the security accreditation package. The Computer Security Division of NIST maintains examples of configuration management plans for operating system software at its Web site http://csrc.nist.gov/fasp under the link entitled ďFASP Areas[ 18 ]

Further, NIST SP 800-53 recognizes the importance of maintaining an organization-defined list of prohibited computer services and configuring IT products to the most restrictive mode consistent with system operational requirements. In addition, NIST SP 800-70, The NIST Security Configuration Checklists Program, contains detailed guidance that should be used when configuring operating system software settings.

Automated Workflow Processes

The CMMI addresses the importance of establishing and maintaining a change management system to store, update, and retrieve configuration management records and other work products subject to configuration management. The change management system includes a change request database wherein change requests are initiated and recorded and configuration management reports are generated. NIST SP 800-53 states that organizations should employ automated mechanisms to (i) document proposed changes to the system, (ii) notify appropriate approval authorities, (iii) identify approvals that have not been received in a timely manner, (iv) inhibit change until necessary approvals are received, and (v) document completed changes to the information system. NIST SP 800-53 also recognizes that organizations should employ automated mechanisms to maintain an up-to-date, complete, accurate, and readily available baseline configuration. Automating change control practices is also consistent with DITís infrastructure services contract, which requires contractor personnel, whenever possible and practical, to consolidate IT platforms and operations, integrate infrastructure requirements into efficient and effective solutions, and capture and deliver information in real or near-real time using electronic means.

OMB Circular No. A-130, Management of Federal Information Resources, dated November 28, 2000, requires agencies to institute performance measures and management processes that monitor actual performance against expected results. The CMMI describes various types of configuration metrics, such as the status of change requests and number of change requests related to software defects, that are part of configuration management. Such metrics would be a valuable asset to DIT management in evaluating the performance of the FDICís IT infrastructure services contractors.

EFFECT

The lack of an organizational policy and system-specific configuration management procedures limited the FDICís assurance that its information systems were configured to minimize the risk of security vulnerabilities and IT service interruptions. Absent appropriate policies and procedures, DIT is overly dependent on the knowledge and experience of individual system engineers and administrators to maintain minimum baseline configuration settings consistently across platforms. The lack of automated workflow processes impaired DITís ability to efficiently and effectively manage system configuration changes throughout the systemsí life cycle and to report meaningful configuration metrics to management. In addition, the lack of periodic self-assessments of configuration management controls limited DITís assurance that key work products were current, accurate, and complete and that configuration management processes were efficient, effective, and achieving intended results.

RECOMMENDATIONS

IBM recommends that the FDIC CIO:

  1. Establish a policy that takes an enterprise approach to defining the roles, responsibilities, and overall principles and management expectations for performing configuration management on operating system software. The policy should address requirements for developing and maintaining configuration management plans and performing periodic self-assessments of configuration management processes and practices.
  2. Develop configuration management plan(s) covering the four operating system software platforms addressed in this report consistent with federal standards and guidelines and industry-accepted practices. DIT should determine whether other operating system software platforms require configuration management plan(s) and develop such plans where appropriate.
  3. Ensure that the certification and accreditation of the FDICís general support systems incorporate an evaluation and testing of the FDICís configuration management policy and plans referenced in recommendations 1 and 2 of this report.
  4. Document the minimum required configuration settings for the Windows® server and desktop operating system platforms and develop procedures to ensure that changes to baseline configuration settings are captured and documented.
  5. Standardize and integrate the recording, tracking, and reporting of operating system software configuration changes to the maximum extent practical. As part of this effort, DIT should consider using automated mechanisms to improve performance metric reporting for configuration changes from a system-specific and enterprise perspective.


APPENDIX A: OBJECTIVE, SCOPE, AND METHODOLOGY

The objective of the audit was to determine whether the FDIC had established and implemented configuration management controls over its operating system software that were consistent with federal standards and guidelines and industry-accepted practices. The scope of the audit focused on four operating system software platforms: (1) Microsoft Windows® for servers,[ 19 ] (2) Microsoft Windows® for desktop (and laptop) computers, (3) Sun Microsystems, Inc.ís Solaris™ for servers, and (4) Cisco IOSģ for telecommunications. IBM chose these four platforms because they support many of the FDICís sensitive and mission-critical business applications. The audit did not include an evaluation of the FDICís configuration management controls over applications running on the four operating system software platforms because DIT was performing an internal assessment of application configuration management controls.

To accomplish the audit objective, IBM interviewed key DIT personnel who had responsibility for maintaining the configuration of the four operating system software platforms. IBM reviewed relevant FDIC policies, procedures, and guidelines and evaluated key documents and reports, such as server and desktop build procedures, system security plans, and DITís Daily Reports for Technical Infrastructure. IBM also reviewed DITís security self-assessment procedures that were designed to ensure the secure configuration of the four selected platforms. In addition, IBM evaluated DITís change control procedures for each of the four platforms by testing a judgmental sample of configuration changes. Using source documentation and DITís automated systems of record, IBM determined whether selected configuration changes had been properly authorized, evaluated, tested, tracked, implemented, and reported. IBM coordinated its work with an ongoing DIT internal assessment of the security patch management process for desktop computers.

IBM used NIST SP 800-53, Recommended Security Controls for Federal Information Systems, and the configuration management principles defined in the CMMI, developed by Carnegie Mellon Universityís SEI, as the primary criteria for conducting the audit. In addition, IBM used relevant provisions of the Government Accountability Officeís Federal Information System Controls Audit Manual[ 20 ] and NIST SP 800-37, Guide for the Security Certification and Accreditation of Federal Information Systems, as supplemental criteria. The OIG evaluated the nature, timing, and extent of work described in IBMís audit program, obtained an understanding of IBMís methodologies and assumptions, attended key meetings, monitored progress throughout the audit, and performed other appropriate procedures. In this manner, the OIG was able to assure itself that, except for the performance of an external peer review, IBM's audit work complied with generally accepted government auditing standards. IBM conducted its field work from October 2004 through June 2005.

Prior Audit Coverage

On September 1, 2000, the OIG issued Audit Report No. 00-038 entitled, Audit of the Information Technology Configuration Management Program. The report states that the then Division of Information Resources Management (now DIT) was in the process of developing a plan to establish a formal IT configuration management program. The report discusses the salient components of effective configuration management and recommends considering these components in developing and implementing a formal configuration management program. FDIC management had taken action sufficient to close the recommendation.

Computer-based Data, Performance Measures, and Illegal Acts

IBM performed appropriate procedures to ensure that computer-based data were valid and reliable when those data were significant to the auditís findings and conclusions. Such procedures included verifying selected automated data to source documentation and corroborating automated data through interviews with appropriate DIT personnel. In addition, IBM evaluated whether DITís configuration management performance metrics for operating system software were consistent with federal and industry guidance. Finally, IBM did not develop specific audit procedures to detect fraud and illegal acts because they were not considered material to the audit objective. However, throughout the audit, IBM was sensitive to the potential of fraud, waste, abuse, and mismanagement.


APPENDIX B: ADDITIONAL INFORMATION ON THE CAPABILITY MATURITY MODEL INTEGRATION

The CMMI combines a carefully chosen set of best practices based on experience in four bodies of knowledge (described in Table 2 below). Organizations from industry, government, and Carnegie Mellon Universityís SEI jointly developed the CMMI to provide organizations a mechanism to effectively appraise their process area capabilities, establish priorities, and implement improvements.

Table 2: CMMI Bodies of Knowledge

Body of Knowledge Description
System Engineering Covers the development of total systems, which may or may not include software. Systems engineers focus on transforming customers' needs, expectations, and constraints into products and supporting these products throughout their life cycle.
Software Engineering Covers the development of software systems. Software engineers focus on applying systematic, disciplined, and quantifiable approaches to the development, operation, and maintenance of software.
Integrated Product and Process Development Provides a systematic approach that achieves a timely collaboration of relevant stakeholders throughout the life of a product to satisfy customers' needs, expectations, and requirements. The processes to support an Integrated Product and Process Development approach are integrated with the other processes in the organization.
Supplier Sourcing Covers the use of suppliers to perform functions or add modifications to products that are specifically needed by a project. Projects benefit from enhanced source analysis and monitoring supplier activities before product delivery.

The CMMI supports two representations: staged and continuous. The staged representation provides a proven sequence of improvements, beginning with basic management practices and progressing through a pre-defined and proven path of successive levels, each serving as a foundation for the next. The staged representation permits comparisons across and among organizations by the use of overall, organization-wide maturity levels. The continuous representation allows an organization to select the order of improvement that best meets its business objectives and mitigates the organizationís areas of risk. The continuous representation enables comparisons across and among organizations on a process-area-by-process-area basis, using the six capability levels depicted in Table 3 on the next page. Both representations are designed to achieve essentially equivalent results.

Table 3: Capability Levels (Continuous Representation) Excerpts From the CMMI

Capability Level Capability Level Description
0 Incomplete. Reflects processes that are either not performed or partially performed.
1 Performed. Reflects processes that support and enable the work needed to produce identified work products.
2 Managed. Reflects processes that are planned and executed in accordance with policy, employ skilled people having adequate resources to produce controlled outputs, involve relevant stakeholders, are monitored, controlled, and reviewed, and are evaluated for adherence to its process description. A critical distinction between a performed process and a managed process is the extent to which the process is managed. A managed process is planned, and the performance of the process is managed against the plan. Corrective actions are taken when the actual results and performance deviate significantly from the plan.
3 Defined. Reflects managed processes that are tailored from the organization's set of standard processes according to the organizationís tailoring guidelines and contributes work products, measures, and other process-improvement information to the organizational process assets. The organizationís set of standard processes, which are the basis of the defined process, are established and improved over time.
4 Quantitatively Managed. Reflects processes that are controlled using statistical and other quantitative techniques. Quantitative objectives for quality and process performance are established and used as criteria in managing the process.
5 Optimizing. Reflects processes that are quantitatively managed and changed and adapted to meet relevant current and projected business objectives. An optimizing process focuses on continually improving process performance through both incremental and innovative technological improvements. Process improvements that would address root causes of process variation and measurably improve the organizationís processes are identified, evaluated, and deployed as appropriate.

IBM used the CMMI Version 1.1 for Systems Engineering, Software Engineering, Integrated Product and Process Development, and Supplier Sourcing (Continuous Representation), dated March 2002, (CMMI-SE/SW/IPPD/SS, V1.1) as a key criterion for conducting the audit because the CMMI defines generally accepted software configuration management principles, and DIT has embraced the CMMI as a means of achieving IT process improvement.


APPENDIX C: LAWS AND REGULATIONS

Below are the key statutes, regulations, standards, and guidelines that were considered during the audit. Statutory and regulatory sources may not be legally binding on the FDIC; see individual references for further information.

Federal Information Security Management Act (FISMA), title III, E-Government Act of 2002, Pub. L. No. 107-347, dated December 17, 2002
http://www.cio.gov/archive/e_gov_act_2002.pdf

Enacted as part of the E-Government Act of 2002, FISMA permanently re-authorized and strengthened the information security program, evaluation, and reporting requirements established by the Government Information Security Reform Act (GISRA), which expired in November 2002. FISMA provides a comprehensive framework for ensuring the effectiveness of information security controls over information resources supporting federal operations and assets. Among its provisions, FISMA requires federal agencies to establish and implement minimally acceptable configuration requirements for their information systems; see section 301(b). For purposes of that section, the FDIC is considered an agency and is, therefore, subject to its provisions.

OMB Circular No. A-130, Management of Federal Information Resources (Transmittal Memorandum No. 4) Appendix III, Security of Federal Automated Information Resources, dated November 2000 (OMB A-130, Appendix III)
http://www.whitehouse.gov/omb/circulars/a130/a130appendix_iii.html

OMB A-130, Appendix III, establishes minimum controls for federal automated information security programs. The FDICís Legal Division has opined that portions of the circular apply to the FDIC, while other portions do not apply. The Legal Division specifically opined that Appendix III of the circular legally requires the FDIC to implement and maintain an information security program consistent with government-wide policies, standards, and procedures issued by the OMB and the U.S. Department of Commerce.

NIST Special Publication (SP) 800-53, Recommended Security Controls for Federal Information Systems, dated February 2005
http://csrc.nist.gov/publications/nistpubs/800-53/SP800-53.pdf

The publication defines minimum recommended security controls for non-national security federal information systems based on the impact levels defined in NIST Federal Information Processing Standard (FIPS) 199, Standards for Security Categorization of Federal Information and Information Systems. NIST SP 800-53 provides guidelines for selecting and specifying minimum security controls for federal information systems until the publication of FIPS 200, Minimum Security Requirements for Federal Information and Information Systems (projected for publication December 2005). The guidelines have been developed to help achieve more secure systems within the federal government. NIST SPs are, by their own terms, guidelines (rather than mandatory requirements) for agencies in implementing their IT operations.


APPENDIX D: ACRONYMS

Acronyms Definition
CIO Chief Information Officer
CMMI Capability Maturity Model Integration
DIT Division of Information Technology
FCMS FDIC Change Management System
FDIC Federal Deposit Insurance Corporation
FISMA Federal Information Security Management Act
IBM International Business Machines Business Consulting Services
IS Infrastructure Services
IT Information Technology
NIST National Institute of Standards and Technology
OIG Office of Inspector General
OMB Office of Management and Budget
POA&M Plan of Action & Milestones
SEI Software Engineering Institute
SMS Systems Management Server
SP Special Publication


APPENDIX E: GLOSSARY OF TERMS

Term Definition
Accreditation Security accreditation is the official management decision given by a senior agency official to authorize operation of an information system and to explicitly accept the risk to agency operations, agency assets, or individuals based on the implementation of an agreed-upon set of security controls.
Baseline A set of configuration items that has been reviewed and agreed upon and, thereafter, serves as the basis for future management, development, or maintenance.
Build Procedures Automated and documented procedures used for installing operating system software on servers and desktops.
Capability Level A capability level consists of related specific and generic practices for a process area that can improve the organizationís processes associated with that process area.
Certification Certification is a comprehensive assessment of the management, operational, and technical security controls in an information system, made in support of security accreditation, to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system.
Configuration Accounting An element of configuration management consisting of the recording and reporting of information needed to manage a configuration effectively.
Configuration Audit A self-assessment conducted to verify that a configuration item conforms to a specified standard or requirement.
Configuration Control An element of configuration management consisting of the evaluation, coordination, approval or disapproval, and implementation of changes to configuration items following configuration identification.
Configuration Identification An element of configuration management consisting of selecting configuration items for a product, assigning unique identifiers to them, and recording their functional and physical characteristics in technical documentation.
Configuration Item An aggregation of work products designated for configuration management and treated as a single entity in the configuration management process.
General Support System An interconnected set of information resources under the same direct management control. This system normally includes hardware, software, information, data, applications, communications, and people.
National Institute of Standards and Technology A non-regulatory federal agency within the U.S. Department of Commerceís Technology Administration. NIST publishes technical, physical, administrative, and management standards and guidelines for the cost-effective security and privacy of sensitive, but unclassified, information in federal computer systems.
Plan of Action and Milestones A plan of action and milestones (POA&M), sometimes referred to as a corrective action plan, is a tool that identifies tasks to be accomplished. It details resources required to accomplish the elements of the plan, milestones in meeting the task, and scheduled completion dates for the milestones. POA&Ms assist management in identifying, assessing, prioritizing, and monitoring the progress of corrective efforts for weaknesses identified in programs and systems.
Roll-back Plan A documented plan (sometimes called a ďback-out planĒ) that describes the system recovery steps to be followed should a configuration change cause an unexpected, negative effect on an organizationís IT operations.
Security Vulnerability A flaw or weakness in the design or implementation of an information system (including the security procedures and security controls associated with the system) that could be intentionally or unintentionally exploited to adversely affect an organizationís operations or assets through a loss of confidentiality, integrity, or availability.
Software Configuration Management The technical and administrative processes of identifying, documenting, and maintaining configuration item integrity; controlling configuration item changes; recording and reporting on configuration item change status; and verifying compliance with policy.
Work Product An artifact produced by a process, such as server and desktop build procedures, ghost imaging procedures, software image files, system inventories, and other files, documents, services, processes, and specifications.



Part II

Corporation Comments and OIG Evaluation



CORPORATION COMMENTS AND OIG EVALUATION

The report contains five recommendations directed to the CIO. The CIOís response to the draft report is presented in its entirety, beginning on page II-6. DIT concurred with two of the reportís recommendations, and partially concurred with the remaining three recommendations. Based on the CIOís response, the five report recommendations are considered resolved but will remain undispositioned and open until we have determined that agreed-to corrective actions have been completed and are effective. The CIOís responses for each of the reportís recommendations are summarized below along with our evaluation of the responses.

Recommendation 1: The CIO should establish a policy that takes an enterprise approach to defining the roles, responsibilities, and overall principles and management expectations for performing configuration management on operating system software. The policy should address requirements for developing and maintaining configuration management plans and performing periodic self- assessments of configuration management processes and practices.

DIT Response: DIT partially concurs with the recommendation. DIT does not believe that a single policy covering all types of software is necessarily the best approach. However, DIT agrees to review its policies to determine how to effectively cover configuration management of the various operating systems and will develop appropriate modifications to existing policies or a new policy, as required, to meet the intent of the recommendation. The new and/or revised policy will be established from a high-level, enterprise approach that will address requirements for configuration management plans and periodic self-assessments.

OIG Evaluation of Response: DITís response meets the intent of our recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.

Recommendation 2: The CIO should develop configuration management plan(s) covering the four operating system software platforms addressed in this report consistent with federal standards and guidelines and industry-accepted practices. DIT should determine whether other operating system software platforms require configuration management plan(s) and develop such plans where appropriate.

DIT Response: DIT concurs with the recommendation. DIT will incorporate current server configuration procedures and practices into configuration management plans consistent with federal standards and guidelines for the four operating systems covered by the audit. DIT will also determine whether other operating system software platforms require configuration management plan(s) and develop such plans as appropriate.

OIG Evaluation of Response: The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.

Recommendation 3: The CIO should ensure that the certification and accreditation of the FDICís general support systems incorporate an evaluation and testing of the FDICís configuration management policy and plans referenced in recommendations 1 and 2 of this report.

DIT Response: DIT partially concurs with the recommendation. DIT stated that the FDICís security test and evaluation (ST&E) process, a component of the certification and accreditation (C&A) program, evaluates configuration management policies, procedures, and plans for compliance with NIST guidance and industry best practices. Once the new configuration management policy and plans are developed, DIT agrees to include evaluation and testing of the policy and plans in future C&A cycles.

OIG Evaluation of Response: DITís response meets the intent of the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.

Recommendation 4: The CIO should document the minimum required configuration settings for the Windows® server and desktop operating system platforms and develop procedures to ensure that changes to baseline configuration settings are documented.

DIT Response: DIT partially concurs with the recommendation. DIT indicated that it has several processes to document required configuration settings for Windowsģ servers and desktop operating systems, including server build procedures. However, DIT agrees to review its current procedures to ensure that standard baseline configurations and approved exceptions to configuration settings are fully documented. DIT will also re-emphasize compliance with operational procedures established to ensure that server and desktop build procedures are consistently applied for each operating system. Additionally, DIT will evaluate the feasibility of adopting automated tool(s) that can facilitate periodic review of configuration settings to monitor compliance with build standards.

OIG Evaluation of Response: DITís response meets the intent of the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.

Recommendation 5: The CIO should standardize and integrate the recording, tracking, and reporting of operating system software configuration changes to the maximum extent practical. As part of this effort, DIT should consider using automated mechanisms to improve performance metric reporting for configuration changes from a system-specific and enterprise perspective.

DIT Response: DIT concurs with the recommendation. DIT stated that it has been working to standardize a single system for tracking and documenting configuration changes and improving performance metric reporting.

OIG Evaluation of Response: The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.


MANAGEMENT RESPONSE TO RECOMMENDATIONS

This table represents the management response on the recommendations in our report and the status of the recommendations as of the date of report issuance.

Rec. Number Corrective Action: Taken or Planned/Status Expected
Completion Date
Monetary Benefits Resolved: [ a ] Yes or No Dispositioned: [ b ] Yes or No
Open or Closed [ c ]
1
DIT will modify existing policies or develop a new policy that addresses configuration management principles from a high-level, enterprise approach and that addresses requirements for configuration management plans and periodic self-assessments.   November 30, 2005   N/A   Yes   No   Open
2
DIT will incorporate operating system software configuration management procedures and practices into configuration management plans consistent with federal standards and guidelines.   March 15, 2006   N/A   Yes   No   Open
3
DIT will include new configuration management policies and plans in future certification and accreditation cycles.   June 30, 2006   N/A   Yes   No   Open
4
DIT will (1) review its current procedures to ensure that standard baseline configurations and approved exceptions to configuration settings are fully documented, (2) re-emphasize compliance with operational procedures for ensuring server and desktop build procedures are consistently applied for each operating system, and (3) evaluate the feasibility of adopting automated tool(s) that can facilitate periodic review of configuration settings to monitor compliance with build standards.   April 15, 2006   N/A   Yes   No   Open
5
DIT will standardize a single system for tracking and documenting configuration changes and improving performance metric reporting.   August 31, 2006   N/A   Yes   No   Open
a Resolved Ė
(1) Management concurs with the recommendation, and the planned corrective action is consistent with the recommendation.
(2) Management does not concur with the recommendation, but planned alternative action is acceptable to the OIG.
(3) Management agrees to the OIG monetary benefits, or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount.

b Dispositioned Ė The agreed-upon corrective action must be implemented, determined to be effective, and the actual amounts of monetary benefits achieved through implementation identified. The OIG is responsible for determining whether the documentation provided by management is adequate to disposition the recommendation.

c Once the OIG dispositions the recommendation, it can then be closed.


DIVISION OF INFORMATION TECHNOLOGY COMMENTS

See Division of Information Technology Comments, page 1 [ D ]


See Division of Information Technology Comments, page 2 [ D ]


See Division of Information Technology Comments, page 3 [ D ]

Last updated 10/12/2005