The New Financial Environment
DATE: March 5, 2003
TO: Fred S. Selby, Director, Division of Finance
FROM: Russell A. Rau [Electronically produced version; original signed by Russell A. Rau], Assistant Inspector General for Audits
SUBJECT: Audit of the New Financial Environment Project Control Framework (Audit Report No. 03-016)
The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) has completed an audit of the New Financial Environment (NFE) project control framework. This audit is the first in a series of reviews that we intend to conduct at critical milestones or decision points during the development and implementation of the NFE. Prior to this audit, the FDIC OIG performed two limited scope evaluations of the NFE project at the request of FDIC management. (Note: The first report, entitled The New Financial Environment Project, dated December 7, 2001 (Evaluation Report No. 01-004), assessed the reasonableness of the NFE cost-benefit analysis and the financial systems architecture. The second report provided observations on selected procedures and documents related to the NFE Request for Proposal.) The objective of this audit was to determine whether the FDIC has established a control framework for ensuring the delivery of a quality system that meets corporate requirements and user needs in a timely and cost-effective manner. A detailed discussion of our audit scope and methodology is contained in Appendix I of this report.
The purpose of this report is to provide observations and recommendations intended to assist in ensuring the success of the NFE project. Providing this information at this point in the project's life cycle will afford the FDIC the opportunity to take timely corrective actions.
The NFE project is a major corporate initiative to enhance the FDIC's ability to meet current and future financial management and information needs. The project involves implementing a new commercial-off-the-shelf (COTS) software package to replace the FDIC's current core financial systems, which are based on the Walker Interactive Systems, Inc.'s Tamris software products. The project also involves extensive re-engineering of the FDIC’s business practices. The FDIC considers the re-engineering of its business practices to be a critical factor in achieving the expected benefits of NFE and avoiding the high maintenance costs associated with software customization.
In July 2000, the FDIC's Division of Finance (DOF) formed a project team to evaluate the FDIC's current and emerging financial needs and recommend alternative solutions. On December 10, 2001, the FDIC's Board of Directors approved contract expenditure authority for the NFE project totaling approximately $28.8 million. (Note: At the time the Board case was approved, the FDIC estimated the total life cycle cost of the NFE, including FDIC staff time, to be approximately $62.5 million over 8 years.) The FDIC executed a multi-year contract with Accenture, LLP (Accenture) on October 1, 2002, to replace its core financial systems with PeopleSoft Financials. The contract contained a 4-year base period not-to-exceed approximately $26 million. The Division of Information Resources Management (DIRM) is responsible for contract oversight. The FDIC plans to implement accounts payable, accounts receivable, general ledger, budget, procurement, treasury management, reporting, and portions of the cost management modules on July 1, 2004. Enhanced cost management functionality is scheduled for implementation in 2005.
Key to the success of any large and complex project, such as NFE, is implementing effective management controls early in the project's life cycle. Such controls include both day-to-day management of the project, such as scope, schedule, and cost controls, as well as controls that reflect the broader environment in which the project operates, such as ensuring the project is effectively coordinated with other related organizational projects. All of these controls collectively comprise a project control framework.
The Project Management Institute (PMI) has conducted extensive research and analysis in the field of project management and published a standards guide in 2000, entitled A Guide to the Project Management Body of Knowledge (PMBOK® Guide). (Note: PMI was established in 1969 as a not-for-profit project management professional association. PMI has over 95,000 members in 125 countries worldwide.) The PMBOK® Guide documents proven practices, tools, and techniques that have become generally accepted in the field of project management, including information systems development and implementation. (Note: The PMBOK® Guide defines the term "generally accepted" as being applicable to most projects, most of the time, and having widespread consensus regarding value and usefulness.) The PMBOK® Guide is an approved standard of both the American National Standards Institute and the Institute of Electrical and Electronics Engineers. The PMBOK® Guide identifies nine distinct knowledge areas associated with successful project management. The nine areas are as follows:
We used the PMBOK® Guide, in conjunction with other government and industry guidance, as the primary criteria for auditing the establishment of NFE project controls. Although the FDIC is not required to comply with the PMBOK® Guide, we used the guide as criteria because it contains sound and prudent practices related to successful project management.
RESULTS OF AUDIT
The FDIC established key controls for ensuring the delivery of a quality system that meets corporate requirements and user needs in a timely and cost-effective manner as described in the nine areas of the PMBOK® Guide. However, improvement is needed in three of the nine PMBOK® areas (integration management, communications management, and risk management). Specifically, the FDIC had not formally defined an integrated control framework for the NFE project. Without an integrated control framework, it will be difficult for the FDIC to ensure accountability and a corporate approach on the project (See Finding A: Defining an Integrated Control Framework for the New Financial Environment). The FDIC can also improve communications management on the project. Without effective communications management, the FDIC runs the risk that critical project information will not be provided to all NFE stakeholders for decision-making in a timely manner (See Finding B: Communications Management). Finally, the FDIC can improve its risk response planning for the NFE project. Without such improvements, the NFE project team may be inadequately prepared for a significant risk event, potentially increasing the impact of the risk (See Finding C: Risk Response Planning).
FINDINGS AND RECOMMENDATIONS
FINDING A: DEFINING AN INTEGRATED CONTROL FRAMEWORK FOR THE NEW FINANCIAL ENVIRONMENT
The FDIC established key controls for ensuring the success of the NFE project, such as a steering committee to manage and oversee the project, planning documents to track the project's progress, and a risk manager to identify, evaluate, and mitigate project risks. However, the FDIC had not fully defined the NFE control framework. Specifically, the FDIC had not documented an overall framework explaining the roles, relationships, and reporting structures among key project players, such as the NFE Steering Committee, project sub-groups, Capital Investment Review Committee (CIRC), and senior management officials. In addition, the FDIC had not developed and approved a charter for the NFE Steering Committee defining its responsibilities, membership, and operating guidelines, although this committee began meeting in August 2000. The NFE Steering Committee is a critical control for ensuring effective coordination and integration on the project. The NFE project team postponed defining a control framework for the NFE or completing a steering committee charter until the implementation phase of the project, when the team could obtain input from Accenture on how these controls should be defined. Absent a formally defined control framework, it will be difficult for the FDIC to ensure accountability and a corporate perspective on the project.
The PMBOK® Guide stresses the need to identify, assign, and document clear roles, responsibilities, and reporting structures for individuals and groups associated with a project in the earliest phases of the project's life cycle. The Guide also describes how project management processes and plans can be effectively integrated. The FDIC recently took a formal approach to promote integration on its Enterprise Architecture (EA) project. (Note: An EA is an institutional systems blueprint that defines in both business and technological terms an organization’s current and target operating environments and how the organization will transition between the two.) In December 2002, senior FDIC management formally approved an EA Blueprint document describing how the EA project is structured and how EA products and processes are managed from a corporate perspective. The EA Blueprint document included formal charters for each of the EA committees.
Similarly, formally defining all of the critical controls established for the NFE project and documenting how these controls interrelate will promote accountability on the project and strengthen management of the project from a corporate perspective. Ensuring a corporate perspective for the NFE is especially important in light of the re-engineering of financial and non-financial business processes that will take place on the project and the extensive coordination that will be required across the FDIC's divisions and offices.
We recommend that the Director, DOF, in conjunction with the NFE project team:
FINDING B: COMMUNICATIONS MANAGEMENT
Since its inception in July 2000, the NFE project team has developed a number of channels for disseminating information about the project throughout the Corporation and continued to refine these channels on an ongoing basis. For example, the project team provided project status reports and periodic briefings to senior FDIC managers and corporate committees and promoted awareness of the project through the NFE Web site and FDIC News. However, communications management could be improved. The NFE project team had not formalized and approved a fundamental component of its communications strategy, a communications management plan. (Note: A communications management plan documents the processes required to ensure timely and appropriate generation, collection, dissemination, storage, and ultimate disposition of project information. It identifies project stakeholders and evaluates their information and communications needs (i.e., who needs what information, when they need it, how it will be provided to them, and who will provide it to them).) A communications management plan had not been completed because the NFE project team and Accenture had not completely defined the communications requirements for the project. Without a formal communications management plan, the FDIC runs the risk that critical project information will not be provided to all NFE stakeholders for decision-making in a timely manner.
The PMBOK® Guide states that identifying the information needs of project stakeholders and determining a suitable means of meeting those needs is an important factor for project success. On most projects, the majority of communications planning is completed during the earliest project phases and reviewed regularly throughout the project. According to the PMBOK® Guide, the primary output of communications planning is a communications management plan that documents the methods for gathering, disseminating, updating, and storing project information. The plan defines who receives what information and the methods for information distribution.
An April 2002 study conducted by the Chief Financial Officers Council and the Joint Financial Management Improvement Program, entitled Building the Work Force Capacity to Successfully Implement Financial Systems, also identifies good communications as a critical success factor in financial systems implementations. The study states that a management document identifying critical actions and the people responsible for deliverables, and holding parties accountable for those deliverables, is critical to the success of the overall project. The need for effective project communications was also cited as a lesson learned in a recent DIRM report on FDIC system development efforts. (Note: DIRM report, entitled Post Implementation Review Trends Report, dated May 29, 2002.)
At the time of our audit, the NFE project team and Accenture had already defined many of the components necessary for a communications management plan and were working on a draft plan. However, according to the NFE project schedule, a communications management plan was to be completed and approved not later than December 15, 2002. Given the criticality of this important document, the NFE project team should make completing and approving a communications management plan a high priority.
We recommend that the Director, DOF, in conjunction with the NFE project team:
FINDING C: RISK RESPONSE PLANNING
The NFE project team began assessing NFE risks early in the project's life cycle and took steps to improve its risk management practices as the project progressed. (Note: The PMBOK® Guide defines risk management as a “systematic process of identifying, analyzing, and responding to project risk." Risk management includes deciding how to plan a project's risk management activities, identifying risks and the measures that determine when a risk is about to occur or has already occurred, and using qualitative and quantitative analysis of risks and their implications on project objectives. Risk management also involves developing procedures to enhance opportunities and reduce threats, such as determining whether risks can be mitigated or a contingency plan is required, as well as ongoing monitoring and control of project risks and results evaluation.) While these represent positive actions, the FDIC could improve its risk response planning. Specifically, the NFE project team did not establish clear measures for determining when project risks categorized as "significant" become a reality. (Note: The FDIC’s risk management methodology used on the NFE project defines significant risks as having a high or moderate likelihood of occurrence with catastrophic impact on the project or having a high likelihood of occurrence with a critical impact on the project.) In addition, the project team did not develop contingency plans, as appropriate, for significant risks before they occur. Such measures and contingency plans did not exist because the FDIC's risk management practices relative to the NFE project represented a new approach to managing risks on corporate IT projects and was evolving. Without clear measures for determining when significant project risks occur and appropriate contingency planning, the NFE project team may be required to react to a significant risk event after it happens, potentially increasing the impact of the risk.
The NFE project team established a risk management process for the NFE project in April 2001 when it completed a formal risk management plan. This process was expanded when the FDIC designated the Director, Office of Internal Control Management (OICM), as the project's risk manager in July 2002. As risk manager for the NFE project, the OICM Director is responsible for administering the NFE risk management plan, identifying, evaluating, and mitigating project risks, and preparing monthly reports to project stakeholders. Using a defined methodology based on industry and government practice, the OICM identified seven specific risk factors related to the NFE project that it classified as "significant." (Note: The seven risk factors are (1) changing requirements, (2) funding estimates are inaccurate, (3) adequate technology is unavailable, (4) significant changes in business practices (workflow), (5) unrealistic project schedule, (6) contract oversight, and (7) simultaneous system development.) To address these seven significant risk factors, the NFE team developed mitigation plans containing actions that management should take to mitigate each risk factor. OICM is responsible for monitoring the implementation of the NFE risk mitigation plans.
Some project risks, such as cost and schedule overruns, cannot be entirely mitigated. For this reason, it is important to establish clear thresholds or measures for determining when a significant risk has occurred or is about to occur. It is also important to consider contingency actions in advance, in case a significant risk becomes a reality. The PMBOK® Guide states that risk triggers, sometimes called risk symptoms or warning signs, should be established to indicate that a risk has occurred or is about to occur. The PMBOK® Guide also states that developing a contingency or fallback plan may be appropriate for some risks, particularly those risks having potentially high impact. The most common type of contingency or fallback plans include the establishment of a contingency allowance or reserve, including amounts of time, money, or resources, to account for known risks. Contingencies can also include development of alternative options if a selected project strategy is not effective. This concept of proactive risk response planning is also recognized in the NFE risk management plan, which states "the risk manager will identify and the team will deal with potential problems early and develop contingencies to avoid a crisis environment."
System projects may not proceed exactly as originally planned. Effective management of any major project such as the NFE requires a well thought-out approach to handle significant project risks and a mechanism for addressing those risks. Unless the NFE project team establishes specific measures and contingencies, as appropriate, for risks classified as significant, it may be left to react to these risk events as they occur. Advance contingency planning can greatly reduce the impact of a significant risk should it become a reality on the NFE project.
We recommend that the Director, DOF, in conjunction with the NFE project team:
CORPORATION COMMENTS AND OIG EVALUATION
On February 21, 2003, the Director of DOF provided a written response to the draft report's five recommendations to enhance and improve the overall project control framework for the NFE. DOF concurred with all five recommendations. Corrective action for one recommendation was completed prior to DOF's written response. DOF plans to complete corrective actions for the remaining four recommendations by April 15, 2003. The following summarizes DOF's response to each recommendation.
DOF concurred with the recommendation. In its response, DOF indicated that the Steering Committee membership had been finalized for the implementation phase of the project and that a Steering Committee charter was being developed. The estimated completion date for development and approval by DOF of a Steering Committee charter is March 15, 2003.
Recommendation 1 is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
DOF concurred with the recommendation. In its response, DOF indicated that it is in the process of developing a “project governance” document to address the above recommendation and identify other relevant project control documents. The estimated completion date for development and approval of the project governance document by DOF is April 15, 2003.
Recommendation 2 is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
DOF concurred with this recommendation. A communications plan was finalized and accepted by the FDIC on January 29, 2003. The OIG confirmed with the contract oversight manager that the communications plan was received and accepted on that date.
Recommendation 3 is resolved, dispositioned, and closed.
DOF concurred with this recommendation. In its response, DOF indicated that the NFE team will meet with the NFE risk manager and Accenture to develop appropriate measurement criteria. The estimated completion date for developing and approving the measurement criteria by DOF is April 15, 2003.
Recommendation 4 is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
DOF concurred with this recommendation. In its response, DOF stated that the NFE team is developing contingency plans for significant risk areas. The estimated completion date for development and approval of contingency plans by DOF is April 15, 2003.
Recommendation 5 is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
A summary chart showing management's responses to all recommendations is presented in Appendix III.
SCOPE AND METHODOLOGY
To accomplish our audit objective, we interviewed Headquarters DOF, DIRM, and OICM officials who were responsible for managing and implementing the NFE project. We also spoke with representatives of the U.S. General Accounting Office to obtain their perspective on the NFE project and its control framework. In addition, we spoke with representatives of Accenture, the consulting firm hired by the FDIC to provide implementation services on the NFE, to become familiar with Accenture’s control processes for managing and implementing the project. Further, we attended NFE Steering Committee meetings and selected project briefings to observe certain aspects of the NFE control framework. We also reviewed key documents related to the NFE control framework, including the quality management plan, risk management and mitigation plans, work breakdown structure, Board of Directors’ case authorizing contract expenditure authority for the NFE, and relevant corporate correspondence.
The scope of our audit was limited to determining whether the FDIC had established a control framework for ensuring the delivery of a quality system that meets corporate requirements and user needs in a timely and cost-effective manner. Our audit did not evaluate the effectiveness of the implementation of internal controls related to the NFE project. Such evaluations will be conducted in future audits of the NFE. In addition, our audit did not assess the FDIC's compliance with applicable laws and regulations because we did not identify specific laws or regulations pertaining to the establishment of project controls.
Our audit included a determination of whether the FDIC had established performance measures to control the NFE project, such as status reporting and budget and schedule variance analysis. We plan to evaluate the adequacy and effectiveness of NFE performance measures as part of our future coverage of the NFE. In addition, we corroborated automated information used to support our audit findings, conclusions, and recommendations with other sources to ensure they were sufficiently reliable. For example, we discussed information contained in project status reports and plans with key project personnel. Throughout the audit, the auditors were sensitive to the possibility of abuse or illegal acts. Finally, we relied on PMI's PMBOK® Guide as the primary criteria for determining whether the FDIC had established a project control framework for the NFE.
We conducted our audit in accordance with Generally Accepted Government Auditing Standards during the period December 2002 through January 2003.
DATE: February 21, 2003
MEMORANDUM TO: Stephen M. Beard, Deputy Assistant Inspector General for Audits
FROM: Fred S. Selby [Electronically produced version; original signed by Fred S. Selby], Director, Division of Finance
SUBJECT: Draft Report Entitled The New Financial Environment Project Control Framework (Assignment Number 2003-011)
We have completed our review of the subject Office of Inspector General (OIG) report. We appreciate the review performed by OIG and its recommendations to enhance and improve the overall NFE project control framework. We concur with all recommendations presented and offer the following specific responses.
Finding A: Integration Management
Finding A recommends that the Director of DOF, in conjunction with the NFE Project Team:
Finding B: Communications Management
Finding B recommended that the Director of DOF, in conjunction with the NFE Project Team, promptly complete and approve a formal communications management plan for the NFE project.
We concur with this recommendation. The Communications Plan was finalized and accepted by FDIC on January 29, 2003.
Finding C: Risk Management
Finding C recommended that the Director of DOF, in conjunction with the NFE Project Team:
If you have any questions regarding our response, please contact James Anderson at (202) 416-7208 or Mike Agresto at (202) 416-6986.
cc: Steven App
MANAGEMENT RESPONSES TO RECOMMENDATIONS
The following presents the management responses that have been made on recommendations in our report and the status of recommendations as of the date of report issuance. The information is based on management’s written response to our report.
Note: The following definitions apply to the Management Responses to Recommendations below.
Resolved: (1) Management concurs with the recommendation and the planned corrective action is consistent with the recommendation. (2) Management does not concur with the recommendation but planned alternative action is acceptable to the OIG. (3) Management agrees to the OIG monetary benefits or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount.
Dispositioned: The agreed-upon corrective action must be implemented, determined to be effective, and the actual amounts of monetary benefits achieved through implementation identified. The OIG is responsible for determining whether the documentation provided by management is adequate to disposition the recommendation. Once the OIG dispositions the recommendation, it can then be closed.
Recommendation Number 1
Recommendation Number 2
Recommendation Number 3
Recommendation Number 4
Recommendation Number 5
|Last Updated 03/21/2003|