New Financial Environment Scope
DATE: September 29, 2003
TO: Steven O. App, Chief Financial Officer
FROM: Russell A. Rau [Electronically produced version; original signed by Russell Rau], Assistant Inspector General for Audits
SUBJECT: New Financial Environment Scope Management Controls (Audit Report No. 03-045)
The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) has completed an audit of the New Financial Environment (NFE) scope management controls. This audit is the second in a series of reviews that we intend to conduct at critical milestones or decision points during the development and implementation of the NFE. The first audit was to determine whether FDIC had established a control framework for the NFE project. (Note: Audit of the New Financial Environment Project Control Framework, dated March 5, 2003 (Audit Report No. 03 016).) Prior to these audits, the FDIC OIG performed two limited-scope evaluations of the NFE project at the request of FDIC management. (Note: The first of these reports, entitled The New Financial Environment Project, dated December 7, 2001 (Evaluation Report No. 01-004), assessed the reasonableness of the NFE cost-benefit analysis and the financial systems architecture. The second report, entitled Preaward Review of New Financial Environment Project, dated October 7, 2002 (Audit Report 03-002), provided observations on selected procedures and documents related to the NFE Request for Proposal.) The objective of this audit was to determine whether the FDIC had implemented adequate controls for ensuring that the scope of the NFE project was effectively managed and any cost or schedule adjustments resulting from project scope changes were properly evaluated and controlled. A detailed discussion of our audit objective, scope, and methodology is in Appendix I of this report.
Readers of this report are also encouraged to review the Corporation’s comments in Appendix III to gain a better understanding of the significant scope of the NFE project. In addition, the comments address in detail management’s efforts to work through and resolve project challenges that have arisen.
The purpose of this report is to provide observations and recommendations intended to assist management’s effort to ensure the success of the NFE project. Providing this information at this point in the project's life cycle will enable the FDIC to take timely corrective actions.
The NFE project is a major corporate initiative to enhance the FDIC's ability to meet current and future financial management and information needs. The project involves implementing a new commercial-off-the-shelf (COTS) software package to replace the FDIC's current financial systems, which are based on the Walker Interactive Systems, Inc. Tamris software products. The project also involves extensive re-engineering of the FDIC’s business practices. The FDIC considers the re-engineering of its business practices to be a critical factor in achieving the expected benefits of the NFE in terms of streamlining business processes and avoiding the high maintenance costs associated with software customization.
On December 10, 2001, the FDIC's Board of Directors approved contract expenditure authority for the NFE project totaling approximately $28.8 million. (Note: When the Board case was approved, the FDIC estimated the total life-cycle cost of the NFE, including FDIC staff time, to be approximately $62.5 million over 8 years.) The FDIC executed a multi-year contract with Accenture, LLP (Accenture) in October 2002 to replace its financial systems with PeopleSoft® Financials. (Note: Accenture is a global management consulting and technology services company with more than 80,000 people in 47 countries.) The contract contained a 4-year base period not to exceed approximately $26 million. The FDIC planned to implement the core financial system on July 1, 2004, that is, accounts payable, accounts receivable, general ledger, budget, procurement, treasury management, reporting, and portions of the cost management modules. The enhanced cost management functionality is scheduled for implementation in 2005. The NFE contract is a fixed-price contract with interim payments based on a schedule of 189 deliverables that define Accenture’s activities and schedule. Of the 189 deliverables, 62 were for key design, development, and implementation activities, and the remaining 127 were for status reports and operations. By August 31, 2003, 32 of the 62 key deliverables were to have been completed and accepted by FDIC.
NFE Functional and Technical Requirements
The FDIC developed 753 functional and technical requirements for the NFE core financial system and included them in the request for proposal and as part of the contract’s statement of work. As part of the core financial system’s general design phase, FDIC and Accenture personnel discussed the functional and technical requirements in Joint Application Development (JAD) and Conference Room Pilot (CRP) sessions. (Note: JAD sessions were conducted using a focus group format with representatives from each FDIC functional area. During these sessions, FDIC business processes were reviewed and verified in order to provide a baseline for planning activities. CRP sessions were conducted after the installation and demonstration of the basic PeopleSoft® Financials application. The purpose of these sessions was to demonstrate that the packaged software solution supported FDIC’s core business processes and to identify additional requirements and necessary modifications. Sessions were conducted in a small conference-room setting, with FDIC subject matter experts validating business processing effects.) Additionally, NFE project teams were to analyze NFE requirements and develop a baseline requirements list that included new requirements identified during the general design phase. Near the end of the general design phase and not later than March 31, 2003, Accenture was to perform an overall schedule and cost re-estimate for implementing the NFE baseline requirements that included any new functional and technical requirements approved by FDIC NFE management.
NFE Management Control Framework
The NFE management control framework includes oversight by a senior management advisory group, the NFE Principals group, and an NFE Steering Committee. The purpose of the NFE Principals group and NFE Steering Committee is to ensure the successful completion of the project. The NFE Principals group is composed of the Chief Financial Officer (CFO) and the directors of the divisions most impacted by NFE. To keep senior FDIC management informed of the project’s progress, the NFE Principal’s group receives a monthly briefing from the NFE project team. Additionally, the Director of the Division of Finance (DOF), the project sponsor and a member of the NFE Principals group, is responsible for keeping the FDIC Board of Directors informed of the NFE project’s status.
The NFE Steering Committee provides direct oversight of the NFE project. The committee is co-chaired by two DOF Deputy Directors and includes FDIC executives from the Division of Information Resources Management (DIRM), Division of Administration (DOA), Division of Resolutions and Receiverships, Division of Supervision and Consumer Protection and the Office of Internal Control Management (OICM). The NFE Steering Committee's purpose is to oversee the planning, development, and successful implementation of the core financial system. The Committee’s functions include the following:
The Steering Committee meets every 2 weeks to discuss issues and receive progress reports from FDIC and Accenture NFE project managers.
The Director OICM, who is the project’s risk manager, ensures that risks are closely monitored and controlled. The risk manager reports monthly to the CFO; Director, DOF; and the NFE Steering Committee on indications that a significant risk event may occur. (Note: A significant risk will have a critical or catastrophic effect on the project if it occurs.) The Director OICM also ensures that the project team develops risk mitigation and contingency plans.
In addition to the executive oversight described above, the NFE team includes three FDIC project managers to run the day-to-day technical, business, systems, and other aspects of the project. The project managers’ duties are to: assign team members, provide contract oversight, identify operational obstacles, and ensure the project is completed on time and within budget. The project managers formed additional teams of FDIC employees and subject matter experts to work with Accenture to ensure the delivered core financial system meets FDIC needs, familiarize FDIC employees with changes caused by NFE, and help keep the project on time and within budget.
Scope and Time Management Control Guidance
Effective scope and time management controls are vital to the success of any large and complex project such as the NFE. These controls include controls that reflect the broader environment in which the project operates, such as ensuring the project is effectively coordinated with other related organizational projects. The Project Management Institute (PMI) has conducted extensive research and analysis in the field of project management and published a standards guide in 2000, entitled A Guide to the Project Management Body of Knowledge (PMBOK® Guide).(Note: PMI was established in 1969 as a not-for-profit project management professional association. PMI has over 95,000 members in 125 countries worldwide.) The PMBOK® Guide identifies nine distinct knowledge areas associated with successful project management. The following two knowledge areas focus on the work to be done and the time needed to do the work.
Although the FDIC is not required to comply with the PMBOK® Guide, we used it as criteria for scope and time management because the guide contains generally accepted industry practices related to successful project management. In response to our audit report on the NFE control framework, the FDIC agreed to implement recommendations for integration, communication, and risk management. The report recommendations were based on guidelines in the PMBOK® Guide. Additional information about the PMBOK® Guide is included in our audit objective, scope, and methodology in Appendix I of this report.
RESULTS OF AUDIT
Key scope management deliverables for the NFE project are significantly behind schedule (see Finding A: Scope Management Deliverables). Improvements in project oversight could have minimized the impact of issues affecting the project deliverable schedule (see Finding B: Project Oversight). In addition, FDIC management did not adopt adequate time management practices to maximize project progress (see Finding C: Time Management). As a result, the project is less likely to be deployed on schedule, which could increase FDIC’s contracting costs and defer the benefits anticipated from an integrated financial system.
FINDINGS AND RECOMMENDATIONS
FINDING A: SCOPE MANAGEMENT DELIVERABLES
Baseline requirements and the general design have not been finalized for the NFE project. Since October 2002, the beginning of the NFE implementation contract, through August 31, 2003, 25 (78 percent) of the 32 design and implementation deliverables were delayed from the original delivery schedule (see Appendix II). Of particular concern, two deliverables that provide the foundation for the remainder of the project, the Requirements Traceability Matrix (RTM) and the General Design Document, were 4½ months behind the original schedule as of August 31, 2003. (Note: The RTM will capture various information for each NFE requirement such as the functional area, whether the requirement will be included as part of the NFE baseline implementation, description of the requirement, etc. Once all the requirements have been reviewed and a final decision has been made on which requirements are in the NFE baseline, Accenture will produce a “Build RTM.” The “Build RTM” will list all the requirements to be included in the baseline NFE. The “Build RTM” will be generated at the end of the NFE General Design and will provide the scope boundaries for the NFE build phase (Detail Design, Programming, Testing, etc.). The General Design Document translates system requirements into specific system processes and elements, conceptualizing the high-level application structure to meet those requirements. The General Design Document also defines server databases and any other outside network connectivity that may impact the application structure and performance. When the General Design Document is finalized, the development team begins to create detailed designs for the system.) During this time, the NFE project team was working with Accenture to minimize the number of new requirements and potential customizations resulting from initial business process review and NFE pilot sessions with representatives of FDIC program offices. However, as discussed in Findings B and C, improvements in project oversight and time management could have been helpful in minimizing the delays that have occurred. Further delays could result in additional costs for implementing the core financial system, continuing the operation of the existing financial management system, and deferring the efficiencies anticipated by a completed NFE.
Requirements Traceability Matrix
The project team has not completed the RTM for the core financial system. After JAD and CRP sessions and other project efforts by Accenture and FDIC, Accenture proposed about 160 new requirements in addition to the 753 requirements defined in the statement of work. The NFE team worked with Accenture to tie the proposed requirements to existing requirements or to eliminate the majority of them. This work, while time consuming, reduced the number of new requirements from 160 to about 30. In addition, the team was able to offset the new requirements by revisiting and eliminating some of the original 753 requirements.
Analyzing and reaching agreement with Accenture project management on the 160 requirements took longer than expected. Specifically, the RTM was originally scheduled to be completed by April 15, 2003. However, the project team reported that it had not made determinations on about 30 of the 160 additional requirements as of June 30, 2003. As of August 31, 2003, 11 months into the 21 month project, FDIC had not finalized its requirements in the form of an RTM for the core financial system.
The General Design Document, a specific contract deliverable, is composed of a series of documents defining how specific differences between the COTS application and FDIC business needs will be met. It provides the foundation for developing detail system specifications and was originally scheduled to be complete by April 15, 2003 but had not been completed as of August 31, 2003. Because the General Design Document incorporates baseline requirements, delays in defining these requirements have also delayed completion of the General Design Document. As a result of delays in completing the General Design Document, other activities dependent on general design were also slipping. For example, the final detail design will not be completed until 4 weeks after completion of the general design.
It is also important to promptly complete the RTM and general design of the project because the Accenture NFE implementation contract is a fixed-price contract. The FDIC Acquisition Policy Manual states that under a fixed-price contract, a contractor is required to deliver the required goods or perform the services at a specified fixed price. A set price is established for delivery of a defined product or service. Payment is normally made upon delivery and acceptance of the goods or services. Successful use of fixed-price contracts requires a clear definition of requirements at the time of contract award and realistic estimates of work to be performed. Further, DOA’s best practices for program divisions state that a fixed-price contract is appropriate when the contracted services have reasonably definable functional and detailed specifications. Delays in finalizing the RTM and General Design Document could result in contract performance issues.
FINDING B: PROJECT OVERSIGHT
Sustained and effective oversight is crucial to the successful implementation of the NFE project. The NFE Steering Committee and FDIC senior management did not fully utilize the NFE project control framework that was established to achieve such oversight. Also, the scope management document that was developed did not address the impact of systems and business processes external to the core financial system. Recognizing that corrective actions may be needed to address the project delays, the project team is conducting a self-assessment of its management practices.
Implementation of Project Control Framework
The General Accounting Office Standards for Internal Control in the Federal Government state that internal (management) control is a continuous built-in component of operations. Management controls are an integral component of a project's management that provide reasonable assurance of effective and efficient operations. The controls are a series of actions and activities that occur throughout the project on an ongoing basis and as part of the project's infrastructure to help managers run the project and achieve the project's goals. The critical oversight controls established for the NFE project promote accountability on the project and strengthen management of the project from a corporate perspective.
The NFE Steering Committee and FDIC senior management did not fully utilize the NFE project control framework. Executive oversight meetings intended to oversee the planning, development, and successful implementation of the core financial system were cancelled when problems were encountered. The NFE Steering Committee co-chairmen cancelled the Steering Committee meetings on May 15 and May 29, 2003, because the project team was working to address scope and schedule issues. When the May 29, 2003 meeting was cancelled, one of the co-chairmen outlined significant project scope management issues and potential schedule delays in an e-mail to committee members and explained that the project team wanted to develop more comprehensive information about the issues before they were presented to the committee for discussion. Although the issues were communicated to the Steering Committee, canceling the meetings bypassed the project’s established process for Steering Committee oversight and interdivisional communication. The NFE Steering Committee co-chairmen also cancelled the June 2003 monthly NFE Principals meeting so that the issues could be discussed with the NFE Steering Committee before they were presented to the principals.
The cancellation of the key meetings limited senior management’s ability to make proactive, informed decisions to ensure that significant issues affecting schedule, scope, and costs are resolved in a timely manner. In short, the NFE Principals group and Steering Committee could be more effective by evaluating the project’s progress and intervening promptly when delays or obstacles become apparent.
Scope Management Plan
According to the PMBOK® Guide, scope management includes the processes required to ensure that the project includes all the work required, and only the work required, to complete the project successfully. Scope management primarily applies to defining and controlling what is or is not included in the project and involves authorizing the project, defining the scope, obtaining stakeholder acceptance of the scope, and controlling scope changes. The scope planning process should result in a scope management plan describing the project scope and how scope changes will be integrated into the project. The scope management plan should also include an assessment of the stability of the project, i.e., how likely it is to change, how frequently, and by how much. This plan should include a clear description of how scope changes will be identified and classified. For example, the plan should describe how suggestions for including and excluding system and business processes in the project are evaluated, prioritized, and eventually approved or rejected. The PMBOK® Guide states this is particularly difficult—and therefore absolutely essential—when the product (in this case the core financial system) characteristics are still being defined. The PMBOK® Guide also states that management of both project and product scope must be well integrated to ensure that project efforts will result in the delivery of the specified product.
The NFE team developed a Scope Management Approach document that described the process for defining the core financial system scope, i.e., completing the RTM. However, the Scope Management Approach document did not contain the processes necessary to identify and evaluate the project scope impact of FDIC systems and business operations external to the NFE project. As of June 30, 2003, the NFE project team had not determined whether 20 of the 221 FDIC systems were impacted by the NFE project. In its June 27, 2003 weekly status report, Accenture warned that the final scope of the system’s interface work could not be determined until all affected systems were identified. By August 25, 2003, the NFE team had completed its effort to identify systems impacted by the NFE.
The NFE team also discovered that coordinating legacy systems changes, existing applications upgrades, and current and emerging system development efforts required more attention than originally anticipated. Recognizing the need to invest additional efforts in managing the project scope, the NFE project team created an NFE Program Management function to address the systems changes that must occur and how they will be coordinated with the NFE core financial system implementation. Although establishing the Program Management function should improve coordination on systems issues, the NFE Steering Committee should work with the project team to develop a scope management plan. The scope management plan should clearly define systems interfaces, changes to the interfaces, and a process to identify, authorize, and integrate changes into the NFE project, as prescribed by the PMBOK® Guide.
NFE Project Self-Assessment
In response to the delays in completing the General Design Document, the Acting Director, OICM reported in the May 2003 NFE-Risk Factor Evaluation that a significant risk, an unrealistic project schedule, had occurred and that contingency plans were being implemented. The contingency plans included (Note: According to the NFE Project Risk Management Plan, completing planned milestones on time is a measure of success for every project. The most common cause for missed schedules is simply poor estimating, which may result from a lack of experience, a lack of estimating tools, or management pressure toward an unrealistic schedule.):
As a result of project delays and the significant risk event, the NFE team is assessing project scope and time management practices and expects to complete an analysis of significant issues by December 1, 2003. These issues include:
The ongoing self-assessment may provide valuable insight into issues affecting the implementation of NFE. Consistent with the project control framework, the results of the assessment should be the impetus behind a senior management review of the project to establish metrics for measuring progress and to establish project re-evaluation criteria if the measures are not achieved.
FINDING C: TIME MANAGEMENT
The NFE project team needs to improve time management controls. Specifically, monthly status reports lacked information needed for decision-making in addressing issues causing or that could cause delays, and there was no time limit for decision-making at any level. Establishing a time management plan would help the project team better control the project schedule.
Time Management Practices
The PMBOK® Guide describes project time management processes that include defining, sequencing, and estimating the duration of activities, developing a schedule, and controlling schedule change. Schedule control involves influencing the factors that create schedule changes to ensure that changes are agreed upon, determining that the schedule has changed, and managing the actual changes. Information needed for a schedule control process includes:
The PMBOK® Guide suggests the use of a change control system that defines the procedures for changing the project schedule--including the paperwork, tracking systems, and approval levels necessary for authorizing changes. Performance measurement and variance evaluation techniques are also recommended to assess the magnitude of variations to determine whether corrective action is necessary and which action is appropriate. An effective schedule control process will provide updates to the schedule information used to manage the project. The schedule control process provides the corrective action to bring expected future schedule performance in line with the project plan. The FDIC Acquisition Policy Manual also emphasized the importance of controlling the schedule, stating that the schedule is one of the concerns in the administration of a fixed-price contract.
For time management of the project, the NFE Steering Committee receives a monthly critical milestone report and a graphical schedule of high-level activities. However, the reports do not provide adequate information to assist the NFE Steering Committee in making well informed decisions. The reports show progress on milestones and activities that are ongoing and whether they are on schedule, less than 2 weeks behind schedule, or more than 2 weeks behind schedule. Because the reports do not include information on actual variances between past activities and a project baseline (i.e., the original contract deliverables schedule), the reports do not provide sufficient information to evaluate performance. Also, the information in the status reports is extracted from the project plan, a Microsoft® Project database, that does not identify all dependencies linking critical activities to be accomplished in order to deploy the core financial system. As a result, the impact of delays on the project implementation date cannot be determined from the monthly status reports—preventing the NFE Steering Committee from anticipating, identifying, and addressing issues that contribute to delays. Empirical data on the schedule variances is essential for the NFE Steering Committee and the project team to determine whether ongoing corrective actions such as fast tracking, additional resources, and design efficiencies will keep the project on schedule. (Note: Working on activities concurrently rather than sequentially--for example, working on detail design issues and requirements definition at the same time.)
Another issue affecting time management of the project is the slow decision-making process. To meet its aggressive timeframe, the NFE project is structured to allow decision-making to begin at the lowest levels of the NFE team and progress to the project managers, NFE Steering Committee, and finally the Director, DOF, for disposition. However, there is no time limit for decision-making at any of the levels. As a result, several unresolved issues have delayed the project. For example, in its June 27, 2003, weekly status report, Accenture states that “For the past  weeks there has been no activity towards the final categorization of the systems in a preliminary category . . . It is important for these categories to be finalized as quickly as possible, as the final scope of the interface work cannot be identified until all systems are in their final category.” (Note: The NFE project team identified 221 FDIC systems and was categorizing their relationship with the NFE project into 6 categories: to be absorbed (35), retired systems (19), impacted by NFE (34), NFE impact to be determined (0), no NFE impact (101), and systems that retrieve information from other systems or databases to prepare reports (32). The preliminary categories of concern to Accenture were the “to be absorbed” and “NFE impact to be determined” categories. The “NFE impact to be determined” category was reduced from 20 on June 30, 2003 to none by August 25, 2003.) Another example is the inability of stakeholders to arrive at a consensus on PeopleSoft® chartfields (accounting elements necessary to appropriately process transactions). In addition, NFE Steering Committee meeting minutes, dated June 26, 2003, state that if decision-making continues to be slow, Accenture resources will not be used at full capacity or will be taken off the project; either outcome is detrimental to the project. Accenture personnel have been working on lower-priority activities to minimize the impact of delays while waiting for decisions on higher-priority items such as completion of the RTM.
To control the project schedule, the NFE Steering Committee should work with the project team to develop a time management plan that identifies where delays in an individual activity will delay completion of the project and that provides a process for evaluating, approving, and managing changes to the project schedule.
CONCLUSION AND RECOMMENDATIONS
Continual delays in key project deliverables will result in significant schedule overruns with corresponding increases in the costs of implementing the core financial system. The FDIC will also incur the costs of continued operation of the existing Walker system—estimated in the NFE cost-benefit analysis at about $250,000 per month more than the cost of an operating NFE system. In addition, the benefits of an integrated financial system, such as data sharing and fewer systems will be deferred. Further, the Corporation will have to adjust maintenance and development schedules for interfacing legacy systems to compensate for the NFE project delays.
The NFE project and project delays have corporate-wide impacts. Because the RTM, General Design Document, and other project deliverables have been delayed, the Principals group should monitor and encourage full and consistent implementation of the NFE management control framework. Further, the Principals group and NFE Steering Committee need to direct and work with the project team to ensure that the project scope is promptly finalized and that impacts to the schedule are adequately managed.
We recommend that the CFO, in conjunction with the Director, DOF:
CORPORATION COMMENTS AND OIG EVALUATION
On September 25, 2003, the CFO and the Director, DOF, provided a written response to the draft report. Their response is presented in its entirety in Appendix III of this report. The CFO and Director, DOF, concurred with both recommendations. DOF plans to complete recommended corrective actions by December 1, 2003. The following summarizes management’s response to the recommendations.
1. Conduct a senior management review of the NFE project to establish metrics for measuring progress and project re-evaluation criteria if measures are not achieved.
The CFO and Director, DOF, concurred with this recommendation. They are working with Accenture to develop a revised project schedule as a result of current delays. The new project plan will provide a critical path that can be monitored by project leadership. Management is also working to improve reporting and monitoring at all levels to support timely and informed decision-making. From these efforts, the CFO and Director, DOF, will develop improved project measurement metrics and re-evaluation criteria. The estimated completion date for development and approval is November 14, 2003.
Management's planned actions are responsive to the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
2a. Require that the RTM and General Design Document are expeditiously approved to avoid project delays.
The CFO and Director, DOF, concurred, indicating that the RTM and the General Design Document are not one-time deliverables. The CFO and Director, DOF, have been working with Accenture to finalize research and resolve remaining issues associated with requirements and open general design items for everything except the Enterprise Performance Management suite, which is on a later date track. All issues have been assigned completion dates and are being actively monitored. The estimated completion date of the RTM and General Design Document deliverables is November 14, 2003.
Management's planned action is responsive to the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
2b. Develop a project scope management plan that clearly defines the overall project scope and the project scope change processes in addition to the existing NFE Scope Management Approach document, which describes the process for developing baseline requirements.
The CFO and Director, DOF, concurred and will develop a broader scope management plan that unifies the existing NFE Scope Management Approach document and the existing Interface Approach document. Tying these together under an integrated plan will ensure a broader scope management focus and allow for better monitoring of all efforts that influence the NFE project success. The estimated completion date for development and approval is December 1, 2003.
Management's planned action is responsive to the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
2c. Develop a time management plan that limits the time for decisions at each project level and has adequate performance measures to evaluate variances from the project schedule, including the impact of delays on the deployment date.
The CFO and Director, DOF, concurred and will work with Accenture to review existing time management documents and execution processes to identify and address where improvements are needed. The estimated completion date for review, revision, and approval is October 31, 2003.
Management's planned action is responsive to the recommendation. The recommendation is resolved but will remain undispositioned and open until we have determined that agreed-to corrective action has been completed and is effective.
2d. Develop a schedule control process that defines the procedures for identifying, requesting, authorizing, and managing changes to the project schedule.
The CFO and Director, DOF, concurred with this aspect of the recommendation. They will work with Accenture to review existing schedule control documents and execution processes to identify and address where improvements are needed. The estimated completion date for review, revision, and approval is October 31, 2003.
A summary chart showing management's responses to the recommendations is presented in Appendix IV.
Corporation Comments on NFE Project
In responding to the draft report, the CFO and Director, DOF, provided additional insights on the scope of the NFE project. They indicated that the evaluation of the 27 business modules/support tools against 753 requirements, 465 issues identified, 221 legacy applications, and the effort needed to finalize the new accounting code control key (known as chartfields in PeopleSoft®) has taken longer than originally contemplated when the project schedule was established in September of 2002. Extensive discussion, research, and evaluation of options have been required on many issues given the magnitude of the change or the complexity of the solution options. As a result, the key project milestones and deliverables (i.e., the RTM and completion of general design documents) associated with this critical initial project phase have been delayed. Although these delays do have a negative impact in the short term relative to project timelines and associated additional implementation costs, FDIC believes the extra effort to date to manage the scope of the NFE will yield significant benefit in the later project phases and post-implementation.
The NFE project leadership also recognized the need to improve scope and time management efforts. A Program Management Office was created to provide a control structure that reflects the broader scope of implementation work that is beyond the design of the existing project management structure. A legacy system change plan was developed to better guide the decision-making process. Also, NFE project leaders are working with Accenture to improve status reporting and deliverables monitoring so that issues and potential impacts are elevated in a more timely manner for higher-level decision-making when required.
We agree that the NFE project scope is significant and has corporate-wide impact. The challenge for NFE project leadership is to ensure that the project scope and time management plans are consistently followed after they are developed to minimize risks of cost overruns and schedule slippages.
APPENDIX I: OBJECTIVE, SCOPE, AND METHODOLOGY
The objective of our audit was to determine whether the FDIC had implemented adequate controls for ensuring that the scope of the NFE project was effectively managed and that any cost or schedule adjustments resulting from project scope changes were properly evaluated and controlled. Specifically, we were to determine whether changes to the functional and technical requirements in the statement of work were adequately supported by the 90-day test, JAD, and CRP results. (Note: In February 2003, the NFE project team completed a 90-day acceptance test of the PeopleSoft® Financials software. The purpose of this test was to verify that the software met FDIC’s business, security, and integration needs.)
We were unable to determine whether changes to the functional and technical requirements were adequately supported because of delays in completing the requirement definitions. The NFE project team was responsible for developing a requirements list at the end of the NFE general design phase in order to identify the scope of the NFE implementation. However, this final requirements list, originally scheduled for completion on April 15, 2003, had not been completed as of August 31, 2003. As a result, we could not conclude whether changes to the proposed requirements were adequately supported. We intend to conduct an audit of the support for changes to the proposed requirements after the requirements list is completed.
To accomplish our audit objective:
We performed the following activities for our audit:
Project Management Criteria
Although the FDIC is not required to comply with the PMBOK® Guide, we used it as the primary criteria for determining whether the FDIC had implemented adequate scope management controls because the guide contains generally accepted industry practices related to successful project management. The PMBOK® Guide documents proven practices, tools, and techniques that have become generally accepted in the field of project management, including information systems development and implementation. (Note: The PMBOK® Guide defines the term "generally accepted" as being applicable to most projects, most of the time, and having widespread consensus regarding value and usefulness.) The PMBOK® Guide is an approved standard of both the American National Standards Institute and the Institute of Electrical and Electronics Engineers. The PMBOK® Guide identifies nine distinct knowledge areas associated with successful project management. The nine areas are as follows:
APPENDIX II: ANALYSIS OF KEY CONTRACT DELIVERABLE DATES
Table in Appendix II: Analysis of Key Contract Deliverable Dates
Notes on Table in Appendix II: For the header “Delayed (Yes or No)” in the table below, a ”Yes” response means that the deliverable was delayed more than 2 weeks from the original October 2002 scheduled due date. For the header “Key Deliverables,” this means deliverables related to design and implementation and due by August 31, 2003. An additional 157 deliverables were periodic status reports and maintenance deliverables or were due after August 31, 2003. For the header “October 2002,” this refers to attachment 7 to the NFE contract initiated October 1, 2002. For the header “Accepted Date,” this means the date the FDIC contract oversight manager formally accepted the deliverable as satisfactorily completed. For the Configuration Management Approach/Initial Package Configuration with an Accepted Date of 3/24/03 in the table below, the Initial Package Configuration was accepted on March 24, 2003. The Configuration Management Approach was accepted on September 8, 2003. Regarding the General Design Document (FINAL) and Detailed Designs (DRAFT) items in the table below with an Accepted Date of 6/3/03, these documents are being delivered, reviewed, and approved in batches. The entire deliverable will not be accepted until all documents have been received and approved.
Source: OIG analysis of NFE contract deliverables.
APPENDIX III: CORPORATION COMMENTS
September 25, 2003
MEMORANDUM TO: Stephen M. Beard, Deputy Assistant Inspector General for Audits
FROM: Steven O. App [Electronically produced version; original signed by Steven O. App], Deputy to the Chairman and Chief Financial Officer; and Fred S. Selby [Electronically produced version; original signed by Fred S. Selby], Director, Division of Finance
SUBJECT: Draft Report Entitled The New Financial Environment ScopeManagement Controls (Assignment Number 2003-034)
We have completed our review of the subject Office of Inspector General (OIG) draft report. We appreciate the ongoing review work performed by the OIG and its recommendations to improve NFE project controls. We concur with the specific recommendations presented in this report and believe that implementing recommended actions will enhance efforts already underway by the project sponsors and project leaders to improve project oversight and control.
To better frame the environment in which the OIG report findings are presented, it is important to describe briefly the scope of the NFE project to emphasize that, while improvements are needed, existing project oversight and scope management efforts have been effective in many ways.
The scope of the NFE project is significant. The FDIC is implementing 27 PeopleSoft financial, performance management, and enterprise service modules; e-business support products; and portal packs. These modules/support tools address the 753 business and technical requirements identified during the planning stages of the project. NFE was chartered under several guiding principles which not only reflect industry best practices, but also lessons learned by the FDIC through long-time experience. These principles include:
Staying true to these principles has been a constant challenge. Implementing 27 modules/support tools of business functionality that directly or indirectly impact nearly every Division and Office in the FDIC has required extensive evaluation of software-delivered processes, data fields, and controls with the business owners to understand the specifics of the changes and agree that business needs can, in fact, be met. Through formal workshops and follow-up meetings involving hundreds of FDIC staff over the course of many months, the implementation team ended up with a list of 465 issues/questions/open items identified as a result of process, policy, or practice change. Each of these issues had the potential to impact requirements or system design decisions and, therefore, needed to be thoroughly evaluated and resolved with the business owner.
In addition to the 465 issues, 221 legacy applications needed evaluation to determine whether the application would be absorbed by NFE or needed to be renovated to accommodate the NFE-driven changes (for example, changes to accounting codes and reporting). As with the issues, each of these legacy applications had the potential to impact requirements or system design decisions.
The evaluation of the 27 business modules/support tools against 753 requirements, 465 issues, and 221 legacy applications, and the effort needed to finalize the new accounting code control key (known as “chartfields” in PeopleSoft), has taken longer than was originally contemplated when the project schedule was established in September of 2002. Extensive discussion, research, and evaluation of options have been required on many issues given the magnitude of the change or the complexity of the solution options. As a result, the key project milestones and deliverables (i.e., the Requirements Traceability Matrix and completion of general design documents) associated with this critical initial project phase have been delayed. While these delays do have a negative impact in the short term relative to project timeline and associated additional implementation costs, we believe the extra effort to date to manage the scope of NFE in line with our guiding project principles will yield significant benefit in the future as we move into the later project phases and post-implementation. This focus on quality outcomes as opposed to forcing date-driven decisions will help us avoid costly rework in the future.
The results of this extra effort can be seen in the numbers. To date, with 91% of the issues resolved, only 40 new requirements have been accepted. Of these 40, 21 resulted in gaps from delivered software. Another 31 potential new requirements are still open and tied to unresolved issues. At the high end, there is a potential of 71 new requirements, which is less than a 10% increase over our original 753. In addition, 19 of the original requirements were eliminated, in some cases resulting in work hours that can be utilized against the software gaps identified. Excluding custom reports, interfaces, and workflow designs, a total of only 30 gaps between business owner needs and software functionality delivered out-of-the-box have been identified. These gaps are associated with critical business needs to enhance control, support required reporting, or minimize manual intervention. General design documents are created for identified gaps to evaluate the scope and impact of the change and to provide a rough cost estimate (if tied to a new requirement). These design documents undergo an additional evaluation process before moving to the next stage to ensure that the business need continues to be justified given cost and impact.
Finally, beyond the numbers, the involvement in issues resolution and decision-making by the business owners and their staff has yielded significant benefit to our change management effort.
While very pleased with the value derived from the significant level of effort undertaken to work through project challenges, the NFE project leadership recognized the need to improve scope and time management efforts and has been implementing changes to our control environment to do so. Efforts such as creating a Program Management Office (to provide a control structure that reflects the broader scope of implementation work beyond what our existing project management structure was designed for), developing a legacy system change plan (to better guide the decision-making process), and working with Accenture and the project leads to improve status reporting and deliverables monitoring (so that issues and potential impacts are elevated in a more timely manner for higher-level decision-making when required) are just a few examples of actions already in place. OIG’s recommendations to expand our scope management plan, to develop a time management plan and a schedule control process, and to establish project metrics will support and enhance our current efforts.
Our specific responses to OIG’s recommendations are as follows:
If you have any questions regarding our response, please contact Karen Hughes at (202) 416-7201 or Steven P. Anderson at (202) 416-6975.
cc: James Anderson
APPENDIX IV: MANAGEMENT RESPONSES TO RECOMMENDATIONS
The following presents the management responses that have been made on recommendations in our report and the status of recommendations as of the date of report issuance. The information is based on management's written response to our report (and subsequent communication with management representatives).
Please note the following definitions that relate to the management responses to the recommendations:
Resolved: (1) Management concurs with the recommendation and the planned corrective action is consistent with the recommendation. (2) Management does not concur with the recommendation but planned alternative action is acceptable to the OIG. (3) Management agrees to the OIG monetary benefits or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides an amount.
Dispositioned: The agreed-upon corrective action must be implemented, determined to be effective, and the actual amounts of monetary benefits achieved through implementation identified. The OIG is responsible for determining whether the documentation provided by management is adequate to disposition the recommendation. Once the OIG dispositions the recommendation, it can then be closed.
Recommendation Number 1
Recommendation Number 2a
Recommendation Number 2b
Recommendation Number 2c
Recommendation Number 2d
|Last Updated 1/26/2004|