Systems and Data Conversion for the New Financial Environment
June 2005
Report No. 05-020
AUDIT REPORT

Background and Purpose of Audit
The FDIC Office of Inspector General (OIG) contracted with KPMG LLP to audit the effectiveness of the New Financial Environment (NFE)
system development data and systems conversion activities. The NFE’s Transition Plan and a Data Conversion Approach and Plan contained
numerous interdependent tasks for NFE system deployment. These tasks included completing user procedures for 23 key business operations,
ensuring data integrity for 35 retiring systems and 23 interfacing legacy systems, and conducting user acceptance testing for the core financial
system. The Transition Plan defined the overall framework for the transition to the NFE system. The Data Conversion Approach and Plan
presented the methodology for data conversion activities (legacy systems and PeopleSoft applications). NFE deployment was scheduled for May 2, 2005.
The audit objective was to determine whether the systems and data conversion plans and activities were adequate to minimize the risk of
errors and omissions during implementation of the NFE.
 |
Results of Audit
The FDIC had developed a general data conversion methodology tailored for the NFE implementation at the FDIC that presents
considerations common among all data conversion activities and a brief description of the approach to converting each application.
KPMG had reservations regarding the lack of detailed data conversion, validation, and clean-up plans for the asset management,
general ledger, vendor, purchase order, accounts receivables and cash management functions. KPMG also noted that performance
testing proceeded without a detailed plan and did not include critical tests. Lack of detailed plans and sufficient tests increased the
likelihood of errors and omissions during the conversion process and limited the FDIC’s ability to identify and resolve issues potentially
impacting or interrupting NFE operations.
The audit was terminated during fieldwork to avoid delaying the NFE implementation schedule. KPMG was unable to collect sufficient,
competent, and relevant evidence in a timely manner, as required by generally accepted government auditing standards, to provide a
reasonable basis for audit conclusions related to the objective. Therefore, KPMG disclaims from providing any assurances. However, KPMG
did suggest that its reservations and the associated risks be mitigated.
Management Response
The FDIC’s Division of Finance (DOF) responded that the conversion activity planning and execution, coupled with the active involvement
of data owners from the impacted business areas in planning, testing, and validation, provided a high degree of confidence that the conversion
of data would result in minimal and manageable operational disruption and conversion errors. Regarding performance testing, DOF indicated that
“tuning” of functions has continued following implementation in those few situations where on-line response time or batch throughput was found
to need improvement. DOF expects this process to continue for several months, but no interruptions or delays in service are anticipated.
Due to the audit being terminated, we cannot confirm or evaluate the adequacy of the various actions that DOF indicates were taken either
in response to KPMG’s reservations or in the course of planned conversion activities.
NFE Conversion Stages
[ D ]
|
TABLE OF CONTENTS
| Part I: |
Report by KPMG LLP
Systems and Data Conversion for the New Financial Environment |
| Part II: |
Corporation Comments and OIG Evaluation
Corporation Comments
|
Part I
Report by KPMG LLP
Systems and Data Conversion for the New Financial
Environment
Prepared for the
Federal Deposit Insurance Corporation
Office of Inspector General
TABLE OF CONTENTS
| EXECUTIVE SUMMARY |
| Results of Limited Review |
| BACKGROUND |
| NFE System Implementation |
| NFE Test Strategy |
| Data and Systems Conversion |
| THE FDIC’S DATA AND SYSTEMS CONVERSION APPROACH AND KPMG’S RESERVATIONS |
| Data Conversion |
| Systems Conversion |
| Conclusion |
| APPENDIX A: OBJECTIVE, SCOPE, AND METHODOLOGY |
| APPENDIX B: RISK ASSESSMENT APPROACH |
| TABLES |
| TABLE 1: NFE Conversion Development Cycle |
| TABLE 2: Reservations Associated With Data Conversion Activities |
| TABLE 3: Reservation Associated With Systems Conversion Activities |
| FIGURE |
| Risk Assessment Matrix |
EXECUTIVE SUMMARY
The Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) contracted with KPMG LLP to provide professional audit services. KPMG was tasked under the contract to audit the effectiveness of the FDIC’s New Financial Environment (NFE) system development data and systems conversion activities. The objective of the audit was to determine whether systems and data conversion plans and activities were adequate to minimize the risk of errors and omissions during implementation of the NFE.
KPMG was unable to conduct its work in accordance with generally accepted government auditing standards. Specifically, KPMG was unable to collect sufficient, competent, and relevant evidence in a timely manner as required by generally accepted government auditing standards to provide a reasonable basis for audit conclusions related to the audit objective. In addition, the scope of work performed did not support an opinion regarding our objective, the adequacy of internal control, or compliance with applicable regulations related to NFE systems and data conversion. Further, FDIC management informed us
that providing access to the OIG would “definitely impact” the NFE implementation schedule. As a result, the audit was terminated on April 6, 2005, and KPMG disclaims from providing any assurances with respect to the audit objective. A detailed discussion
of our audit objective, scope, and methodology is provided in Appendix A of this report. This report provides information on our reservations with regard to the objective based on the limited review that KPMG performed.
KPMG evaluated data and systems conversion activities according to guidelines established by the National Institute of Standards and Technology (NIST), the Capability Maturity Model Integration (CMMI) for systems engineering, and Joint Financial Management Improvement Program (JFMIP) for government financial systems. KPMG also considered relevant guidelines from the Government Accountability Office (GAO) and the Office of Management and Budget (OMB) that are related to the implementation of the Federal Financial Management Improvement Act (FFMIA) and OMB Circular No. A 127, Financial Management Systems.
Results of Limited Review
The FDIC had developed a general data conversion methodology that was tailored for the NFE implementation at the FDIC that presents considerations common among all data conversion activities (legacy systems and PeopleSoft applications) and a brief description of the approach to converting each application. The FDIC also has reported considerable progress in completing most data and systems conversion activities considered critical to the NFE deployment, which occurred on May 2, 2005.
Although KPMG cannot express an overall opinion regarding the systems and data conversion activities, the limited work performed did identify several reservations related to the data conversion process, data validation testing, and data clean-up planning that
warranted management attention. Additionally, KPMG noted performance test concerns in preparation for systems conversion to NFE.
BACKGROUND
On December 10, 2001, the FDIC’s Board of Directors approved the purchase and implementation of a commercial-off-the-shelf solution to support an enterprise-wide financial environment for the FDIC. The decision was based on the need to modernize
the FDIC’s complex and aging legacy financial systems. In October 2002, the FDIC contracted with Accenture LLP (Accenture) to assist the Corporation in replacing its financial systems with a PeopleSoft financials solution. This effort, called the New Financial Environment project, is jointly managed by the FDIC’s Division of Finance (DOF) and Division of Information Technology (DIT). The current NFE project timeline is separated into two phases of deployment. The first phase called for deployment of core PeopleSoft financial modules in May 2005. Phase II deployment includes the PeopleSoft Enterprise Performance Management suite to assist in FDIC strategic decision-making activities in the areas of Budgeting and Receivership Service Billing by July 1, 2005 and Activity Based “Cost” Management by September 1, 2005.
NFE System Implementation
The implementation of the NFE system affects many systems throughout the FDIC. Therefore, the deployment of the system will, in varying degrees, involve users from each division and office of the Corporation. The NFE project team has coordinated the transition activities with the business owners. These activities include understanding and documenting re-engineered business processes and preparing for data conversion.
To prepare for NFE implementation, Accenture developed a Transition Plan and a Data Conversion Approach and Plan, which contained numerous interdependent tasks that had to be performed for NFE system deployment. Some of these tasks included completing user procedures for the 23 key business operations, ensuring data integrity for 35 retiring systems and 23 interfacing legacy systems, and conducting user acceptance testing for the core financial system. The Transition Plan defined the overall framework for the transition to the NFE. The plan listed the transition activities; stakeholder
responsibilities; communication methods for stakeholders; NFE and other FDIC system interfaces; and the management, control, and reporting mechanisms for transition
progress. The Data Conversion Approach and Plan presented the methodology for the conversion of data from the legacy systems to the PeopleSoft applications. Although this plan was general in nature, more detailed design documents for application-specific data conversions had been prepared.
The plan and design documents described above also defined activities for pre-conversion and cutover phases of data and systems conversion. Pre-conversion activities included tasks prior to and leading up to the conversion, such as determining the scope and
approach or method, developing the conversion plan, performing data clean-up and validation, ensuring data integrity, and conducting necessary analysis and testing. Cutover tasks to convert the legacy data to the new system were to include testing system process and data edits; testing system interfaces, both incoming and outgoing; managing the critical path of system implementation; supervising workload completion; and performing data reconciliation.
NFE Test Strategy
The FDIC had developed a rigorous multi-stage test strategy and schedule for NFE to ensure that the system will function as designed and meet user’s needs. Key components of this test strategy included Systems Integration Testing (SIT) and User Acceptance Testing (UAT). Additionally, the FDIC had established independent quality assurance testing for NFE that was performed by the FDIC Configuration and Quality Management Staff (CQMS).
- SIT ensures that all business functions perform as designed on an end-to-end basis across the NFE applications and platforms. SIT verifies that the application modules interact correctly within PeopleSoft financial modules, including all interfaces that send or receive transactional data to/from the NFE.
- UAT is the final round of NFE testing, and its purpose is to secure the agreement of all business process owners that the PeopleSoft modules, as modified and configured, and the impacted FDIC legacy application interfaces meet the business owners’ current, stated business requirements when used in conjunction with processes and procedures developed by the business owners and the NFE business planning team.
- CQMS performed quality assurance test activities over a 3-week period starting on November 1, 2004. The scope of this review, as stated in the CQMS NFE Test Plan dated September 22, 2004, was to verify the effectiveness of the SIT performed against NFE core financial modules and interfaces.
Data and Systems Conversion
Of the many critical tasks necessary to successfully implement a new financial system, data conversion is one of the most frequently underestimated. From the outset, it should be understood that financial systems data conversion is a complex and difficult task that requires highly skilled staff for successful completion. If data conversion is done right,
the new system has the opportunity for success. However, converting data incorrectly has lengthy and long-term repercussions.
Data conversion is defined as the automated or manual modification of existing data to enable the data to operate with similar functional capability but in a different
environment. Automated conversion is the process of transferring selected transactions from the legacy system to the new one through the use of an automated tool or custom-developed software. The volume of data and difficulty in performing an automated
conversion are examples of factors considered in determining whether to use a manual or automated conversion process. Other specific issues that apply uniquely to the replacement of a financial system include the identification of specific open transactions and beginning balances to be established, consideration of data conversion approaches and implications, and analysis and reconciliation to validate transactions and data.
Systems conversion relates to activities to prepare a new system for deployment in its operational environment, including the planning and tests of system performance as a means of addressing any performance-related issues that may impact the availability of a new system to its user community.
THE FDIC’S DATA AND SYSTEMS CONVERSION APPROACH AND KPMG’S RESERVATIONS
The FDIC’s approach in addressing data and systems conversion activities and KPMG’s reservations are addressed in the following sections.
Data Conversion
The FDIC had developed a high-level Data Conversion Approach and Plan, initially dated October 31, 2003, for the NFE implementation at the FDIC. The approach provided guiding principles common among all data conversion from legacy systems to PeopleSoft applications that included, for example:
- determination of what data will be converted based on the “To-Be” processes defined, rather than the “As Is” legacy data;
- data conversion will be automated when justified (volume, reusability, or cost savings);
- legacy data will undergo data cleansing to reduce data volume and minimize data integrity issues;
- data conversion methods will be tested for their limitations in “mock” conversions prior to the actual conversion; and
- converted data will be used in system and user acceptance testing to help identify data conversion and data integrity issues.
Additionally, as shown in Table 1, the approach identifies major development stages and the objectives for each stage of the conversion process that each major NFE conversion area would undergo.
Table 1: NFE Conversion Development Cycle
 |
 |
Assess |
 |
Analyze |
 |
Design |
 |
Build |
 |
Implement |
 |
 |
 |
Identify the legacy systems data |
 |
Conduct further analysis into the data types identified during the Assess stage |
 |
Create translation |
 |
Develop conversion program modules |
 |
Schedule conversion |
 |
 |
 |
Define the requirements for historical data |
 |
Create data maps & validation rules |
 |
Create manual conversion procedures (if applicable) |
 |
Verify that data clean-up efforts are on schedule |
 |
Extract clean legacy data |
 |
 |
 |
Assess the data quality in legacy systems |
 |
Identify specific data extraction criteria from legacy systems |
 |
Prepare detailed design specifications |
 |
Execute test plans |
 |
Execute official conversion |
 |
 |
 |
Determine data volumes |
 |
Finalize selection of specific conversion tools |
 |
Verify that data clean-up efforts are on schedule |
 |
Define conversion cutover plan |
 |
Reconcile converted data to legacy data, and obtain sign-off |
 |
 |
 |
Identify crosswalks |
 |
Develop templates and standards for the design stage |
 |
Develop verification and sign-off procedures |
 |
Execute “mock” conversions and verify |
 |
|
 |
 |
 |
Identify the best method of conversion |
 |
Update the work plan, and revise time estimates |
 |
Update the work plan, and revise time estimates |
 |
Resolve problems with data and/or conversion specifications |
 |
|
 |
 |
 |
Estimate analysis time, and prepare an overall work plan |
 |
|
 |
|
 |
Update the work plan, and revise time estimates |
 |
|
 |
 |
 |
KPMG noted that the NFE Data Conversion Approach and Plan also provided a brief description of the approach for converting each
application where automated conversions and data clean-up were planned for asset management, general ledger, vendor, and purchase
order functions. KPMG also noted that manual conversions were planned for accounts receivable and cash management functions.
NFE Project Management documentation shows that performance of conversion development activities began in October 2003 with
the development of an overall strategic development plan, development of design documents from November 2003 through April 2004,
and mock conversions from May 2004 through March 2005.
KPMG has summarized its reservations associated with data conversion activities in Table 2 on the next page and has assigned a risk
ranking based on risk-management-assessment criteria defined for the NFE project. See Appendix B for a description of the risk assessment
approach.
Table 2: Reservations Associated With Data Conversion Activities
 |
 |
 |
Risk |
 |
 |
 |
 |
 |
Reservations |
 |
Potential Impact |
 |
High |
 |
Medium |
 |
Low |
 |
 |
 |
Data Conversion Process |
 |
 |
 |
- No detailed conversion plan or approach for major automated conversion (AM, GL, PO, and Vendor).a
|
 |
- Data conversion requirements not adequately defined or addressed, increasing likelihood of errors and omissions.
|
 |
|
 |
|
 |
|
 |
 |
 |
- No detailed plan or approach for manual conversions (Accounts Receivable, Cash Management).
|
 |
- Risk of errors and omissions increases without appropriately defined, critical pre-data conversion activities.
|
 |
|
 |
|
 |
|
 |
 |
 |
- Data conversion detail design documents for most automated conversion areas lack both functional and technical information (e.g., reasons for approaches, references to data clean-up/ validation approaches, data mapping of source/target fields).
|
 |
- Account balances not traceable to audited sources from the legacy systems.
|
 |
|
 |
|
 |
|
 |
 |
 |
Data Validation Testing |
 |
 |
 |
- No standards or formal planning regarding data validation for most major automated conversion areas.
|
 |
- Data integrity issues may exist without validation rules defined and implemented.
|
 |
|
 |
|
 |
|
 |
 |
 |
- Testing of converted data may not be adequately addressed. UATb combines new and converted data into the same test script (e.g., focus in AM appears to have been on newly created scripts).
|
 |
- Without the separation, testers cannot easily identify additional testing efforts required for data validation, increasing risks of errors and omissions (e.g., appears that FDIC incorporated KPMG validation list suggestions into AM for testing converted assets from April 1 through April 7, 2005).
|
 |
|
 |
|
 |
|
 |
 |
 |
Data Conversion Results |
 |
 |
 |
- Purchase orders are converted into NFE without supporting detail, and the amount of time to search supporting detail from the legacy systems, including impact on users’ workload is unknown. Consequently, users will not be able to determine through NFE whether an invoice for a converted PO was previously paid.
|
 |
- Approximately 1,000 out of 5,000 partially paid POs are potentially at risk of duplicate payment.
- Potential inability to pay vouchers in a timely manner may cause the FDIC to be in violation of the Prompt Payment Act.
- Having to maintain the legacy systems and the PeopleSoft application may create extreme confusion and hardship from both an operational and technical support perspective.
|
 |
|
 |
|
 |
|
 |
 |
 |
Data Clean-up Results |
 |
 |
 |
- No data clean-up plans for major automated conversion areas.
|
 |
- Errors may be perpetuated in NFE, causing major post-reconciliation efforts to occur.
|
 |
|
 |
|
 |
|
 |
 |
 |
aAM – Asset Management, GL – General Ledger, PO – Purchase Order.
bUAT – User Acceptance Testing.
Systems Conversion
FDIC management stated that the NFE performance testing did not follow a typical system development life cycle’s performance test approach.
For the NFE, high-level performance measures were needed in the statement of work (SOW) and were used as the basis for performance planning
throughout all levels of testing. Specifically, the SOW stated that results are acceptable for:
- 5 seconds or less for online transactions on the FDIC’s metropolitan area network and 15 seconds or less on the wide area network;
- 2 minutes or less for queries of little complexity;
- 5 minutes or less for queries of moderate complexity; and
- 10 minutes or less for queries of high complexity.
Additionally, the SOW stated that the estimated range of transaction volume for the NFE is from a low range of 11,896,500 with 25 concurrent
users to an estimated high range of 21,274,000 with 150 concurrent users during a crisis. According to FDIC management, the criteria were based
on the current legacy systems’ performance. The metrics described form the basis for five cycles of performance testing where each is intended to
do the following:
- Cycle One – NFE On-line Stress Test
- Cycle Two – Corporate Human Resources Information System, including Supplemental Payments
- Cycle Three – Major PeopleSoft Tuning
- Cycle Four – Query/Report
- Cycle Five – Batch Window Tuning
KPMG has summarized its reservation associated with systems conversion activities in Table 3 below and has assigned a risk ranking based on risk
management assessment criteria defined for the NFE project. See Appendix B for a description of the risk assessment approach.
Table 3: Reservation Associated With Systems Conversion Activities
 |
 |
 |
Risk |
 |
 |
 |
 |
 |
Reservations |
 |
Potential Impact |
 |
High |
 |
Medium |
 |
Low |
 |
 |
 |
Systems Conversion – Performance Testing |
 |
 |
 |
- System performance testing proceeded without a detailed test plan and omitted critical tests of performance.
|
 |
- Unpredictable system performance could seriously impact or interrupt NFE operations.
|
 |
|
 |
|
 |
|
 |
 |
 |
Conclusion
KPMG recognizes that the reservations identified may not be addressed prior to NFE scheduled implementation. Also, audit work was not
completed due to the scope limitation previously discussed. Therefore, the report contains no recommendations. However, KPMG suggests
that the Directors of DOF and/or DIT and the NFE project management team review the risks identified and develop risk mitigation procedures
as outlined in the NFE risk management plan for addressing the reservations at an appropriate time.
APPENDIX A: OBJECTIVE, SCOPE, AND METHODOLOGY
Objective
The objective of the audit was to determine whether the systems and data conversion
plans and activities were adequate to minimize the risk of errors and omissions during implementation of the NFE.
Scope
The scope of coverage in meeting the audit objective focused on data and systems conversion
activities critical to NFE deployment, which included the following:
- Evaluate the effectiveness of NFE data and systems conversion planning and test activities critical to NFE deployment for NFE
modules where automated conversions were to take place.
- Focus on performance test issues related to systems conversion activities.
- Evaluate data conversion activities for NFE interfaces to legacy systems for three of eight selected NFE legacy interfaces considered
critical to deployment. The interface systems included the Control Totals Module as the primary receivership and subsidiary financial
reporting system, Electronic Travel Voucher System, and Dividend Processing System.
- Evaluate effectiveness of “mock” data conversions for accuracy and completeness.
- Evaluate effectiveness of system performance tests as an accurate predictor of production performance in identifying any performance
problems for corrective action prior to system deployment.
Scope Limitation
The audit of NFE systems and data conversion was terminated on April 6, 2005 before it
was completed. As previously stated, KPMG was unable to collect sufficient, competent, and relevant evidence in a timely manner as required
by generally accepted government auditing standards to provide a reasonable basis for audit conclusions related to our objective. More specifically,
the scope of audit work performed did not support an opinion regarding our objective, the adequacy of internal control, or compliance with
applicable regulations related to NFE systems and data conversion. In addition, had the audit been completed, other matters may have come to
our attention concerning the adequacy of overall NFE implementation.
Beyond our concerns regarding compliance with auditing standards, our goal was to complete our audit work and provide management with findings,
conclusions, and recommendations, if any, that management could address prior to NFE implementation to achieve maximum benefit. However, due
to NFE implementation activities, NFE project managers and team members were often not available to respond in a timely manner to our requests
for interviews or documentation. In fact, FDIC management informed us that providing access to the OIG would “definitely impact” the NFE
implementation schedule. With data conversion activities still in process and NFE implementation imminent, we could not obtain sufficient information
or perform necessary procedures regarding the data conversion activities to achieve our goal or to accomplish our stated audit objective.
Methodology
KPMG performed the following work related to the audit objectives before terminating the effort:
- Conducted interviews with DIT and DOF officials who were responsible for managing and implementing the NFE project as well as
representatives from Accenture LLP, the consulting firm hired by the FDIC to provide NFE implementation services, including performance
of system development test activities. We also spoke with business process leads from several divisions to determine their involvement in
data conversion activities such as data reconciliation and to obtain an understanding of NFE conversion activities, including procedures and practices.
- Reviewed NFE system performance, systems conversion, and data conversion documentation.
- Performed data analysis to sample the completeness and accuracy of asset conversion.
- Obtained and reviewed conversion test script and results for AM including review of reconciliation processes applied.
- Obtained a data extract of purchase orders from the Financial Information Management System to perform data analysis to determine conversion volume, feature workload, and risk of duplicate payments.
KPMG also determined the risk levels for the NFE project related to data and systems conversion activities where specific risks are likely to occur.
Prior Audit Coverage
The OIG has issued the following reports related to the NFE:
- Audit Report No. 05-019 entitled, FDIC’s New Financial Environment Testing, dated June 6, 2005, which addressed the adequacy of NFE’s test and defect management processes.
- Audit Report No. 05-007 entitled, Management Controls Over the Re-baselined New Financial Environment Project, dated February 18, 2005, which addressed whether the FDIC has established adequate management control over the re-baselined NFE project.
- Audit Report No. 03-045 entitled, New Financial Environment Scope Management Controls, dated September 29, 2003, which addressed whether the FDIC had implemented adequate controls for ensuring that the scope of the NFE project was effectively managed.
- Audit Report No. 03-016 entitled, The New Financial Environment Project Control Framework, dated March 5, 2003, which addressed whether the FDIC had established a control framework for the NFE project.
- Audit Report No. 03-002 entitled, Preaward Review of the New Financial Environment Project, dated October 7, 2002, which provided observations on selected procedures and documents related to the NFE Request for Proposal.
- Evaluation Report No. 01-004 entitled, The New Financial Environment Project, dated December 7, 2001, which assessed the reasonableness of the NFE cost-benefit analysis and the financial systems architecture.
KPMG conducted field work in Washington, D.C., and at the FDIC’s Virginia Square facility from January through March 2005. Based on the scope limitation described earlier, KPMG was unable to conduct this audit in accordance with generally accepted government auditing standards. Therefore, KPMG disclaims from providing any assurances with respect to the objective of the audit.
APPENDIX B: RISK ASSESSMENT APPROACH
Risk Ratings
Per CMMI and industry standard practices, software projects should establish a risk management strategy that includes categorization of risks identified to develop a mitigation strategy that reduces risks to levels acceptable to management. KPMG
assessed the potential impact of risks identified in this review based on professional judgment and applicable risk management criteria defined for the NFE project by the FDIC. The NFE project assesses risks based on probability of occurrence and impact as follows:
Probability
The likelihood of risk occurrence is quantitatively or qualitatively rated on the following scale:
|
Probability
|
Uncertainty Statement
|
Evaluation of Impact
(see Impact)
|
|
> 80%
|
Extreme, Almost
certain
|
|
|
61%-80%
|
High, Likely
|
4
|
|
41%-60%
|
Medium
|
3
|
|
21%-40%
|
Low
|
2
|
|
1%-20%
|
Very Low, Highly
unlikely
|
1
|
Impact
Impact is an estimate of the overall scale of the impact following an occurrence of each risk.
Impact measures the severity of adverse affects, or the magnitude of a loss, if the risk comes to pass and is
rated on the following scale:
5 - Critical impact; threatens overall success of NFE on a long-term basis.
4 - High impact; significant disruption to successful delivery of NFE objectives, products, and benefits.
3 - Medium impact; significant disruption to NFE schedule, cost, and products over the medium term.
2 - Low impact; progress disrupted with moderate to low extensions to schedule and cost, across short term.
1 - Very low impact; slight exposure.
The two variables, impact and probability, are combined to assess the overall risk category as displayed in the following matrix.
Risk Assessment Matrix
Source: The FDIC New Financial Environment Risk Management Plan developed by Accenture.
[ D ]
Risk categorization is based upon factors where specific risks are likely to occur including technical, operational, external,
resource/cost, and schedule. Overall risks assigned by KPMG focused on issues impacting the FDIC’s ability to achieve NFE
objectives from both a technical and operational nature. These factors, referred to as risk drivers, may impact both Cost and Schedule risks.
Each risk is described further below:
Technical
Technology-based risks consider the non-achievement of the application specifications and benefits expected. These risks include
new/non-standard platform technology, integration problems with existing systems, migration problems, performance expectations
not achieved, environment complexity and functionality, and system operability.
Operational
Operational-based risks focus on the peripheral organizational and business operational re-engineering changes arising from the NFE
implementation effort. These risks consider both the transitional and the long-term effects of the NFE’s introduction, including the
organizational and behavioral changes required, the human and physical resource planning, and communication required to facilitate
a smooth transition to the new structure.
External
External-based risks consider the environmental factors largely outside of the control of the NFE Project Management
that can directly or indirectly affect the successful delivery of the NFE. Risks arising from legislative regulations,
legal requirements, and the strategic direction and priority conflicts of a controlling body are profiled under this category.
Resource/Cost
Cost-based risks outline the non-achievement of the financial benefits of NFE. These cost risks include additional
costs in changing or solving design, application program, or operational problems.
Schedule
Schedule-based risks focus on the non-achievement of the biggest system benefits within the specified
time frame. These schedule-based risks arise from extensions as a result of scope changes, resource unavailability,
and additional schedule extensions for solving the risks as discussed earlier in Resource/Cost.
Part II
Corporation Comments and OIG Evaluation
CORPORATION COMMENTS AND OIG EVALUATION
We provided the Director, DOF, a draft of this report dated April 22, 2005. The report did not contain a
recommendation, and a written response was not required. However, DOF provided a written response, which
is presented in its entirety, beginning on page II-2, and is summarized below, along with our evaluation of the response.
DOF Response: DOF management responded that it believes that the NFE conversion approach was
well planned and well executed. Further, DOF stated that where known data conversion weaknesses existed, such as
those associated with the purchase order conversion described in our report, manual controls and actions were taken to
minimize the risks to the Corporation.
With respect to NFE performance testing, DOF indicated that “tuning” of some functions has continued during the period
immediately following implementation in a few situations where on line response time or batch throughput needed improvement.
As with any major new implementation, this process is expected to continue for several months, but no interruptions or delays in
service are anticipated.
DOF also indicated that since the OIG was completing its field work on NFE testing, the timing of the additional audit on NFE
systems and data conversion efforts was less than optimal as the entire NFE project team was focused on completing the necessary
tasks in anticipation of NFE’s May 2, 2005 go-live date. DOF expressed appreciation for our decision to terminate the audit and believed
it was appropriate under the circumstances.
OIG Evaluation of Response: Due to the audit being terminated, we cannot confirm or evaluate the adequacy
of the various actions that DOF indicates were taken either in response to KPMG’s reservations or in the course of planned conversion activities.
With regard to the timing of our two NFE-related audits, the FDIC’s concurrent scheduling of two critical system development
activities — testing and conversion — made it necessary for the OIG to schedule the two audits in a manner that resulted in overlapping
timeframes. However, as discussed earlier in the report, we terminated this audit when FDIC management informed us that providing
access to the OIG would “definitely impact” the NFE implementation schedule. While we are pleased that DOF agreed with our decision
to terminate the audit, it is important to note that we independently made the decision after carefully assessing the cost/benefit of
continuing the audit, including the impact of the audit on the NFE project team’s ability to meet the NFE go-live date.
[ D ]
[ D ]