To: Stephen M. Beard,
            Deputy Assistant Inspector General for Audits

From: Michael J. Zamorski
            Director

Concur: John F. Bovenzi
            Deputy to the Chairman and Chief Operating Officer

Subject: Draft Report Entitled, "DSC's Process for Tracking and Evaluating the Impact of the MERIT Guidelines" (Assignment Number 2004-039)

The Division of Supervision and Consumer Protection (DSC) appreciates the opportunity to respond to this Draft Report prepared by the FDIC's Office of Inspector General (OIG). We agree with your observation that utilization of Maximum Efficiency, Risk-focused, Institution Targeted (MERIT) guidelines has contributed to more efficient use of examination resources. However, we disagree with the Report's implication that the scaling back of loan portfolio reviews in the lowest-risk institutions represents a potential risk to the integrity of safety and soundness examinations. DSC already utilizes a number of measures to ensure the quality and integrity of the examination process. The Report does not consider the breadth and depth of these existing risk management processes, nor does it offer compelling reasons to implement a comprehensive monitoring system that segregates MERIT examinations from other examinations for special reviews.

As the Report notes, the FDIC, as well as the other federal banking agencies, has for many years conducted all examinations on a risk-focused basis to enhance examination efficiencies and to reduce regulatory burden. The risk-focused approach makes use of the bank's own internal control and risk management systems, and emphasizes the accuracy of overall results as opposed to trying to capture each possible item subject to review. Risk-focused examinations rely on the professional expertise of personnel on multiple levels. The examiners-in-charge (EICs) determine the scope of examinations based on past examinations, results of off-site monitoring systems, discussions with bank management, contact with external auditors, and financial performance, among other things. Examiners have the duty to expand examination activities if red flags are present indicating potential increased risk. Further, supervisors also have the duty to assure that examinations are appropriately conducted. At completion, all reports of examination receive at least one level of review and obtain supervisory concurrence before transmission to the institution.

MERIT guidelines should not be interpreted as a new examination program or a different type of examination. The MERIT guidelines are entirely consistent with the risk-focused examination approach, and examinations conducted using MERIT guidelines are, by any measure, full-scope examinations. In fact, the MERIT guidelines merely define the reduced level of transaction testing expected for institutions with low-risk profiles. To be eligible for MERIT guidelines, suitable banks must display solid performance and meet relevant criteria for a period of time, thereby filtering out banks involved in high-risk activities, new banks, banks with new management, or reflecting other concerns. Such strong performance must be demonstrated for two consecutive examinations, which more than likely will have two different EICs and may alternate with a state banking department examination.

It is important to note that the development of MERIT was an interdivisional effort. Extensive research and analysis, using data from numerous prior years, was undertaken to define the MERIT criteria. Rigorous filtering processes and controls were incorporated into the program to ensure that only banks with low-risk profiles were eligible. Field Supervisors and Examiners on the MERIT Development Team diligently developed criteria for loan penetrations based on their extensive experience, data, and analysis of prior examinations and workpapers.

Examiner judgment remains an important element in the proper implementation of the program. The removal of approximately 300 banks from MERIT-eligibility in 2004 clearly indicates that the program is working as intended and that MERIT examinations are not being performed on banks that reflect elevated risk. As the Report states, DSC amended its Summary Analysis of Examination Report in 2004 to more readily capture the reasons that these banks are no longer eligible. This information will provide another useful tool to DSC in its assessment of MERIT and in DSC's overall quality control process.

Given the extensive filtering process to become MERIT-eligible, and the existing controls and review over all examination products, the value of creating a comprehensive quantitative monitoring system for low-risk banks is highly questionable. The Report does not include an evaluation of any of the examination controls and systems in place, yet recommends DSC create an additional quantitative system to measure MERIT examinations, which constitute our lowest risk portfolio of banks and those in which we have the highest degree of confidence.

Response to Specific Recommendations

Recommendation:

1. Implement a monitoring process for tracking and evaluating the impact of reduced loan coverage at MERIT-eligible institutions.

   After careful consideration, DSC does not concur with this recommendation. DSC maintains a strong risk management process, in the form of a comprehensive quality control program, for all examinations. In addition to a sound, risk-focused examination program and utilization of a well-trained and experienced examination staff, there are a number of other processes that provide assurances with regard to the integrity of examination findings, including those conducted using the MERIT guidelines. These include:

   1. An alternating examination schedule with state banking departments,
   2. Rotation of EICs,
   3. Multi-tier staff involvement in the examination process,
   4. Examination review processes,
   5. Examination surveys,
   6. Off-site monitoring programs,
   7. Banker outreach initiatives, and
   8. Strong internal review and quality assurance evaluations.

   With regard to most state non-member banks, DSC alternates examination responsibility with state banking authorities. Alternating the agencies which conduct examinations provides a significant check-and-balance for examination procedures performed and findings reached. It is also customary to alternate EICs at consecutive examinations, which also provides a cross-check regarding the quality of previous examinations.

   Field supervisors, supervisory examiners, and case managers maintain a thorough knowledge of banks operating within their designated areas. These individuals are cognizant of the macro- and micro-economic issues impacting their institutions, and actively participate in determining the examination scope and procedures to be utilized during the examination. Final examination reports are reviewed by field office management and/or case managers to ensure the examination scope and procedures documented within the pre-examination planning memorandum were followed (with exceptions documented) and to ensure that the examination findings are sufficiently supported.

   DSC is currently expanding the Field Territory internal review program designed to enhance the evaluation of adherence to established operating policies and procedures. This action has been taken in response to increased delegation of examination responsibilities and will ensure qualitative controls are maintained. During these reviews, a sample of examination reports and related work papers is reviewed to ensure that the examination scope (including loan review) was appropriate, identified examination procedures were followed, and the bank's risk profile was properly identified and addressed.

   At the conclusion of every examination, surveys concerning the examination are provided to bank management. Bank management is asked to comment on a number of issues concerning the examination process, including whether the examination appropriately identified risks facing the institution.

   During the interim period between examinations, the operational status of the bank is monitored through DSC's existing off-site supervisory programs, which include the Growth Monitoring System (GMS), Statistical CAMELS Off-site Rating (SCOR) system, and the Multi-Flag system. These systems monitor quarterly financial data and alert DSC to potential emerging risk areas.

   Finally, field and regional staff have regular contact with banks via banker outreach initiatives, which provide for direct feedback from the banking community concerning the examination process, including the use of MERIT guidelines.

Possible Approaches Suggested by the OIG for Monitoring MERIT Examinations

In conjunction with the recommendation to implement a comprehensive monitoring process, the OIG suggests that the following approaches could provide useful information with respect to risk assessment:

• Track and compare various financial ratios, such as Past Due Loan percentages, to help monitor whether the quality of the overall MERIT bank loan portfolios are declining.

  As noted above, DSC already monitors various financial ratios for all banks continuously through the use of the SCOR off-site monitoring tool. Changes in loan quality indicators detected by SCOR are assessed by field and regional staff. Any material concerns are documented, and this information is utilized in determining the scope of loan reviews at the next safety and soundness examination.

  We do not believe it would be beneficial to implement a comprehensive program that segregates and analyzes the collective financial ratios of MERIT-eligible institutions over time. It has been our experience that, collectively, external economic factors tend to play a much more significant role in elevated loan delinquency ratios and other loan quality indicators. Given the rigorous MERIT eligibility criteria, which would automatically exclude most other identifiable sources of deterioration from the universe of MERIT-eligible institutions, these external factors--which tend to be localized--would overshadow any subtle trends that might otherwise be detected when analyzing MERIT-eligible institutions on an aggregate basis.

• Establish a process to determine a periodic sample of MERIT-eligible institutions and conduct full-scope examinations of these institutions. This would establish a control group for which the results could be statistically benchmarked to all MERIT examinations.

  In considering whether to implement this approach, DSC consulted with the FDIC's Division of Insurance and Research, including at least one PhD statistician on staff. To produce a reliable statistical sample with reasonable confidence intervals, the sample size for such a control group would have to be relatively large (i.e., 100 examinations per year). Additionally, to avoid tainting the sample, examiners could not be told that a particular examination was to be part of the control group until all of the pre-examination scoping had been completed. This would be extremely disruptive to entire examination schedule (and that of our State counterparts); thus this option is not feasible.

  Moreover, the value added by establishing a control group for benchmarking MERIT examinations is unclear. Control group comparisons are useful in stable environments where examinations (both control group and others) occur under similar conditions. Such a technique does not account for shifts in risk profiles that can occur due to external factors. Further, the use of a control group implies comparison of distinctly different approaches. The use of MERIT guidelines is nothing more than further refinement of long-standing, risk-scoping concepts, and is therefore not a distinctly different approach. OIG in essence is recommending substantially expanded transaction testing in selected institutions, which is already an option at every examination if conditions warrant.

• Establish a process to determine periodic sample of MERIT-eligible institutions and utilize statistical loan sampling techniques for the associated institutions in lieu of judgmentally selected samples. This would establish a benchmark for selecting loan samples for MERIT examinations.

  DSC has tested loan sampling programs in various sizes and types of institutions. The benefits and risks associated with utilizing statistical loan sampling are not limited to MERIT-eligible institutions, although our pilot testing of such programs has found them less effective than judgmental samples for banks with total assets of less than $100 million. DSC will continue to explore the use of statistical loan sampling techniques; however, MERIT-eligible institutions should neither be singled out nor excluded from any future testing

Recommendation:

2. Require examiners to justify variances on the Pre-Examination Planning Memorandum and/or in the Confidential-Supervisory Section page of the Report of Examination if the loan penetration levels fall below the established ranges for Category 1 and Category 2 MERIT examinations.

   We concur with this recommendation. We will provide clarification to examiners indicating that variances both above and below the recommended thresholds should be explained in the Pre-Examination Planning Memorandum and/or in the Confidential-Supervisory Section page of the Report of Examination, where appropriate. This clarification, via written memorandum, will be issued by the June 30, 2005.

   DSC notes that in the sampling of banks included in the Report, only two of the examinations had loan penetration levels below the MERIT-recommended ranges. The Report does not comment on whether the loan penetration levels in these two instances were addressed and justified in the pre-examination planning memoranda. Subsequent to the receipt of the draft report, we reviewed examination reports and other supervisory documentation for these two institutions, and the reasons for the LPRs used are clearly evident. In one of the two, the examiner documented that the loan portfolio consisted entirely of homogenous1-4 family residential mortgages and consumer loans, which were appropriately classified according to the Uniform Retail Credit Classification and Account Management Policy. In the second of the two examples cited in the draft report, the LPR of 11 percent was fully discussed in the report of examination. These two examples do not indicate any weakness or cause for concern.

   Additionally, DSC provides the following statistical summary containing a sample of examination data covering the past two years. The data in the attached table show that LPRs correlate most closely with asset quality and institution size. For example, in a large sample of MERIT examinations where asset quality was rated strong (1 or 2), the range for average MERIT LPRs was 23.17 percent to 27.00 percent, and 21.94 percent to 25.56 percent on an asset-weighted basis. Compare those figures with non-MERIT examinations with similarly high asset quality ratings, and we see that the range is 28.34 percent to 34.36 percent, and 19.86 percent to 22.88 percent on an asset-weighted average basis. Overall, where examination ratings are comparable for strong institutions, LPR variance is not significant between MERIT and non-MERIT guideline usage. In fact, on a weighted average basis non-MERIT guidelines account for lower overall LPRs. The attached graphs, which represent the LPRs plotted for MERIT sample statistics, continue to show that our emphasis on appropriate risk-focused procedures is stable, balanced, and effectively managed.

From a large representative sample of 4,298 Safety and Soundness Examinations completed primarily in 2003 and 2004, of which 1,961 utilized MERIT guidelines, the following chart displays average and weighted average Loan Penetration Ratios compared to Asset Quality Ratings. Scatter charts on the succeeding pages show the loan penetration ratios for the 1,961 MERIT guideline examinations.

	Search | Accessibility | Privacy | Information Quality | Contact Us | Site Map | Home