Federal Deposit Insurance Corporation
Office of Inspector General
Federal Deposit Insurance Corporation - Office of Inspector General

The FDIC's Risk Monitoring of Systemically Important Financial Institutions' Proximity and Speed to Default or Danger of Default

This is the accessible text file for FDIC OIG report number EVAL-17-003 entitled 'The FDIC’s Risk Monitoring of Systemically Important Financial Institutions’ Proximity and Speed to Default or Danger of Default' .

This text file was formatted by the FDIC OIG to be accessible to users with visual impairments.

We have maintained the structural and data integrity of the original printed product in this text file to the extent possbile. Accessibility features, such as descriptions of tables, footnotes, and the text of the Corporation’s comments, are provided but may not exactly duplicate the presentation or format of the printed version.

The portable document format (PDF) file also posted on our Web site is an exact electronic replica of the printed version.

Office of Audits and Evaluations Report No. EVAL-17-003

The FDIC’s Risk Monitoring of Systemically Important Financial Institutions’ Proximity and Speed to Default or Danger of Default

January 2017

Executive Summary

Why We Did The Evaluation

The FDIC is charged under the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 (Dodd-Frank Act) with responsibility for liquidating failing financial companies that pose a significant risk to the financial stability of the U.S. These financial companies are commonly known as systemically important financial institutions (SIFIs). This report presents the results of our evaluation of the FDIC’s progress in implementing systemic risk monitoring that would identify the financial companies’ risk of default so that the FDIC could undertake necessary preparatory actions for their resolution.

Our objective was to evaluate the FDIC’s progress in developing criteria and a process for assessing SIFIs’ proximity and speed to default or danger of default.

Background

To fulfill its responsibility, the FDIC’s Division of Risk Management Supervision-Complex Financial Institutions (RMS-CFI) has undertaken numerous initiatives, including risk monitoring of larger institutions for which FDIC is not the primary federal regulator (PFR). This monitoring includes understanding SIFIs’:

- structure, business activities, and resolution/recovery capabilities to inform FDIC resolution planning efforts; - business activities and risk profiles to gauge both proximity to a resolution event and the speed at which an institution’s condition could potentially deteriorate to a resolution event; and - recovery plans, early warning signals and triggers, escalation, and the range of FDIC remedial actions to be taken should a triggering event occur.

As of June 2016, RMS-CFI monitors 16 SIFIs in its financial institution portfolio with assets over $13 trillion.

Evaluation Results

We determined that the FDIC’s RMS-CFI has made steady progress in developing criteria and a process, namely the Systemic Monitoring System (SMS), for assessing the proximity and speed to default for the 16 large and complex SIFIs in RMS-CFI’s portfolio. The SMS gathers and analyzes SIFI supervisory reports and market information using standardized metrics that are then combined with RMS-CFI onsite Institution Monitoring teams’ (IM team) perspectives and analyses of the risks shown by those metrics. Ultimately, an RMS-CFI committee assesses the indicated risks from IM team submissions and other sources to assign a quarterly risk rating for each SIFI on its proximity and speed to default. As the proximity to default increases, the FDIC may take a number of actions, including increased monitoring and a resolution strategy refresh.

Our evaluation found that:

- the FDIC followed select SMS requirements and controls established in the Systemic Monitoring System – Description of the Framework and the Quarterly Process; - RMS-CFI plans to expand SMS to cover all SIFIs in its portfolio in 2017; - RMS-CFI should develop more detailed SMS tool documentation on its inputs and methodology for users and reviewers; and - RMS-CFI needs to independently evaluate the SMS tool’s output.

Recommendations and Corporation Comments

The report contains three recommendations addressed to the Director, RMS, to improve SMS documentation and to independently evaluate the SMS tool’s output. In a written response, dated December 30, 2016, the Director, RMS, concurred with the recommendations and provided planned corrective actions and targeted completion dates for each.

[End of Executive Summary]

Contents

Background

Evaluation Results

RMS-CFI Implemented the SMS to Assess Certain SIFIs’ Proximity and Speed to Default

RMS-CFI Followed Select SMS Requirements and Controls

RMS-CFI Plans to Expand the SMS to Cover Remaining SIFIs in Its Portfolio

RMS-CFI Should Develop More Detailed SMS Tool Documentation

RMS-CFI Needs to Independently Evaluate the SMS Tool’s Output

Other Matter: RMS-CFI and OCFI Operating Protocols

Corporation Comments and OIG Evaluation

Appendices 1. Objective, Scope, and Methodology 2. SIFIs and RMS-CFI Monitoring 3. RMS-CFI Business Lines and Their Interaction 4. Glossary of Terms 5. Abbreviations and Acronyms 6. Corporation Comments 7. Summary of the Corporation’s Corrective Actions

Tables 1. SIFIs Monitored by RMS-CFI, as of June 2016 2. Results of Testing Select SMS Requirements 3. Select Controls Observed in the SMS

Figures 1. RMS-CFI Sections Involved with SIFI Risk Monitoring 2. Process Map of the Quarterly SMS

[FDIC OIG letterhead, FDIC logo, Federal Deposit Insurance Corporation, Federal Deposit Insurance Corporation, Office of Inspector General 3501 Fairfax Drive, Arlington, VA 22226]

DATE: January 26, 2017

MEMORANDUM TO: Doreen R. Eberley, Division of Risk Management Supervision /Signed/

FROM: E. Marshall Gentry, Assistant Inspector General for Evaluations

SUBJECT: The FDIC’s Risk Monitoring of Systemically Important Financial Institutions’ Proximity and Speed to Default or Danger of Default (Report No. EVAL-17-003)

This report presents the results of our evaluation of the FDIC’s risk monitoring of systemically important financial institutions’ (SIFI) proximity and speed to default or danger of default.1 The FDIC is charged under the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010 (Dodd-Frank Act) with responsibility for liquidating failing financial companies that pose a significant risk to the financial stability of the United States.2 This report presents the results of our evaluation of the FDIC’s progress in implementing systemic risk monitoring that would identify the financial companies’ risk of default so that the FDIC could undertake necessary preparatory actions for their resolution.3

Footnote 1 The term SIFI is commonly used to refer to bank holding companies with $50 billion or more in total consolidated assets and nonbank financial co:mpanies designated by the Financial Stability Oversight Council for Federal Reserve supervision and enhanced prudential standards, but the Dodd-Frank Act does not use the term.

Footnote 2: The Dodd-Frank Act provides that the Secretary of the Treasury may appoint the FDIC as receiver following a statutorily prescribed appointment process, which includes the written recommendation of the Board of Governors of the Federal Reserve System and either the FDIC, the Securities and Exchange Commission or the Federal Insurance Office, as specified under the law. Following receipt of such recommendation, the Secretary, in consultation with the President of the United States, would then make certain statutorily required determinations regarding the company, including whether the company is in default or in danger of default. Upon such determinations, and subject to either the acquiescence or consent of the company’s board of directors or, if contested, subject to a limited and accelerated judicial review process, the FDIC would then be appointed receiver.

Footnote 3: Dodd-Frank Act §203(c)(4) defines default or danger of default as (1) a case has been, or likely will promptly be, commenced with respect to the financial company under the Bankruptcy Code, (2) the financial company has incurred, or is likely to incur, losses that will deplete all or substantially all of its capital, and there is no reasonable prospect for the company to avoid such depletion, (3) the assets of the financial company are, or are likely to be, less than its obligations to creditors and others, or (4) the financial company is, or is likely to be, unable to pay its obligations in the normal course of business.

Our objective was to evaluate the FDIC’s progress in developing criteria and a process for assessing systemically important financial institutions’ (SIFI) proximity and speed to default or danger of default. This report focuses on the 16 large and complex SIFIs monitored by the Division of Risk Management Supervision-Complex Financial Institutions (RMS-CFI). To address our objective, we reviewed the Systemic Monitoring System (SMS), a system used by RMS-CFI to assess risk and to provide risk assessment ratings on certain SIFIs’ proximity and speed to default.4 The SMS is one tool of many used by RMS to monitor SIFIs.

Footnote 4: The FDIC’s Division of Risk Management Supervision promotes stability and public confidence in the nation’s financial system through examining and supervising insured financial institutions, evaluating resolutions plans, and monitoring and mitigating systemic risks.

In particular, we reviewed SMS process documentation and risk assessments for three judgmentally selected SIFIs covered by the SMS. Our review of the risk assessments included testing of the SMS quantitative tool which “synthesizes data from numerous sources, evaluates the level and change in metrics that serve as important barometers of overall risk, produces a preliminary risk assessment, and identifies areas requiring further follow-up.”5 We interviewed RMS-CFI officials who conduct follow-up on site at the SIFIs on the results of the quantitative tool. We also interviewed members of the Risk Assessment Committee (RAC) who determine SIFI final risk ratings. Finally, we assessed compliance with select internal controls in the system, including documentary evidence supporting the risk assessments and reviews and approvals within the process.

Footnote 5: Systemic Monitoring System – Description of the Framework and the Quarterly Process, FDIC – Complex Financial Institutions, March 17, 2016.

We conducted this evaluation in accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for Inspection and Evaluation. Appendix 1 of this report includes additional details on our objective, scope, and methodology. Appendix 2 contains an overview of RMS-CFI’s SIFI monitoring activities. Appendix 3 contains a description of RMS-CFI’s business lines and their interactions. Appendix 4 contains a glossary of key terms,6 and Appendix 5 contains a list of acronyms.

Footnote 6: Terms that are underlined when first used in this report are defined in Appendix 4, Glossary of Terms.

Background

RMS-CFI monitors SIFIs through, for instance, its insured depository institution back-up examination authority,7 on-site monitoring, and the SMS, as detailed in Appendix 2. It does so because under Title II of the Dodd-Frank Act, the FDIC may be called upon to resolve failing SIFIs. To fulfill this responsibility, RMS-CFI has undertaken numerous initiatives to understand and evaluate SIFIs’:

Footnote 7: The Dodd-Frank Act §172 grants the FDIC back-up examination authority for systemic nonbank financial companies and bank holding companies if the FDIC Board determines examination is necessary to implement the FDIC’s authority to provide for orderly liquidation of the company. The Dodd-Frank Act also allows the FDIC to bring back-up enforcement actions against depository institution holding companies if the conduct, or threatened conduct, of a depository institution holding company poses a foreseeable material risk of loss to the FDIC’s Deposit Insurance Fund.

- structure, business activities, and resolution/recovery capabilities to inform FDIC resolution planning efforts;

- business activities and risk profile to gauge both proximity to a resolution event and the speed at which an institution’s condition could potentially deteriorate to a resolution event; and

- recovery plans, early warning signals and triggers, escalation, and the range of FDIC remedial actions to be taken should a triggering event occur.

As of June 2016, RMS-CFI monitored 16 SIFIs in its portfolio with assets over $13 trillion, as shown in Table 1.

Table 1: SIFIs Monitored by RMS-CFI, as of June 2016



Row 1 SIFI Name (Universal Banks)a: Bank of America Total Assets (USD billions): $2,186.6

Row 2 SIFI Name (Universal Banks)a: Citigroup Total Assets (USD billions): 1,818.8

Row 3 SIFI Name (Universal Banks)a: JP Morgan Chase Total Assets (USD billions): 2,466.1

Row 4 SIFI Name (Universal Banks)a: Wells Fargo Total Assets (USD billions): 1,889.2

Row 5 SIFI Name (Investment Banks)b: Goldman Sachs Total Assets (USD billions): 896.8

Row 6 SIFI Name (Investment Banks)b: Morgan Stanley Total Assets (USD billions): 828.9

Row 7 SIFI Name (Custody Banks)c: Bank of New York Mellon Total Assets (USD billions): 372.4

Row 8 SIFI Name (Custody Banks)c: Northern Trust Total Assets (USD billions): 121.5

Row 9 SIFI Name (Custody Banks)c: State Street Total Assets (USD billions): 255.4











Row 10 SIFI Name (Foreign Bank Operations)d: Barclays Total Assets (USD billions): 265.5

Row 11 SIFI Name (Foreign Bank Operations)d: Credit Suisse Total Assets (USD billions): 266.1

Row 12 SIFI Name (Foreign Bank Operations)d: Deutsche Bank Total Assets (USD billions): 353.2

Row 13 SIFI Name (Foreign Bank Operations)d: HSBC Total Assets (USD billions): 295.5

Row 14 SIFI Name (Foreign Bank Operations)d: UBS Total Assets (USD billions): 254.9

Row 16 SIFI Name (NonBanks)e: AIG Total Assets (USD billions): 510.3

Row 17 SIFI Name (NonBanks)e: Prudential Total Assets (USD billions): 796.5

Source: OIG-generated based on FDIC-provided data. Notes: a Banks that engage in commercial banking, investment banking, and other financial services. b Financial institutions that act as underwriters or agents that serve as intermediaries between issuers of securities and the investing public. c Insured depository institutions with previous calendar year-end trust assets of at least $50 billion, or those insured depository institutions that derived more than 50 percent of their revenue (interest income plus non-interest income) from trust activity over the previous calendar year. d Acquired or established (by a foreign financial institution) freestanding banks or bank holding companies in the U.S. These entities are regulated and supervised as domestic institutions. e Companies, other than banks or holding companies, that are incorporated or organized under the laws of the U.S. or any State, and that are predominantly engaged in financial activities. [End of Table]

RMS-CFI conducts its monitoring through sections responsible for on-site and off-site SIFI risk monitoring activities. Figure 1 below provides an overview of these sections and their activities.

Figure 1: RMS-CFI Sections Involved with SIFI Risk Monitoring

Risk Surveillance •Analyzes industry conditions •Identifies emerging risks and trends •Aggregates financial information •Measures proximity to a resolution event

Financial Products & Practices •Analyzes specific financial products and activities •Identifies excessive concentrations •Understands unique risks posed by new products or business strategies

Quantitative Modeling •Evaluates quantitative risk models and model risk management •Conducts special studies of economic, banking, and financial markets

Supervisory Program I & II •Directly supports examination activities •Ensures quality and consistency of risk evaluations, including SMS, supervisory strategies, and risk analytics •Informs resolution strategy development

Institution Monitoring •Monitors firm-specific financial condition •Performs risk monitoring of a firm's proximity to a resolution event and speed to default •Supports Dodd-Frank Act Title II orderly liquidation resolution authority

Source: RMS-CFI 2016 Business Plan. [End of Figure 1]

Of particular importance to this evaluation, RMS-CFI created the SMS to provide a quarterly independent assessment of supervisory reports and market information—by using its own methodology to assess SIFI risks. The SMS concludes with a risk rating on SIFI proximity to a resolution event (remote, low-moderate, moderate, or imminent) and the direction of the risk (increasing, decreasing, or stable). The Dodd-Frank Act §203(c)(4) in part defines default or danger of default, in terms of capital depletion and insolvency, and the SMS analyzes data that pertains to both proximity and speed to default for the purpose of monitoring the institution.

Evaluation Results

The FDIC’s RMS-CFI has made steady progress in developing criteria and a process, namely the SMS, for assessing SIFIs’ proximity and speed to default. This system, implemented for certain SIFIs in 2014, contains standardized metrics and triggers (criteria) to measure SIFI performance. These metrics are used to identify changes in SIFI liquidity, credit, capital, and market risk, among other risk factors, necessary to evaluate SIFI proximity and speed to default. The risks shown by these metrics are analyzed by RMS-CFI’s onsite Institution Monitoring teams (IM team) as well as by the RAC. Ultimately, the RAC assesses the identified risks from IM team submissions and other sources to assign a quarterly risk rating for each SIFI in RMS-CFI’s portfolio on its proximity and speed to default. As the proximity to default increases, the FDIC may take a number of actions, including increased monitoring and resolution strategy refresh.8 SMS has provided coverage for universal banks since 2014. SMS coverage has since expanded to include investment banks and custody banks, and RMS-CFI plans to begin using the system for foreign banking organizations (FBOs) in 2016 and nonbank financial companies in 2017.

Footnote 8: Many FDIC divisions are involved in preparations for a SIFI resolution, such as the Division of Resolutions and Receiverships, the Legal Division, the Division of Finance, the Office of Communications, OCFI, and the Office of Legislative Affairs. However, these preparations are beyond the scope of this evaluation.

Based on our testing, RMS-CFI is complying with SMS process requirements. Nevertheless, RMS-CFI can do more to enhance the SMS by providing risk monitoring coverage and risk ratings for the remaining SIFIs in the RMS-CFI portfolio as planned, improving system documentation, and independently evaluating the system’s output.

RMS-CFI Implemented the SMS to Assess Certain SIFIs’ Proximity and Speed to Default

In 2014, RMS-CFI implemented the SMS to augment its ongoing SIFI monitoring efforts by providing a quantitatively-driven approach for risk monitoring the SIFIs in its financial institution portfolio. The SMS monitors these SIFIs’ liquidity, credit, capital, market, securities, broker-dealer, earnings/capital, and custody risks to arrive at quarterly SIFI risk ratings on the firms’ proximity and speed to default. As illustrated in Figure 2, the SMS rating assignment process begins with the SMS tool which uses metrics for each of these risk areas. The data for metric value calculation comes from supervisory reports and market information. Each metric value is assigned trigger thresholds which, if exceeded, represent potential risk. These trigger thresholds are determined using historical data analysis and expert judgment. RMS-CFI performs additional analyses that ultimately produce a risk rating of 1 to 4 (low to high risk). Scores in the 3 to 4 range may require further investigation, the next step in the process.

The RAC reviews the results of the SMS tool and determines which metrics require further investigation by the on-site IM teams. The IM teams investigate these metrics using various information sources, such as the SIFI’s management information system and information obtained from meetings with SIFI management, and complete an SMS Ratings Determination Form. The form documents the work performed, conclusions, proposed supervisory action, and provides an overall SIFI rating recommendation to the RAC. These ratings conclude on SIFI proximity to a resolution event and the perceived direction of risk. The RAC reviews the Ratings Determination Forms, along with information from other sources, to determine a final risk rating (A for best, B, C, or D for worst) for each SIFI. The review identifies a SIFI’s proximity and speed to default as well as opines on supervisory and resolution-related actions the FDIC may consider. If the RAC concludes the risk is severe, a rating of a C or D, it will alert OCFI, which may take any number of actions, including developing a written recommendation for receivership.

In cases where there is a significant disagreement between the IM teams and the RAC on a firm’s overall final risk assessment rating, the Associate Director of the Risk Analytics Branch has final authority to override the RAC’s rating determination and assign a final risk assessment rating to the firm. The Deputy Director, RMS-CFI, has the final authority to assign final risk assessment ratings to firms.

In addition to providing the FDIC with a system that facilitates assessing proximity and speed to default, SMS provides staff with another tool to assess individual SIFI supervisory risk quarterly as well as across the firms and the change in risk over time. It also provides a forum for discussing risk within RMS-CFI, the SIFI, and the PFR, resulting in thorough comprehension of SIFI risks.

Figure 2: Process Map of the Quarterly SMS

Development and Distribution of SMS Reporting Packages (flow chart) (1) Regulatory Data Sources -OR- (1) Industry Data Source (2) SMS DG performs quantitative data enhancement and categorization (3) SMS DG evaluates the level and change in the quantitative data against metrics (4) SMS DG develops preliminary SMS reporting package (5) SMS DG review of preliminary SMS reporting package results (6) Quality review of preliminary SMS reporting package data (go back to step 2) (7) RAC initial review of preliminary SMS reporting packages to determine risk areas it wants reviewed. (8) CFI Management reviews preliminary SMS reporting packages (9) SMS DG to incorporate key metrics and scope comments from RAC and management feedback, if any, into the reporting packages (10) Final SMS Reporting Package posted to SharePoint

Review of Material Risks Identified by the SMS Reporting Packages/ Qualitative Factors Considered (10) Final SMS Reporting Package posted to SharePoint (11) IM teams utilize both quantitative and qualitative knowledge gathered through off-site monitoring systems (including follow-up on metrics designated as in scope by the RAC), firm MIS, ongoing onsite monitoring activities, as well as target examination participation to develop an independent assessment of potential proximity to default and speed to default. 12) IM teams communicate their assessment to the RAC utilizing the SMS Ratings Determination Form. When necessary, teams will develop recommendations for appropriate supervisory- and resolution-related responses (in consultation with the PFR and OCFI). (13) Final SMS Ratings Determination Form posted to SharePoint

Assignment of Final Overall Firm SMS Risk Assessment Rating (13) Final SMS Ratings Determination Form posted to SharePoint (14) RAC reviews finalized Ratings Determination Forms (15) RAC votes to assign overall risk rating (16) CFI Management reviews final ratings, and makes changes if necessary. (17) CFI Management coordinates next steps for supervisory- and-resolution-related actions (18) Supervisory and resolutionrelated actions occur as necessary

Source: OIG-generated based on FDIC documents and interviews. [End of Figure 2]

RMS-CFI Followed Select SMS Requirements and Controls

We assessed SMS using RMS-CFI’s internal guidance as well as the Government Accountability Office’s (GAO) Standards for Internal Control in the Federal Government (September 2014). RMS-CFI’s Systemic Monitoring System – Description of the Framework and the Quarterly Process (March 2014) provides the system overview, scoring methodology, quarterly process of assigning SIFI ratings, and the supervisory- and resolution-related responses to the SIFI ratings. The document also establishes the roles and responsibilities of RMS-CFI officials in the process, including points of review and approval, and documentary requirements, which constitute controls in the process.

The GAO’s Standards for Internal Control in the Federal Government describes internal control as the process effected by an entity’s oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. In this case, management’s objective in instituting SMS is to develop an independent risk assessment through the analysis of quantitative data generated by the SMS tool and qualitative analysis provided by RMS-CFI officials to quarterly assess SIFI proximity to default and speed to default. Our review focused on select compliance controls relevant to the SMS and to producing a quarterly risk rating. In particular, the Standards provide that management should establish an organizational structure, assign responsibility, and delegate authority to achieve the entity’s objective; management should design control activities to achieve objectives and respond to risks; and management should implement control activities through policies.

We found that RMS-CFI complied with its established SMS process requirements. Specifically, RMS-CFI ran SMS tool reports quarterly for our three judgmentally selected SIFIs for the two selected quarters and documented the reports. The RAC reviewed the reports to identify metrics for investigation by the IM teams and communicated those metrics to the teams. The IM teams completed their investigations and assigned a preliminary risk rating for each SIFI and documented the results of their assessment. The RAC assigned a final risk rating for each selected SIFI, documented its conclusion, and met quarterly deadline requirements. Table 4 summarizes the process requirements we tested.

Table 2: Results of Testing Select SMS Requirements

Row 1 Process Requirements: SMS preliminary risk report generated for Q4 2015 and Q1 2016 Financial Institution 1: cheeck Financial Institution 2: cheeck Financial Institution 3: cheeck

Row 2 Process Requirements: RAC review of preliminary report Financial Institution 1: cheeck Financial Institution 2: cheeck Financial Institution 3: cheeck

Row 3 Process Requirements: IM team analysis of RAC-identified metrics and completion of Ratings Determination Form Financial Institution 1: cheeck Financial Institution 2: cheeck Financial Institution 3: cheeck

Row 4 Process Requirements: RAC assessment of information from all sources and completion of final risk assessment Financial Institution 1: cheeck Financial Institution 2: cheeck Financial Institution 3: cheeck

Source: OIG-generated from its review of RMS-CFI documentary evidence. [End of Table 2]

Furthermore, we found that RMS–CFI complied with select SMS process controls, as described in Table 3 below, and took appropriate action consistent with SMS guidance for the derived risk rating. We also concluded that these controls provided reasonable assurance of compliance with the SMS. Nevertheless, we found that the SMS tool documentation needs further detail to support the metric value calculation, data sources, and triggers, an issue which is discussed later in the report.

Table 3: Select Controls Observed in the SMS

Row 1 GAO Internal Control Principle: Principle 3 – Assignment of Responsibility and Delegation of Authority Description of Control: Established roles and responsibilities for implementing the SMS Observed: Check

Row 2 GAO Internal Control Principle: Principle 10 – Control Activities Description of Control: System of oversight and review and approval of SMS tool output, IM team investigation and risk assessment, and RAC risk assessment Observed: check

Row 3 GAO Internal Control Principle: Principle 10 – Control Activities Description of Control: Documented SMS tool transactions, including input, metrics, triggers, and output Observed: check

Row4 GAO Internal Control Principle: Principle 10 – Control Activities Description of Control: Documented risk assessment conclusions Observed: check

Row 5 GAO Internal Control Principle: Principle 10 – Control Activities Description of Control: SMS tool documentation (Appropriate documentation of transactions and controls over information processing) Observed: No

Row 6 GAO Internal Control Principle: Principle 12 – Documentation of Responsibilities through Policies Description of Control: Established policies and procedures for the SMS Observed: Check

Source: OIG-generated from its review of RMS-CFI documentary evidence. [End of Table 3]

RMS-CFI Plans to Expand the SMS to Cover Remaining SIFIs in Its Portfolio

SMS covered 9 of the 16 SIFIs in RMS-CFI’s portfolio, including universal banks, investment banks, and custody banks, as of July 31, 2016. RMS-CFI has plans in place for 2016 and 2017 to add the remaining seven SIFIs, the FBOs and non-bank firms, so that full coverage of its portfolio is provided through SMS. Full coverage will help RMS-CFI in fulfilling its role in assessing a SIFI’s performance and its proximity and speed to default.

Roll-out of SMS has been planned and conducted over several years because of the complexity and breadth of the program, including differences in SIFI operations, challenges in identifying appropriate metrics and sources of information, or obtaining necessary information. For example, a Federal Reserve Board regulation implemented enhanced prudential standards for certain companies, including FBOs. This regulation required FBOs with $50 billion or more in total U.S. non-branch assets as of June 30, 2015, to establish a U.S. intermediate holding company and transfer ownership interest in the substantial majority of its U.S. subsidiaries to the U.S. intermediate holding company by July 1, 2016. The intermediate holding company must begin complying with applicable regulatory reporting requirements as of September 30, 2016. This information will be then available for use by the SMS.

The FDIC is monitoring these SIFIs’ risk through other means, including the IM teams assigned to the FBOs and non-bank firms, to assess risks and has in place FBO and nonbank monitoring program frameworks. We are not making a recommendation at this time as RMS-CFI has plans in place for the SMS’s expansion to provide coverage for all institutions in its portfolio.

RMS-CFI Should Develop More Detailed SMS Tool Documentation

The current SMS Data Dictionary: Ratios/Metrics Included in SMS lists the metrics by type and title, and provides a description of the metric. However, the description did not always provide enough detail on the source of the metric or, if the source was identified, it did not provide enough detail to locate the data within the source from which it was obtained. The data dictionary also did not typically describe the rationale for the inclusion of the metric. In addition, while the quantitative triggers used to evaluate the metric values are documented in SMS, the rationale for the trigger values and the methodology for their calculation are not documented.

The GAO’s Standards for Internal Control in the Federal Government provides for appropriate documentation of transactions and controls over information processing. These include policies, manuals, and data checks that support the integrity of the information used by management for decision-making purposes.

RMS-CFI’s focus has been on designing and implementing the SMS. As an interim measure, the current abbreviated data dictionary was developed to provide a general understanding of the metrics. With SMS maturing, RMS-CFI has recognized the need for a detailed data dictionary and has undertaken efforts to provide more detail in the forthcoming version.

Without a detailed data dictionary, not all metric values can be recalculated nor can all individual components making up the metric values be identified. We judgmentally selected 60 metrics for testing purposes, and although we were able to recalculate some of them, many could not be recalculated. Also, we could not recalculate trigger values because the methodology used for determining them is not documented. In our interviews, IM team members did not always understand how certain metrics were calculated, and some stated that a further understanding of the metrics would be beneficial in their analysis of SIFIs and in their discussion with SIFI management. A clearer understanding of metric values and triggers could:

- increase participant satisfaction and buy-in of SMS use; - benefit RMS-CFI succession planning and knowledge management; and - support quality assurance review by providing necessary information to reviewers.

Recommendations. We recommend that the Director, RMS: (1) Update the Systemic Monitoring System data dictionary and provide rationales for the metrics and sufficient descriptive detail to permit recalculation of metric values. (2) Document the rationale and methodology for determining metric trigger values.

RMS-CFI Needs to Independently Evaluate the SMS Tool’s Output

We found that RMS-CFI has not independently evaluated the SMS tool’s output though it has done its own reviews of the tool. The GAO’s Standards for Internal Control in the Federal Government provides that management should establish and operate monitoring activities and evaluate the results. Further, separate evaluations provide objectivity when performed by reviewers who do not have responsibility for the activities being evaluated.

Our review found that RMS-CFI did not evaluate the tool because it is still under development. Also, because the SMS tool is not viewed as a “Model,” it did not fall under FDIC Circular 1170.1, FDIC Model Risk Management Policy, Office of Corporate Risk Management, October 1, 2014, which provides that “Divisions and Offices are responsible for developing, using, validating, and managing the Models used or created in their Divisional and Office operations” and “will submit a written Validation Program Summary document plan for regular evaluation, monitoring, and analysis.”

Nevertheless, RMS-CFI completed an SMS Tool Validation Program Summary, dated January 8, 2016, that planned for a code review in 2016 and the Chief Risk Officer approved the completed form. According to the Tool Validation Program Summary, an analyst in RMS-CFI would check program logic for errors and ensure that associated documentation is accurate and readily understood. Evaluation of the SMS tool would provide RMS-CFI assurance when using the tool outputs to guide its investigations and help ensure an accurate depiction of risks with SIFIs.

Recommendation. We recommend that the Director, RMS:

(3) Independently evaluate the Systemic Monitoring System tool to verify the accuracy of the tool’s output.

Other Matter: RMS-CFI and OCFI Operating Protocols

In a draft of this report, we noted differences in the RMS-CFI and OCFI Operating Protocols – Business Activities Collaboration document located on the FDIC’s internal website and actual information sharing practices. We recommended that RMS-CFI and OCFI coordinate and discuss the sharing of information necessary for achieving the organizations’ objectives and revise the operating protocols document accordingly. In response to the draft report, RMS-CFI clarified that the operating protocols document was intended for the 2013 transition of OCFI risk monitoring staff to RMS and was no longer in effect. Thus, it would not make sense to revise the document. RMS-CFI provided examples of reports and other means through which RMS-CFI and OCFI share information. We removed the recommendation from our final report, but we encourage RMS-CFI and OCFI to continue to look for opportunities to share information efficiently and effectively.

Corporation Comments and OIG Evaluation

The Director, RMS-CFI, provided a response, dated December 30, 2016, to a draft of this report. The response is presented in its entirety in Appendix 6. The Director concurred with the three recommendations and provided planned corrective actions and targeted completion dates for each from January 31 through June 30, 2017. These recommendations will remain open until planned actions are completed. A summary of the Corporation’s corrective actions is presented in Appendix 7.

Appendix 1

OBJECTIVE, SCOPE, AND METHODOLOGY

Objective

Our objective was to evaluate the FDIC’s progress in developing criteria and a process for assessing systemically important financial institutions’ proximity and speed to default.

We performed our evaluation from May through September 2016 in accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for Inspection and Evaluation.

Scope and Methodology

The scope of this evaluation included RMS-CFI’s SMS for assessing proximity and speed to default for four universal banks, two investment banks, and three custody banks. Our review included compliance testing with the SMS for three judgmentally-selected institutions (one universal bank, one investment bank, and one custody bank) and analysis of metric values, triggers, and risk-scoping scores used by RMS-CFI to assess changes in SIFI performance.9

Footnote 9: The results of non-statistical samples cannot be projected to the intended population by standard statistical methods.

We performed our work at the FDIC’s offices in Arlington, Virginia; Washington, D.C.; and the New York Regional Office. To address our objective, we obtained an understanding of the process by interviewing officials in RMS-CFI’s Risk Analytics and Complex Financial Institutions Branches, including officials in the Risk Surveillance, Financial Practices and Products, Supervisory Program, and Institution Monitoring (IM) Sections. For each of the selected institutions, we:

- Judgmentally selected 10 metrics (from the 3 institutions) from the fourth quarter 2015 and from the first quarter 2016 for a total of 60 metrics. The selected metrics consisted of those that had been “flagged” by the SMS tool as exceeding their assigned trigger values.

- Recalculated the metric values using the Risk Analytics Branch data dictionary and the risk scoping scores using Systemic Monitoring System – Description of the Framework and Quarterly Process, 3/17/2016.

- Interviewed judgmentally-selected IM managers, team leaders, and team members for our sample institutions to obtain their input on the SMS program, and in particular with the SMS tool.

- Interviewed judgmentally-selected RAC members to obtain their input on the SMS process, and in particular with assigning a final risk assessment rating on SIFI proximity and speed to default.

- Traced RAC-identified metrics for IM team investigation to IM team response to the committee.

- Verified that documentation required by the Systemic Monitoring System – Description of the Framework and Quarterly Process was completed.

- We did not validate SIFI risk-rating determinations made by the IM teams or RAC or the performance of the SMS to assess speed and proximity to default.

- Consistent with the stated objective, we did not assess the RMS-CFI’s overall internal control or management control structure beyond what we include in this report. We obtained data from the SMS tool which obtains data from non-FDIC information systems; however, we did not assess the effectiveness of information system controls

Appendix 2

SIFIs AND RMS-CFI MONITORING

[Table]

Row 1 Universal Banks - Monitoring - RMS-CFI Monitored SIFIs: Citigroup Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 2 Universal Banks - Monitoring - RMS-CFI Monitored SIFIs: JP Morgan Chase Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 3 Universal Banks - Monitoring - RMS-CFI Monitored SIFIs: Bank of America Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 4 Universal Banks - Monitoring - RMS-CFI Monitored SIFIs: Wells Fargo Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 5 Investment Banks - Monitoring - RMS-CFI Monitored SIFIs: Goldman Sachs Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 6 Investment Banks - Monitoring - RMS-CFI Monitored SIFIs: Morgan Stanley Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 7 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Bank of New York Mellon Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 8 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Northern Trust Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 9 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: State Street Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: check

Row 10 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Barclays Monitoring - IDI Back-Up Supervisiona: Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 11 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Credit Suisse Monitoring - IDI Back-Up Supervisiona: Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 12 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Deutsche Bank Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 13 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: HSBC Monitoring - IDI Back-Up Supervisiona: check Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 14 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: UBS Monitoring - IDI Back-Up Supervisiona: Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 15 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: AIG Monitoring - IDI Back-Up Supervisiona: Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No

Row 16 Custody Banks - Monitoring - RMS-CFI Monitored SIFIs: Prudential Monitoring - IDI Back-Up Supervisiona: Monitoring - On-site SIFI Monitoringb: check Monitoring - SMS Monitoringc: No



Source: OIG-generated. Notes: a Insured Depository Institution (IDI) back-up supervision helps the FDIC to develop an independent FDIC risk assessment, including the appropriateness of assigned IDI ratings; ensure a reduction of unreasonable risk; ensure that the FDIC deposit insurance pricing is appropriate; and understand the IDI and affiliate relationships and the effect of such relationships on the IDI. b IM teams perform on-site risk monitoring to include bank and non-bank firms to understand structure, business activities, and resolution/recovery capabilities to inform FDIC resolution planning efforts. c SMS provides ongoing risk monitoring of systemically important bank holding companies and non-bank financial companies to assist in early warning of proximity and speed to default for various types of financial institutions. d RMS-CFI plans to add FBOs to SMS in 2016 and non-bank financial institutions in 2017 [End of Table]

Appendix 3

RMS-CFI BUSINESS LINES AND THEIR INTERACTION

[Table]

Too Much Risk Could Trigger a Potential Corrective Action Event

Too Much Risk Could Trigger a Potential Resolution Event

Row 1 Business Line: Statute IDI Back-up Supervision 11 SIFIs: - FDI Act Sec. 10(b)(3) – Special examination of any IDI - FDI Act Sec. 10(b)(4) – Examination of affiliates of depository institutions SIFI Risk Monitoring All SIFIs: - FDI Act Sec. 10(b)(3) – Special Exam for Title II - DFA Sec. 172(b) – BHC or NBFC Enforcement - DFA Sec. 203(c) – Default or Danger of Default Resolution Planning Support All SIFIs: - DFA 165(d) – Living Will - Joint FDIC-FRB Resolution Plan Rule - Section 10(d) IDI Resolution Plan Rule

Row 2 Business Line: Purpose IDI Back-up Supervision 11 SIFIs: - Independently assess IDI’s ratings - Ensure appropriate corrective actions - Ensure insurance pricing is appropriate - Understand IDI and affiliated relationships SIFI Risk Monitoring All SIFIs: Understand bank and nonbank firms: - Structure and activities for Title II planning - Resolution/recovery capabilities - Speed and proximity to a resolution event Resolution Planning Support All SIFIs: - Develop resolution plan assessment framework - Inform Title II resolution strategies - Inform and support potential resolution of IDIs

Row 3 Business Line: Activities IDI Back-up Supervision 11 SIFIs: - Large IDI program analysis - Insurance pricing support - Exercise exam and enforcement authorities - Work with other PFRs on individual and horizontal reviews - Support individual institution monitoring SIFI Risk Monitoring All SIFIs: - Monitor firm‐specific risks and financial condition - Assess proximity and speed to a resolution event - Participate in CCAR and CLAR - Analyze industry conditions and trends - Support policy considerations (FSOC, Basel, etc.) Resolution Planning Support All SIFIs: - Participate in the OCFI‐led reviews of 165(d) and IDI plans - Facilitate information collection - Verify and validate key plan assertions

Source: OIG-generated. [End of Table]

Appendix 4

Glossary of Terms

Dodd-Frank Act (DFA) - Legislation signed into law on July 21, 2010, that promotes the financial stability of the United States by improving accountability and transparency in the financial system, to end ‘‘too big to fail,’’ to protect the American taxpayer by ending bailouts, to protect consumers from abusive financial services practices, and for other purposes.

Failing - The closing of a financial institution by its chartering authority, which rescinds the institution’s charter and revokes its ability to conduct business because the institution is insolvent, critically undercapitalized, or unable to meet deposit outflows.

Primary Federal Regulator (PFR) - The state or federal agency with principal supervisory responsibility for a financial institution.

Receivership - The legal procedure for winding down the affairs of an insolvent institution.

Risk- Exposure to uncertain change.

Risk Assessment - Generally, the identification and quantification of risk types, levels, and locations in a process or organizational unit.

Systemic Risk - Risk associated with the general health or structure of the financial system which would have serious adverse effects on economic conditions or financial stability.

Appendix 5

Abbreviations and Acronyms

BHC - Bank Holding Company CCAR - Comprehensive Capital Analysis and Review CLAR - Comprehensive Liquidity and Asset Review DG - Development Group FBO - Foreign Banking Organization FDI Act - Federal Deposit Insurance Act FRB - Federal Reserve Board FSOC - Financial Stability Oversight Council GAO - Government Accountability Office IDI - Insured Depository Institution IM team - Institution Monitoring team MIS - Management Information System NBFC - Non-banking Finance Company OCFI - Office of Complex Financial Institutions OIG - Office of Inspector General PFR - Primary Federal Regulator RAC - Risk Assessment Committee RMS-CFI - Division of Risk Management Supervision-Complex Financial Institutions SIFI - Systemically Important Financial Institution SMS - Systemic Monitoring System

Appendix 6

Corporation Comments

[FDIC letterhead, FDIC logo, Federal Deposit Insurance Corporation, Division of Risk Management Supervision, 550 17th Street NW, Washington, D.C. 20429-9990]

DATE: December 30, 2016

MEMORANDUM TO: E. Marshall Gentry Assistant Inspector General for Evaluations

FROM: Doreen R. Eberley Director, Division of Risk Management Supervision

SUBJECT: Management Response to Draft Evaluation Report Entitled, The FDIC’s Risk Monitoring of Systemically Important Financial Institutions’ Proximity and Speed to Default or Danger of Default (Assignment No. 2016-028)

The Division of Risk Management Supervision (RMS) has reviewed the Office of Inspector General’s (OIG) draft evaluation report entitled, The FDIC’s Risk Monitoring of Systemically Important Financial Institutions’ Proximity and Speed to Default or Danger of Default (Assignment No. 2016-028) dated December 9, 2016. We appreciate the OIG’s evaluation and recognition of the progress made by the RMS Complex Financial Institutions (RMS CFI) group to develop a Systemic Monitoring System (SMS) tool for assessing the proximity and speed to default of systemically important financial institutions (SIFIs). We have reviewed and concur with the OIG’s recommendations. Below, we outlined our plan for addressing each of the three recommendations to include target completion dates.

Recommendation 1: Update the Systemic Monitoring System data dictionary and provide rationales for the metrics and sufficient descriptive detail to permit recalculation of metric values.

Management Response: RMS concurs with this recommendation.

Corrective Action: As noted in the OIG Report, RMS-CFI has recognized the need for a detailed data dictionary and has undertaken efforts to provide more detail in the forthcoming version. RMS CFI will enhance the existing data dictionary with specific variable names and the exact formulas used to calculate the metrics so that an outside analyst would be able to replicate the values using underlying data sources. The existing metric descriptions contained in the data dictionary will be augmented to ensure an outside analyst will understand the risk being described by the metric and the reason for the metric’s inclusion in SMS.

Targeted Completion Date: March 31, 2017.

Recommendation 2: Document the rationale and methodology for determining metric trigger values.

Management Response: RMS concurs with this recommendation.

Corrective Action: Within SMS, metric trigger values are set using one of two approaches: judgment-based or algorithmic. The triggers for “standard” metrics are set using analyst judgment taking into account the historical time series available for a given metric and judgment-based thresholds for identifying risk escalation. The triggers for “exception-based” metrics are generally set using an algorithm as the number of metrics considered are too numerous to evaluate separately. RMS CFI will develop a document that describes the logic underlying judgment-based thresholds and a description of the algorithms underlying exception-based metrics.

Target Completion Date: June 30, 2017.

Recommendation 3: Independently evaluate the Systemic Monitoring System tool to verify the accuracy of the tool’s output.

Management Response: RMS concurs with this recommendation.

Corrective Action: As noted in the OIG Report, RMS CFI included a thorough review of the SMS code as part of its Validation Program Summary as approved by the FDIC’s Chief Risk Officer on April 29, 2016. The intent of this code review is to ensure that SMS tool outputs accurately depict risk levels conveyed through SMS risk metrics.

A staff member of RMS CFI who was not involved in the SMS code development will perform an independent review of the code to ensure that output can be replicated from and matches to an independent calculation of source data. The code will be moved to the Team Foundation Server, which is a secure repository that will facilitate ongoing regular reviews of further code development.

Target Completion Date: January 31, 2017.

Appendix 7

Summary of the Corporation’s Corrective Actions

This table presents corrective actions taken or planned by the Corporation in response to the recommendations in the report and the status of the recommendations as of the date of report issuance.

[Table]

Row 1 Rec. No.: 1 Corrective Action: Taken or Planned: RMS-CFI plans to provide more detail in the SMS data dictionary. Expected Completion Date: March 31, 2017 Monetary Benefits: $0 Resolved:a Yes or No: Yes Open or Closedb: Open

Row 2 Rec. No.: 2 Corrective Action: Taken or Planned: RMS-CFI plans to develop a document to describe the setting of the SMS trigger values. Expected Completion Date: June 30, 2017 Monetary Benefits: $0 Resolved:a Yes or No: Yes Open or Closedb: Open

Row 3 Rec. No.: 3 Corrective Action: Taken or Planned: RMS-CFI plans to conduct an independent SMS code review Expected Completion Date: January 31, 2017 Monetary Benefits: $0 Resolved:a Yes or No: Yes Open or Closedb: Open

a Resolved – (1) Management concurs with the recommendation, and the planned, ongoing, and completed corrective action is consistent with the recommendation. (2) Management does not concur with the recommendation, but alternative action meets the intent of the recommendation. (3) Management agrees to the OIG monetary benefits, or a different amount, or no ($0) amount. Monetary benefits are considered resolved as long as management provides the amount. b Recommendations will be closed when (a) Corporate Management Control notifies the OIG that corrective actions are complete or (b) in the case of recommendations that the OIG determines to be particularly significant, when the OIG confirms that corrective actions have been completed and are responsive. [End of Table]

[End of Appendix 7]

[End of Report]

Print Print
Close