Dear Mr. Gianni:
We have reviewed the system of quality control for the audit function of the Federal Deposit Insurance Corporation (FDIC) Office of Inspector General (OIG) in effect for the year ended March 31, 2004. We conducted our review in conformity with standards and guidelines established by the President's Council on Integrity and Efficiency (PCIE). We tested compliance with the FDIC OIG's system of quality control to the extent we considered appropriate. These tests included a review of the audits identified in the Enclosure.
In performing our review, we considered the Policy Statement on Quality Control and External Reviews, dated February 2002, issued by the PCIE. According to that Statement, an OIG's quality control policies and procedures should be appropriately comprehensive and suitably designed to provide reasonable assurance that the objectives of quality control will be met. The Statement also recognizes that the nature, extent and formality of an OIG's system of quality control depends on various factors such as the size of the OIG, the location of its offices, the nature of the work, and its organizational structure.
In our opinion, the system of quality control for the audit function of the FDIC OIG in effect for the year ended March 31, 2004, has been designed in accordance with the quality standards established by the PCIE and was being complied with for the year then ended to provide the OIG with reasonable assurance of material compliance with professional auditing standards in the conduct of its audits. Therefore, we are issuing an unqualified opinion on your system of audit quality control.
We have identified, in a separate Letter of Comments dated August 31, 2004, other matters that came to our attention which do not affect our overall opinion.
Scope and Methodology
We conducted the Peer Review in accordance with the PCIE Guide for Conducting External Quality Control Reviews of the Audit Operations of Offices of Inspector General, issued during February 2002, and subsequently revised through March 2004.
We reviewed the last peer review report on the FDIC OIG audit function, issued by the U.S. Agency for International Development on January 25, 2002, and the related peer review working papers. To obtain an understanding of the audit operation and the internal quality control system, we reviewed the FDIC OIG’s audit policies and procedures. We considered controls such as those in place to ensure that only qualified staff is hired, that continuing professional education requirements are met, and that independence is maintained. To obtain an understanding of the FDIC OIG’s internal quality assurance program, we reviewed related policies and procedures, interviewed responsible staff, and reviewed internal quality assurance reports.
We tested compliance with the FDIC OIG’s system of quality control to the extent that we considered appropriate. These tests included a review of 6 of 36 audit reports issued between April 1, 2003, and March 31, 2004. We did not review the financial statement audit and monitoring activities covering the FY 2003 financial statements for the FDIC OIG since that audit is performed by the U.S. General Accounting Office.
The report numbers and titles of the selected audits are listed in the table below.
Dear Mr. Gianni:
We have reviewed the system of quality control for the audit function of Federal Deposit Insurance Corporation Office of Inspector General (FDIC OIG) in effect for the year ended March 31, 2004, and have issued our report thereon dated August 31, 2004, in which we rendered an unqualified opinion on the FDIC OIG system of quality control for its audit function. This letter should be read in conjunction with that report.
Our review was for the purpose of reporting whether the OIG’s internal quality control system was designed in accordance with the quality standards established by the President’s Council on Integrity and Efficiency (PCIE) and was being complied with that system for the year reviewed to provide reasonable assurance of material compliance with professional auditing standards in the conduct of its audits. We conducted our review in conformity with standards and guidelines established by the PCIE. Our review would not necessarily disclose all weaknesses in the system or all instances of noncompliance with it because our review was based on selective tests.
There are inherent limitations that should be recognized in considering the potential effectiveness of any system of quality control. In the performance of most control procedures, departures can result from misunderstanding of instructions, mistakes of judgment, carelessness, or other personal factors. Projection of any evaluation of a system of quality control to future periods is subject to the risk that one or more procedures may become inadequate because of changes in conditions or that the degree of compliance with procedures may deteriorate.
As a result of our review, we identified reportable conditions, which we considered in determining our opinion set forth in our report dated August 31, 2004. A reportable condition for peer review purposes represents a significant deficiency in the design or operation of the reviewed organization’s internal control that could adversely affect the organization’s ability to comply with applicable auditing standards and established auditing policies and procedures.
The two reportable conditions that we identified are discussed in the following paragraphs. During our review, we provided additional information about these conditions, including a listing of the applicable audits, to your office.
We believe that the conditions addressed in this section represent opportunities for improvement of the FDIC OIG’s system of quality control. While we did not identify a material effect to the reported audit findings and conclusions resulting from the conditions discussed below, we believe that the potential exists for these conditions to adversely affect audit operations.
It should be noted that the FDIC OIG's internal quality assurance program identified some of the same issues that we raise as a result of our peer review. We were advised that corrective actions had been taken, as detailed in the recent concurrence by the Director of Supervision and Insurance Audits to the internal quality assurance report dated April 20, 2004.
In performing our work, we considered the results of the prior peer review of the FDIC OIG, for which the U.S. Agency for International Development issued a report dated January 25, 2002. We found that the nine open recommendations contained in the Letter of Comments from that peer review have been addressed.
Finding 1. Quality Control – Review and Resolution of Referencing Points
The FDIC OIG considers referencing to be one of the key aspects of its internal control system, and an essential step in ensuring overall audit report quality. Policies and procedures have been established for indexing reports to supporting assignment documentation and the subsequent verification of the supporting evidence by an independent referencer as a control to assure compliance with the Government Auditing Standards requirement for reporting accuracy.
The FDIC OIG Audit Manual 330.2 – Indexing and Referencing of Reports requires the Auditor-in-Charge to respond to points raised by the independent referencer on the Referencing Point Sheet with comments as to the action taken to indicate whether changes were made in response to issues/questions identified. Any remaining open points should be submitted to the Director for review and possible resolution. After all the points have been resolved, the Auditor-in-Charge, independent referencer, and Director are required to sign the Referencing Point Sheet. If the comments cannot be resolved between the referencer, Auditor-in-Charge, and Director, the comments should be elevated to the Deputy Assistant Inspector General for Audits (DAIGA) for resolution. Only the DAIGA can pass on points identified by the referencer. In addition to the Referencing Point Sheet, the Audit Manual also requires completion of the Referencing Checklist to ensure that referencing requirements have been performed. This checklist has separate sections for the Director, Auditor-in-Charge, and referencer to complete and sign certifying that all referencer’s comments have been adequately resolved. The Director has overall responsibility for ensuring that reports are fully indexed to supporting documentation and independently referenced.
We noted that four of the six audits reviewed did not have the Auditor-in-Charge’s response or actions taken to resolve all of the points raised by the referencer. In addition, one audit had no signature on the Referencing Checklist to indicate final approval by the Director. FDIC OIG management explained that there were other certification forms signed by the Director, Auditor-in-Charge and referencer that are designed to provide assurance on the quality of the report. These certification forms are meant as an additional compensating control. However, we feel if the required detailed documentation of the process they are intended to certify is incomplete, the additional assurance that outstanding referencer points are addressed is lacking.
Recommendation. FDIC OIG management should reemphasize its policy on referencing and completion of the Referencing Point Sheet and Referencing Checklist. Specific emphasis should be placed on existing requirements for the Auditor-in-Charge to document all responses and/or actions taken to resolve the points raised by the referencer and the Director to sign off certifying that all referencing points have been adequately resolved.
Views of Responsible Officials. FDIC OIG management concurred and plans to issue a staff advisory by October 22, 2004, reemphasizing the policy on referencing, including completion of the Referencing Point Sheet and the Referencing Checklist. On August 10, 2004, the Referencing Point Sheet was enhanced to include an additional column for the referncer to initial his/her agreement to the Auditor’s individual responses/actions; and an "All Cleared" box is now included on the form for the referencer’s initials and date satisfying such action has occurred. In addition, training to cover the independent referencing process is planned for November 2004.
Finding 2. Supervisory Review and Documentation
The second field work standard for performance audits under Government Auditing Standards is staff is to be properly supervised. Supervision involves directing the efforts of auditors and others that are involved in the audit to determine whether the audit objectives are being accomplished. The FDIC OIG Audit Manual expresses the view that supervision is one of the most important aspects of ensuring assignment quality and that these reviews of assigned work must be documented and maintained in the assignment documentation. By conducting reviews, supervisors can also satisfy themselves that the staff clearly understands what work is expected to be performed, why the work is being conducted and what the work is expected to accomplish. There should be documented evidence of supervisory reviews in the assignment documentation. To be beneficial, the supervisory involvement must start early in the audit and continue in a timely manner to ensure that the efforts are redirected, when appropriate, and that all of the necessary work is performed. In our review, three issues surfaced involving documentation to support that all assigned work was performed and that the staff was being provided with timely supervision.
The FDIC OIG Audit Manual 320.6 – Preparation and Review of Assignment Documentation requires supervisory review of assignment documentation be documented and retained. The Auditor-in-Charge is responsible for monitoring and oversight of assigned staff and for reviewing all assignment documentation they prepare. The Office of Audits Director is responsible for reviewing all assignment documentation prepared by the auditor-in-charge. These supervisory reviews must be made periodically to ensure that the work is progressing satisfactorily and supports the reported findings, opinions, conclusions and recommendations.
Our review noted that not all work papers had the required documentation as evidence of supervisory reviews. We found assignment documentation that was (i) not signed off by the preparer and reviewer; (ii) dated as being reviewed before the date they were prepared; (iii) not indexed and cross-indexed or was incorrectly indexed; and, (iv) prepared and reviewed by the same person. We also found open reviewer notes, signatures missing from checklists, and incomplete checklists. Since the previous peer review, the FDIC OIG implemented an electronic work paper software program to document audits. Most of the lacking documentation of timely supervisory reviews can be attributed to the recent implementation of TeamMate and the fact that supervisory reviews are required each time a work paper is modified.
We did find evidence of adequate levels of supervision from planning to audit report issuance on key control documents. However, the missing sign-off dates and checklists indicate that documentation related to supervision needs improvement to ensure evidence that supervisory reviews were performed.
The FDIC OIG Audit Manual 320.3 – Assignment Programs states that the assignment program represents a contract between the Office of Audits Director and the team concerning the work to be performed. The purpose of assignment programs is to provide early guidance and an understanding between a supervisor and staff on the detailed procedures and techniques for collecting and analyzing sufficient, competent, and relevant evidence to address the assignment objectives within the approved milestones. The Manual requires all assignment program steps be indexed to the supporting documentation. Any step not performed should be annotated to provide a brief explanation to why the step was not performed. The Manual also states that if major changes in the scope occur as the assignment proceeds, the program should be modified and the Office of Audits Director should approve any major changes.
For two of the six audits reviewed, we found some assignment program steps that were not indexed to supporting documentation. Even though workpaper indexing on the approved assignment program was absent, we noted that the program steps were sufficiently performed to answer the audit objectives. However, by not indexing the workpapers to the assignment program, steps appeared to be omitted from the audit and there is an increased risk that all the required work needed to support coverage of the audit objective was not completed during the assignment.
The FDIC OIG Audit Manual 320.6 – Preparation and Review of Assignment Documentation requires that assignment documentation must contain the purpose for preparing the documentation, the source of the information, the scope of the review, applicable criteria, sufficient explanation of analytical methods and formulas used, the results of the review, and the assignment team’s conclusions.
We found workpapers that did not always contain the required elements for the six audits reviewed. However, none of these examples affected the accuracy of the final audit reports. Audit documentation (1) provides the principal support for the auditor's report; (2) aids auditors in conducting and supervising the audit; and, (3) allows for the independent review of audit quality. The preparation of audit documentation should be appropriately detailed to provide a clear understanding of its purpose, source and the conclusions the auditors reached. It should also be organized to provide a clear link to the findings, conclusions, and recommendations contained in the audit report.
Recommendation. – FDIC OIG management should reemphasize its existing policy (1) to index supporting documentation to completed steps in the audit program; (2) that requires proper documentation to show evidence of supervision in all workpaper files; and (3) to include all four elements on each workpaper or include a reference to where the four elements can be found elsewhere in the workpaper files.
Views of Responsible Officials. FDIC OIG management concurred and stated after the audits subject to peer review were completed, additional guidance was issued to staff that discussed procedures for preparing audit documentation (work paper files) as the assignment progresses and archiving audit documentation upon assignment completion. Specific actions included staff advisories on Assignment Documentation (dated March 26, 2004) and Supervision (dated April 13, 2004), and administrative procedures on Archiving Office of Audits Assignment Documentation (dated March 31, 2004). In addition, training to reemphasize the policy on supervisory reviews and documentation, to include the specific elements identified above is planned for November 2004.
We appreciate the cooperation and courtesies extended by your audit executives and staff to our peer review team.
The Department of Energy’s Office of Inspector General (DOE OIG) conducted an external peer quality assurance review of the Office of Audits (OA) and concluded that our system of quality control has been designed in accordance with the quality standards established by the President's Council on Integrity and Efficiency and provided reasonable assurance of conforming with applicable professional standards, including Government Auditing Standards. In a separate Letter of Comments (LOC) dated August 31, 2004, the DOE OIG discussed matters that came to its attention in the course of its review that did not affect its overall opinion but required corrective action. The LOC provided two recommendations for corrective action, and the OA has completed steps necessary to resolve and disposition both of them.
A description of each of the DOE OIG’s recommendations, our response to their recommendations, and the corrective action implementation chronology are provided in the attachment.
Please let me know if you have any questions.
Office of Audits
Matrix of 2004 Peer Review Recommendations and Corrective Action Status
1 Training provided at the Office of Audits staff conference, December 1-3, 2004.