O OF FF FI IC CE E O OF F T TH HE E S SP PE EC CI IA AL L I IN NS SP PE EC CT TO OR R G GE EN NE ER RA AL L F FO OR R I IR RA AQ Q R RE EC CO ON NS ST TR RU UC CT TI IO ON N M M ME E ET T TH H HO O C C CO O OS S ST T T- - -T T OD D DO O OL L LO O OG G GI I IE E ES S S F F FO O OR TO O O- - -C C CO O OM M MP P PL L LE E ET T TE E E R R R R RE E EP E E ES S ST T TI P PO O OR R RT T TI I IN N NG G G I IM M MA A AT T TE E ES S S S S SI I IG G GI I IR R R- - -0 0 05 5 J J JA A AN N NU U UA A AR R RY Y Y 2 2 27 7 7, , , 5- - -0 0 02 2 27 7 7 2 2 20 0 00 0 06 6 6 SPECIAL INSPE CTOR GENE RAL FOR IRAQ RECONSTRUCTION January 27, 2006 MEMORANDUM FOR DIRECTOR, IRAQ RECONSTRUCTION MANAGEMENT OFFICE COMMANDING GENERAL, GULF REGION DIVISION, U.S. ARMY CORPS OF ENGINEERS COMMANDING GENERAL, MULTI-NATIONAL SECURITY TRANSITION COMMAND-IRAQ MISSION DIRECTOR-IRAQ, U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT SUBJECT: Audit Report on Methodologies for Reporting Cost-to-Complete Estimates (SIGIR-05-027) We are providing this audit report for your information and use. We performed the audit in accordance with our statutory duties contained in Public Law 108-106, as amended, which mandates the independent and objective conduct of audits relating to the programs and operations funded with amounts appropriated or otherwise made available to the Iraq Relief and Reconstruction Fund. Public Law 108-106, as amended, requires that we provide for the independent and objective leadership and coordination of and recommendations on policies designed to promote economy, efficiency, and effectiveness in the administration of such programs and operations and to prevent and detect waste, fraud, and abuse. We considered comments from the Iraq Reconstruction Management Office; Gulf Region Division, U.S. Army Corps of Engineers; Multi-National Security Transition Command-Iraq; and U.S. Agency for International Development on the draft of this report when preparing the final report. We appreciate the courtesies extended to the staff. For additional information on this report, please contact Mr. Joseph T. McDermott at (914) 822-4618, or by email at joseph.mcdermott@iraq.centcom.mil; or Ms. Karen Bell at (703) 428-0147, or by email at karen.bell@sigir.mil. For the report distribution, see Appendix D. Stuart W. Bowen, Jr. Inspector General 400 Army Navy Drive • Arlington, Virginia 22202 Special Inspector General For Iraq Reconstruction SIGIR-05-027 January 27, 2006 Methodologies for Reporting Cost-to-Complete Estimates Executive Summary This report is one of a series of reports that addressed the reporting of cost-to-complete information for projects funded by IRRF. The report discusses the need for improvements to the processes and procedures used to estimate and report the costs to complete projects funded with Iraq Relief and Reconstruction Funds. Section 2207 of Public Law 108-106 requires a quarterly report to Congress that includes estimates of the cost required to complete each project. Introduction. In November 2003, $18.6 billion was appropriated under the Emergency Supplemental Appropriations Act for Defense and the Reconstruction of Iraq and Afghanistan (Public Law 108-106). The law created the Iraq Relief and Reconstruction Fund (IRRF), of which $18.4 billion of the appropriation was designated for Iraq reconstruction. Section 2207 of this law requires a report to Congress every three months that provides updates on the uses of all Iraq Relief and Reconstruction funds on a project by project basis, including estimates of the cost required to complete each project. The report, entitled “Section 2207 Report on Iraq Relief and Reconstruction,” is compiled by the Department of State’s Iraq Reconstruction Management Office (IRMO) from information provided by the Department of State, the Department of Defense, and the U.S. Agency for International Development (USAID). The cost-to-complete information is reported through a companion report to the Section 2207 Report called the “Project Assessment Report” (PAR). Objective. The objective of this audit was to determine the adequacy of methodologies used to estimate and report the costs to complete for projects funded with IRRF. Specifically, we reviewed the cost-to-complete methodologies in the Gulf Region Division-Project and Contracting Office’s (GRD-PCO) Facilities and Transportation sector, specifically those used in the October 2005 Section 2207 Report. We also reviewed similar USAID projects. We also conducted follow-up on our previous recommendation to IRMO regarding the formalization and finalization of cost-to- complete procedures.1 Results. The Gulf Region Division-Project and Contracting Office (GRD-PCO)2, the Multi-National Security Transition Command-Iraq (MNSTC-I), and USAID failed to estimate and report reliable and transparent cost-to-complete information for the IRRF projects we reviewed. MNSTC-I did not submit a report for the September 30, 2005, PAR, and GRD-PCO and USAID submitted reports with errors that were significant enough to undermine users’ confidence in the reporting. 1 SIGIR audit report, “Management of Iraq Relief and Reconstruction Fund Programs: Cost-to-Complete Estimate Reporting,” Report Number SIGIR-05-021, October 24, 2005. 2 The responsibilities of the GRD were consolidated with those of the PCO on December 4, 2005. This report uses the term GRD-PCO for the consolidated entity.i This condition occurred because the IRMO did not provide formal written guidance to the reporting agencies, as we had previously recommended. The condition also occurred because the reporting agencies failed to: • develop and communicate effective internal guidance • identify and correct errors in the reports before issuance • maintain adequate documented support for the calculation of reported estimates- at-completion • adequately report project scope changes Failure to effectively compile and report cost-to-complete information as required by Public Law 108-106 excludes important project visibility essential for project management and Congress to make informed management decisions during IRRF program execution. Because the October 2005 Section 2207 Report did not contain accurate cost-to-complete data, Congress could not use the information to assess IRRF program progress or the availability of funds. Internal Control Weaknesses. We found internal control weaknesses. IRMO and the reporting agencies did not develop and communicate effective guidance, maintain documentation supporting significant numbers in the reports, and adequately review reports before issuance. Management Actions. Since our last cost-to-complete review on October 24, 2005, progress has been made in improving cost-to-complete reporting. However, challenges remain. For example, GRD-PCO program management stated that their cost-to-complete reports are now a more effective project management tool, as a result of improvements to the reporting over the last few months. In addition, since the beginning of the audit, GRD-PCO initiated a draft of guidance detailing methodology for compiling the cost-to- complete reports. GRD-PCO has also decided not to use computer modeling as the methodology for determining future costs. Also, IRMO has recently done a more thorough analysis of data from the cost-to- complete reports, and IRMO management stated that the result is that the reporting is now being used more effectively as a project management tool. IRMO management stated IRMO is now requiring the reporting agencies to limit the use of program contingency for close-out and emergencies, rather than transferring it directly to projects with a funding shortfall. IRMO management stated that this new requirement helps provide a better picture of cost-to-complete. IRMO management also stated that their analysis identified $7 million from closed projects that had not been transferred back to the pool of available funds. Those funds will now be available for use in ongoing projects. Recommendations. In order to make informed management decisions, IRRF management and Congress require timely, accurate and comprehensive cost-to-complete information. As such, we recommend that the: 1. Director, Iraq Reconstruction Management Office, require IRMO management to take these actions: a. Formalize its cost-to-complete action plan by issuing a formal policy to finalize consistent procedures across all supporting IRRF organizations for the collection and compilation of the cost-to-complete information. ii b. In the next Department of State Section 2207 Report, provide data to Congress on the adequacy of cost-to-complete methodologies in the other sectors. 2. Commanding General, Gulf Region Division, require the GRD-PCO sector management to: a. Finalize the draft of official written guidance covering the methodology, including roles and responsibilities, for generating cost-to-complete reports. The guidance should be issued as a policy directive and should cover all of the construction sub-sectors, as well as non-construction projects. The guidance should include a future costs projection element to assure full consideration of potential costs. The guidance should be consistent with the guidance provided by IRMO. b. Ensure that the decision not to use computer modeling in the calculation of cost-to-complete is updated and reflected in all current and future guidance. c. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. d. Create and maintain a permanent central file to document the calculation of estimates-at-completion (EAC) by project. The requirement to create and maintain the central file should be written into the guidance. e. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedules to be sufficiently descriptive to inform users of differences between what is expected to be completed relative to what was planned. 3. Commanding General, Multi-National Security Transition Command-Iraq, require MNSTC-I management to: a. Add the future cost projections component to guidance relating to the calculation of the EACs for construction projects. Develop guidance for non-construction projects including the future cost projections component and the requirement to document calculation of the EACs. The guidance should be consistent with the guidance provided by IRMO. b. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. c. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedules to be sufficiently descriptive to inform users of differences between what is expected to be completed relative to what was planned. Identify all projects to be completed with funds other than IRRF by adding footnotes to the cost-to-complete report. iii 4. USAID Mission Director-Iraq, require USAID management to: a. Develop and distribute official written guidance covering the entire methodology, including roles and responsibilities, for generating cost-to- complete reports. The guidance should be issued as a policy directive. The guidance should cover both construction and non-construction projects, and should include a future costs projections component to assure full consideration of potential costs. This guidance should be consistent with guidance provided by IRMO. b. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. c. Create and maintain a permanent central file to document the calculation of EACs by project. The requirement to create and maintain the central file should be written into the guidance. d. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedules to be sufficiently descriptive to inform users of differences between what is expected to be completed relative to what was planned. Management Comments and Audit Response. We did not receive written comments on this report from IRMO. However, in discussions with IRMO senior managers they stated that they would develop formal policies and procedures for the participating agencies. They also said that the January 2006 Section 2207 Report would report the adequacy of cost-to-complete methodologies in the other sectors. The Commanding General, Gulf Region Division, concurred with the recommendations, and provided information on activities that had occurred since the preparation of the draft of this report. In addition, GRD-PCO officials stated that because of the number of scope changes to projects, rather than footnoting the automated cost-to-complete report, GRD- PCO will develop a supporting schedule summarizing the scope changes by quarter. The Mission Director-Iraq, USAID, concurred with our recommendations and provided some technical corrections to the report. MNSTC-I officials concurred with most of our recommendations, but did not concur that it should report cost-to-complete information for non-construction projects. The management comments that concurred with the findings and recommendations are fully responsive. Additional information and corrections provided were considered and the report was changed accordingly. However, we did not agree with the non- concurrence by MNSTC-I regarding reporting for non-construction projects, because without information on actual program costs, decision makers cannot determine where future investments should be made. iv Table of Contents Executive Summary i Introduction Background 1 Objective 3 Finding Methodologies for Reporting Cost-to-Complete Information - Facilities and Transportation Sector 4 Appendixes A. Scope and Methodology 13 B. Definition of Reporting Terms 15 C. Acronyms 17 D. Report Distribution 18 E. Audit Team Members 20 Management Comments Commanding General, Gulf Region Division Mission Director-Iraq, U.S. Agency for International Development Commanding General, Multi-National Security Transition Command-Iraq 21 22 27 Introduction Background In November 2003, $18.6 billion was appropriated under the Emergency Supplemental Appropriations Act for Defense and the Reconstruction of Iraq and Afghanistan (Public Law 108-106). The law created the Iraq Relief and Reconstruction Fund (IRRF), of which $18.4 billion of the appropriation was designated for Iraq reconstruction. Section 2207 Report. Section 2207 of Public Law 108-106 requires a report to Congress every three months that updates the proposed uses of all IRRF funds on a project by project basis, including estimates of the cost required to complete each project. The most recent report, entitled, “Section 2207 Report on Iraq Relief and Reconstruction” (Section 2207 Report), released in October 2005, was prepared by the Iraq Reconstruction Management Office (IRMO). The Section 2207 Report is compiled by IRMO from data provided by the Department of State, the Department of Defense, and the U.S. Agency for International Development (USAID). The cost-to-complete information is reported quarterly through a companion report to the Section 2207 Report called the “Project Assessment Report” (PAR). Iraq Reconstruction Management Office (IRMO). National Security Presidential Directive 36, “United States Government Operations in Iraq,” May 11, 2004, delegated responsibility for the continuous supervision and general direction of all assistance for Iraq to the Secretary of State. The Directive also created a temporary organization within the U.S. Mission in Iraq, called the Iraq Reconstruction Management Office, to facilitate the transition in Iraq. Project and Contracting Office (GRD-PCO). National Security Presidential Directive 36, “United States Government Operations in Iraq,” May 11, 2004, also established the PCO and directed the PCO to provide acquisition and project management support for activities in Iraq. On June 22, 2004, the Deputy Secretary of Defense established the PCO within the Department of the Army and directed the PCO to provide support for all activities associated with financial, program, and project management for both construction and non-construction IRRF activities. On December 4, 2005, PCO was consolidated with the Corps of Engineers, Gulf Region Division. As of December 15, 2005, the total IRRF funding apportioned to GRD-PCO F&T sector was $1.49 billion. Multi-National Security Transition Command-Iraq (MNSTC-I). MNSTC-I is a component of the U.S Central Command’s Multi-National Forces-Iraq Command. MNSTC-I’s general mission is to help Iraq organize, train, equip, and mentor Iraqi Security Forces in order to transition security tasks to Iraqi control. The Coalition Military Assistance Training Team trains and mentors the provision of garrison support services at eight major division and brigade bases throughout Iraq. Provision of services includes constructing/refurbishing facilities, life support, communications, force protection, and medical support. The Civilian Police Assistance Training Team staffs, equips, organizes, trains, and mentors the Iraqi Police Service forces and Department of Border Enforcement forces. All of the projects are accounted for under the F&T sector, since they are all Security and Justice projects. As of December 15, 2005, the total IRRF funding apportioned to MNSTC-I was $5.26 billion. 1 Corps of Engineers Financial Management System (CEFMS). GRD-PCO Finance and MNSTC-I use the CEFMS for IRRF financial accounting. The CEFMS was designed as a single entry system so the transactions update, in real time, the general ledger and subsidiary ledgers. In CEFMS, as in other financial accounting systems, general ledger amounts should be in agreement with and supported by subsidiary ledgers and transactions detail amounts. U.S. Agency for International Development (USAID). USAID assists Iraqis in reconstructing their country by working with Iraq's interim government. USAID programs are implemented in coordination with the United Nations, World Bank, International Monetary Fund, coalition country partners, nongovernmental organizations, and private sector partners. The USAID Mission in Iraq carries out programs in education, health care, food security, infrastructure reconstruction, economic growth, community development, local governance, and transition initiatives. As of September 30, 2005, USAID had oversight over two construction projects that were similar to F&T, a hospital and a communications network. As of December 15, 2005, the total IRRF funding apportioned to USAID for those projects was $120 million. Phoenix Accounting System. USAID formerly obtained financial data from the Mission Accounting Control System (MACS). Currently, USAID is transferring to the Phoenix financial accounting system, which is replacing the MACS. These systems are the USAID-wide finance and accounting systems. Facilities and Transportation (F&T) Sector. IRMO reports project data grouped by ten sectors: Electric Sector; Oil Infrastructure; Water Resources and Sanitation; Security and Law Enforcement; Justice, Public Safety Infrastructure, and Civil Society; Transportation and Telecommunications Projects; Roads, Bridges, and Construction; Health Care; Private Sector Employment Development; and Education, Refugees, Human Rights, Democracy, and Governance. In April, 2005, the Gulf Region Division-Project and Contracting Office (GRD-PCO) consolidated the seven smaller sectors into one larger sector called Facilities and Transportation Sector (F&T). All, except the Oil, Water, and Electricity sectors, are considered to fall under F&T. Multi-National Security Transition Command-Iraq (MNSTC-I) projects are within the security and justice sectors; therefore, we reviewed those projects since they are similar to F&T. We also reviewed two USAID projects similar to F&T, a hospital and a communications network. As of December 15, 2005, the total IRRF funding apportioned to all F&T and similar reportable projects was approximately $6.8 billion. The total comprises GRD-PCO ($1.45 billion), USAID ($120 million), and MNSTC-I ($5.26 billion). Cost-to-Complete Estimates. The cost to complete a project may be defined as the total estimated cost of the project less the total actual cost of work performed to date: • The estimated cost of a project is the actual direct cost, plus indirect costs or allocable costs to the contract, plus the estimate of costs (direct and indirect) for authorized work remaining. • The actual cost of work performed to date is the cost (direct and indirect) to date of completed work packages and the completed portion of work packages begun and not yet completed. 2 A process for reporting cost-to-complete estimates is an essential component for managing contract performance. Cost-to-complete estimates should be developed as follows: • Determine the progress toward contract completion to date. • Determine the cost of the contract work completed to date. • Estimate the amount of work remaining to be completed. • Estimate the cost of the work remaining to be completed. • Calculate variances between the estimated value of work remaining and the remaining budget. • Determine the reasons for variances from initial estimates and take appropriate action to correct the causes for the variances. Objective The objective of this audit was to determine the adequacy of methodologies used to estimate and report the costs to complete for projects funded with IRRF. Specifically, we reviewed the cost-to-complete methodologies in the GRD-PCO Facilities and Transportation sector, specifically those used in the October 2005 Section 2207 Report. We also reviewed similar USAID projects. We also conducted follow-up on our previous recommendation to IRMO regarding the formalization and finalization of cost-to- complete procedures.3 For a discussion of the audit scope, methodology, and a summary of prior coverage, see Appendix A. For definitions of the acronyms used in this report, see Appendix C. For a list of the audit team members, see Appendix E. 3 SIGIR audit report, “Management of Iraq Relief and Reconstruction Fund Programs: Cost-to-Complete Estimate Reporting,” Report Number SIGIR-05-021, October 24, 2005. 3 Methodologies for Reporting Cost-to-Complete Information - Facilities and Transportation Sector For the Department of Defense, GRD-PCO and MNSTC-I; as well as the USAID failed to estimate and report reliable and transparent cost-to-complete information for IRRF projects – F&T sector. MNSTC-I did not submit a report for the September 30, 2005, PAR, and GRD-PCO and USAID submitted reports with errors that were significant enough to undermine users’ confidence in the reporting. This condition occurred because the IRMO did not provide formal written guidance to the reporting agencies. The condition also occurred because the reporting agencies failed to: • develop and communicate effective internal guidance • identify and correct errors in the reports before issuance • maintain adequate documented support for the calculation of reported estimates- at-completion • adequately report project scope changes Failure to effectively estimate and report cost-to-complete information as required by Public Law 108-106 excludes important project visibility essential for program and project management and Congress to make informed management decisions during IRRF program execution. As we have reported, without current and accurate cost-to-complete data, the funds available for the completion of the IRRF program cannot be determined and the availability of funds for the initiation of new projects cannot be projected. SIGIR requested cost-to-complete data from GRD-PCO in February, 2005, and then again, in May. The first monthly cost-to-complete report was produced in June, 2005. GRD-PCO and USAID submitted to IRMO cost-to-complete information dated September 30, 2005 as part of the quarterly project assessment report (PAR). Cost-to-complete information was not reported quarterly before the September 30, 2005 PAR because: • there was a lack of guidance • tasks for a number of projects were not identified, or the parties had not reached agreement on all of the tasks that should be included • cost estimates for some task orders and projects had not yet been definitized • there was a re-prioritization of tasks to be accomplished as a result of reprogramming of funds • software was not implemented to efficiently track estimated and actual costs • the security situation slowed the pace of the design-and-build teams in visiting job sites, resulting in unanticipated costs. The full effect of those costs was not known Internal Control Weaknesses. We found internal control weaknesses. IRMO and the reporting agencies did not develop and communicate effective guidance, maintain 4 documentation supporting significant numbers in the reports, and adequately review reports before issuance. Iraq Reconstruction Management Office In the Special Inspector General for Iraq Reconstruction Audit Report Number SIGIR- 05-021, dated October 24, 2005, “Management of Iraq Relief and Reconstruction Fund Programs: Cost-to-Complete Estimate Reporting,” we recommended that the Director of the IRMO take action to develop procedures for eliciting cost-to-complete information. IRMO officials agreed and commenced discussions with reporting agencies to develop a formal process to provide the necessary cost-to-complete reports on a monthly basis. Our recommendation in the report stated that IRMO should formalize its cost-to-complete action plan by issuing a formal policy to finalize consistent procedures across all supporting IRRF organizations for the collection and compilation of cost-to-complete information. The lack of guidance resulted in inconsistency among the different activities in preparing quarterly reports of cost-to-complete information. For example, GRD-PCO includes associated close-out costs in project EACs, but USAID reported associated close-out costs separately and did not include them when calculating the EACs. IRMO has not completed its action plan to issue a formal cost-to-complete reporting policy. However, IRMO has recently done a more thorough analysis of data from the cost-to-complete reports, and IRMO management stated that the result is that the reporting is now being used more effectively as a project management tool. IRMO management stated IRMO is now requiring the reporting agencies to limit the use of program contingency for close- out and emergencies, rather than transferring it directly to projects with a funding shortfall. IRMO management stated that this new requirement helps provide a better picture of cost-to-complete. IRMO management also stated IRMO’s analysis identified $7 million from closed projects that had not been transferred back to the pool of available funds. Those funds will now be available for use in open projects. Appendix B represents the agreement of IRMO, OMB, and the Special Inspector General for Iraq Reconstruction as to the data fields needed to ensure complete and transparent reporting of cost-to-complete. During the course of this review, IRMO has updated the format of the PAR in anticipation of preparation of the report for the quarter ending December 31, 2005. Project and Contracting Office GRD-PCO prepared monthly cost-to-complete reports for every month from June 2005 through September 2005, and submitted the reports to IRMO. GRD-PCO has worked to improve methodologies and the quality of the reporting, and we found there has been improvement since we last reported on GRD-PCO cost-to-complete data in October 2005. GRD-PCO program management stated their cost-to-complete reports are now a more effective project management tool, as a result of improvements to the reporting over the last few months. However, we found that GRD-PCO has not formalized or issued policy or guidance, and is not validating the data it is reporting to IRMO. Guidance for Cost-to-Complete Reporting. In July 2005, GRD-PCO issued an executive summary entitled, “Cost To Complete Reporting,” that accompanied the July monthly cost-to-complete report. The executive summary provided an overview of GRD-PCO methodology for compiling cost-to-complete data. The document provides 5 general information about cost-to-complete reporting including report format. The document states the EAC is developed based on the budget to complete plus the earned value calculations, plus future cost projections validated through computer generated modeling. However, the document is not official guidance because it is not signed or dated. GRD-PCO F&T staff stated there was no formal written guidance regarding the compilation of the cost-to-complete report. One staff person stated he learned the methodology from a predecessor and “typed up my own guidance.” In September, 2005, GRD-PCO F&T sector provided to SIGIR another document entitled, “Standard Practice for Developing Estimate at Completion (EAC) for Construction Task Orders”. Estimate-at-completion is a crucial number used to calculate cost-to-complete. GRD-PCO F&T personnel also stated the standard practice document was distributed to all F&T personnel responsible for calculating the EACs for construction projects. The new document was more detailed and provided information about individual responsibilities. However, the document was presented by GRD-PCO as a “memorialization” or “standard practice”, not a policy directive. The document is unsigned and is not dated. In addition, the document only covers development of the EACs for construction task orders, not the methodology for developing the entire cost-to- complete reports. The methodology for the non-construction element has not been documented. Further, the standard practice document does not specifically require consideration of future cost projections. Also, the executive summary document states amounts are validated through computer modeling. However, GRD-PCO personnel stated computer modeling does not work for the F&T sector because, based on experience, the numbers generated are not realistic. GRD-PCO personnel stated while they do consider future cost projections, they do not validate data with computer models. Report Anomalies. The Project and Contracting Office F&T sector failed to satisfactorily review the September 30, 2005, cost-to-complete report submitted for the PAR to correct anomalies. We identified twenty-seven anomalies in the cost-to-complete field of the PAR for the GRD-PCO F&T projects. We defined an anomaly as any reported amount at least three percent different than the expected amount for the field, based on the arithmetic formula for cost-to-complete (EAC less cumulative expenditures). For example, the project code 32000 (Penal Facilities) contains an anomaly in the cost-to- complete field. The reported cost-to-complete is $39,209,525. The EAC is $82,734,929. The cumulative expenditures are $15,407,550. We would expect cost-to-complete to be $67,327,379 (82,734,929 – 15,407,550). The difference between what is expected and what is reported is an apparent understatement of $28,117,854 (67,327,379 – 39,209,525) or 41.8 percent of the expected cost-to-complete amount. The reason for the error could not be determined by analyzing the report. GRD-PCO also provided an additional data field called “extended cost-to-complete”. GRD-PCO defined extended cost-to-complete as cost-to-complete plus costs of sustainability for fiscal year 2006. However, we identified 228 anomalies out of 295 possible lines in the report, where the reported extended cost-to-complete was different than the expected extended cost-to-complete. For example, the project code 15000 (Public Safety Training & Facilities) contains an anomaly in the extended cost-to-complete field. The reported extended cost-to-complete 6 is $39,279,411. The reported cost-to-complete is $25,202,866 and the fiscal year 2006 sustainability cost is approximately $921,703. We would expect extended cost-to- complete to be $26,124,569 (25,202,866 + 921,703). The difference between what is expected and what is reported is an apparent overstatement of $13,154,842 (39,279,411 – 26,124,569) or 50.4 percent of the expected amount. It appears, in many cases, that the spreadsheet adds the EAC, instead of the cost-to-complete, to the sustainability cost to arrive at extended cost-to-complete. The GRD-PCO cost-to-complete report had fields for contract completion date, scheduled completion date, and actual completion date. However, some projects contained an actual completion date after the September 30, 2005, report date. For example, task order 18635 in the health sector is identified as 46 percent complete, but the report lists an actual completion date of January 20, 2006. It is illogical to report an actual completion date after the report date. GRD-PCO F&T staff acknowledged there were errors in the reporting of actual completion dates. In addition to a few MNSTC-I projects contracted through GRD-PCO, the GRD-PCO cost-to-complete report also included data for MNSTC-I projects not contracted through GRD-PCO. When assembling the data for those projects, GRD-PCO simply used EACs that would result in a zero variance. The EACs were not properly calculated, and the inclusion of these projects will result in duplicate reporting when MNSTC-I starts issuing cost-to-complete reports. GRD-PCO personnel stated they intend to remove the non- GRD-PCO MNSTC-I projects from the reporting. GRD-PCO personnel agreed there were errors in the report. GRD-PCO personnel stated they are working to improve the reporting spreadsheets and eliminate anomalies. Documentation of Reported Amounts. GRD-PCO failed to maintain adequate documentation to support the calculation of the EAC for each project. Cost-to-complete is simply the result of an arithmetic equation (EAC less cumulative expenditures). However, EAC, which is the total expected cost of the project, must be calculated to arrive at cost-to-complete. GRD-PCO program management stated that each project manager has primary responsibility for the calculation of the EAC for each project under his or her authority. However, there is no permanent central file containing documentation of the EACs, and GRD-PCO personnel did not know if project managers were maintaining documentation of the calculations. In addition, there is no guidance requiring each project manager to provide written documentation supporting his or her calculation. The cumulative expenditures amount is drawn from a report run in CEFMS. GRD-PCO personnel in the F&T sector stated they do not have documentation to support cumulative expenditures amounts reported in the cost-to-complete reports. GRD-PCO relies upon the accuracy of CEFMS4 . Reporting project scope changes. GRD-PCO failed to provide information in the September 30, 2005, PAR alerting readers to significant scope changes in the projects. For example, the scope of a project in the health sector was significantly changed when GRD-PCO decided to construct fewer facilities. GRD-PCO issued monthly cost-to- 4 SIGIR has not audited CEFMS; however, for more information on the reliability of data drawn from CEFMS, see GAO report 01-89 “Significant Weaknesses in Corps of Engineers’ Computer Controls”, October, 2000, and GAO follow-up report 02-589 “Corps of Engineers Making Improvements But Weaknesses Continue”, June, 2002. 7 complete reports for July and August 2005 that identified the scope changes in footnotes. However, the submission for the September 30, 2005, PAR contained no footnotes or any other numbers or text to explain the change in scope of the health project. Multi-National Security Transition Command-Iraq MNSTC-I failed to submit a cost-to-complete report for the September 30, 2005, PAR. However, MNSTC-I developed procedures and plans to submit a report for the December 31, 2005, PAR. Guidance for Cost-to-Complete Reporting. MNSTC-I issued authoritative guidance detailing the process for reporting cost-to-complete information for construction projects. The MNSTC-I guidance directs the contracting agencies, Air Force Center for Environmental Excellence and Gulf Region Division, regarding how to calculate the EACs. However, MNSTC-I failed to include a provision for future cost projections. Future cost projections are a critical component for assessing the total cost of the project. MNSTC-I has not issued guidance relating to non-construction projects. Documentation of Reported Amounts. Personnel at MNSTC-I stated that the contracting agencies are responsible for documenting calculation of the EACs. While MNSTC-I does not keep support for the reported EAC amounts, the issued guidance requires contracting agencies to maintain documentation of the calculation. Like GRD-PCO, MNSTC-I uses CEFMS to obtain the cumulative expenditures data. MNSTC-I personnel stated that support for the amounts drawn from CEFMS reports are maintained by the contracting agencies. MNSTC-I personnel stated that the contracting agencies are expected to keep invoices on file, and that the agencies can produce the invoices upon request to support a cumulative expenditures number, if necessary. U.S. Agency for International Development USAID submitted a cost-to-complete report for the September 30, 2005 PAR. However, we found that USAID has not formalized or issued policy or guidance, and is not validating the data it is reporting to IRMO. Guidance for Cost-to-Complete Reporting. The US Agency for International Development has not developed and communicated effective guidance for estimating and reporting cost-to-complete information. On June 2, 2005, SIGIR made a specific request for policies and procedures relating to cost-to-complete reporting. USAID communicated their procedures orally with SIGIR auditors, but USAID stated they would not provide cost-to-complete methodology in writing until they received instruction from IRMO. USAID obtains financial data from USAID-wide financial systems. However, the USAID contractor employee primarily responsible for compiling the USAID cost-to- complete submission for the September 30, 2005, PAR stated he used data provided by the contractor, rather than data from USAID systems. The lack of formal guidance resulted in an over-reliance on information provided by a contractor. Report Anomalies. USAID failed to thoroughly review the September 30, 2005, cost- to-complete reports to correct anomalies. USAID reported two construction task orders 8 that were similar to the F&T sector, a hospital and a communications network, with a total value of $120 million. We identified an anomaly in the cost-to-complete field for the totals row (for all sectors) in the USAID PAR. The reported cost-to-complete in the totals line is $961,389,057. The total EACs are $1,331,211,678. The total cumulative expenditures are $581,194,360. We would expect cost-to-complete to be $750,017,318 (1,331,211,678 – 581,194,360). The difference between the reported cost-to-complete and the calculated cost-to-complete is an apparent overstatement of $211,371,739 (961,389,057 – 750,017,318). The report was in error by including indirect costs of $211,371,739 in the total cumulative expenditures, but not including the amount in the total EACs. We also identified an anomaly in the PAR where USAID separated programmatic close- out costs from the EAC. The programmatic close-out costs are reported in a separate column, and the cell formula for the close-out costs is three percent of EAC. This means the close-out costs are not being included in the EACs, which are undervalued by three percent for all USAID projects as a result. The total amount of the separated close-out costs for all projects was approximately $19.3 million. The EAC is the total cost to complete the project, and the associated programmatic close-out costs are part of the total costs. Documentation of Reported Amounts. USAID failed to maintain supporting documentation for the calculation of the EAC for each project. It appears USAID simply used the budgeted amount for the project as the value of the EAC. The cumulative expenditures amount reported in the September 30, 2005, PAR was derived from data provided by the contractor managing the projects. USAID personnel stated they have copies of invoices on file and can support any cumulative expenditures number in the cost-to-complete report. Reporting Project Scope Changes. USAID failed to provide information in the September 30, 2005, PAR alerting readers to significant scope changes in the projects. Because we only reviewed two projects that were similar to the F&T sector and both were in the early stages of construction, it is possible neither had a significant scope change. Conclusion GRD-PCO, MNSTC-I, and USAID were required by Public Law 108-106 to submit cost- to-complete information to IRMO for the PAR for the quarter ending September 30, 2005. However, MNSTC-I did not submit a cost-to-complete report for the quarter. In addition, the three reporting entities failed to develop proper guidance, adequately review reports before submission, document the calculation of critical numbers, and note significant scope changes in the reporting. We conclude that IRMO, as well as GRD-PCO, MNSTC-I, and USAID (the reporting entities) failed to develop methodologies to assure reliable and transparent cost-to- complete reporting. Failure to effectively compile and report cost-to-complete information as required by Public Law 108-106 excludes important project visibility essential for project management and Congress to make informed management decisions during IRRF program execution. Because the October 2005 Section 2207 Report did not 9 contain accurate cost-to-complete data, Congress could not use the information to assess IRRF program progress or the availability of funds. Recommendations, Management Comments, and Audit Response In order to make informed management decisions, IRRF management and Congress require timely, accurate and comprehensive cost-to-complete information. As such, we recommend the: 1. 2. 3. Director, Iraq Reconstruction Management Office, require IRMO management to take these actions: a. Formalize its cost-to-complete action plan by issuing a formal policy to finalize consistent procedures across all supporting IRRF organizations for the collection and compilation of the cost-to-complete information. b. In the next Department of State Section 2207 Report, provide data to Congress on the adequacy of cost-to-complete methodologies in the other sectors. Commanding General, Gulf Region Division, require the GRD-PCO sector management to: a. Finalize the draft of official written guidance covering the methodology, including roles and responsibilities, for generating cost-to-complete reports. The guidance should be issued as a policy directive and should cover all of the construction sub-sectors, as well as non-construction projects. The guidance should include a future costs projection element to assure full consideration of potential costs. The guidance should be consistent with the guidance provided by IRMO. b. Ensure that the decision not to use computer modeling in the calculation of cost-to-complete is updated and reflected in all current and future guidance. c. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. d. Create and maintain a permanent central file to document the calculation of estimates-at-completion (EAC) by project. The requirement to create and maintain the central file should be written into the guidance. e. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedules to be sufficiently descriptive to inform users of differences between what is expected to be completed relative to what was planned. Commanding General, Multi-National Security Transition Command-Iraq, require MNSTC-I management to: a. Add the future cost projections component to guidance relating to the calculation of the EACs for construction projects. Develop guidance for 10 4. b. c. non-construction projects including the future cost projections component and the requirement to document calculation of the EACs. The guidance should be consistent with the guidance provided by IRMO. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedules to be sufficiently descriptive to inform users of differences between what is expected to be completed relative to what was planned. Identify all projects to be completed with funds other than IRRF by adding footnotes to the cost-to-complete report. USAID Mission Director-Iraq, require USAID management to: a. Develop and distribute official written guidance covering the entire methodology, including roles and responsibilities, for generating cost-to- complete reports. The guidance should be issued as a policy directive. The guidance should cover both construction and non-construction projects, and should include a future costs projections component to assure full consideration of potential costs. This guidance should be consistent with guidance provided by IRMO. b. Develop a thorough review process to eliminate errors in the reporting spreadsheets. The review process should be written into the guidance. c. Create and maintain a permanent central file to document the calculation of EACs by project. The requirement to create and maintain the central file should be written into the guidance. d. Develop policies to define significant scope changes. For example, variances reflecting a 15 percent change in schedule, scope, or budget could be used as a metric to define “significant”. The changes should be reported in the quarter the changes occurred by adding supporting schedules to the cost-to-complete report. Require the supporting schedule be sufficiently descriptive to inform users of differences between what will be completed relative to what was planned. Management Comments and Audit Response. We received written comments to the draft of this report from officials at GRD-PCO, USAID, and MNSTC-I. We did not receive written comments from IRMO, but discussed the report and its recommendations with senior IRMO managers. IRMO managers told us that they concurred with our recommendation to issue a formal policy to finalize consistent procedures across all supporting IRRF organizations. They also said that the January 2006 Section 2207 Report would discuss the adequacy of cost- to-complete methodologies in the other sectors. The Commanding General, Gulf Region Division, concurred with the recommendations, and provided more recent information on activities that had occurred since the preparation of the draft of this report. In addition, GRD-PCO officials stated that because of the number of scope changes to projects, rather than footnoting the automated cost-to- 11 complete report, GRD-PCO will develop a supporting schedule summarizing the scope changes by quarter. Comments were fully responsive. The Mission Director-Iraq, USAID, concurred with our recommendations and stated that they would develop a written mission policy and procedures for estimating CTC that included controls for validating and revalidating estimates. USAID also said that it would seek to coordinate with IRMO to develop a mutually agreeable methodology for calculating CTC estimates. However, they said that any methodology used by USAID would reflect USAID’s definition of a project and USAID’s requirements, policies, procedures and limitations. USAID also stated that it would create and maintain a permanent central file to document the calculation of EACs by project. USAID did question our use of the term facilities and transportation sector projects, stating that it is not a term used by USAID. Consequently, they questioned whether the anomalies we identified pertained to USAID projects. We agree, and have changed our report as appropriate. However, standardized reporting still applies to USAID activities, as it has construction projects that must be included in Section 2207 Reports. These include one hospital project and one communication project. Comments were fully responsive. MNSTC-I officials concurred with most of our recommendations, but did not concur that it should provide cost-to-complete information for non-construction projects. MNSTC-I stated that it already reported cost-to-complete information in the quarterly Section 2207 Reports in the form of obligation and outlay data. However, we believe this information does not provide decision makers with the necessary transparency as to the actual cost to complete non-construction projects. For example, MNSTC-I sometimes uses non-IRRF funds to complete projects after IRRF funds are fully expended. Without information on actual program costs, decision makers cannot determine where future investments should be made. 12 Appendix A. Scope and Methodology In March 2005, we initiated an audit to determine whether the information systems used by U.S. government organizations result in effective and reliable program management information. This audit report is the third report on cost-to-complete (Project No. SIGIR- 2005-08.2). It discusses procedures and processes used to develop and report cost-to- complete estimates for IRRF projects. Due to time and resource considerations, we limited the scope of the audit to GRD- PCO’s F&T sector and similar projects in MNSTC-I and USAID. Furthermore, we limited the scope of the audit to the three organizations that received the largest apportionment of funds – GRD-PCO, USAID, and MNSTC-I. To gain an understanding of each entity’s operations and processes for developing and reporting cost-to-complete information, we interviewed management personnel from IRMO, GRD-PCO; MNSTC-I and USAID. We also reviewed organization charts and websites to obtain background information and to determine responsibilities. We met with IRMO management, GRD-PCO Project Management, GRD-PCO Financial Management, USAID Directors, and MNSTC-I Commanders to discuss the contractual requirements and agency methodologies used to determine project status and to calculate and report cost-to-complete estimates. We also held discussions with OMB and IRMO on management requirements of cost-to-complete. Appendix B represents the agreement of IRMO, OMB, and the Special Inspector General for Iraq Reconstruction as to the data fields needed to ensure complete and transparent reporting of cost-to-complete. We reviewed available procedures, both formal and in-progress, that described the processes, methodology, responsibilities, documentation standards and formats for cost- to-complete reporting. We analyzed GRD-PCO F&T and USAID cost-to-complete reports submitted to IRMO for the September 30, 2005, PAR to determine if data is properly reflected in the report, if the arithmetic calculations were correct, and if significant project scope changes were reported. We also analyzed GRD-PCO F&T cost-to-complete monthly reports for July and August, 2005. We met with representatives from GRD-PCO and Air Force Center for Environmental Excellence to review contract management procedures. We performed this audit from May 14, 2005 through December 31, 2005, in accordance with generally accepted government auditing standards. Use of Computer-Processed Data. We reviewed cost-to-complete reports that were compiled in Excel spreadsheets based on data taken from reports run in financial accounting computer systems. The data in the GRD-PCO cost-to-complete report was taken from reports run in CEFMS. We did not audit CEFMS5. USAID’s financial 5 For more information on the reliability of data drawn from CEFMS, see GAO report 01-89 “Significant Weaknesses in Corps of Engineers’ Computer Controls”, October, 2000, and GAO follow-up report 02-589 “Corps of Engineers Making Improvements But Weaknesses Continue”, June, 2002. 13 accounting system was formerly the MACS, but USAID is currently transitioning to the Phoenix accounting system. We did not audit MACS or Phoenix. Moreover, the person primarily responsible for compiling the USAID submission for the September 30, 2005, PAR stated he used data provided by the contractor managing the projects. Prior Coverage. Special Inspector General for Iraq Reconstruction Audit Report Number SIGIR-05-021, dated October 24, 2005, “Management of Iraq Relief and Reconstruction Fund Programs: Cost-to-Complete Estimate Reporting”, concluded the three organizations responsible for IRRF projects – PCO, USAID, and the MNSTC-I – have been required, since January 2004, to report cost-to-complete information for their IRRF projects in quarterly reports to the Congress. However, these organizations did not begin providing reasonably comprehensive cost-to-complete data to IRMO until the summer of 2005. Special Inspector General for Iraq Reconstruction Audit Report Number SIGIR-05-011, dated July 26, 2005, “Cost-to-Complete Estimates and Financial Reporting for the Management of the Iraq Relief and Reconstruction Fund”, included a review of PCO’s input to the April 2005 Section 2207 Report and found that PCO did not provide cost-to- complete information to IRMO for the Section 2207 Report. PCO maintained that (1) project data was not sufficiently mature to develop reasonable estimates at completion; an (2) they could not consolidate information from their management information systems because they were not integrated. 14 Appendix B. Definition of Reporting Terms6 DATA FIELD DEFINITION Task Order An order for the performance of a task during the period of a task order indefinite delivery contract for services that do not procure or specify a firm quantity of services (other than a minimum or maximum quantity) Sector PL 108-106 Section 2207 report sector Sub-Sector PL 108-106 Section 2207 subsector URI Unique Requirements code Project Name/ Description Name and short description of project DB/Prime Contractor Name of prime contractor to which US Agency awarded contract Sub or other contract Name(s) of any subs or other contractors the prime is using Original Start Date Original date when project was supposed to begin under the original Project Identification Form (PIF) or original estimate Actual Start date Date by which project is currently scheduled to begin Target completion date Original date by which project was to have been completed Projected completion date Current date by which project is projected to be complete Authorized Amount Amount Authorized and Apportioned for the Project Code Original PIF Value Project Identification Form Amount budgeted for project when the original PIF was completed (for the first 2207 report) Definitized (Y/N) Yes, if a project has been completely definitized Definitized Value The dollar amount budgeted for the project, according to the definitization of the project Contingency % Percent of PIF, revised estimate or definitized amount that includes built-in contingency amount. Committed to Date Amount committed for the project Obligated to Date Amount of Contract Award Reserve Any reserve (security, strategic, floating, etc.) that is not included in current project estimate but is allocated against each specific project. If such a reserve is not allocated by project, but by sector or task order (ATO), please add a separate line item to data call, labeling it as a reserve. Associated Close Out Costs The current estimate of project close-out costs, or contract close-out costs that will be booked against each project. Demobilization Costs The current estimate of project demobilization costs, or contractor demobilization costs that will be booked against each project. Cumulative Expenditures Amount of Contract Expenditures (includes accounts payable plus disbursements, work that is completed) Certified Commitments Requirements not yet obligated (project contingencies or pending modifications) Undelivered Orders Amount of Unexpended Balance on the Contract (work yet to be accomplished) Budget to Complete Undelivered Orders Amount plus Certified Amount cost-to-complete Estimated cost to complete the project based on the difference between the Estimate to Complete and the Current cost incurred to date. Variance Indicates sufficiency of program funds for estimated cost to complete Earned Value or % Complete The % of the project complete, as measured against the baseline schedule, cost and scope Most Recent Estimate Date The date of the most recent estimate at completion 6 The list is current as of the September 30, 2005, PAR. As a result of efforts by IRMO to improve cost-to- complete, the reporting terms list will change for the December 31, 2005, PAR. 15 DATA FIELD DEFINITION Value of Most Recent Estimate The dollar amount budgeted for the project, according to the most recent estimate (if no new estimate has been done from the original PIF, leave blank) Estimate At Completion The estimate of what the project is expected to cost by the time it is completed Variance explanation Rationale of variance between estimate/definitized value and EAC 16 Appendix C. Acronyms CEFMS U.S. Army Corps of Engineers Financial Management System EAC Estimate at Completion F&T Facilities and Transportation Sector GRD-PCO Gulf Region Division-Project and Contracting Office IRMO Iraq Reconstruction Management Office IRRF Iraq Relief and Reconstruction Fund MACS Mission Accounting Control System MNSTC-I Multi-National Security Transition Command-Iraq PAR Project Assessment Report USAID U.S. Agency for International Development 17 Appendix D. Report Distribution Department of State Secretary of State Senior Advisor to the Secretary and Coordinator for Iraq U.S. Ambassador to Iraq Director, Iraq Reconstruction Management Office Inspector General, Department of State Department of Defense Secretary of Defense Deputy Secretary of Defense Director, Defense Reconstruction Support Office Under Secretary of Defense (Comptroller)/Chief Financial Officer Deputy Chief Financial Officer Deputy Comptroller (Program/Budget) Inspector General, Department of Defense Department of the Army Assistant Secretary of the Army for Acquisition, Logistics, and Technology Principal Deputy to the Assistant Secretary of the Army for Acquisition, Logistics, and Technology Deputy Assistant Secretary of the Army (Policy and Procurement) Director, Project and Contracting Office Commanding General, Joint Contracting Command – Iraq/Afghanistan Assistant Secretary of the Army for Financial Management and Comptroller Chief of Engineers and Commander, U.S. Army Corps of Engineers Commander, Gulf Region Division Auditor General of the Army U.S. Central Command Commanding General, Multi-National Force – Iraq Commanding General, Multi-National Security Transition Command – Iraq Commander, Joint Area Support Group – Central Other Defense Organizations Director, Defense Contract Audit Agency 18 Other Federal Government Organizations Director, Office of Management and Budget Comptroller General of the United States Inspector General, Department of the Treasury Inspector General, Department of Commerce Inspector General, Department of Health and Human Services Inspector General, U.S. Agency for International Development Mission Director – Iraq, U.S. Agency for International Development Congressional Committees and Subcommittees, Chairman and Ranking Minority Member U.S. Senate Senate Committee on Appropriations Subcommittee on Defense Subcommittee on State, Foreign Operations and Related Programs Senate Committee on Armed Services Senate Committee on Foreign Relations Subcommittee on International Operations and Terrorism Subcommittee on Near Eastern and South Asian Affairs Senate Committee on Homeland Security and Governmental Affairs Subcommittee on Federal Financial Management, Government Information and International Security Subcommittee on Oversight of Government Management, the Federal Workforce, and the District of Columbia U.S. House of Representatives House Committee on Appropriations Subcommittee on Defense Subcommittee on Foreign Operations, Export Financing and Related Programs Subcommittee on Science, State, Justice and Commerce and Related Agencies House Committee on Armed Services House Committee on Government Reform Subcommittee on Management, Finance and Accountability Subcommittee on National Security, Emerging Threats and International Relations House Committee on International Relations Subcommittee on Middle East and Central Asia 19 Appendix E. Audit Team Members This audit report was prepared and the audit work was conducted under the direction of Joseph T. McDermott, the Assistant Inspector General for Auditing, Office of the Special Inspector General for Iraq Reconstruction. The staff members who contributed to the report include: Glenn Furbish William Shimp 20 Management Comments Commanding General, Gulf Region Division, U.S. Army Corp of Engineers 21 Management Comments Mission Director-Iraq, U.S. Agency for International Development 22 23 24 25 26 Management Comments Commanding General, Multi-National Security Transition Command-Iraq 27 28 29 30