Follow Us:

Goal One Performance Measures, Analysis and Review


Table of Contents >

PERFORMANCE MEASURE 1.1.1.2


PERFORMANCE MEASURE 1.1.1.2 Implement automated surveillance alerts and a case management system. FY 2011 – Implement four automated market alerts.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Integrated Survelilance System (ISS)
Verification: Internal Review of Daily Output from Alerts
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
70% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission was able to develop and implement two alerts and four automated reporting programs. The two alert programs were developed around price and concentration, while the automated reports focused on position, position changes, option expiration, and early data releases. These programs highlight unusual and alarming situations and market conditions that initiate further review and analysis throughout the agency. While the target for FY 2011 was set at four automated market alerts, automated reports provide a similar benefit to those who review and analyze the output within the Commission, much like an official alert program. Therefore, the actual performance towards completion of four alerts was 70 percent of the target versus 50 percent. However, it should be noted that alerts and automated reports rarely remain in a static state. The Commission is continuously reviewing output to enhance the tools available to better assist with surveillance activity. The automation of these alerts improves the efficiency at which the Commission conducts its surveillance activity, an important step at a time when the responsibilities of the agency continue to expand. The Commission will continue to work diligently on the remaining and additional alert programs and automated reports, seeking the most efficient methods of conducting operations. However, staffing levels and budget constraints will continue to be a large obstacle towards success, going into FY 2012 and FY 2013.

PERFORMANCE MEASURE 1.1.1.3


PERFORMANCE MEASURE 1.1.1.3 Implement automated trading violation alerts and a case management system. FY 2011 – Implement five automated trading violation alerts.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Actimize
Verification: Internal Review of Daily Output from Alerts
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
20% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission implemented one automated trading violation alert around market profiling and made several enhancements to existing models, such as the trading ahead model, during the fiscal year. The planned outcomes for period covered by the Strategic Plan were based on an assumption that the Commission would make significant progress on obtaining order book and ownership and control data. However, with previous and current budget constraints, a continuing resolution lasting most of the fiscal year, expected progress in this area was limited. Yet still, the Commission continues to make progress in spite of this resource shortfall, by working with data that currently is available – transactional data.

The automated alerts are increasingly becoming important as the roles and responsibilities of the agency continue to expand. At minimum, to remain efficient and effective, the Commission's technology must keep pace with that of the market. In the absence of these improvements, the CFTC does not have all the data available to analyze the full audit trail, from order entry to account ownership, and runs the risk of missing trade practice violations.

As with most technical solutions the process of acquiring the necessary data, and building the most effective alert programs, is resource intensive. The Commission will continue to work diligently on the remaining and additional alert programs, seeking the most efficient methods of conducting operations. However, staffing levels and budget constraints will continue to be a large obstacle towards success, going into FY 2012 and FY 2013.

PERFORMANCE MEASURE 1.1.2.1


PERFORMANCE MEASURE 1.1.2.1 Review information requirements of current and proposed forms. FY 2011 – Conduct internal review and update current reporting forms. Collaborate with industry committee to develop recommendations for ownership and control information related to exchange-traded futures and options. Percent complete.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Forms
Verification: Final Rule
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
50% 100% 100% N/A N/A N/A

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission continues to make great strides towards the large trader reporting forms and is currently preparing its proposals for review. After the Notice of Proposed Rulemaking (NPRM), comments will be reviewed and work will continue on a proposed final rule. This is a dependency that adversely impacts the remaining steps in the process to form enhancement. Therefore, no more than 50 percent complete will be recorded for this measure.

The expectation is to see the Commission adopt final rules and amendments to the existing forms during the first and second quarters of FY 2012. The team anticipates incorporating ownership and control reporting standards into the revised forms. To date, the review of Forms 102 & 40 and the related regulations has been done on a separate track than the implementation effort for the Part 20 swaps large trader rules. Due to the nature of the performance measure, it should be noted that this data is estimated for reporting purposes, however remains adequate for this level of communication.

In FY 2011, the Commission faced a challenge of a limited workforce. As a result, Division functions outside the rulemaking process, including the development of requirements for current and future forms, were limited during the fiscal year. Without updating these forms, the Commission and market participants will continue to experience inefficiencies in oversight and compliance by way of incomplete or inaccurate form filing, untimely form filing, and lack of automated review of form filings. As the Commission remains determined to complete its rulemaking during the upcoming FY 2012, Divisions and Offices can expect to face similar challenges and a prolonged environment of budget and human resource constraints.

PERFORMANCE MEASURE 1.1.3.1


PERFORMANCE MEASURE 1.1.3.1 Transmit information and consult with the Office of Information Technology Services (OITS) to implement electronic filing of forms. FY 2011 – Transmit information requirements to the ODT for revised trader reporting forms. Percent complete.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Forms
Verification: Final Rule
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
50% 100% 100% 100% N/A N/A

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission continues to make great strides towards the large trader reporting forms and is currently preparing its proposals for review. This is a dependency that adversely impacts the remaining steps in the process to form enhancement, including the transmission of requirements and collaboration with the Office of Data and Technology (ODT, formerly known as OITS). Once the proposals become final and all reviewed periods complete, the Commission will deliver the necessary requirements to the technology team and continue to work with industry representatives to implement the rules. As stated in the previous, related performance measure narrative, the Commission remains determined to complete its rulemaking during the upcoming FY 2012. Divisions and Offices can expect to face continued challenges and a prolonged environment of budget and human resource constraints. Given this level of performance, this strategic effort will likely remain relevant both in FY 2012 and a part of FY 2013. While these time frames and the progress are estimated, they remain adequate for this level of reporting.

PERFORMANCE MEASURE 1.1.4.1


PERFORMANCE MEASURE 1.1.4.1 Percentage of contracts that are reviewed, in a timely manner, following a finding of market significance, and determined to be in compliance with core principles or referred back to exchange for modification.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: FILAC
Verification: Internal Review of Dates Entered into System; Review of ISS Database to determine significance
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
2% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. During this fiscal year, the Commission completed reviews of 57 certified contracts, 27 of which were completed within 90 days of the filing and considered timely for this performance measure. However, staff and resource limitations leave the Commission with over 2,000 contracts to be reviewed leading into FY 2012. Furthermore, there were an additional 1,400 new contracts self-certified by Designated Contract Markets (DCMs) that did not exceed volume and open interest thresholds in or to be considered significant, but could change in the coming year.

The Commission reviews a contract's terms and conditions, and the position limits and accountability standards, to ensure that a contract is not readily susceptible to manipulation and standards are appropriate. Often, reviews include an analysis of the assumptions made by the particular exchange staff submitting the new contract regarding supply and demand of the underlying commodity. In the absence of Commission due diligence, undetected contract flaws or faulty assumptions could lead the contract to be readily susceptible to manipulation, disruptions in the cash market, or excessive speculation.

As staff members are released from rule-writing responsibilities, review of contracts that exhibit market significance will resume. The Dodd-Frank Act rulemaking effort will continue through most of FY 2012 and is expected to further impact future targets set in the current Strategic Plan. In response to these external conditions, the Commission is considering modified targets that more accurately reflect the Commission's capabilities given the existing staffing levels and continued budget constraints.

PERFORMANCE MEASURE 1.1.5.1


PERFORMANCE MEASURE 1.1.5.1 Rule submissions are reviewed and a determination is made regarding compliance with the CEA, or referred back to the exchange for correction, as amended by the Dodd-Frank Act and Commission regulations within the required 10-day or 90-day time period.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: FILAC
Verification: Internal Review of Dates Entered into System
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
77% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission was able to complete reviews of 330 submissions for certification or approval of trading rules, 82.5 percent of the total submitted throughout the year. In addition, the Commission was also able to complete 47 submissions for certification or approval of product rules, approximately 53 percent of the total on the year. A combined total between trading and product rules suggest the performance of an estimated 77 percent reviewed by the Commission during the fiscal year. However, limited staffing and resource constraints throughout the fiscal year continued to inhibit the level of reviews of rule amendment filings as planned.

It is significant to note, that the backlog of rule amendment certification reviews has grown so much that, this goal as written, most likely will not be met with the existing staff levels. To date, there are 70 submissions for trading rules and 130 substantive rule amendments on backlog for review. While the Commission continues to seek the most efficient methods of conducting operations, staffing levels and budget constraints will continue to be the largest obstacle to success going into FY 2012 and FY 2013.

PERFORMANCE MEASURE 1.1.6.1


PERFORMANCE MEASURE 1.1.6.1 DCM and SEF applications are reviewed and a determination is made regarding compliance with core principles within statutory time frames. Percentage of applications reviewed.
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Memorandum Initiating Review
Verification: Final Report
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
100% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was met for FY 2011. The Commission performed one review during the reporting period on Eris Exchange, LLC application, formally submitted in the third quarter of FY 2011. CFTC is aware of one DCM application that will be filed in FY 2012 and expect at least one or two others will be filed during the year. Current performance levels should be expected during FY 2012.

PERFORMANCE MEASURE 1.2.1.1


PERFORMANCE MEASURE 1.2.1.1 Percentage of major DCMs and SEFs reviewed, during the year. (Structural Sufficiency)
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Memorandum Initiating Review
Verification: Final Report
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
40% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission completed a joint rule enforcement review of core principles relating to NYMEX's and COMEX's trade practice surveillance, audit trail, and disciplinary programs. This review also examined the Exchanges' staffing levels to ensure that they had sufficient staff to perform their self-regulatory responsibilities for each of the reviewed programs. Although it was found that NYMEX and COMEX were in compliance with the relevant core principles, recommendations for improvement were made regarding the Exchanges' staffing and disciplinary programs. As direct result of limited resources and available staff for this activity, the remaining three major DCMs were not reviewed during this reporting cycle (CME, CBT, and ICE Futures U.S.). In addition, regulations for SEF registration were incomplete and thus, there were no major SEFs to be reviewed as a part of the performance measure.

In FY 2011, the Commission faced a challenge of a limited workforce, an inability to hire from a continuing resolution lasting seven months, and a high-priority rulemaking directive. As a result, Division functions outside the rulemaking process, including the review of both major and non-major DCMs and SEFs, were extremely limited during the fiscal year. Not reviewing all major DCMs is an oversight risk of exchanges that are responsible for the vast majority of U.S. futures trading volume. Timely rule enforcement reviews are intended to ensure market integrity in order to foster open, competitive, and financially sound markets. If major exchanges are not reviewed annually, it is difficult to provide any assurance to the public or other regulators of the exchanges' ongoing core principle compliance. Moreover, the Dodd-Frank Act modified several existing core principles and adopted five new core principles.

As the Commission remains determined to complete its Dodd-Frank Act rulemaking during the upcoming FY 2012, Divisions and Offices can expect to face similar challenges and a prolonged environment of budget and human resource constraints.

PERFORMANCE MEASURE 1.2.1.2


PERFORMANCE MEASURE 1.2.1.2 Percentage of non-major DCMs and SEFs reviewed, during the year. (Structural Sufficiency)
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Memorandum Initiating Review
Verification: Final Report
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
20% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission initiated three rule enforcement reviews of non-major DCMs for core principles related to market surveillance, trade practice surveillance, audit trail, and disciplinary programs. Due to insufficient staff resources, none of these reviews were completed during the 2011 fiscal year. No review was initiated on any of the remaining nine non-major DCMs during this reporting cycle.

A review of NYSE Liffe's trade practice surveillance program will be completed in the first quarter of FY 2012, while the review of the KCBT's market surveillance, trade practice surveillance, audit trail, and disciplinary programs will be completed during the second quarter of FY 2012. The review of CBOE Futures' audit trail program had been suspended (and as of the end of FY 2011, considered complete) due to the significant issues identified. A 38.5 Exchange Letter has been sent to CBOE and they are in the process of implementing the necessary changes. A follow-up review on CBOE will be initiated in the fourth quarter of FY 2012. It should be noted that CBOE Futures and NYSE Liffe both contract with the NFA for regulatory services. As a result, the reviews require reviewing work performed by NFA as well as the exchanges.

Although non-major exchanges do not pose the same risks as major exchanges, there are serious consequences of not performing reviews at least every two or three years. Generally, the non-major exchanges' compliance programs are not as sophisticated as the major exchanges and typically result in more recommendations for improvement from a rule enforcement review. However, past and current resources have been insufficient for timely review of these non-major DCMs, with some DCMs going 5 years or longer without a review. It can be expected that delays and deficiencies will continue to grow in the current operating environment.

Regulations for SEF registration were incomplete and thus, there were no non-major SEFs to be reviewed as a part of the performance measure.

PERFORMANCE MEASURE 1.2.2.1


PERFORMANCE MEASURE 1.2.2.1 Percentage of major DCMs and SEFs reviewed, during the year. (Automated Systems and Business Continuity)
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Market Continuity Program
Verification: Final Report
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
80% 100% 100% 100% 100% 100%

Performance Analysis & Review

The performance target was not met for FY 2011. The Commission initiated and conducted on-site visits for all five major DCMS (CME, CBOT, NYMEX, COMEX, and ICE) during FY 2011. System Safeguards Examination's (SSE) involve an assessment of the compliance on a DCM's automated systems and business continuity-disaster recovery (BC-DR) plans. Each SSE includes review of some or all of the six principal categories of proper risk management controls: (1) information security; (2) BC-DR and pandemic planning; (3) capacity planning processes and testing; (4) computer and network operations; (5) systems development methodology and quality assurance; and (6) physical security and environmental controls.

One System Safeguards Examination (SSE), initiated in the third quarter FY 2011, was conducted for CME Group. The examination evaluated compliance with the technology-related aspects of applicable CEA provisions and regulations with respect to CME Group's Globex trading system and related systems, as well as general controls for management of information technology resources. It was found that CME Group's implementation of the general controls for management of information technology resources supporting the Globex trading environment adequately met the requirements of the Designation Criterion and Core Principles. However, the inspection also identified four low-risk findings and recommendations for improvement (three for information security and one for enterprise risk management).

While examinations (SSEs) were initiated and visits were conducted for all five major DCMs, final examination reports for CME, CBOT, NYMEX, and COMEX will be completed during the second quarter and ICE during the third quarter, of FY 2012. Regulations for SEF and SDR registration are incomplete and thus, there were no major SEFs or SDRs to be reviewed as a part of the performance measure.

PERFORMANCE MEASURE 1.2.2.2


PERFORMANCE MEASURE 1.2.2.2 Percentage of non-major DCMs and SEFs reviewed, during the year. (Automated Systems and Business Continuity)
Lead Program Office: Division of Market Oversight (DMO)
Data Source: Market Continuity Program
Verification: Final Report
ACTUAL
FY 2011
PLAN
FY 2011
PLAN
FY 2012
PLAN
FY 2013
PLAN
FY 2014
PLAN
FY 2015
N/A 33% 33% 33% 33% 33%

Performance Analysis & Review

The performance target was not met for FY 2011. System Safeguards Examination's (SSE) were not conducted for non-major (Tier 2) DCMs during FY 2011 due to limited staff resources. The Market Continuity Program presently does not have enough systems risk analysts to conduct SSEs for non-major DCMs in addition to conducting major DSM SSEs, assisting with Dodd-Frank Act rulemakings, other automated system-related Commission rulemakings and policy development, and responding to market interruptions and cyber threats at DCMs.

While automated system malfunctions, cyber incidents, or trading interruptions at non-major DCMs do not pose the same degree of risk to the U.S. financial system as similar problems at a major DCM, automated systems and BC-DR resources are critical for any DCM to maintain a comprehensive audit trail, publish timely market data, conduct adequate market and trade practice surveillance, provide large trader reporting, and monitor and enforce compliance with DCM rules and Commission regulations. Non-major DCMs are more likely to experience catastrophic systems failures or security breaches because they often have less capital to invest in systems development, testing, and maintenance, and cannot afford the most current hardware, software, or security measures. Frequently, technology staff at the non-major DCMs have multiple responsibilities across the organization; in many cases, the roles held are in conflict with each other, and do not represent best practices in organizational separation of duties. Examination of non-major DCMs on at least a biennial basis is important to the Commission's regulatory mission.

 

< Previous Page | Table of Contents | Next Page >