Appendix 4. Industry Trends and CFTC Resource Requirements

Industry Trend 5: Market Participants Need to Manage Large Volumes of Data in Order to Adapt to Evolving Derivatives Environment

Data, and the ability to analyze and report data, is now more important than ever in the financial industry.  To remain competitive and viable, firms must aggregate data from across asset classes, access data from a wide variety of data sources and markets, calculate real-time risk analytics, and optimize collateral / margin requirements.  For example, the low-tech, high-touch bilateral voice transactions era for swaps trading has begun to make way to electronic trading and clearing.  Furthermore, Dodd Frank regulations require additional data reporting requirements in order to remain compliant.  These changes in the marketplace are requiring additional investments in technology and data infrastructures as well as business processes.  The complexities of adapting to these changes present technical, operational, and business process challenges and opportunities to the industry.  In response to the influx of new types of data from new and existing registrants, the CFTC must build its own information infrastructure and analytical capabilities to support its responsibilities as a first line regulator.  While all CFTC mission activities are impacted, CFTC's biggest requirements are in the surveillance and data infrastructure activities.

Surveillance (Market, Financial and Risk, and Business Analytics)

• The Commission performs three broad types of surveillance: market, financial and risk, and business analytics. All three types of surveillance will require additional resources to support the new regulatory regime.
• Market oversight and surveillance, in particular, are dependent on the ability to acquire large volumes of data and the development of sophisticated analytics to identify trends and/or outlying events that warrant further investigation.
• Equally important to CFTC's surveillance activities is the need for subject matter experts, to interpret the output of automated surveillance systems.  Benefits of the balance between human expertise and technology investments will depend of the need to:
  • Develop new surveillance approaches that will be subsequently operationalized;
  • Analyze data and focus on participants, issues, and trends that have the biggest market impact; and
  • Record, track, and refer potential violations to other divisions (e.g., enforcement) to ensure appropriate follow-up.
• Increasing availability of price information and electronic trade and settlement activity will create the need for analytic capability across the Commission's surveillance areas.

Data Infrastructure

• Currently CFTC extracts, transforms, and loads more than half a billion rows of data per day in support of business analytics, including:
  • Trade record data (every trade on every futures exchange);
  • Large-Trader end-of-day position data;
  • SDR data, including primary economic terms, confirmations, and open swaps positions;
  • Price, volume, and open interest data for futures, options, and swaps risk array data from DCOs;
  • Margin data from DCOs for individual counterparties and clearing members;
  • Segregation Information from DCOs;
  • Product reference data from DCMs;
  • Swap and futures account identification data; and
  • Registration data.
• Data understanding and ingestion is the first priority for the Commission's resources in data infrastructure.  The CFTC has an imperative to aggregate various types of data from multiple industry sources (e.g., DCMs, SDRs, and DCOs) across multiple markets (e.g., futures, exchange-traded swaps, and off-exchange swaps.  The new swaps data is an order of magnitude more complex than futures, independent of notional value).  The increasing complexity, volume, and interrelations of the data set will require significantly more powerful hardware such as massively parallel processing systems to support business analytics.
• Data related to this aggregated market is expanding in size, shape, and complexity and industry adaptation to the Dodd-Frank environment cannot be precisely predicted.  Receipt and analysis of the first wave of registrant reporting will give Commission staff insight into the markets, which can be used to firm-up requirements and designs for internal surveillance systems.  Likewise, the same business analytics tools used for data understanding and ad hoc mining of mature datasets will also be used to automate transparency reporting.
• Ensuring data quality, consistency, and standardization is an important component in enabling accurate data analytics to support the Commission's surveillance activities. Implementation of internationally acceptable and used data standards is key for regulators around the globe to aggregate, analyze, and measure risk.  The Commission will require both subject matter expertise and technology to fulfill these requirements.  This capability must include the ability to define standards for data submission by firms and ingestion by CFTC, managing data quality through a combination of automated and expertise validation, and the creation of automated feedback loops with industry data sources.